Systemic Reviews Of RCTs

Question 1

Los

Answer 1

• Revisit framework for using population based research in clinical practice (Assess-Access-Appraise-Act)
• Introduce the need for and importance of systematic reviews in healthcare
• Discover what systematic reviews are
• Learn where to find systematic reviews on the
effectiveness of a treatment (Access)
• Understand key methodological aspects of a systematic review on the effectiveness of a treatment (Appraise – validity)
• Consider numerical results of a systematic review on effectiveness (Appraise – results)
• Understand what to consider with regard to applying findings (Act)

Question 2

What are the key steps to using population based research to help with clinical decisions

Answer 2

➢ Assess
– Formulate a clear clinical question (PICO)
– Identify the study design that would provide the most
accurate answer
➢ Access
– Search for and retrieve the research
➢ Appraise
– Identify the strengths and weaknesses of the research you find and understand what the results mean
➢ Act
– Apply the evidence, taking into account its limitations, alongside other information eg patient preference

Question 3

Why should we not use only one trial for research

Answer 3

There are differences between trials
Many trails available

Question 4

Why may there be differences between RCTs

Answer 4

• chance
• differences in people
• differences in intervention / comparator
• differences in care other than that being investigated
• differences in the way treatment effects are assessed
• differences in follow-up
• reporting of the available evidence
• selection from the available evidence

Question 5

What is publication bias

Answer 5

• When the publication of research results depends on their nature and direction.
– Positive results bias
- outcome reporting bias

Question 6

What is positive result bias

Answer 6

– Positive results bias - when authors are more likely to submit, or editors accept, positive than negative or inconclusive results

Question 7

What is outcome reporting bias

Answer 7

– Outcome reporting bias - when several outcomes within a trial are measured but are reported selectively depending on the strength and direction of those results.

Question 8

What are the issues with using trails

Answer 8

• Large Volume of Information
• Low-quality information drowns out high-quality information?
• Cannot find/read the right evidence for my/my patient’s/my organisations specific needs
• Must critically evaluate the methodological rigor and statistical analyses of a study,

Question 9

What is the definition of review and its benefits

Answer 9

Any attempt to summarise information

• Volume of literature condensed
• New information made accessible
• Some reviews are extremely good and have taken considerable time to find all the relevant information and review the topic well

Question 10

What is a systematic review

Answer 10

the research method generally used to synthesis
multiple pieces of research evidence on the effectiveness of a treatment
(and other types of research question)

• Review of all literature
• On one particular topic
• using scientific methods

Top of the “hierarchy of evidence”

Question 11

Features of a systematic review

Answer 11

• Clear question being addressed by the review
• An explicit and detailed statement of the methods used
• Comprehensive ascertainment of the literature relevant to the review question
• Steps to reduce bias in the review process /documenting of any bias
Can be with or without:
• An appropriate numerical summary of the size of effect with confidence intervals

Question 12

What are the stages of a systematic review

Answer 12

Define question (pico) - may need to reformulate
Write a protocol
Search for evidence/ studies
Select Relevant Studies from Search
Appraise Studies
Extract, Analyse & Summarise Data
Interpret Review Results
Discussion & Conclusions

Question 13

How are components of a question clearly defined

Answer 13

Through pico

P How would I describe the group of patients?
I What main intervention am I considering?
C What is the main alternative
intervention?
O What outcome(s) are important?
- context

Question 14

Why is a protocol written

Answer 14

To describe methods
Ensure methods and problems are considered
Avoids introducing bias

Question 15

How to select relevant studies from search

Answer 15

Unbiased selection of studies for
review.

Two reviewers independently using pre-defined criteria (based on question(PICO)) to select relevant studies from the search Reduce errors

Question 16

How to extract, analyse and summarise data

Answer 16

Two reviewers independently using pre defined approach to identify relevant data in studies
Predefined analysis of data
Use of validated appropriate methods of numerical data synthesis (meta analysis)
Pre defined exploration of possible effect modifiers

Question 17

What does a systematic review essentially do

Answer 17

Combine data from similar studies to produce a Summary Effect of Intervention (I) vs Control (or Comparator) (C) on each Outcome (1,2,…O) in the specified Population (P)
Can do this for many outcomes
To give stronger and more precise measure of te intervention

Question 18

What will a meta analysis contain

Answer 18

Will have a list of:
Study
Intervention
Comparator
Forest Plot
Relative risk

Question 19

What is the central horizontal line in forest plots

Answer 19

If box on the line - Signifies No difference in the groups of the trials for the outcome
Intervention vs comparator

Position of horizontal line depends on metric
Relative risk or odds ratio = 1
Mean difference = 0

Question 20

What is on the x axis

Answer 20

On each side of the x axis
Should be intervention and comparator
To see which results favour which

Question 21

What do squares represent on the forest plot

Answer 21

Middle of square represent the results
Each square has horizontal line through it which represents confidence interval
Size of square relates to weight the study is given in the meta analysis e.g. bigger sample size = bigger square

Question 22

What is the diamond in the forest plot

Answer 22

Central position is the Summarised result from all the data in the studies
Width is confidence interval around summary estimate

Question 23

What should be addressed in discussions and conclusions

Answer 23

Considers implications of the review findings in the context of current practice and knowledge and outlines specific future research
Summary of findings

Question 24

What is the difference btw meta analysis and systematic reviews

Answer 24

• Systematic review:
– explicit methods to identify and review all of the literature on a specific topic in order to answer a specific question.
• Meta-analysis:
– statistical technique(s) used to combine the quantitative results from a number of studies

Not all systematic reviews contain a meta analysis - sometimes too much variation btw the studies to combine them together

Question 25

Advantages of systematic reviews

Answer 25

• Can be the best source of evidence on the effects and effectiveness of healthcare interventions
• Assimilation of large amounts of research evidence
• Provide reliable unbiased estimates of effect
2 independent reviewers
• Increase precision of estimates of effects
Width of diamond small. Narrower confidence intervals around summary estimate. Increases precision.
• Provide information about the generalise ability and consistency of effect
• Identify what information is missing
• Useful for making decisions

Question 26

What do systematic reviews prevent

Answer 26

Unnecessary repetition. Stops patients going into unnecessary trails.
The more studies added to the review the narrower the confidence intervals get. Increased precision.
But still the same results and conclusion

Therefore need systematic review before any new research is done.

Question 27

Where to find systematic reviews

Answer 27

Epistemonikos
– Systematic review records from many sources including Pubmed/MEDLINE, Embase,
Cochrane Database of Systematic Reviews

Cochrane library
– Cochrane Database of Systematic Reviews (CDSR) - includes all Cochrane Reviews (and
protocols) prepared by Cochrane Review Groups.

MEDLINE (and/or EMBASE) (via FindIt)
– need to use a “filter” for reviews

NHS evidence

Question 28

What is critical appraisal

Answer 28

• The systematic identification of strengths and weaknesses of pieces of information &
• The application of that information within the limits identified
Broken down into internal and external validity

Question 29

What is internal validity

Answer 29

extent to which a research study establishes a trustworthy cause-and-effect relationship.1 This type of validity depends largely on the study’s procedures and how rigorously it is performed.

Question 30

What is external validity

Answer 30

– The degree to which the reviews conclusions apply to other persons in other places and at other times
Internal validity needs to be strong before considering the strength of external validity

Question 31

What checklist is used for critical appraisal

Answer 31

CASP

Three part process:
1. Is the review trustworthy / valid (internal validity)? Screening questions
Detailed questions
2. What were the review’s results ?
3. How will the results help (external validity)?  Relevance

Question 32

What are the screening questions for critical appraisal

Answer 32

1. Did the review ask a clearly-focused question?
2. Did the review include the right type of study?
3. Did the reviewers try to identify all relevant studies?
4. How did the reviewers decide which studies to include and did they assess the quality (validity) of the included studies?
   5.If the results of the studies have been combined together (e.g. meta-analysis) was it reasonable to do so?
   6.How are the results presented and what is the main result?
   7.How precise are these results?
   8.Can the results be applied to the local population?
5. Were all important outcomes considered
6. Should policy or practice change as a result of the evidence
   contained in this review?

Question 33

Did the review ask a clearly-focused question?

Answer 33

Consider if the question is ‘focused’ in terms of:
– the population studied
– the intervention given
– the comparator
– the outcomes considered

Question 34

Did the review include the right type of study?

Answer 34

Consider if the included studies:
– address the review’s question
– have an appropriate study design

Question 35

Did the reviewers try to identify all relevant studies?

Answer 35

Consider:
– which bibliographic databases were used
– if there was follow-up from reference lists
– if there was personal contact with experts
– if the reviewers searched for unpublished studies
– if the reviewers searched for non-English-language studies

Question 36

How did the reviewers decide which studies to include and did they assess the quality (validity) of the included studies?

Answer 36

Consider:
– if a clear, pre-determined strategy was used to determine which studies were included in the review and how their quality was assessed
Look for:
– defined selection criteria (based on PICO)
– defined validity process (e.g using CASP checklist)
– more than one assessor

Question 37

If the results of the studies have been combined together (e.g. meta-analysis) was it reasonable to do so?

Answer 37

Consider whether:
– the studies have similar characteristics
– the results of each study are clearly displayed – the results were similar from study to study
Heterogeneity (difference)
Methodological – different study designs combined
Clinical – different patient characteristics, settings, interventions
Statistical – differences in reported effect size between the trials – look for statistical tests (Chi2, I2,)
– the reasons for any variations in results are discussed

Question 38

How are the results presented and what is the main result?

Answer 38

Consider:
– how the results are expressed (e.g. relative risk, mean difference etc.)
– how large this size of result is and how meaningful it is
– how you would sum up the bottom-line result of the review in one sentence

Question 39

How precise are these results?

Answer 39

Consider:
– if a confidence interval were reported. Would your decision about whether or not to use this intervention be the same at the upper confidence limit as at the lower confidence limit?
– if a p-value is reported where confidence intervals are unavailable
On the forest plot = width of the diamond/ horizontal line through the square

Question 40

Can the results be applied to the local population?

Answer 40

Consider whether:
– the population sample covered by the review could be different from your population in ways that would produce different results
–your local setting differs much from that of the review – you can provide the same intervention in your setting

Question 41

Were all important outcomes considered?

Answer 41

Consider outcomes from the point of view of the: – individual
– policy makers and professionals
– family/carers
– wider community

Question 42

Should policy or practice change as a result of the evidence
contained in this review?

Answer 42

Consider:
– whether any benefit reported outweighs any harm and/or cost
– if this information is not reported can it be filled in from elsewhere?