S6) Reviews of Evidence Flashcards
Literature reviews of studies are used in evidence-based healthcare.
Describe the difference between these studies
- Narrative reviews: implicit assumptions, opaque methodology, not reproducible → biased, subjective
- Systematic reviews: explicit assumptions, transparent methodology, reproducible → unbiased, objective
Systematic reviews involve a clearly focused question with explicit statements about which factors?
- Types of study
- Types of participants
- Types of interventions
- Types of outcome measures
What is a systematic review?
A systematic review is an overview of primary studies that used explicit and reproducible methods
Identify the four steps involved in a systematic review
⇒ Systematic literature search
⇒ Selection of the materials
⇒ Appraisal
⇒ Synthesis
What is a meta-analysis?
A meta-analysis is a quantitative synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way
Illustrate the relationship between a systematic review and a meta-analysis
A systematic review will not necessarily include a meta-analysis if, for example, clinical heterogeneity is too great
State the four purposes of a meta-analysis
- To facilitate the synthesis of a large number of study results
- To systematically collate study results
- To reduce problems of interpretation due to variations in sampling
- To quantify effect sizes and their uncertainty as a pooled estimate
To ensure quality criteria, a meta-analysis should have a formal protocol.
What should be specified?
- Compilation of complete set of studies
- Identification of common variable or category definition
- Standardised data extraction
- Analysis allowing for sources of variation
How does one interpret a forest plot?
- Individual odds ratios (squares) with their 95% CI (lines) are displayed for each study
- Size of square is in proportion to the weight given to the study
- The (diamond) is the pooled estimate with the centre indicating the pooled odds ratio (dotted line) and the width representing the pooled 95% CI
- The (solid line) is the null hypothesis OR
What are the three major problems with a meta-analysis?
- Heterogeneity between studies
- Variable quality of the studies
- Publication bias in selection of studie
There are two approaches to calculating the pooled estimate odds ratio (OR) and its 95% CI.
Describe both the fixed effect model and the random effects model
- Fixed effect model – assumes that the studies are estimating exactly the same true effect size
- Random effects model – assumes that the studies are estimating similar, but not the same, true effect size
Compare and contrast the fixed effect model with the random effects model in light of the following:
- Point estimate
- 95% Confidence Interval
- Weighting of the studies
- Hypothesis test for heterogeneity
- Point estimate e.g. odds ratio – is often similar in both models
- 95% CI – is often wider in the Random Effects Model
- Weighting of the studies – is more equal between the studies in the Random Effects Model i.e. greater weighting towards small studies
- Hypothesis test for heterogeneity – low statistical power to detect heterogeneity, often use 10% significance level
What are the causes for variable quality?
- Poor study design
- Poor design protocol
- Poor protocol implementation
What is the reason for publication bias?
Studies with statistically significant / ‘favourable’ results are more likely to be published than those studies with non-statistically significant / ‘unfavourable’ results
What are the consequences of publication bias?
- Any systematic review / meta-analysis can be flawed by such bias
- Publication bias leads to a biased selection of studies towards demonstration of effect