Study Design: Meta-Analysis and Systematic Reviews Flashcards
What is a systematic review?
Aims to answer a defined research question by collecting and summarising empirical evidence – usually published in the scientific literature - that fits pre-specified eligibility criteria. Looking at more than one study.
What is a meta-analysis?
Refers to statistical techniques used in a systematic review to integrate the results of studies matching the eligibility criteria.
What is the purpose of a systematic review?
Individual studies usually provide insufficient power to answer a research question.
Furthermore, multiple studies of the same research question often lead to inconsistent or even opposite results.
In order to provide more generalizable conclusions, researchers can conduct a systematic review of the primary studies on a particular research question to provide a comprehensive summary of our knowledge at the time of the review.
What are the main advantages of a systematic approach?
Transparent process because of the explicit methods in identifying and rejecting studies.
A meta-analysis will increase the power of the study and enhance the precision of estimates of treatment effects, accounting for sample size, and uncertainties.
Systematic reviews may demonstrate the lack of adequate evidence and thus identify areas where further studies are needed.
What is stage I of a systematic review?
Planning the review
– The authors need to clearly define the research question to be addressed.
– This question is usually framed around the definition of study participants, intervention (exposure), outcomes and study designs of interest.
What is stage II of a systematic review?
– Identification of research : requires clearly defined search criteria and a thorough search of all published literature (including exhaustive searches of reference lists, conference proceedings and contact with researchers in the field).
– Selection of studies : inclusion and exclusion criteria should be defined a priority; these are likely to be based on factors such as study design, year, sample size, completeness of information, study quality etc.
– Study quality assessment : study quality can be assessed against recognized or user-defined criteria, usually to establish whether various biases are likely to exist in the in study (e.g. selection bias, measurement bias, attrition bias/loss to follow-up).
What is stage III of a systematic review?
– Reporting and dissemination : study details need to be abstracted from each eligible study along with the effect estimate (or details that allow an effect estimate to be calculated).
– These details need to be tabulated in a meaningful way, including, where appropriate, details of populations, interventions/exposure, outcomes and study design, and a summary of the findings.
– The last step consists of estimating an overall effect by combining the data, if a meta-analysis is deemed appropriate.
What are the advantages of meta-analyses?
– More subjects can be included than any single constituent study, producing a more reliable and precise estimate of effect;
– Differences (heterogeneity) between published studies can be identified and explored.
Generate a pooled overall risk estimate
– Produce a more reliable and precise estimate of effect
– Explore differences(heterogeneity) between published studies
– Identify whether a publication bias is occurring
What are the limitations of meta-analyses?
– If the studies are too heterogeneous, it may be inappropriate, even misleading to statistically pool the results from separate studies.
– Publication bias
– Labour intensive
– Inconsistency of results (studies differs with respect to populations interventions/exposure, outcomes, study design, clinical differences, methodological differences, unknown study characteristics)
– Low study quality
How are statistics visualised in a meta-analysis?
A forest plot is the most common way of visually summarizing the results of a meta-analysis. This is a graphical representation of the results from each study included in a systematic review, together with the combined meta-analysis result.
What is publication bias?
Refers to the greater likelihood of research with statistically significant results being published in peer-reviewed literature in comparison to those with null or non-significant results. Failure to include all relevant data in a meta-analysis may mean the effect of an intervention/exposure is over- or under-estimated.
What are funnel plots used for?
Publication bias in meta-analyses can be explored using funnel plots, which show whether there is a link between study size (or precision) and the effect estimate.
How is heterogeneity explored in studies?
Studies that are trying to answer the same question may still differ with respect to the exact population, interventions/exposure, outcomes and designs used.
Even where these factors are homogeneous, heterogeneity may still exist because of clinical differences, methodological differences or unknown study characteristics.
Heterogeneity can be explored using Galbraith (radial) plots. But remember, if too much heterogeneity exists, it might not be appropriate to pool the studies.
What are the limitations of conducting systematic reviews?
If there are too few studies matching the eligibility criteria defined, a systematic review might not add much to the field.
If the methodological quality of studies is inadequate, then the findings of reviews of this material may also be compromised.
Publication bias can distort findings because studies with statistically significant results are more likely to get published.
What questions need to be answered to critically appraise a systematic review?
- Was a clear, predefined question answered?
- Was a comprehensive search for relevant literature carried out?
- Was methodological quality of each study assessed appropriately?
- Was heterogeneity explored?
- How credible is the evidence?
- Check guidelines for reporting
