CAT: SYSTEMATIC REVIEW Flashcards
Pros of systematic reviews?
Brings together all of the best available evidence for a question
Allows recognition of important treatment effects by combining the results of small trials that individually might have lacked the power to consistently demonstrate differences among treatments
Explicit methods which limit bias in identifying and rejecting studies so conclusions are more reliable and accurate
Large amounts of infromation can be assimilated quickly by healthcare providers, researchers and policy makers
Reasons for heterogeneity can be identified and new hypotheses can be generated about particular subgroups
They have more statistical power than individual studies
Utilises results from RCTs which are the best evidence
Recognised as the strongest design
They can make a meta-analysis to increase the precision of the overall results
Cons of systematic reviews?
Reporting quality varies
Poor external validity
High risk of publication bias
Time and resource intensive
Important data can often be overlooked e.g. grey literature
What is the difference between dichotomous and continuous data?
Dichotomous data is where each outcome is one of only two possible categorical responses
Continuous data can take any value
How can dichotomous results be expressed?
Relative risk (RR)
Odds ratio
Risk different
ARR/NNT
How can continuous results be expressed?
Mean difference or standardised mean difference
What is clinical heterogeneity?
differences in the characteristics of the study with respect to population, intervention or outcomes
In exam practice it said “forms of treatment varied between individual groups”
What is statistical heterogeneity?
variability in estimates of treatment effectiveness when examining a similar outcome or measuring the same effect in multiple studies.
What is a meta-analysis?
When the results of more than 1 study are pooled statistically to create a summary estimate of treatment effectiveness to give a more precise estimate of how well a treatment works than any individual trial
Why are RCTs used for systematic reviews involving therapy?
Less susceptible to selection and information bias so considered the most robust and reliable
What is publication bias?
Failure to publish the results of a study on the basis of issues other than the quality of the study e.g. based on the nature and strength of the findings
What is language bias?
Studies with positive findings are more likely to be published in an England language journal and also more quickly than studies with inconclusive or negative findings
So a thorough SR should not restrict itself to journals written in English
How should studies/data be reviewed in a systematic review?
By at least 2 reviewers working independantly from each other and any differences should be settled by consensus or by a third person
What are the 2 statistical models to combine results for a meta analysis?
Random effects model
Fixed effects model
How can the robustness of a meta-analysis be tested?
By a sensitivity analysis
What is the random effects model?
This model assumes that thr studies are estimating different effects due to random error i.,e. Chance but ALSO due to heterogeneity which is always present
This is used when there is substantial heterogeneity among studies
What is the fixed effects model?
This model assumes all studies are estimating the same underlying effect and variability amongst the studies is just due to random error i.e. Chance. It does not recognise any heterogeneity
This model is used with heterogeneity is low!
What is impprtant about selecting relevant studies for systematic reviews?
Multiple data bases should be used - cochrane, medline, embase
Include unpublished studies, non-English studies, grey literature, made personal contact with experts, raw data from published trials
Reviewing reference lists of included studies
Give clear examples of search terms, sources searched and the retrieval process
They should have a prisma/flow diagram
What should the authors do to assess quality of included studies in a systematic review?
Each trial should be evaluated in terms sof its methodological quality, precision and external validity
Methodological quality - is there likely to be bias?
Precision - is there a measure of likelihood of random errors i.e. confidence interval width
External validity - how generalisable are the results
using a checklist for this can increase the replicability - this is super important!!!
Quality criteria commonly used for RCTs includes: concealed, random allocation; groups with similar known prognostic factors; equal treatment of groups; ITT analysis
what is selection bias?
Systematic differences between baseline characteristics of the groups that are compared due to proper randomisation not being achieved. This means the sample is not representative
What is internal validity?
The extent to which the observed results represent the truth in the population being studied i.e. free from bias. It’s the degree of confidence that the causal relationship you are testing is not influenced by other factors or variables.
What is external validity?
The extent to which you can generalise the findings of a study to other situations, people, settings and measures
What is performance bias?
Systematic differences that occur due to knowledge of interventions allocation, in either the researcher or the participant
Effective blinding reduces this
What is detection bias?
systematic differences between groups in methods for detecting outcomes
E.g, 1 group is screened more regularly
Aka ascertainment and observer bias
What is attrition bias?
Systematic differences between study groups in the number and way participants are lost from a study
What is reporting bias?
a type of bias that occurs when certain results, outcomes, or information are selectively reported or highlighted, while others are suppressed or underreported.
When can a meta-analysis not be done?
If studies are too heterogeneous
If studies have too high a risk of bias e.g. publication
If studies are too low a quality or there is not enough data from a study
If there’s a limited number of studies
What is homogeneity?
What will be seen on a forest plot?
When the results of each individual trial are compatable with the results of others
On a forest plot all of the confidence intervals would overlap
How can you assess heterogeneity in studies?
Cochran’s Q test - which is based off the chi-squared test (X^2)
I^2 statistic
Eyeball test - looking for overlap of CI of trials with the summary estimate
L’Abbe plots
How can you investigate for cause of heterogeneity?
Subgroup analysis
Meta-regression
What is the line of no effect on a forest plot?
This corresponds to the value of 1 for the binary outcome i.e. the risk ratio
This is the position at which there is no clear difference between the intervention and control group
What is a standardised mean difference?
A summary statistic in meta-analysis when the studies all assess the same outcome but measure it in a variety of ways (for example, all studies measure depression but they use different psychometric scales).
So it is necessary to standardize the results of the studies to a uniform scale before they can be combined. The standardized mean difference expresses the size of the intervention effect in each study relative to the variability observed in that study.
What is I squared? How do we interpret it?
This is a measure of the % of variation in effect sizes between studies due to heterogeneity rather than chance
Anything over 25% consider as high!!
What is a subgroup analysis?
When all participants data included in the meta analysis is split into subgroups according to the characteristics of pts and a meta analysis is performed on the sub set
This can investigate heterogeneous results or answer specific questions about a particular characteristic
What is a sensitivity analysis?
A form of subgroup analysis done by quality of the individual studies
systematically varying different elements of the review process to observe how sensitive the overall results are to this change.
If the conclusion results remain consistent across different scenarios then it adds strength to the reliability of the findings and suggests robustness
Advantages of a meta-analysis?
Improves precision by increase the sample size
Allows for objective appraisal of evidence
Heterogeneity between study results can be explained
Increases the generalizability
Answers questions not posed by individual studies
Establishes questions for future RCTs
Reduces time and energy decision-makers spend looking at research
What is a systematic review?
A review that summarises the evidence on a clearly formulated review question according to a predefined protocol, using systematic and explicit/transparent/reproducible methods to identify, select and appraise relevant studies, and to extract, analyse, collate and report their findings.
What does it mean if, on a forest plot, the horizontal line of a trial does not cross the line of no effect?
There is a 95% chance that there is a real difference between the groups
What does it mean if, on a forest plot, the horizontal line of a trial does cross the line of no effect?
Either there is no significant difference between the treatments or that the sample size was too small to allow us to be confident where the true result lies
Outline why a systematic review of RCTs is an appropriate match to a clinical question on treatment effectiveness?
RCTs are an appropriate match for treatment effectivenes because it reduces bias via randomisation which allows for exmination of cause-effect relationships between an intervention and outcome
Systematic reviews bring together the best available evidence and may allow greater precision in estimating effectiveness if combined with a meta-analysis
Why is intention to treat analysis important?
The intention-to-treat analysis preserves the prognostic balance afforded by randomization, thereby minimizing any risk of bias. important for preventing attrition bias
they also mirror actual practice, when not everyone adheres to the treatment, and the treatment people have may be changed according to how their condition responds to it.
What do you get marks for when in the exam they ask you to explain a forest plot?
Outline that the figure shows e.g. figure 2 shows the results of a pooled analysis of 7 RCTs
Identifying individual effect and confidence intervals for each study e.g. results are expressed in terms of whether treatment was associated with an increased rate of xyz… 95% confidence intervals are presented for each study and these represent…
Identifying the null effect and say how it represents the RR of 1 so no association between different interventions
Identifying and correctly interpreting the overall summary
Heterogeneity or weighting
Examples for answering: what other outcomes would you like to see in studies?
Patient outcome measures
Side effects of drugs
Quality of life
Patient acceptibility/satisfaction
What is GRADE?
Grading of Recommendations, Assessment, Development, and Evaluations
A systematic approach to rate the certainty of evidence in systematic reviews and other evidence syntheses
What is methodological heterogeneity?
variability in study design and quality across different studies investigating a similar topic
In systematic reviews, why are outcomes generally not used as part of the eligibility criteria for including studies?
As systematic reviews aim to include all rigorous studies of a particular comparison of interventions in a particular population of participants, regardless of the outcomes measured or reported.
Why might a systematic review avoid using newer diagnostic criteria for defining diseases or conditions of interest?
Newer diagnostic criteria might not have been used in earlier studies, leading to the exclusion of valuable data.
What are examples of inactive control interventions, and how do they differ from active control interventions?
Inactive control interventions include placebo, no treatment, standard care, or a waiting list control. They differ from active control interventions, which involve an active element, such as a different variant of the same intervention, a different drug, or a different kind of therapy.
Why is it important to consider variation in the intervention in a systematic review?
As significant differences in dosage/intensity, mode of delivery, frequency, or duration can lead to substantially different effects on the participants and outcomes of interest. This could affect theinternal validity and reliability of the review’s results, so it might be necessary to restrict the variation to ensure consistency across the included studies.
Why is it essential to consider both adverse effects and beneficial effects in a systematic review’s outcome assessment?
It is crucial to assess both adverse and beneficial effects to ensure a balanced and comprehensive understanding of the intervention’s impact. This helps in determining the overall risk-benefit ratio and informs clinical decision-making.
What differentiates primary outcomes from secondary outcomes in a systematic review?
Primary outcomes are the key focus of the review and form the basis for major conclusions about the intervention’s effects. Secondary outcomes are those that aren’t chosen as primary outcomes but still provide important additional insights. They may include outcomes specific to certain comparisons or additional outcomes that the review intends to address, such as surrogate measures or laboratory tests.
How does publication bias impact meta analysis findings?
Abstracts reporting insignificant findings are excluded so there will be an overestimation of the effects = reduces the internal validity
Reduces the number of studies so will reduce the power of the study
Name some methods used in systematic reviews to check for the existence of publication bias?
Funnel plots
Egger regression test
Fail Safe sample size
Begg-Mazumdar’s test
Cumulative meta analysis
In a systematic review, what condition must be met before individual study results can be combined in a meta-analysis?
There must be a reasonable degree of homogeneity among thr studies
Examples of groups of outcomes in a study
patient outcomes
clinical outcomes
HRQoL outcomes
health economics
What is Egger’s test?
A statistical method to detect publication bias
Uses linear regression to detect assymetry in a funnel plot!
What is important about the specific search strategy in systematic reviews?
There should be a clear search strategy with a description of the approach to searching
There should be evidence of a wide range of logical and relevant keywords and terms
There should be evidence that subject headings and indexing terms were used
Any limits on the search should be described and their potential impact explained e.g. dates, language
What is important about study eligibility in a systematic review?
The review should have pre-defined objectives and eligibility criteria
Is there were any restrictions on the eligibility criteria, were they appropriate?
Did the types of included studies match the review question?