Meta-Analysis Flashcards
What is the logic of a meta analysis?
Change the focus to the direction and magnitude of the effects across studies - effect size
What is the significance of effect size in a meta-analysis?
It is the DV
Standardises findings across studies
Independent of sample size
Represents magnitude and direction of effects
Comparable
What is the effect of statistical significance?
Highly dependent on sample size - small = larger variability than large
Effect can be non-sig with high variability and sig when low
What are the criteria for a meta analysis?
Quantitative results
Same constructs and relationships
Findings configured in comparable statistical form
Measure from studies comparable
What is the replication continuum?
Direct replication - same study
Conceptual replications - tests same fundamental hypothesis differently
Less directly related
Closer to pure replication = easy to argue comparability
How do you decide what to include in a meta analysis?
Studies have explicit inclusion and exclusion criteria
Study quality
Grey literature may include publication bias - important failures to replicate missing
What are the strengths of meta analyses?
Systematic appraisal of scientific RQ
Provides quantitative measure of effects across studies
Reveal relationship obscured in other approaches
Handle large number of studies
What are weaknesses of meta analyses?
Effortful and time consuming
Comparability often difficult to achieve
Selection bias is a risk - null effects not reported
Can include weak studies
What are forest plots?
Used to depict individual effect sizes and compute aggregate effect
Display effect size and confidence interval
Show direction of effects relative to null
Weigh contribution of each study
What are funnel plots?
Used to plot effect size relative to sample size
Show overall pattern of effects relative to null effects
Used to compute aggregate effect size
Used to investigate publication bias
What are the features of funnel plots?
Effect size vs sample size
Small = more variability - inverted funnel
Asymmetrical plot suggests missing studies
Can occur if smaller studies have large effects
Test for heterogeneity
What is the many labs approach?
Uses meta analysis to combine independent studies conducted independently across many labs internationally
What is the file draw problem?
Selectively reporting findings
Tend to report more positive than negative
