Flashcards in SCC Chapter 2 Statistical Conclusion Validity & Internal Validity Deck (14)
Validity is a property of inferences, not...
Designs or inferences
Three questions to ask in exploring threats to validity
1. How would this threat apply in this case?
2. Is there evidence that the threat is plausible rather than just possible?
3. Does the threat operate in the same direction as the observed effect, so that it could partially or totally explain the observed findings?
Ruling out threats is a ________ enterprise
Statistical conclusion validity concerns what two related statistical inferences?
1. whether the presumed cause and effect covary
2. how strongly they covary
What are 8 threats to statistical conclusion validity?
1 Low statistical power
2 Violated assumptions
3 Fishing & the error rate problem
4 Unreliability of measures
5 Restriction of range
6 Unreliability of treatment implementation
7 Extraneous variance in experiment
8 Heterogeneity of units
9 Inaccurate effect size estimation
Fishing & the error rate problem refers to
inflated type I & II error rates
Restriction of range refers to
floor or ceiling effects
Unreliability of treatment implementation refers to
poorly standardized treatment
Extraneous variance in experiment refers to
distracting noises, administrative changes, etc.
Heterogeneity of units refers to
obscures covariation between treatment and outcome
Inaccurate effect size estimation refers to
outliers, wrong effect size measure
What are 8 Methods to increase power?
1. Use matching, stratifying, blocking
2. Measure and correct for covariates
3. Use larger samples
4. Use equal cell sample sizes
5. Improve measurement
6. Increase the strength of treatment
7. Increase variability of treatment
8. Use a within-participants design
What is Internal Validity
inferences about whether observed covariation reflects a causal relationship in the form in which the variables were manipulated or measured.