SCC Chapter 2 ​Statistical Conclusion Validity & Internal Validity Flashcards Preview

ECPY 767 (Final) > SCC Chapter 2 ​Statistical Conclusion Validity & Internal Validity > Flashcards

Flashcards in SCC Chapter 2 ​Statistical Conclusion Validity & Internal Validity Deck (14):

Validity is a property of inferences, not...

Designs or inferences


Three questions to ask in exploring threats to validity

1. How would this threat apply in this case?
2. Is there evidence that the threat is plausible rather than just possible?
3. Does the threat operate in the same direction as the observed effect, so that it could partially or totally explain the observed findings?


Ruling out threats is a ________ enterprise



Statistical conclusion validity concerns what two related statistical inferences?

1. whether the presumed cause and effect covary
2. how strongly they covary


What are 8 threats to statistical conclusion validity?

1 Low statistical power
2 Violated assumptions
3 Fishing & the error rate problem
4 Unreliability of measures
5 Restriction of range
6 Unreliability of treatment implementation
7 Extraneous variance in experiment
8 Heterogeneity of units
9 Inaccurate effect size estimation


Fishing & the error rate problem refers to

inflated type I & II error rates


Restriction of range refers to

floor or ceiling effects


Unreliability of treatment implementation refers to

poorly standardized treatment


Extraneous variance in experiment refers to

distracting noises, administrative changes, etc.


Heterogeneity of units refers to

obscures covariation between treatment and outcome


Inaccurate effect size estimation refers to

outliers, wrong effect size measure


What are 8 ​Methods to increase power?

1. Use matching, stratifying, blocking
2. Measure and correct for covariates
3. Use larger samples
4. Use equal cell sample sizes
5. Improve measurement
6. Increase the strength of treatment
7. Increase variability of treatment
8. Use a within-participants design


What is Internal Validity

inferences about whether observed covariation reflects a causal relationship in the form in which the variables were manipulated or measured.


What are 11 threats to internal validity?

1. ambiguity of temporal precedence
2. cohort effects (cross sectional data)
3. inadequate comparison or control group(s)
4. selection
5. history
6. maturation
7. statistical regression
8. differential attrition/mortality
9. testing
10. instrumentation
11. additive and interactions effects of threats to internal validity