Precision and Statistical Issues Flashcards
(22 cards)
What are two components of Accuracy
- Validity
2. Precision
What is Accuracy
The degree to which a measurement represents the true value of what is being measured.
Validity
타당성
the degree to which the results and conclusions of a study reflect the true state of nature.
Also the degree to which the study is free of systematic error (is unbiased)
Precision
정확/신중
the degree to which random error affects the parameter estimates within a study
The degree to which statistical estimates would be reproducible with repeated sampling
two types of validity
- internal validity
2. external validity
internal validity
the degree to which a study is free of bias or systematic error
internal validity is prerequisite of external validity
external validity
the extent to which the results of a study may be applied to populations or groups that were not subjects of the study
bias
systematic error in study design or conduct that leads to results that deviate from the truth
If study is unbiased…
repeatedly conducting that study will produce correct results on verage
potential sources of bias
systematic measurement error (as distinguished from random sampling error)
flaws in study conception, design or analysis
conscious or unconscious selection in obtaining or interpreting results
measure of precision
reciprocal of the VARIANCE of a measure or estimate
increasing the precision of an estimate is equivalent to
reducing its variance
measure of imprecision
standard error: series of repeated estimates of a single quantity
larger standard deviation/error/variance means
less precision in your estimates
measure of both precision and imprecision
confidence interval: looking at the CI gives an idea of how precise or imprecise the measures are.
testing for the presence of effects
point estimates significance test (Fisher) Hypothesis test (Neyman-pearson) p-cause Bayesian analyses
measuring effect sizes
point estimates and CI
p-value functions (= ci functions)
regression modeling
Bayesian analyses
H0
there is no effect (lack of difference), so if outcome is positive, reject H0
p-value
probability of obtaining a value of the test statistic for that association equal to, or more extreme than, the value actually observed, if H0 is true and if in fact the statistical model used to derive the test statistic is valid and no bias is present
p-value is sensitive to sample size
very small effect in very large sample can be statistically significant even they don’t mean anything
p-value is affected by precision
but provides no or very little information about precision
p-value nd effect size and sample size
p-value mixes information about effect size and sample size; it is difficult to tell which is more important in a given case
