significance and power Flashcards
(11 cards)
what is the 3 steps to test the null hypothesis significance?
Assume H0 is true
Fit a model to data and get a test statistic
Calculate the probability of getting test statistic, assuming H0 is true
what are the 3 principles in the misuse of NHST?
- P values are not measuring the probability of getting results by chance or that a specific hypothesis is true
- Statistical significance is not the same as practical importance
- The P value alone is not a good measure of evidence regarding a model or hypothesis
what is power?
the probability of finding an effect assuming one exists in the population
what is B
probability of not finding the effect and is usually 0.2
1- b
what affects power?
- effect size
- no of ppts
- alpha level
what is effect size?
An objective and standardised measure of the magnitude of an effect
A larger value = bigger effect size
Depends on test conducted : cohens d, Pearsons r, partial eta squared
what is number of ppts?
More ppts = more signal and less noise
But you should choose sample size depending on expected effect size
Larger effect size = fewer ppts needed to get a real affect
Smaller effect size = more ppts needed to detect a real effect
what is the size of alpha?
the probability of obtaining a type I error
we contain our P value to this criteria when testing significance
what is the problem with alpha testing?
if we run multiple tests this will increase the rate at which we get a type I error also known as familywise experiment error rate
can account for this by limiting the number of yeses or by using corrections such as bonferroni correction
what is a one tailed test?
we hypothesise there will be a difference in scores and we’re specific about which score will be higher
what is a two tailed test?
we hypothesise there will be a difference in scores but this could be at either direction
