Hypothesis Testing Flashcards
Step 1 of hypothesis testing
Define the null hypothesis
H0 = no difference in …
Step 2 of hypothesis testing
Define alternative hypothesis
HA = there is a difference in…
Statistical notation of hypothesis testing
H0: μ1 – μ2 = 0
HA: μ1 – μ2 =/= 0
Step 3 of hypothesis testing
Choose a significance level for the test to determine if the result is statistically significant
Typically 0.5 (5%)
Step 4 of hypothesis testing
Perform an appropriate statistical test
Statistical test for numeric outcomes
T-test
ANOVA/ANCOVA
Linear regression
Statistical tests for categorical outcomes
Chi-squared test
Logistic regression
P-value
probability of seeing an effect of the observed magnitude or greater if the null hypothesis were true
Step 5 of hypothesis testing
Use p-value to weigh up strength of evidence against null hypothesis
-If the p-value is high the result is probable under the null hypothesis… so it is likely the null hypothesis is true
The smaller the p-value, the less likely it is we would see our observed result under the null hypothesis
If the p-value is smaller than our significance level (so < 0.05 in our example) we reject the null hypothesis and declare the result statistically significant
Disadvantages of statistical significance
does not mean a difference/association/effect is clinically important
Type I error
False positive
occurs if an investigator rejects a null hypothesis that is actually true in the population
Type II error
False negative
occurs if the investigator fails to reject a null hypothesis that is actually false in the population
Odds ratio null value
1
Hypothesis testing assumptions
Values are normally distributed
Sample size is sufficiently large
Central limit theorem
Means of repeat samples are normally distributed around population mean, even if population isn’t normally distributed
