Statistical Tests Flashcards
1
Q
What is a hypothesis?
A
- allows you to state your idea in a specific testable form
- used to describe a working theory about the data sets you’re considering
2
Q
Define ‘null hypothesis’
A
- a working assumption that there’s no difference between the data sets you wish to compare
- i.e. there is no difference/relationship/association
3
Q
Define ‘alternative hypothesis’
A
- assumption that there is a difference between data sets
- i.e. there is a difference/relationship/association
4
Q
Define ‘significance’
A
- a measure of likelihood that the NULL hypothesis is correct
5
Q
Define ‘two-sided hypothesis’
A
- states the difference could be in either direction
- null = NO difference between methods A and B
- alternative = A difference between methods and B
6
Q
Define ‘one-sided hypothesis’
A
- states a difference in a specific direction
- alternative = test results using method A are HIGHER/LOWER than those using method B
- null = test results using method A are NOT HIGHER/LOWER than those using method B
7
Q
What is a ‘type I error’?
A
- false positive (reject the NH when it is true)
8
Q
What is a ‘type II error’?
A
- false negative (accept the NH when it is false)
9
Q
What is the type II error rate (b)?
A
- the probability of incorrectly retaining a false null hypothesis
10
Q
What is meant by ‘power’?
A
- the probability of correctly rejecting a false null hypothesis
- power = 1 - b (type II error rate)
11
Q
How can we reduce the chance of type I error?
A
- choose a lower probability/higher significance
- e.g. P = 0.01
12
Q
What is the issue of choosing a higher significance?
A
- critical value of test statistic increases
- thus probability of type II error increases
13
Q
What are the significance levels (P-values) and what they mean?
A
- P > 0.05 = not significant
- P < or = 0.05 = significant
- P < or = 0.01 = highly significant
- P < or = 0.001 = very highly significant
14
Q
What is a parametric test?
A
- makes particular assumptions about mathematical nature of population distribution from which the samples were taken
- better able to distinguish between true and marginal differences between sample (have greater ‘power’)
15
Q
What is a non-parametric test?
A
- doesn’t assume that data fit a particular pattern, but may assume some characteristics of their distributions
16
Q
What is ‘effect size’ (ES)?
A
- measures the strength of the result is solely magnitude-based
- it does not depend on sample size
17
Q
What is meant by ‘A priori power analysis’?
A
- done in planning phase to determine N (study size)
- necessary to justify project resources in funding processes
- minimise use of animals or risk to patients in clinical research
- involves estimating the sample size required for a study based on predetermined maximum tolerable Type I and II error rates and the minimum effect size that would be clinically, practically, or theoretically meaningful
18
Q
What is meant by ‘Post-hoc power analysis’?
A
- done on completion of study to determine Observed Power
- Necessary to check that your expected and measured ES align well
- i.e. Did you have sufficient subjects to detect differences reliably?
19
Q
What is meant by ‘Sensitivity power analysis’?
A
- done in planning phase when the sample size is predetermined by study constraints e.g. if there are only 20 subjects available in a pilot study
- instead we determine what level of effect we might be able to find, referred to as the minimal detectable effect (MDE) or minimum clinically important difference (MCID)
20
Q
How does sample size affect the effect size?
A
- as sample size increases, the effect size decreases
21
Q
What is the relationship sample size and power?
A
- increase in sample size increases power
- but NOT a linear relationship
22
Q
When would you use a t-test?
A
- comparing means from TWO independent samples
23
Q
What other situations would you use a t-test?
A
- comparing means of paired data
- comparing a sample mean with a chosen value
24
Q
When would you use ANOVA?
A
- comparing means from TWO OR MORE samples
25
Similarity between t-test and ANOVA
- both assume data has a Gaussian distribution and variances of the samples are homogeneous
26
What are the 2 chief non-parametric tests for comparing locations of 2 samples?
- Mann-Whitney U-test
- Kolmogorov-Smirnov test
27
What does the Mann-Whitney U-test assume and what sample's size should it be?
- assumes that the frequency distributions of samples are similar
- sample's size must more or equal to 4
28
What sample's size must Kolmogorov-Smirnov test have?
- more or equal to 4
- samples must have equal sizes
29
Significant differences found with the Kolmogorov-Smirnov test could be due to what?
- differences in location
- or shape of distribution
- or both
30
What are the 2 suitable non-parametric comparisons of location of paired data?
- Wilcoxon's signed rank test
- Dixon and Mood's sign test
31
What is the Wilcoxon's signed rank test?
- used for quantitative data
- assumes that distributions have similar shape
- sample size equal to or more than 6
32
What is the Dixon and Mood's sign test used for?
- paired data scores where one variable is recorded as 'greater than/better than' the other
- sample size equal to or more than 6
33
What are the 2 suitable non-parametric comparisons of location for 3 or more samples?
- Kruskal-Wallis H-test
- Friedman S-test
34
What is the Kruskal-Wallis H-test?
- number of samples is without limit and can be unequal in size
- underlying distributions are assumed to be similar
