Statistical Hypothesis Tests

Question 1

What are hypothesis tests?

Answer 1

• tests one possible value (unlike how CI tests a range)
• determine the strength of evidence, provided by data, against the proposition the single value is the true mean
• work by comparing an estimate obtained from data with what we expect to find under H0

Question 2

What is the research hypothesis?

Answer 2

The hypothesis that the research is designed to investigate.

Question 3

What is the null hypothesis?

Answer 3

H0: hypothesis we test, typically a skeptical reaction to the research hypothesis.

Question 4

What is the alternative hypothesis?

Answer 4

H1: specifies a departure from H0, typically corresponds to the research hypothesis.

Question 5

What is a test-statistic?

Answer 5

Used to quantify how much our data-estimate differs from the value in our H0.

Question 6

What does the t-test statstic show us?

Answer 6

t-stat will be small if H0 is true.

Question 7

When comparing two means, what is the null hypothesis?

Answer 7

That the difference between the values is 0.

Question 8

How is the p-value obtained?

Answer 8

Compare t-stat to a reference t-distribution, p-value is obtained.

Question 9

What is the p-value?

Answer 9

The probability of observing a test statistic at least as extreme as the one observed, given H0 is true.

Question 10

What p-values count as strong evidence against H0?

Answer 10

0. 1 - no evidence
1. 05 - weak evidence
2. 01 - some evidence
3. 001 - strong evidence

Question 11

When do we reject H0?

Answer 11

When the p-value is small. We reject H0 in favour of H1.

Question 12

What is the non-parametric alternative to the t-test?

Answer 12

The Mann-Whitney test.

Question 13

What does non-parametric mean? What is another name for it?

Answer 13

Methods to deal with non-normal data.

“distribution-free”

Question 14

How is the Mann-Whitney test done?

Answer 14

• data is converted into ranks:
  H0: there is NO difference between the ranks of each group
• if H0 true, average ranks of each group about equal
• if ranks are very different we have reason to believe that the samples were not drawn from the same population

Question 15

What is the procedure for the Mann-Whitney test?

Answer 15

1. combine the scores from both groups and rank them in order of increasing size
2. take each group and calculate the sum (Wn, n is the group)
3. find the test stat for each group (Un)
4. choose either Un value as the test stat (Ustat), usually the smaller value is used
5. the test stat is then compared to the relevant distribution
6. U is approx normally distributed, standard value (z) can be found
7. we can now compare this test stat to the distribution of the test stat under H0.

Question 16

How is the standardised value for the Mann-Whitney test found?

Answer 16

z = (U - 0.5 - μU)/σU

Question 17

What is statistical significance?

Answer 17

H0 is rejected, indicates that differences in group means are not likely due to sampling error.

Question 18

What is practical significance?

Answer 18

Asks are the differences between samples big enough to have real meaning?

Question 19

What is a type I error?

Answer 19

Conclude there is a genuine difference when there is in fact not (reject H0 when its true).

Question 20

What is a type II error?

Answer 20

Conclude there is no difference when in reality there is (fail to reject H0 when it’s false).

Question 21

What does ANOVA mean?

Answer 21

ANalysis Of VAriance

Question 22

What is an ANOVA test?

Answer 22

One test that compares all means simultaneously.

• H1: at least one mean is different from at least one of the others
• H0 is true: expect group means to be similar and any differences to be from sampling variation alone

Question 23

What type of statistic does an ANOVA test produce?

Answer 23

F-test statistic.

Question 24

What is s^2B and how is it calculated (for F-test stat)?

Answer 24

• difference between group mean and overall mean
• mean sq between groups (ANOVA table)

s^2B = (Σ ni (x̄ i - x̄)^2)/k-1 = SSB/ k - 1

Question 25

What is s^2W and how is it calculated (for F-test stat)?

Answer 25

• variability between groups
• mean sq residuals (ANOVA table)

s^2W = (Σ (ni - 1) si^2)/n(tot) - k = SSW/n(tot) - k

Question 26

How does the F-stat look when H0 is false?

Answer 26

Should be large.

Question 27

What is Tukey’s Difference test?

Answer 27

• used for multiple comparrisons

- output: difference, lower and upper bounds of CI, adjusted p-value

Question 28

What is the Kruskal-Wallis test?

Answer 28

• does not assume normality (non-parametric)
• assumes different groups have the same stand. deviation
• H0: all medians are the same
• data is assigned ranks: smallest score = 1, ties = mean of the ranks concerned (eg 1.5)
• average rank calculated for each group

Question 29

Give a summary of Hypothesis Testing.

Answer 29

1. state H0, H1
2. statstical test (eg. F-test, Wilcoxon)
3. significance level (α)
4. sampling distribution
5. finding p-value
6. do we reject H0?
7. interpret results based on 6.

Question 30

What is the z-test used for?

Answer 30

• comparing probabilities

- test statistic can be calculated

Question 31

What is the null hypothesis in a χ^2 goodness of fit test?

Answer 31

H0: expected count = total x specified cell probability

Question 32

When do we reject H0 for a χ^2 test?

Answer 32

The larger the test stat (χ0^2), the stronger the evidence against H0.

Question 33

How do we know if a χ^2 test stat is extreme under H0?

Answer 33

Compare χ^2 test stat (χ0^2) with a value obtained from a reference chi-squared dist (χdf^2).

Question 34

Explain how the values in an ANOVA table are found from the first two columns (Df and Sum Sq)?

Df Sum Sq Mean Sq F-value Pr(>F)

Answer 34

Mean Sq - Sum Sq/Df

F-ratio - Mean Sq Group/Mean Sq Residuals

p value - F ratio (above) associated with Df Group and Df Residuals

Question 35

What do the columns of a Tukey’s HSD test mean?

diff lwr upr p adj

Answer 35

diff - estimated difference of means
upr - upper bound of CI
lwr - lower bound of CI
p adj - evaluating H0, Pr that the difference between means is 0