Week 4

Question 1

Statistical power

Answer 1

• what is the probability that a study will detect an effect when there is an effect there to be detected

Question 2

Type 1 and Type 2 errors

Answer 2

Type 1 there is none but you’ve said there is

Type 2 - blind to the actual difference, you’ve said there is none

Question 3

Alpha

Answer 3

• Alpha is the probability that we will reject the null hypothesis when we shouldn’t
• That we say there is an effect when there isn’t one
• This is our Type I Error!
  <5% (

Question 4

Why not use a tiny alpha value

Answer 4

• Because there is a relationship between alpha and beta
• If we choose a tiny alpha value, we will make it difficult to reject the null hypothesis (Type II errors very common!)
• If we choose a larger alpha value, Type II errors will be less common
• Sensitivity vs Specificity
• The risk of false positive (Type I) vs false negative (Type II)

Question 5

Power

Answer 5

• Statistical power is the likelihood that a study will detect an effect when there is an effect there to be detected
• What is the probability of a correct decision of rejecting the null hypothesis when it is false

Power = 1 – Probability of a false negative (Type II error)
Power = 1 – β

Question 6

Factors affecting statistical power

Answer 6

Alpha level
Error variance
Sample size
Effect size

Question 7

Sample size and statistical power

Answer 7

• Sample size works in the same way that error variance works
• As we test more people we are able to better describe a distribution
• Our hypothetical distributions (based on our samples) gets smaller

Question 8

Effect size

Answer 8

• Effect size is the relative distance between our null and true distributions
• This distance is measured in standard deviation units
• An effect size of 0 (zero) would mean no difference between groups (a “perfect” null result)
• Effect size increases as two or more groups become “more” different from each other
• This can help tell us if differences are practically meaningful

Question 9

Effects size measurements

Answer 9

Main Effect (ANOVA)

• Eta Squared
• Omega Squared

Multiple Comparisons (Planned contrast or Post-hoc)

• r
• Cohen’s d

Question 10

Eta squared

Answer 10

• Used for main effect
• Small (.01); Medium (.09); Large (.25)

𝜂^²= (𝑆𝑆)_𝐵𝑒𝑡𝑤𝑒𝑒𝑛 / (𝑆𝑆)_𝑇𝑜𝑡𝑎𝑙

Question 11

Omega squared

Answer 11

• Used for main effect
• Most accurate measure of effect size for main effect
• Small (.01); Medium (.06); Large (.14)

𝜔^2= (𝑆𝑆_𝐵 − (𝑑𝑓_𝐵∗𝑀𝑆_(𝑊))) / (𝑆𝑆_𝑇 + 𝑀𝑆_𝑊 )

Question 12

Effect size for planned contrasts

Answer 12

• r
• Used for follow-up tests
• Particularly useful for planned contrasts
• Small (.10); Medium (.30); Large (.50)

𝑟= √ (𝑡^2 / (𝑡^2+𝑑𝑓))

Question 13

Effect size for post-hoc tests

Answer 13

• Cohen’s d
• Used for follow-up tests
• Can be used for Tukey’s post-hoc tests
• Small (.20); Medium (.50); Large (.80)

Step 1:
𝑆_𝑝𝑜𝑜𝑙𝑒𝑑 = √ (((𝑛_1−1) 𝑠_1^2 + (𝑛_2−1) 𝑠_2^2) / (𝑛_1+𝑛_2 ))

Step 2:
𝑑 = (𝑋̅_1 − 𝑋̅_2) / 𝑠_𝑝𝑜𝑜𝑙𝑒𝑑