Effect Size and Power

Question 1

What do effect sizes measure?

Answer 1

They indicate the proportion of the variance explained

Question 2

What effect size measure do we use for T-Tests?

Answer 2

Cohen’s d

Question 3

What effect size measure do we use for ANOVAs?

Answer 3

Eta and Partial Eta Squared

Question 4

What is Cohen’s d for ANOVA?

Answer 4

The difference between the largest and the smallest group means scaled by standard deviation

Question 5

What assumptions are made about the standard deviations when calculating effect sizes?

Answer 5

That it is constant across groups

Question 6

What is Eta Squared and when do we use it?

Answer 6

• One-Way ANOVAs
• Same as R Squared
• Proportion of variance explained by your experiment

Question 7

What is Partial Eta Squared and when do you use it?

Answer 7

• Factorial ANOVAs

- Proportion of variance that is uniquely explained by each IV

Question 8

What is a power analysis?

Answer 8

Shows the power of a statistical test by checking its:

• Ability to detect an effect when it is actually there
• Ability to correctly reject the null hypothesis

Question 9

What does the power of a test depend on?

Answer 9

• Sample size
• Effect size
• Criteria for significant

Question 10

What are the dangers of underpowered studies?

Answer 10

• Lack of power to detect effect
• More errors likely
• Estimating powers across many studies is worryingly low
• Low power explains failure to replicate

Question 11

Explain power as a function of sample size.

Answer 11

As sample size increases, so does power.

- More participants means increased chance of finding a significant effect if there is one

Question 12

What number is considered good for power?

Answer 12

0.8

Question 13

Explain power as a function of effect size.

Answer 13

Smaller effect sizes need more participants to achieve higher power.
If we know two of those things we can estimate the other.

Question 14

What 4 ways can we use to estimate effect sizes?

Answer 14

• Guess
• Do a pilot study
• Find previous studies
• Find or conduct a meta-analysis

Question 15

When would you guess an effect size?

Answer 15

When there is not much literature on your phenomenon

• Could use Cohen’s heuristics
• Not likely to be informative

Question 16

When would you do a pilot study to estimate effect size?

Answer 16

• Run a pilot with fewer PPs to estimate effect size
• Doesn’t matter if the results are significant
• Can use this estimate to project how many PPs needed

Question 17

How do you use previous studies to estimate effect size?

Answer 17

• Use their results to work out an expected effect size for your experiment
• Not exact but better than guessing

Question 18

How do you use a meta-analysis to estimate effect sizes?

Answer 18

• If there are many studies you can calculate an average effect size across all the results
• Common in drug trials

Question 19

What problems are there with power analyses?

Answer 19

• Garbage in - garbage out technique
• For really complicated designs (e.g. factorial) it is unlikely you will have sufficient precision in estimates of effect size

Question 20

What is the garbage in - garbage out technique?

Answer 20

If you make up the numbers you enter, what you get out is not meaningful

Question 21

What do you need to calculate how many PPs are required for a study?

Answer 21

• Effect size
• Power
• Alpha

Question 22

What do you need to calculate minimum requires effect size detectable?

Answer 22

• Power
• Sample size
• Alpha

Question 23

What do you need to calculate observed power?

Answer 23

• Sample size
• Effect size
• Alpha