Week 4 - Lecture 4

Question 1

what is statistical power?

Answer 1

The probability that a test will detect an effect if one exists (ie. correctly rejecting a false null hypothesis)
- formula: power = 1 - β (Type II error rate)
- power increases the chances of detecting a true effect

Question 2

what does the null hypothesis (H0) state in a study?

Answer 2

That there is no effect or no difference

Question 3

what does the alternative hypothesis (H1) state?

Answer 3

That there is an effect or a difference exists

Question 4

What is a Type I error?

Answer 4

Saying there is an effect when there isn’t one (false positive)

Question 5

What is the symbol used to represent a Type I error?

Answer 5

Alpha (α)

Question 6

What is the common value set for alpha (α) in research?

Answer 6

0.05 (or 5% chance of making a Type I error)

Question 7

Give an example of a Type I error

Answer 7

A pregnancy test says someone **is pregnant **when they are **not **

Question 8

What is a Type II error?

Answer 8

Saying there is no effect when there actually is one (false negative)

Question 9

What is the symbol for a Type II error?

Answer 9

Beta (β)

Question 10

Give an example of a Type II error

Answer 10

A pregancy test says someone is not pregnant when they actually **are **

Question 11

What does **power **refer to in significance testing?

Answer 11

The ability to correctly detect an effect when it exists (1 - β).

Question 12

What happens to power when beta increases?

Answer 12

power decreases (you’re more likely to miss real effects)

Question 13

In terms of errors, why do we not always reduce alpha to 0.01?

Answer 13

Because it would increase beta and make us more likely to miss real effects (Type II errors)

Question 14

What does it mean if power = 0.80?

Answer 14

There is an 80% chance of correctly finding a real effect if it exists

Question 15

complete the table:

Answer 15

refer to photo

Question 16

Does statistical significance always mean the result is practically important?

Answer 16

No. A result can be statistically significant but still be** too small to matter in real life**

Question 17

How does a large sample size affect statistical significance?

Answer 17

It can make tiny effects appear **statistically significant **

Question 18

What can happen with a small sample size?

Answer 18

It might miss real effects because there’s not enough data

Question 19

What is an example of a study with a statistically significant but practically unimportant result?

Answer 19

Facebook’s 2014 study - effect size was d = 0.02, which is very small.

Question 20

What happens to power if you lower alpha (eg. from 0.05 to 0.01)?

Answer 20

Power decreases (but you reduce the chance of a Type I error).

Question 21

What happens to power if you increase alpha (eg. from 0.05 to 0.10)?

Answer 21

power increases (but so does the chance of a Type I error)

Question 22

What does “effect size” tell us?

Answer 22

How big the difference or relationship is, measured in standard deviation units

Question 23

What are Cohen’s effect size benchmarks for small, medium and large effects?

Answer 23

• small = 0.2
• medium = 0.5
• large = 0.8

Question 24

How does sample size affect power?

Answer 24

larger sample size gives more power by reducing standard error

Question 25

What is the effect of increasing sample size from N = 25 to N = 100 on significance?

Answer 25

It can change a non-significant result (Z=1.5) to a significant one (Z=3)

Question 26

What happens to power when variance in the population is low?

Answer 26

power increases because it becomes easier to detect real differences?

Question 27

How can you reduce variance in your sample?

Answer 27

By using a homogeneous (similar) group of participants

Question 28

What does effect size (d) measure?

Answer 28

The **difference between group means**, expressed in standard deviation (SD) units

Question 29

What does statistical effect (δ or delta) represent?

Answer 29

How many standard errors apart the populations are

Question 30

What's the formula for δ in a one-sample t-test?

Answer 30

refer to photo

Question 31

What’s the formula for δ in a two-sample t-test (equal group sizes)?

Answer 31

refer to photo

Question 32

What is the formula to calculate effect size (d)?

Answer 32

refer to photo

Question 33

What do we use δ for once it’s calculated?

Answer 33

We use it to look up power in a power table, given a chosen α level.

Question 34

If d = 0.333 and N = 25, what is δ and power approximately?

Answer 34

- δ = 1.665 - Power ≈ 0.36 (not sufficient)

Question 35

If d increases to 0.667, what happens to power (with same N)?

Answer 35

δ increases to 3.335 and power ≈ 0.91 → power improves

Question 36

What is the formula to find sample size N for one-sample tests?

Answer 36

refer to photo

Question 37

What value of δ is commonly used to achieve 80% power?

Answer 37

δ = 2.8 (when α = 0.05)

Question 38

If d = 0.333 and δ = 2.8, what is the required N?

Answer 38

refer to photo

Question 39

Why must you always round up when calculating N?

Answer 39

You can't test a fraction of a person, and rounding down gives less power than needed

Question 40

In two-sample tests, what does lowercase n represent?

Answer 40

The number of people in each group

Question 41

What is the formula for n in a two-sample t-test?

Answer 41

refer to photo

Question 42

If d = 0.5 and δ = 2.8, how many people are needed in each group? How many total participants are needed in that example?

Answer 42

63 per group Total N = 63 x 2 = 126 participants