ANOVA

Question 1

What is ANOVA?

Answer 1

Statistical test to compare means of more than two groups

Question 2

Multiple pairwise comparison is…

Answer 2

repeating multiple t-tests (this gives the same results as an ANOVA)

Question 3

How many IV levels are there in an ANOVA?

Answer 3

at least 3

Question 4

Notation for number of levels in an ANOVA

Answer 4

k

Question 5

A pairwise comparison is…

Answer 5

One possible t-test out of the many that could be done

Question 6

A pairwise comparison is done to…

Answer 6

determine which treatment affects the dependent variable

Question 7

Formula for how many pairwise comparisons there will be

Answer 7

k(k-1)/2

Question 8

Each pairwise comparison needs a pooled Standard deviation in the denominator, we call this the…

Answer 8

Mean Squared Error

Question 9

What do we use for the Mean Squared Error for ANOVA?

Answer 9

All the sample standard deviations of the ANOVA (gives a more accurate p-value)

Question 10

Formula for an individual pairwise comparison

Answer 10

Question 11

The risk for a type I error increases for each per comparison you make…

Answer 11

we call this the per comparison (PC) error rate

• Type I error occurring somewhere in at least one of the comparisons is higher than the PC rate

Question 12

The risk of committing a Type I error at least once in a whole family is called the…

Answer 12

Family wise( (FW) error rate

• This is substantially higher than the PC rate
• This is because each test you add to the family of tests you expose a fresh possibility of committing and error

Question 13

Statiticians perfer the FW error rate, not PC rate to be set to…

Answer 13

Alpha=0.05

Question 14

Two ways to control family wise error

Answer 14

1) Use smaller PC error rate in each test so FW doesn’t blow up as much

2) Limit the number of comparisons

Question 15

Bonferroni formula is used for…

Answer 15

Familywise error rate

FW = PC x C

e.g.,
= 0.05 x 10 = 0.5

Question 16

Bonferroni formula with PC error rate being smaller example

Answer 16

FW = 0.005 x 10 = 0.05
Using a more stringent error rate for the per comparisons makes the FW error rate 0.05 which is more reasonable

Question 17

Bonferroni correction formula

Answer 17

Used to determine the PC rate we want

PC=FW/c
e.g., PC = 0.05/2 = 0.025

This new corrected amount is called the Bonferroni correction

Question 18

When using the Bonferroni formula it controls for family wise, but it is achieve at the price of reduced _______

Answer 18

power

Question 19

When you decrease the alpha on a statistical test you also decrease the ______

Answer 19

power

Question 20

There is a balance between controlling for family wise and retaining _____

Answer 20

power

Question 21

There is another procedure that uses a formula for estimating FW that is more accurate than Bonferroni. This is because Bonferroni….

Answer 21

overestimates the true FW

Question 22

Sidak formula

Answer 22

A better estimate of FW is given this formula
FW=1-(1-PC)^c

e.g., FW=1-(1-PC)^c
FW=1-(1-0.5)^30=.785

Unlike the Bonferroni formula the Sidak formula never predicts error rates greater than 100%

Therefor for large number of comparisons you should use sidak

Question 23

Sidak rearranged to find the proper PC error rate in order to find any FW error rate that we want
Sidak Correction

Answer 23

The benefit of this is that they don’t require the PC error rate to be reduced as much as Bonferroni. Retaining power

Question 24

Sidak vs Bonferroni. How many comparions?

Answer 24

Bonferroni for 10 or less comparisons

Sidak for more than 10 comparisons

Question 25

What do you have to do before making comparisons?

Answer 25

Choosing how many you are going to compare. This is done before looking at any data. If you look at the data you have to run ALL of the tests

Question 26

P-hacking

Answer 26

Choosing to only run certain tests to get significant results. This leads to type I errors

Question 27

Type I error

Answer 27

False positive

Question 28

Type II error

Answer 28

False Negative

Question 29

A priori coparisons

Answer 29

The comparisons that are planned to test before hand

Question 30

Post-hoc comparisons

Answer 30

Comparisons that are not planned a head of time. Procedure involving post hot tests must perform all possible comparisons

Question 31

A priori tests have...

Answer 31

Slightly inflated FW rate It is still common to reduce FW error by reducing PW in some way The way in which you choose to reduce PC to bring residual FW under control determines what a priori test you are performing

Question 32

Post hoc tests...

Answer 32

Force you to perform all comparisons amoung means There are still many methods of reducing PC in order to limit FW error rate. The method you choose determines what post hoc procedure you are performing You could use either Sidak or Bonferroni

Question 33

A prior procedures

Answer 33

Systematic methods of planning comparisons for apriori tests 1. Multiple comparisons against control group (k-1) pairwise comparisons (called Dunnett's test) 2. Contrasts which are generalization of pairwise comparisons

Question 34

What are contrasts?

Answer 34

Compare 2 groups of means rather than just two individual means Lumping together certain IVs then performing a t-test

Question 35

What must the sum of contrasts coefficients add to?

Answer 35

0

Question 36

calculating a contrast value

Answer 36

Question 37

Calculating a contrast value example

Answer 37

Question 38

If contrasts had no affect the value would equal what

Answer 38

0 If its not 0 a t-test will further explain whether it is an accident

Question 39

T test formula for a contrast

Answer 39

Question 40

The contrast df is quite large becasue it depends on the design using Spooled

Answer 40

N-k

Question 41

Continue at Learning Spooled