What effect does performing multiple statistical tests on a set of data have on error likeliehood?

It makes us more likely to make a type 1 error (false discovery)

What is a per comparison error

A type 1 error (false discovery) made involving a single statistical test

What is familywise error rate?

the chance of making one or more false discoveries (type 1 errors) when performing multiple statistical tests familywise error rate = per comparison error rate multiplied by number of comparisons (tests) you’ve done It’s a probability - the familywise error rate for 3 tests might come out as 1, but we may make 0, 1, 2 or 3, it is just most likely to be 1.

What is the equation for familywise error rate?

a_{fw}=c(pc)

What do we do with planned comparisons when we have used multiple statistical tests?

We ignore the theoretical increase in type 1 errors and simply reject or accept the null hypothesis as usual.

What do we do with post-hoc tests when we have done multiple statistical tests?

We cannot ignore the theoretical increased chance of making a type 1 error.

This is why we use post-hoc tests that decrease the likeliehood of making a type 1 error - but this does mean that you have an increased chance of making a type 2 error (incorrect rejection)

MAKE SURE TO SAY THIS IN YOUR RESULTS SECTION! IT WILL LOOK REALLY GOOD!