# Null Hypothesis significance testing Flashcards

1
Q

What is the scenario we are going to be focusing on here?

A
• Imagine you are interested in anxiety levels in younger and older adults. You decide to class <45 as younger and >45 as older
• You collect data from a sample of 25 younger and 25 older people on an anxiety questionnaire which produces normal scores, has been thoroughly validated in the UK and for which the population parameters are known
2
Q

Why can’t we assess whether our sample and population means are different from the UK average?

A

due to sampling errors

3
Q

What are the questions we could ask of this data?

A

• Does the sample mean look very different from the UK population average?
- Hypothesis: The sample came from a population with higher/ different population mean
- Same as asking ‘does the sample mean look inconsistent with a sample of the appropriate size from this population?’
• Do the older vs. younger sample means look different from each other?

4
Q

What is our null hypothesis significance testing example we are going to be using?

A
• Imagine there is a carefully validated test which measures statistics ability in 1st year UK undergraduates
• Assume that the population test scores follow N(u, sigma)
• You have good reason to hypothesise that your cohort is better at stats than the general UK 1st year undergrad population
5
Q

What happens in step one of NHST and apply this to our example?

A
• generate hypotheses:
• Null hypothesis (H0): My cohort is not better at stats than the UK population of 1st year UG students and my sample is typical for sample from that population
• Put in another way: Performance of a sample of students from my year will be comparable to a random sample of the same size from the population of test takers – the sample from my year will not be particularly different from any other random sample of that size from the population of test takers
• Put another way: the sample will not have an atypically high mean
• (H0): the sample came from a population with mean uh = u
• (uh = hypothetical value for population mean from which this sample is drawn)
• (u = population mean for UK 1st year UG students)
• Research Hypothesis (H1): My cohort is better at stats than the UK population of 1st year UG students
• Put another way: performance of a sample of students from my year will not be comparable to a random sample of the same size from the population of test takers – the sample from my year will be markedly higher than other random samples of that size from the population of test takers
• Put another way: the sample mean will be more consistent with that of a random sample of the same size taken from a population with a higher mean
• (H1): the sample came from a population with mean uh> u)
6
Q

What happens in step 2 of NHST and apply this to our example?

A
• data
• Our ability to test hypotheses rests fundamentally on collecting good data
• Experiment needs to be well designed with as much control as possible of extraneous variables
• In our example, data collection requires us to get an appropriate random sample (N=25) from our cohort and get their stats test results
• We will then be able to work out sample statistics (e.g. mean m, s.d.)
• We also know about the population parameters and so about the SDM
7
Q

What happens is step 3 of NHST?

A
• evaluate inconsistency with H0
• how inconsistent are the data with H0
• related to where our sample mean is in relation to our parent population mean (because uh = sample mean)
• if our sample mean is closer to the parent population mean then you would say that it is more consistent with our null hypothesis and less consistent with our research hypothesis
• you have to assume that H0 is true
8
Q

What happens in step 4 of NHST?

A
• reject or fail to reject
• Based on step 3 we now know how inconsistent the data are with H0
• If data are sufficiently inconsistent with H0 we say ‘we reject H0 in favour of H1’
• If data are not sufficiently inconsistent with H0 we say ‘we fail to reject H0’ (this is not the same as finding evidence for H0)
9
Q

What happens in step 5 of NHST?

A
• interpret
• We now interpret our findings in terms of our hypotheses
• If we could reject the null (H0) we say that we have evidence for the research hypothesis (H1) and that our sample came from a population with mean uh>u
• In our case that would mean we have evidence our cohort is better at stats than the UK population of 1st year UGs
• If we failed to reject the null (H0) then we can not claim to have evidence our cohort is better than the UK population of 1st year UGs
• This is not the same as finding evidence for H0 – we can not conclude that our cohort has the sample population mean as the UK population of 1st year UGs. We are looking for evidence for the research hypotheses, not the null hypotheses.
10
Q

In step 3 if our sample mean is not particularly high what would you say about inconsistency?

A

Not very inconsistent: p is not small here – i.e. it seems entirely plausible to get a sample mean here if null is true so don’t reject the null

11
Q

In step 3 if our sample mean is uncharacteristically high what would you say about inconsistency?

A

Very inconsistent: p is rather small here – i.e. it seems too implausible to get a sample mean here if null is true so reject the null

12
Q

When do you fail to reject the null?

A
• when values of p>a

- suggest not inconsistent with H0

13
Q

When do you reject the null?

A
• When values of p<a></a>
14
Q

What value is a? (how small should p be before we decide to reject H0)

A
• we use a value of a = 0.05 (5%)
15
Q

What is the conditional probability associated with you sample statistic assuming that the null hypothesis is true sometimes called?

A

The p-value

16
Q

is p the probability that the null hypothesis is true?

A

No. We can never know this. We are just looking for evidence of inconsistency

17
Q

if p<0.05 does that mean that H1 (our research hypothesis) is correct?

A
• no

- by chance (i.e. random sampling error) you could get a sample mean that was extreme (leading to small p)

18
Q

What is the z-test?

A

A null hypothesis significant test in which in step 2 you find out the z-score for your data and hence the conditional probability of obtaining that value. If p < 0.05 then we can reject the null hypothesis in favour of the research hypothesis as our data is inconsistent with the null hypothesis - it is not likely to get this sample mean from the parent population.

19
Q

When do you use a z-test?

A
• whenever we want to check a sample mean we have obtained is different from a population mean?
• does it look like our sample came from that population or a population with a different mean?
20
Q

Why don’t we usually use a z-test?

A

Because we don’t usually have the parent population parameters

21
Q

What is a one tailed hypothesis?

A
• only interested in one tail of the graph
• It can be interested in the conditional probability of having got a sample mean as low or lower or as high or higher than our population mean
• used when making a directional prediction
• only formulate a one tailed hypothesis if you have good reason to predict the effect will be in a specific direction
22
Q

What is a two tailed hypothesis?

A
• when you are interested in both tails of the graph
• when we believe our sample is different from the general population but we don’t know in which direction
• used when making a non-directional prediction
• two tailed hypotheses are more conservative as you need a more extreme sample mean to be able to reject the hypothesis (there needs to be 2.5% on either side of the tail as opposed to 5% on one side of the tail)
23
Q

How does does step 3 of z two tailed z-test work?

A
• Work out the z-score for our sample means
• Work out the probability
• Then double the probability but not the z score to work out the area either side of the tail
24
Q

What is the key difference when working out the conditional probability for a two tailed z-test?

A

You need to double the probability to get it on both sides.

25
Q

what is a type 1 error?

A

when you reject the null hypothesis when it is true

26
Q

What is a type 2 error?

A

when you fail to reject the null hypothesis when it is false

27
Q

When do type 1 errors and type 2 errors occur?

A
• Type 1 errors occur because even if your p-value is small there is still a (small) chance that your data were unusually extreme (and so you rejected the NULL) just due to sampling error
• Type II errors often arise because of a problem with your study
 Perhaps your sample was biased