Null Hypothesis significance testing Flashcards
What is the scenario we are going to be focusing on here?
- Imagine you are interested in anxiety levels in younger and older adults. You decide to class <45 as younger and >45 as older
- You collect data from a sample of 25 younger and 25 older people on an anxiety questionnaire which produces normal scores, has been thoroughly validated in the UK and for which the population parameters are known
Why can’t we assess whether our sample and population means are different from the UK average?
due to sampling errors
What are the questions we could ask of this data?
• Does the sample mean look very different from the UK population average?
- Hypothesis: The sample came from a population with higher/ different population mean
- Same as asking ‘does the sample mean look inconsistent with a sample of the appropriate size from this population?’
• Do the older vs. younger sample means look different from each other?
What is our null hypothesis significance testing example we are going to be using?
- Imagine there is a carefully validated test which measures statistics ability in 1st year UK undergraduates
- Assume that the population test scores follow N(u, sigma)
- You have good reason to hypothesise that your cohort is better at stats than the general UK 1st year undergrad population
What happens in step one of NHST and apply this to our example?
- generate hypotheses:
- Null hypothesis (H0): My cohort is not better at stats than the UK population of 1st year UG students and my sample is typical for sample from that population
- Put in another way: Performance of a sample of students from my year will be comparable to a random sample of the same size from the population of test takers – the sample from my year will not be particularly different from any other random sample of that size from the population of test takers
- Put another way: the sample will not have an atypically high mean
- (H0): the sample came from a population with mean uh = u
- (uh = hypothetical value for population mean from which this sample is drawn)
- (u = population mean for UK 1st year UG students)
- Research Hypothesis (H1): My cohort is better at stats than the UK population of 1st year UG students
- Put another way: performance of a sample of students from my year will not be comparable to a random sample of the same size from the population of test takers – the sample from my year will be markedly higher than other random samples of that size from the population of test takers
- Put another way: the sample mean will be more consistent with that of a random sample of the same size taken from a population with a higher mean
- (H1): the sample came from a population with mean uh> u)
What happens in step 2 of NHST and apply this to our example?
- data
- Our ability to test hypotheses rests fundamentally on collecting good data
- Experiment needs to be well designed with as much control as possible of extraneous variables
- In our example, data collection requires us to get an appropriate random sample (N=25) from our cohort and get their stats test results
- We will then be able to work out sample statistics (e.g. mean m, s.d.)
- We also know about the population parameters and so about the SDM
What happens is step 3 of NHST?
- evaluate inconsistency with H0
- how inconsistent are the data with H0
- related to where our sample mean is in relation to our parent population mean (because uh = sample mean)
- if our sample mean is closer to the parent population mean then you would say that it is more consistent with our null hypothesis and less consistent with our research hypothesis
- you have to assume that H0 is true
What happens in step 4 of NHST?
- reject or fail to reject
- Based on step 3 we now know how inconsistent the data are with H0
- If data are sufficiently inconsistent with H0 we say ‘we reject H0 in favour of H1’
- If data are not sufficiently inconsistent with H0 we say ‘we fail to reject H0’ (this is not the same as finding evidence for H0)
What happens in step 5 of NHST?
- interpret
- We now interpret our findings in terms of our hypotheses
- If we could reject the null (H0) we say that we have evidence for the research hypothesis (H1) and that our sample came from a population with mean uh>u
- In our case that would mean we have evidence our cohort is better at stats than the UK population of 1st year UGs
- If we failed to reject the null (H0) then we can not claim to have evidence our cohort is better than the UK population of 1st year UGs
- This is not the same as finding evidence for H0 – we can not conclude that our cohort has the sample population mean as the UK population of 1st year UGs. We are looking for evidence for the research hypotheses, not the null hypotheses.
In step 3 if our sample mean is not particularly high what would you say about inconsistency?
Not very inconsistent: p is not small here – i.e. it seems entirely plausible to get a sample mean here if null is true so don’t reject the null
In step 3 if our sample mean is uncharacteristically high what would you say about inconsistency?
Very inconsistent: p is rather small here – i.e. it seems too implausible to get a sample mean here if null is true so reject the null
When do you fail to reject the null?
- when values of p>a
- suggest not inconsistent with H0
When do you reject the null?
- When values of p<a></a>
What value is a? (how small should p be before we decide to reject H0)
- we use a value of a = 0.05 (5%)
What is the conditional probability associated with you sample statistic assuming that the null hypothesis is true sometimes called?
The p-value
