Null Hypothesis Significance Testing

Question 1

Why does sample size matter?

Answer 1

The larger our sample size, the closer our sample resembles the distribution

Question 2

Define replication

Answer 2

Attempt to reproduce a finding from another study in a new sample using identical methodology

Question 3

What is the focus of confirmatory research?

Answer 3

Testing specific hypotheses
“Null Hypothesis Significance Testing” (NHST)

Question 4

What is exploratory research?

Answer 4

Doesn’t start with a hypothesis
Can collect new data or work with existing data
Useful for generating hypotheses, but statistical tests can be difficult to interpret

Question 5

What are three qualities of a good research question?

Answer 5

1. Researchable and realistic
2. Informed by prior research
3. Not too broad and not too narrow - proportional to the project at hand

Question 6

Define hypothesis

Answer 6

A testable prediction - a statement about what we reasonably believe our data will show

Question 7

What are three different levels a hypothesis can be defined on?

Answer 7

Conceptual, operational, statistical

Question 8

What is a conceptual hypothesis?

Answer 8

Describes our prediction in conceptual terms. Can be defined in terms of the direction of the effect that we’re studying

Question 9

What is meant by operationalisation?

Answer 9

The process of translating concepts into measures

Question 10

What does a statistical hypothesis predict?

Answer 10

What do we expect to happen numerically?

Question 11

How are the null and alternative hypotheses denoted?

Answer 11

N = H0 (small 0)
A = H1 (small 1)

Question 12

When using Null Hypothesis Significance Testing, what are we trying to decide?

Answer 12

Whether we can reject the null hypothesis

Question 13

What is meant by the 𝛼 level?

What should the decision be based on?

Answer 13

The rate of false-positive findings we’re willing to accept if we’re living in a reality where the null hypothesis is true (If we were to repeat our experiment over and over again, how often are we willing to incorrectly reject the null hypothesis?)
The decision should be based on a cost benefit analysis

Question 14

What do psychologists often use as the 𝛼 rate?

Answer 14

𝛼 rate of 5%

Question 15

When p is what value, we reject the null hypothesis

Answer 15

0.05/5%

Question 16

What exactly does the p value tell us?

Answer 16

The probability of observing a test statistic at least as large as the one we observed in our sample if the null hypothesis is true
(How likely the detected effect is IF the null is true)

Question 17

Define the following terms:
1. Effect sizes
2. Confidence intervals
3. Power analysis

Answer 17

1. How large/meaningful is the effect (e.g.mean difference) that we found
2. what are the plausible limits of our effect?
3. determining the necessary sample size before we begin data collection

Question 18

What is statistical power?

Answer 18

The probability of detecting an effect of a certain size as statistically significant, assuming the effect exists

Question 19

Finish the sentences about power analysis:
1. Makes non-_____ effects easier to interpret
2. Reduces over-______ and wasting resources
3. Reduces the ________ of missing an important finding
4. A statistical power of __% is considered the standard to aim for

Answer 19

1. significant
2. sampling
3. probability
4. 80