9 Hypothesis Testing

Question 1

What is hypothesis testing?

Answer 1

Hypothesis testing is trying to guess whether or not your hypothesis is true.

Question 2

What are the two main types of hypotheses in hypothesis testing?

Answer 2

Null hypothesis and alternative hypothesis.

Question 3

What does the null hypothesis (𝐻𝐻0) state?

Answer 3

There is no difference between the two groups.

Question 4

What does the alternative hypothesis (𝐻𝐻1) state?

Answer 4

There is a statistically significant difference between the two groups.

Question 5

What is the purpose of hypothesis testing?

Answer 5

To determine if the perceived difference between groups is due to variance or if it is statistically significant.

Question 6

What is statistical significance?

Answer 6

The ability to say, with a very small margin of error, that there is an actual difference beyond pure chance between two groups.

Question 7

List the general steps of the hypothesis testing process.

Answer 7

• Ask or receive a question
• Create a hypothesis
• Decide what analysis is appropriate
• Gather resources for analysis
• Run the analysis
• Answer the question

Question 8

True or False: In hypothesis testing, both the null and alternative hypotheses can be true at the same time.

Answer 8

False.

Question 9

What is a p-value?

Answer 9

A p-value is the probability that the difference between the groups is due to chance.

Question 10

What does a p-value of 0.05 indicate?

Answer 10

There is a 5% chance that the observed difference is just random.

Question 11

What is alpha in hypothesis testing?

Answer 11

Alpha, or the significance level, determines the cutoff for statistical significance.

Question 12

If p-value < alpha, what decision do you make regarding the hypotheses?

Answer 12

Accept 𝐻𝐻1 and reject 𝐻𝐻0.

Question 13

What does it mean when p-value > alpha?

Answer 13

Accept 𝐻𝐻0 and reject 𝐻𝐻1.

Question 14

Fill in the blank: The null hypothesis states there is ______ significant difference.

Answer 14

NO

Question 15

Fill in the blank: The alternative hypothesis states there ______ significant difference.

Answer 15

IS

Question 16

What is the first step in the hypothesis testing process?

Answer 16

Ask or receive a question.

Question 17

What is the significance of a larger sample size in hypothesis testing?

Answer 17

It can provide a more reliable estimate of the difference between groups.

Question 18

What is the relationship between the null hypothesis and alternative hypothesis?

Answer 18

They are mutually exclusive; one is always true while the other is false.

Question 19

What is the main goal of hypothesis testing?

Answer 19

To determine if there is a statistically significant difference between two groups.

Question 20

What is the interpretation of a p-value less than 0.05?

Answer 20

The perceived difference is statistically significant.

Question 21

What is a common mistake in hypothesis testing?

Answer 21

Not clearly stating which hypothesis is accepted and which is rejected.

Question 22

What is the p-value in hypothesis testing?

Answer 22

The p-value indicates the probability that the observed difference is due to random chance.

Question 23

What does it mean if the p-value is smaller than alpha?

Answer 23

It means we accept the alternative hypothesis (𝐻𝐻1) and reject the null hypothesis (𝐻𝐻0).

Question 24

What is the default type of hypothesis test run by most analytical tools?

Answer 24

Two-tailed test.

Question 25

In a two-tailed test with alpha 0.05, how is alpha distributed?

Answer 25

Half is at the far left of the distribution and half is at the far right.

Question 26

What is a one-tailed test?

Answer 26

A test that puts all of the alpha on one side of the distribution.

Question 27

What is type I error?

Answer 27

Falsely accepting the alternative hypothesis (𝐻𝐻1) and rejecting the null hypothesis (𝐻𝐻0).

Question 28

What is type II error?

Answer 28

Falsely accepting the null hypothesis (𝐻𝐻0) and rejecting the alternative hypothesis (𝐻𝐻1).

Question 29

What does alpha specifically describe?

Answer 29

The probability of a type I error.

Question 30

What happens to beta as alpha decreases?

Answer 30

Beta increases, indicating a higher probability of type II error.

Question 31

What is a common value for alpha in hypothesis testing?

Answer 31

0.05.

Question 32

What is the relationship between type I and type II errors?

Answer 32

As the probability of type I error decreases, the probability of type II error increases.

Question 33

What are the two parts of a good question in hypothesis testing?

Answer 33

* What two groups you want compared * The metric you want to use to compare those groups.

Question 34

What qualities should a good question have?

Answer 34

* The question is short * The question is easy to understand * The question is specific * The question is something you can answer.

Question 35

What should you do if you receive a vague question from your boss?

Answer 35

Ask for clarification to understand what they truly want to know.

Question 36

What is the risk of answering a vague question with one massive visualization?

Answer 36

It may lead to misunderstandings and require more explanation.

Question 37

What does a p-value of 0.04 indicate with an alpha of 0.05?

Answer 37

You should accept the alternative hypothesis and reject the null hypothesis.

Question 38

What type of error occurs when you claim no difference when there actually is one?

Answer 38

Type II error.

Question 39

Fill in the blank: A type I error is also known as a _______.

Answer 39

false positive.

Question 40

Fill in the blank: A type II error is also known as a _______.

Answer 40

false negative.

Question 41

What is the null hypothesis in a hypothesis test comparing two groups?

Answer 41

The null hypothesis states that the two groups are the same and there is no statistically significant difference between them.

Question 42

When should you accept the alternative hypothesis in hypothesis testing?

Answer 42

You should accept the alternative hypothesis and reject the null hypothesis when your p-value is equal to or smaller than your alpha.

Question 43

What type of error occurs when the null hypothesis is accepted incorrectly?

Answer 43

Type II error.

Question 44

Which p-value indicates no difference between the new system and the old system when alpha is 0.05?

Answer 44

0.4.

Question 45

What is a Type I error in hypothesis testing?

Answer 45

A Type I error occurs when you falsely accept the alternative hypothesis and reject the null hypothesis.

Question 46

Fill in the blank: A p-value of _______ would lead you to believe there is no difference between the new system and the old system.

Answer 46

0.4.

Question 47

True or False: A p-value smaller than 0.05 leads to the acceptance of the null hypothesis.

Answer 47

False.

Question 48

What is indicated by a p-value larger than the alpha level in hypothesis testing?

Answer 48

We accept the null hypothesis and reject the alternative hypothesis.

Question 49

What happens when you find a difference between the new system and the old system when there actually isn’t?

Answer 49

Type I error.