22 Sample Size

Question 1

What’s a
Type I error risk (alpha)
also called & why?

Answer 1

“Producer risk”
* If H0 is true but rejected due to a Type I error, then material that is within specification is rejected
* This causes waste, extra cost, and lower employee morale

Question 2

What’s a
Type II error risk (Beta β)
also called & why?

Answer 2

“Consumer risk”
* H0 accepted while false due to Type II error, so material outside of specification is send to consumer
* This causes returns costs, unsatisfied customers, poor brand reputation

Question 3

What info is required for
choosing sample size?

Answer 3

• Alpha: typically 0.05
• Beta (β): The beta level can be set by the experimenter and a sample size calculated from that number. If the sample size is fixed, then the experimenter usually sets alpha and calculates the beta risk from the sample size
• Delta: The practical difference the experimenter wants to detect using the test
• Standard deviation: The estimated population standard deviation
• Type of Data: Discrete or continuous
• Type of test: Depends on type of data

Question 4

What is
Alpha (α)?

Answer 4

• The level of significance
• Commonly α is set to .05 / 5%, but can take any value <.05
• The probability of obtaining a results by chance
• Represents the chance of making a Type I Error (rejecting H0 while true)
• The smaller α, the more “unusual” is the results

Question 5

What is the
power of a test?

Answer 5

• The probability of avoiding a Type II error (accepting H0 while false)
• Power = 1−β

Question 6

What needs to be considered
when selecting Alpha (α)?

Answer 6

What costs are associated with an unnecessary change, if the team makes a Type I mistake?
* Costs when rejecting materials that fit specifications

Question 7

What needs to be considered
when selecting beta?

Answer 7

What costs are associated with a Type II error, if the team doesn’t reject H0?
* What is the potential damage or cost if defective materials are passed to the customer?
* Do costs for lost time or resources occur?

Question 8

What needs to be considered
when selecting Delta
(aka critical difference)?

Answer 8

What margin of error is tolerable?
* How small can a difference be to be insignificant to the customer?
* What is the smallest possible delta to provide all benefits while not being so small as to be unfeasible (as the smaller delta, the larger the sample has to be, which comes at costs)

Question 9

Guidelines for testing
means for continuous data

Answer 9

Alpha (α) 0.05
Beta (β) 0.10 or 0.20
Delta as absolute value or as a function of std dev (.5σ , 1σ , 2σ)

Question 10

Guidelines for testing
variance for continuous data

Answer 10

Alpha (α) 0.05
Beta (β) 0.10 or 0.20
Delta >1 or as a function of std dev (.5σ , 1σ , 2σ)

Question 11

Guidelines for testing
proportions for discrete / binomial data

Answer 11

Alpha (α) 0.05
Beta (β) 0.10 or 0.20
Delta logically or as a function of std dev (.5σ , 1σ , 2σ)

Question 12

What’s an aspect of the
sample variance test (like the F-test)
sample size calculation?

Answer 12

• Comparing variance from two independent samples / sets of data
• For sample sizing it’s important how much variance is expected in each group

Question 13

What’s an aspect of the
Design of Experiment (DOE)
sample size calculation?

Answer 13

• It’s about the effects of multiple factors / independent x variables on a response variable
• For sample sizing it’s important how many factors are tested and how many levels each factor has

Question 14

What assists in setting appropriate
Alpha (α) and Beta (β) values?

Answer 14

• Real-world understanding
• The costs, risks, and implications of errors guide how strict or lenient you should be with α and β.

Question 15

What concept is key in selecting sample sizes
for various hypothesis testing?

Answer 15

Hypothesis testing error(s)

Question 16

What’s the
Confidence Interval (CI)?

Answer 16

• Margin of error (ME)
• As the CI provides a range of values within which the true population parameter is likely to fall.
• The margin of error defines how wide that range is, based on the sample data and confidence level (e.g., 95%).

Question 17

What’s the
Confidence Interval (CI)
also known as?

Answer 17

• Margin of error (ME)
• As the CI provides a range of values within which the true population parameter is likely to fall.
• The margin of error defines how wide that range is, based on the sample data and confidence level (e.g., 95%).

Question 18

What’s the
Confidence Level?

Answer 18

• Think: How sure am I?
• It’s a percentage that expresses how confident I am that the confidence interval contains the true population parameter.
• Common values: 90%, 95%, 99% sure
• It reflects the reliability of the estimation process.