Wk 7: Evidence from Data

Question 1

When is less participants required for sampling?

Answer 1

Less participants - can detect large effect, but hard to detect small effect

Question 2

When is more participants required for sampling?

Answer 2

More participants -can detect smaller effect, but large sample size is impractical, costly, time -consuming, hard to do double blinding (need assistant), ethical reasons (if unsure about side effects of a treatment).

Question 3

What does an experiment involve?

Answer 3

Experiment involves actively applying treatments to subjects and observing their responses.

Question 4

What is an experimental treatment?

Answer 4

Experimental treatment is a combination of factors at different levels.

• e.g. Caffeine 0 mg and 15 mg is 1 factor at 2 levels.

Question 5

What are independent (explanatory) variables?

Answer 5

describe the treatments - you control them.

Question 6

What are dependent (response) variables?

Answer 6

is the variable of interest - you measure them.

Question 7

What are confounding factors?

Answer 7

are the factors other than independent factors that could affect the dependent factors

• Random sampling could ensure that on average, both groups have the same confounding factors, so it does not affect the dependent factors

Question 8

What are comparative experiments?

Answer 8

are desirable to eliminate the placebo effect.

Question 9

What is randomization?

Answer 9

helps remove possible bias in a comparative experiment.

Question 10

What is a blind experiment?

Answer 10

is when the subjects do not know which treatment they are getting.

Question 11

What is a double-blind experiment?

Answer 11

is when the experimenters and subjects both do not know which treatment the subjects are receiving.

• Ask an assistant to note down which treatment the subjective are getting, and do not tell the researcher
• Cannot do double-blind experiment on manual treatment

Question 12

What is the natural suspicion threshold?

Answer 12

5% is the natural suspicion threshold

• In coin flipping, people typically become suspicious when you get 4-6 heads in a roll
• Transfer this into a probability is about 5% (1/2^5)

Question 13

What are the 3 characteristics of hypothesis testing?

Answer 13

1. Randomisation test is one example of statistical hypothesis testing
2. Null hypothesis (H0) predicts that “nothing is really happening”
3. P-value (expected value) is the probability of obtaining such unusual data if H₀ is true
   
   • Small P-value suggests H₀ is more likely incorrect, so there is an experimental effect.
   • Large P-value suggests H₀ is more likely correct, so there is not an experimental effect.
   • We cannot say “the probability that H₀ is true is…”
4. The data are random. Whether a hypothesis is true or not is not random

Question 14

What are the 2 types of the strength of evidence?

Answer 14

1. Small P-value = stronger evidence for experimental effect
2. Large P-value = weaker evidence for experimental effect because H0 is likely correct

Question 15

What does a small P-value indicate?

Answer 15

stronger evidence for experimental effect

Question 16

What does a large P-value indicate?

Answer 16

weaker evidence for experimental effect because H₀ is likely correct

Question 17

If a decision is required, then a threshold for evidence needs to be _____. What is an example?

Answer 17

set

• e.g. If we find a P-value less than 0.05, we say that “the results were significant at the 5% level” and write “p <0.05”
• This practice is overused in research where a dichotomy is usually not required
• Report the actual P-value where possible

Question 18

What are 3 characteristics in inference in practice?

Answer 18

1. Randomisation test is simple but can be limited in practice.
2. Different methods will be required for different data types and different study designs.
3. Important to connect the study design to the appropriate statistical analysis.