Lecture 6

Question 1

What are the ANOVA assumptions?

Answer 1

1. Normality
2. Homogeneity of variance
3. Independence of observations
4. DV measured on an interval or ratio scale
5. X (IV) & Y (DV) are linearly related

Question 2

Explain the normality assumption of ANOVA.

Answer 2

• For any value of x (the IV aka the raw scores) are approximately normally distributed.
• In other words, the raw scores are normally distributed with in each group. Do a frequency distribution of the raw scores for each group.

Question 3

What is the effect of violation of the normality assumption of ANOVA on type I and type II errors?

Answer 3

Type I error:

• non normality only has a slight effect on type I error.
• even for very skewed, or kurtotic (peakedness) distributions.
  e. g. nominal alpha (what we set alpha at = type I errror when all assumptions met) vs actual alpha (type I error if one or more assumptions are violated)

Question 4

In really non-normal populations, when nominal alpha = .05, actual alpha = .055 or .06. If nominal alpha ~ actual alpha, what do we say?

Answer 4

We say F is robust to violations of the assumptions.

Therefore F is robust with respect to the normality assumption.

Question 5

What are the reasons that F is robust with respect to the normality assumption?

Answer 5

The sampling distribution of the mean will be normally distributed if:

a) the raw scores are normally distributed in the population.
b) The raw sores in the population are skewed, the sampling distribution of the mean will approach a normal distribution as n increases (n greater than or equal to 30 or so).

Question 6

Define standard error of the mean:

Answer 6

The standard deviation o the sampling distribution of the mean.

Question 7

When would you use a non-parametric test? Why?

Answer 7

When the population is very skewed. Because non-parametric tests are distribution free, which means they don’t have the normality assumption.

Question 8

What effect does lack of normality have on power?

Answer 8

• Only a light effect (a few 100ths)

- Lack of normality due to platy kurtosis (flattened distribution) does affect power, especially if n is small.

Question 9

How does one check for normality?

Answer 9

• Check via frequency distributions

- If big violation of normality with small n –> conduct a non-parametric test –> i.e. distribution doesn’t matter.

Question 10

What are some examples of non-parametric tests?

Answer 10

Chi square 
Mann whitney
Wilcoxon
Kruskal-Wallace
Friedman

Question 11

Describe the Homogeneity (homoscedasticity) of variance assumption

Answer 11

i.e. variance (refers to error variance aka within group variance) is unaffected by the treatment i.e. the IV
i.e. MSerror
MSwithin
S/A
error due to chance
variability due to chance
etc…
i.e. σ²1 = σ²2 = σ²3 etc
In other words, for every value of x, the variance of y is the same.

Question 12

Illustration of heteroscedasticity

Answer 12

Scores (y axis) and independent variable (x axis)

Each group’s scores grouped together above each group

Question 13

Under what circumstances is F robust for unequal variances?

Answer 13

If n’s are equal or approximately equal.

Question 14

When is heterogeneity of variance an issue?

Answer 14

Only an issue if:

- n’s are sharply unequal and a test shows that the variances are sharply unequal.

Question 15

What is meant by approximately equal n?

Answer 15

largest n/smallest n < 1.5

Question 16

What is meant by approximately equal σ²? (variance)

Answer 16

Largest variance/smallest variance > 3
If ratio is greater than 3, we have sharply unequal σ².
If Fmax > 3, then the variances are sharply unequal

Question 17

When is heterogeneity an issue for type I error? (case 1)

Answer 17

Case 1: If the largest variance is associated with the group with smallest n
F is liberal. i.e. actual alpha is going to be greater than nominal alpha.
i.e. falsely reject H0 too often
Solution: adjust nominal alpha downwards. e.g. .025 –> therefore actual alpha is approximately .05

Question 18

When is heterogeneity an issue for type I error? (case 2)

Answer 18

Case 2: If largest variance is associated with the group with largest n
F is conservative.
i.e. actual alpha is less than nominal alpha.
So people usually don’t make an adjustment.

Question 19

Explain the independence of observations assumption of ANOVA

Answer 19

• Observations within each group are independent of one another.
• Usually satisfied if unrelated subjects run individually and alone.
• Usually satisfied if subjects run individually and alone.
• REALLY IMPORTANT*

Question 20

Why is the independence of observations assumption of ANOVA so important?

Answer 20

Because even small violations have a substantial effect on both alpha and power.

Question 21

How is dependence measured?

Answer 21

Intraclass correlation

Question 22

Explain the DV is measured on an interval or ratio scale assumption of ANOVA

Answer 22

Check definitions against actual DV used.

If DV is nominal or ordinal, conduct a different type of statistical test. e.g. Chi square test

Question 23

Explain the X (IV) & Y (DV) are linearly related assumption of ANOVA

Answer 23

i. e. a subject’s score is comprised of 3 parts:
1. general effect (grand mean)
2. an effect that is unique and constant within a given treatment.
3. An effect that is unpredictable (random error & individual differences).

Question 24

Give the linear model of the fifth assumption for ANOVA

Answer 24

μ + alphaj + eij
where μ = grand mean
alphaj = treatment effect for the jth group
and eij = random error for the ith subject in the jth group
so:
general effect + treatment effect + error

Question 25

Define an outlier

Answer 25

a data point which is very different from the rest of the data.
Outliers can have a dramatic effect on results

Question 26

When removing outliers, what must be done?

Answer 26

Must explain why they were removed, this information must be shared.

Question 27

What causes outliers?

Answer 27

1. Human error (eg data entry)
2. instrumentation
3. Subjects significantly different from the rest of the sample –> perhaps from a different population.

Therefore need to detect and remove outliers.

Question 28

How do you detect outliers for small samples?

Answer 28

• The largest possible z score of a data set is bounded by: (n-1)/√n
  eg. for n=10, largest possible z score is 2.846, therefore for small samples, scrutinize any data point greater than or equal to z=2.5

Question 29

How d you detect outliers for large, normally distributed samples?

Answer 29

• Approximately 99% of scores are within three standard deviations of the mean.
  Therefore z scores >3 should be scrutinized
  Note: If n>100 will get some z scores >3 by chance.
  A criteria of z>3 is also reasonable for non-normal distributions, but could extend it to z>4.

Question 30

What happens when subjects are run after analyses?

Answer 30

Tends to increase variability, therefore decrease probability of finding significance AND if N is really large, tend to get statistical significance no matter what, even if no practical significance.

Question 31

Why does running participants after analyses and having a very large N get statistical significance almost no matter what?

Answer 31

Because as N increases, standard error of the mean (sigmaYbar) decreases.