Week 4 ANCOVA

Question 1

When do we use ANCOVA?

Answer 1

• ANCOVA may be used as an extension to any type of ANOVA design, including MANOVA
• It is used when you want to statistically control for a covariate(s).

Question 2

What is a Covariate?

Answer 2

• A covariate (CV) is a variable that must be related to the DV, which research suggests may impact on the findings.
• when we say control we are saying we are removing the effect of that variable (the CV) on the DV.

Question 3

What does it mean to Statistically control for Covariates?

Answer 3

*when we say control we are saying we are removing the effect of that variable (the CV) on the DV.

Question 4

Which is more Powerful ANOVA or ANCOVA?, and why?

Answer 4

*ANCOVA is more powerful than ANOVA because it removes the amount of error variance apportioned by the CV on the DV – think about this and remember the f-test statistic of ANOVA.

Question 5

When & Why do we use ANCOVA?

Answer 5

ANCOVA can be used as part of:

• One way between groups ANOVA – One-way ANCOVA
• Two way between groups ANOVA – Two-way ANCOVA
• Multivariate ANOVA – MANCOVA

Question 6

What does ANCOVA actually test for?

Answer 6

To test for differences between group means when we know that an extraneous variable affects the outcome variable. Used to control known extraneous variables.
*We use ANCOVA to test differences between group means when we know that another variable (CV) may impact on outcome variable (DV).

Question 7

In what way are the principles of ANCOVA are quite similar to ANOVA

Answer 7

• ANOVA’s equation of the value of variance apportioned to the numerator (between subjects) divided by the value apportioned to the denominator (within subject effect).
• Removing error variance from the denominator will increase the overall value. Resulting equation’s value will be larger.

Question 8

How is ANCOVA different from ANOVA?

Answer 8

*You can control for a known extraneous variable or in repeated measures, where you want to control for the baseline measure in a 2nd administration of a variable. *ANCOVA assesses change in the outcome with the removal of the impact of the baseline to see true differences.

Question 9

What are the advantages of ANCOVA?

Answer 9

• ANCOVA reduces Error Variance
• By explaining some of the unexplained variance (SSR) the error variance in the model can reduce the within/ error variance (noise).

Question 10

How does ANCOVA provide Greater Experimental Control?

Answer 10

By controlling known extraneous variables, we gain greater insight into the effect of the predictor variable(s).
Increases power to detect significant differences

Question 11

How do you decide which Covariates to control for?

Answer 11

• decisions are derived from the theoretical and empirical literature
• I need to know my research area well, have a strong theoretical basis to argue for covariate use.

Question 12

What properties should Covariates have?

Answer 12

• Covariates should be continuous variables that are measured reliably and need to be correlated significantly with the DV.
• Logically - this makes sense. Having a covariate that is not significantly correlated would do little in allowing adjustment of the DV to occur.

Question 13

What points should I consider when using more than one Covariate?

Answer 13

• Each covariate should only share a moderate amount of variance
• Each should contribute uniquely to the variance
• They must be measured before the treatment or experimental manipulation, so the covariates are not influenced by the treatment . That is, you don’t unintentionally control for some of the treatment effect in the analysis.

Question 14

What are the assumptions of ANCOVA?

Answer 14

• ANCOVA assumes covariates are measured reliably & are valid.
• Covariates should be suitable for the intended population
• Covariates should correlate well with the DV but not with each other.
• since overlapping covariates would contribute little to a reduction in error variance
• Should be a linear relationship between the DV and covariates as well as between covariates

Question 15

What needs to happen with the Homogeneity of regression slopes?

Answer 15

• The relationship between covariate & DV for each group must be the same across groups.
• Unequal slopes indicate an interaction between covariate and treatment.
• adjustments are made on the DV according to the overall correlation with the CV. Different slopes across the groups would make the adjustments uninterpretable.

Question 16

When is ANCOVA often used?

Answer 16

ANCOVA is often used to evaluate the impact of an intervention while controlling for pre-test scores

Question 17

The example in the lecture introduced ANCOVA as a means of evaluating the impact of an intervention while controlling for pre-test scores. Can you remember the design?

Answer 17

Variables:
IV – Categorical - (2 groups Maths Building, Confidence Building)
*DV – 1 Continuous - Fear of Statistics after intervention
*Covariate (CV) - Baseline measure of Fear of Statistics prior to participating in the intervention groups

Question 18

The IV was the intervention, what was that?

Answer 18

IV – Student Gp (1) receives maths skills 	intervention
	Student Gp (2) receives confidence 	building 	intervention

Question 19

The DV was the fear of stats after the intervention (time 2), why was the covariate CV – Scores on Fear of Statistics – Test Time1?

Answer 19

By controlling the Scores on Fear of Statistics – Test Time1, i.e. reducing this to er for all participants, the effect on each intervention is clearly identifiable, without the confusion of the original fear of stats (the CV)

Question 20

What would I do if my covariate and DV had a very weak correlation?

Answer 20

Reconsider, if it is not going to reduce error variance? What’s the point of using it?

Question 21

How do I check for interactions between covariate and treatment?

Answer 21

Inspect scatterplot – are the 2 lines similar in their slope? Alternatively, we can also test this statistically if we write the syntax OR build terms in GLM

Question 22

What does Hills suggest when requesting scatterplots?

Answer 22

• I can request a scatterplot for each group, and compare them
• I can look at the 2 groups simultaneously to see if there is a difference in slopes.

Question 23

How do I test Homogeneity of Regression slopes?

Answer 23

either by graph or by building a custom model.
e.g. UNIANOVA fost2 BY group WITH fost1
/METHOD=SSTYPE(3) /INTERCEPT=INCLUDE
/CRITERIA=ALPHA(0.05) /DESIGN=group fost1 fost1*group

Question 24

What would I do if the Assumption – Homogeneity of regression slopes is violated?

Answer 24

Consider again – not going to reduce error variance? So little point of using it.

Question 25

When I have run the Homogeneity of Regression Assumption, what should I do?

Answer 25

NB: Remember to turn off custom in “model”, after you run this analysis or otherwise you will get very different results in the actual analysis

Question 26

What am I looking for when I view the results of the Homogeneity of Regression slope test?

Answer 26

• I am only interested in the interaction when testing for homogeneity of regression slopes.
• I ask if the interaction significant at p<.05?
• If not, this is good & meets the assumption - otherwise if sig would indicate baseline (covariate) group differences existed (Bad)

Question 27

So what does it mean when the assumption of Homogeneity of Regression is met?

Answer 27

*The slopes do not significantly deviate between the 2 groups.

Question 28

A note from Field (pg 374): What is the difference between Tukey’s Least Significant Difference (LSD) & performing multiple t-tests on the data?

Answer 28

• The LSD pairwise comparison makes no attempt to control for the Type 1 error and is equivalent to performing multiple t-tests on the data
• LSD requires the overall ANOVA to be significant
• Tukey can be used when you have equal sample sizes & group variances are similar

Question 29

What table do we look at to check for significance in the Covariate? & how does Field help us interpret this?

Answer 29

We use the “Tests of Between-Subjects Effects” table

• If the covariate is significant it means that the covariate significantly predicts the DV
• when the effect of the covariate is controlled (held at 0), we are then able to observe the effect of the IV on the DV WITHOUT the influence of the CV

Question 30

What might I check for if I run an ANOVA with no covariate included in the analysis?

Answer 30

• Running an ANOVA first can inform me whether there is a significant difference between groups
• I can also determine whether the significance between groups is higher or lower without the covariate

Question 31

What can I check for if I run the EM Means Output with the Covariate removed?

Answer 31

*I can check to see if there is a difference in means when the covariate is removed

Question 32

When running a two-way ANCOVA (an additional IV), what will the “Tests of Between-Subjects Effects” output tell me?

Answer 32

*Whether there is significance in the separate IV’s, in the Covariate, & in the Interaction between the 2 IV’s

Question 33

The output then produces 3 separate tables, one for each IV, & one for the interaction between IV’s - what is happening to the Covariate at this time?

Answer 33

The covariate is controlled (i.e. is set to zero for all participants)
*each table has a little sentence underneath it to indicate this: “a. evaluated at covariates appeared in the model (CV name) = — (value) for each IV and for the interaction:
“a. covariates appearing in the model are evaluated at the following values (CV name) = … Value

Question 34

What can producing an “Estimated Marginal Means of (CV Name)” graph indicate?

Answer 34

The graph will depict the impact of IV on the other.

*If they cross over there is an interaction

Question 35

How can ANOVA / ANCOVA be conceived as Regression?

Answer 35

*We can run a regression with Reading Score as the outcome variable (DV) and Training (Maths Skills or Confidence Building) as the predictor (IV), Note:
Intercept is the mean of the group coded as 0 (Maths Skills)
b for the Dummy Variable is the difference between the means of the Maths Skills or Confidence Building groups
*You could extend the regression example by including the covariate into the model at the 1st step as you would, if you used this in HMR (Hierarchical Multiple Regression).

Question 36

ANCOVA shares the same assumptions as ANOVA, what are these?

Answer 36

• Data should be interval or ratio (Scale SPSS).
• Scores randomly sampled from the population.
• Scores are independent.
• Scores on the dependent variable are normally distributed.
• Homogeneity of Variance (Levene’s) needs to be met

Question 37

What are the assumptions specific to ANCOVA?

Answer 37

*Covariate needs to be reliable and normally distributed.
*Needs to be linear relationship between covariate & DV for each group.
&Homogeneity of Regression across groups. There shouldn’t be a difference in the slopes across the groups. *The covariate is filtered across the entire sample and does not differentiate across groups.

Question 38

What should one remember when checking normality assumptions?

Answer 38

• When checking for normality – don’t forget to do this for each group, by splitting the file by group prior to running the explore function.
• I also obtain z-scores while the file is split
• next assumption to rule out any possible outliers (values ±3.29 SD are seen as true outliers.

Question 39

Just remind me, what do I use to check for normality?

Answer 39

• Kolmogorov-Smirnov or Shapiro-Wilk

* I don’t forget to run z scores for each group individually on the DV and the CV

Question 40

What else can I do to check assumptions?

Answer 40

I can Check linear relationship between CV & DV and correlation between CV and DV should be significant. Homogeneity of regression obtained through scatterplot

Question 41

Coming back to Homogeneity of Regression, what do I want to see when I look up my “test of between-subjects effects” table?

Answer 41

*I look at the interaction between the CV & IV.
*If they are not significant, the covariate may be used
(not significant means: there is no difference among the groups in their slopes)

Question 42

What do the “descriptives” tables tell me about my conditions?

Answer 42

The descriptives show me the means prior to the covariate adjustment

Question 43

What does the “Estimates” and “Pairwise Comparisons” tables tell me about my results?

Answer 43

• Estimates: The means after covariate adjustment

* Pairwise Comparisons: whether any interaction is significant after removing the effects of the covariate

Question 44

What can we do to the alpha value when running a Bonferroni adjustment on Tukey’s LSD?

Answer 44

• start with alpha = .01 (in view of loss of power using small sample sizes)
• to avoid a type 2 error: divide the alpha level of .01 / 3 for each of the post hoc analysis