WEEK 8: Two-Way ANOVA II Flashcards
Learning Objectives:
- Understand and be able to explain when a factorial ANOVA would be used
- Be able to report the results of a 2x2 factorial ANOVA
- Understand and be able to explain main effects in a factorial ANOVA
- Understand and be able to explain the interaction in a factorial ANOVA
- Understand how to use effect sizes in an ANOVA
A factorial ANOVA overview
So last week were looking at 2 factors each with 2 levels and each level had different participants, this week the same participants are taking part in every condition in a within participants design. We are using the same example of drink and drugs on driving simulation. Still interested in looking at the main effects (how factor 1 and factor 2 affect the outcome overall) and interactions (how as one factor changes, the other changes).
(Factor 1) Drink – (Levels 1/2) alcohol/ water
(Factor 2) Drug – (Levels 1/2) caffeine/ placebo
The main effect for factor one compares alcohol and water
The main effect for factor two compares caffeine and placebo
We calculate the main effect by calculating marginal means (e.g. for drink by taking the mean of alcohol and the mean of water and averaging them, ignoring the drug levels). We would do the same for factor 2, comparing caffeine and placebo using marginal means, ignoring the drink levels.
We then look at the interaction by looking at the mean values for each of the 4 conditions and comparing the differences between the differences. Put it in a table to make things easier when doing the exam. We want to know whether there’s a different pattern of results between those who had alcohol and caffeine compared to alcohol and placebo / and between water and caffeine and water and placebo. And then again between caffeine and alcohol and caffeine and water / and between placebo and alcohol and placebo and water. Compare across the grid horizontally and vertically, not diagonally.
We are looking for differences in patterns of results when looking at interactions.
So, when looking at a within design, we are still looking at the same things as between design but there are different calculations going on.
Effect size
Effect size – specific standardised measure of effect, allows to compare across different samples and outcomes, different to the effect we calculate in t-tests and anovas (effects vs error).
Socrative questions
Which conditions will the main effect of drink compare? Alcohol and water
Which conditions will the main effect of drug compare? Caffeine and placebo
(Some graph Qs)
Is it likely that there is a main effect of LoC? Explain why
- The main effect of LoC would compare success and failure responsibility. Success marginal mean is 13.225, failure is 12.6. There could be a significant main effect as there is a small difference between the marginal means. There also could not be a main effect as the difference between the marginal means may not be large enough to be significant.
Is there likely to be a main effect of time? Explain why
- The main effect of time would compare levels 1 and 2. Time level 1 marginal mean is 12.9, level 2 is 12.925. There is unlikely to be a main effect of time as the difference between the marginal means is likely too small to be significant.
Is it likely that there will be an interaction? Explain why
- There is a different response pattern in the two levels for factor 1 (LoC) and factor 2 (time) meaning that there is a smaller difference between the levels for factor 1 in level one of factor 2, and a larger difference between the levels of factor one at level two of factor 2. This means that it is likely that there is an interaction between the factors because as the time condition changes, the data for the locus of control condition changes (the lines will overlap).
REMEMBER: The interaction looks at the values at the end of the lines, not the marginal means or main effects directly.
Interaction – if we have an interaction which simple effects tests do we do?
- The interaction tells us that there is a different pattern of data depending on the levels and the factors, what we need to establish is where the significant differences lie. We also need to assess whether there are significant differences between the mean values, just looking at responsibility for success and just looking at responsibility for failure. We compare the points at the end of the lines (the 4 points), because we are comparing 4 things, we do 4 tests, so Bonferroni correct to a level of .05/4= so use an alpha/sig cut off of 0.0125.
Interaction
What does it mean to have an interaction between two variables?
“There will be a different pattern in the data for level 1 compared to level 2 of Factor #2 in level 1 and 2 for Factor #1”
“There will be a different pattern in the data for Caffeine compared to Placebo in the group given Alcohol and the group given Water”
> If the interaction is significant we do follow up tests
Also called ‘post hoc tests’
Post hoc tests following a significant interaction are called ‘simple effects tests’
They are all t tests with adjusted alpha (the p value cut off) using a Bonferroni Correction (Divide .05 by the number of tests)
If the interaction is NOT significant don’t do the t tests
What are simple effects tests?
The post hoc/ follow up tests conducted after finding a significant interaction between factors. They are all t tests with adjusted alpha cut off using an appropriate bonferroni correction (divide .05 by the number of tests being conducted).
If the interaction between factors is not significant the tests don’t need to be conducted
In the exam, the output will give the simple effect/ follow up t tests regardless of whether they are needed or not, so be careful and check for a significant interaction.
Example 1: Drug and Drink
Interpreting main effects in the presence of an interaction
Interpreting the interaction
***Main Effects:
These are the graphs we’ve looked at but with the factors on the other axis.
Which one would result in no significant main effects?
Again, a main effect is the comparison of the marginal means. So graph B would likely have no significant main effects because the marginal means for both the drug factor and the drink factor would not have differences big enough to likely be significant. Whereas the marginal mean difference between the time conditions would be very different.
Left we can see the marginal means for the drink condition, they are almost the same so likely no main effect.
Right is the marginal means for the drug condition, again, almost the same so likely no main effect of drug.
***Interaction:
More errors are made in the placebo condition compared to the cocaine condition when people drink alcohol But more errors are made in the cocaine condition compared to the placebo condition when people drink water
Summarise: Cocaine is protective against errors when people are drunk, but results in higher error rates when people are sober
Example 1: Time and LoC
Interpreting main effects in the presence of an interaction
Interpreting the interaction
***Main effects:
Left: the marginal means for time are sitting really close together, so there is unlikely to be a significant main effect for time.
Right: when looking at the marginal means for LoC, the means sit with a noticeable different in value between them, meaning there is more likely to be a main effect of LoC
*** Interaction:
Levels of responsibility for failure dropped at time level 2 compare to level one, whereas responsibility for success increased between time level 1 and 2. This means that, following the intervention participants were more willing to take responsibility for their successes and less willing to take responsibility for their failures.
Reporting a within ppts design factorial ANOVA
Sources of variance in a between participants ANOVA
On the output for a between ppt design, the variance values are concise and brief
The within ppts design output looks pretty complicated, go for the bits in the red squares, the top lines of each box. This gives the main effects and the df for each of the main effects and interactions
REPORTING a within ppts design factorial ANOVA
order of values
F(df1,df2) = [F value], p = [p value]
So here this would be F(1,9) = 38.68, p= Report effect sizes for all analyses
> For tests with 2 comparisons – cohen’s d. (p.217 D & R)
> Provides a measure of the ‘distance’ between 2 means in standard deviations
> Samples that overlap will be close to each other in SD
> Samples that do not overlap will be further away from each other
Effect size
Cohens d and partial eta squared
- Report effect sizes for all analyses
- For tests with 2 comparisons – cohen’s d. (p.217 D & R)
- Provides a measure of the ‘distance’ between 2 means in standard deviations
- Samples that overlap will be close to each other in SD
- Samples that do not overlap will be further away from each other
***Cohen’s d
Cohens d standardises things by putting it into a standard deviation, so a measure of how far apart the two means of the sample are in terms of SDs
Samples that overlap will have means closer together.
The amount of overlap results in different values of cohens d. Large overlap = small d (e.g. 85% overlap, Cohen’s d = 0.2). With small overlap = high d (e.g. 32% overlap, Cohen’s d = 1.4).
The bigger the cohen value, the smaller the overlap the bigger the difference.
Cohen decided a small effect size = .2, medium = .5, large = .8.
Effect size is illustrated via a partial eta squared value -ηp2 (p. 342 D & R)
*** Partial ETA squared
This is the calculate of effect size we use for an ANOVA. Gives an overall proportion of the variance accounted for by the different main effects Global magnitude of the difference between groups or conditions
- Provides a measure of how much of the total variance is accounted for by the treatment effect
- Can convert the value provided by SPSS into a % e.g. 0.375 = 37.5 % or 38%
This is really useful because you could have a p value that’s the same for all of the main effects and interactions, this isn’t helpful in terms of finding out what’s having the most impact on the DV, partial eta squared gives you that information.
You report partial eta squared on the end of the main effects and interactions throughout the reporting of the ANOVA.
**Write up example (p.314 D & R)
The main effect of Drink was significant with participants who drank alcohol performing worse (M = 23.3) than those who drank water (M = 18.10; F(1,9) = 38.68, p < .001, ηp2 = .81). This shows that 81% of the variance in error scores can be attributed to different levels of alcohol.
Reporting the ANOVA: Example 1
Spaces left blank
Part 2:
- Simple effects (follow up tests with bonferroni correction)
Bonferroni corrected post-hoc t-tests showed that [Factor 1 level 2 with Factor 2 level 1] participants were less socially anxious (M = ____, SD = _____) compared to [Factor 1 level 2 with Factor 2 level 2] participants (M = ___, SD = _____; t(df) = [t value], p = [p value]), whereas [Factor 1 level 1 with Factor 2 level 2] demonstrated less social anxiety (M = _____, SD = ____) than [Factor 1 level 1 with Factor 2 level 1] (M = _____, SD = ______; t(df) = [t value], p = [p value]).
The contrast between [Factor 2 level 2 with Factor 1 level 1] and [Factor 2 level 2 with Factor 1 level 2] was also significant (t(df) = [t value], p = [p value]) and demonstrated that [Factor 2 level 2 with Factor 1 level 1] are less socially anxious than [Factor 2 level 2 with Factor 1 level 2]. However, the contrast between [Factor 2 level 1 with Factor 1 level 1 and level 2] was not significant (t(df) = [t value], p = [p value]) .
This suggests that _________________________
Reporting the ANOVA: Example 2
Blanks filled: Therapy and Primed Mood
Part 1:
- Design and factors
- Main effects
- Interaction
The efficiency of two contrasting types of therapy (cognitive vs. person-centred) was assessed in relation to mood. In a 2*2 within participants design the total time spent in sessions (hours) was measured and participants were put into a good or bad mood before the session began.
Analyses revealed that the time people spent receiving cognitive or Person centred therapy (22.35 vs.23.2 hours respectively) was not significantly different (F(1, 36) = 0.53, p = .473, ηp2=.014). Nor was there any significant difference in the number of therapeutic hours between those in a good (22.3 hours) or bad (23.25 hours) mood (F(1, 36)= 0.66, p = .422, ηp2=.018).
However, there did appear to be a significant therapy*mood interaction (F(1,36) = 90.69, p
Reporting the ANOVA: Example 1
Spaces left blank
Part 2:
- Simple effects (follow up tests with bonferroni correction)
Bonferroni corrected post-hoc t-tests showed that [Factor 1 level 2 with Factor 2 level 1] participants were less socially anxious (M = ____, SD = _____) compared to [Factor 1 level 2 with Factor 2 level 2] participants (M = ___, SD = _____; t(df) = [t value], p = [p value]), whereas [Factor 1 level 1 with Factor 2 level 2] demonstrated less social anxiety (M = _____, SD = ____) than [Factor 1 level 1 with Factor 2 level 1] (M = _____, SD = ______; t(df) = [t value], p = [p value]).
The contrast between [Factor 2 level 2 with Factor 1 level 1] and [Factor 2 level 2 with Factor 1 level 2] was also significant (t(df) = [t value], p = [p value]) and demonstrated that [Factor 2 level 2 with Factor 1 level 1] are less socially anxious than [Factor 2 level 2 with Factor 1 level 2]. However, the contrast between [Factor 2 level 1 with Factor 1 level 1 and level 2] was not significant (t(df) = [t value], p = [p value]) .
This suggests that _________________________
Reporting the ANOVA: Example 2
Blanks filled: Therapy and Primed Mood
Part 1:
- Design and factors
- Main effects
- Interaction
The efficiency of two contrasting types of therapy (cognitive vs. person-centred) was assessed in relation to mood. In a 2*2 within participants design the total time spent in sessions (hours) was measured and participants were put into a good or bad mood before the session began.
Analyses revealed that the time people spent receiving cognitive or Person centred therapy (22.35 vs.23.2 hours respectively) was not significantly different (F(1, 36) = 0.53, p = .473, ηp2=.014). Nor was there any significant difference in the number of therapeutic hours between those in a good (22.3 hours) or bad (23.25 hours) mood (F(1, 36)= 0.66, p = .422, ηp2=.018).
However, there did appear to be a significant therapy*mood interaction (F(1,36) = 90.69, p
