Factorial Between-Participants ANOVA Learning Objectives

Question 1

Describe what an interaction is and how interactions can extend upon our understanding of established effects or relationships:

Answer 1

An interaction is where a particular effect or relationship between variables changes as a function of another variable. Interactions add nuance to established effects, by indicating the conditions under which the effect is likely to be stronger, weaker, non-existent, or even reversed.

Question 2

Define a factorial design and explain its advantages over a one-way design:

Answer 2

A factorial design is a research methodology that allows for the investigation of the main and interaction effects between two or more independent variables on one or more outcome variables. It has an advantage over a one-way design, as it holds the opportunity to investigate simultaneous effects of two or more factors, as well as the interactions between factors.

Question 3

Describe the research questions that can be addressed in two-way (A x B) factorial design:

Answer 3

Research questions that can be addresses in two-way factorial designs are those that are interested in the interaction of two factors, as well as the effects of each factor. For example does playing video games and/or deep breathing while playing, have an effect on anxiety levels?

Question 4

Explain what it means to say that a significant interaction qualifies a significant main effect:

Answer 4

By saying that a significant interaction qualifies a significant main effect, it is meant that the relationship between the independent variable and dependent variable changes depending on the level of another independent variable.

Question 5

Explain how marginal means are calculated, and specify the types of effects they are used to test in two-way ANOVA:

Answer 5

Marginal means are calculated by averaging all scores on each level of one factor, collapsing over levels of the other factor/s. In two-way ANOVA, marginal means are used to test whether there is a significant/non-significant main effect of each factor - essentially, by comparing marginal means, you can see if there is a significant difference somewhere between them.

Question 6

Explain how cell means are calculated, and specify the types of effects they are used to test in two-way ANOVA

Answer 6

Cell means are calculated by averaging all scores at each level of one factor, at each level of the other factor/s. In two-way ANOVA, cell means are used in comparison, to test for simple effects (if a significant interaction has been found).

Question 7

Explain the difference between a main effect and simple effect

Answer 7

A main effect uses marginal means to indicate if there is a significant difference somewhere between the levels of one factor (not using the other factor/s), whereas a simple effect utilises cell means to indicate the effect of one factor at only one level of the other factor.

Question 8

How is a significant interaction interpreted?

Answer 8

An initial significant interaction must be interpreted that there is a significant interaction SOMEWHERE within the results - significant interactions must be followed up.

Question 9

Describe how two-way interactions are plotted

Answer 9

Two-way interactions are typically plotted using a line graph, with the DV on the y-axis, the x-axis being the factor with either the most levels, or the most theoretically important factor. The less important factor is then represented by separate lines.

Question 10

In a graph of a two-way ANOVA, explain how we would identify:
- Main effects of each factor
- An interaction between the two factors
- Simple effects of each factor

Answer 10

Main effects can be identified by averaging across each factor, at each level of that factor, to see if there is a difference between each level. A potential interaction between two factors can be identified by seeing if the lines are non-parallel (suggesting there may be an interaction). The simple effects of each factor can be identified by looking at one line/set of points in isolation, and seeing if that line/points are at the same point, or a different point.

Question 11

Specify the general and specific notations for a factorial between-participants design

Answer 11

The general notations for a factorial between-participants design is the type of design, then number of factors, then numbers of levels each factor has. For example, a two-way between-participants factorial design. Specifically, the levels of each factor are notated, for example a 3 x 2 between-participants factorial design.

Question 12

Describe the characteristics of a fully between-participants design

Answer 12

A fully between-participants design means that each participant is exposed to only one condition (one factor at one level of that factor).

Question 13

List the various names used to describe “between-participants” designs/factors

Answer 13

Between-participants, between-groups, independent groups, independent measures

Question 14

Describe the research question that can be addressed in one-way ANOVA

Answer 14

Is there a main effect of factor A on the DV? Is there a main effect of factor B on the DV? Is there a factor A x factor B interaction?

Question 15

Explain how variance is partitioned in two-way between-participants ANOVA

Answer 15

In two-way between-participants ANOVA, variance is partitioned into within-groups variance (random variation in DV within the groups that can’t be explained by the factors or their interaction), and between-groups variance. Between-groups variance is further partitioned into variance due to factor A (systemic variation in DV between the different groups/levels/treatments of factor A), variance due to factor B (systemic variation in DV between the different groups/levels/treatments of factor B), and variance due to A x B interaction (systemic variation in DV between the different cells of A x B that can’t be explained by the main effects of factor A and B).

Question 15

Explain the structural model (also known as the linear or conceptual model) of two-way between-participants ANOVA

Answer 15

Each participants’ score is the sum of three components:
ijk = g.. ßk + + Eijk
x
Xijk score of person t at level J (of Factor A) and level k (ot Factor B) — i.e., cell jk
g.. grand mean
treatment effect of levelj lot Factor A)
ßk treatment effect ot level k lot Factor B)
aßjk treatment effect of cell 1k (based on interaction between Factor A and B)
Eijk error associated with person t in cell 1k

Question 16

In conceptual terms (i.e., in words), explain what is represented by each of the following in two-way between-participants ANOVA:
- SS for each main effect
- SS for the interaction SS error

Answer 16

In two-way between-participants ANOVA, SS for each main effect is the amount of variability that is accounted for by a specific factor, while holding the other factor constant. SS for the interaction is the amount of variability in the interaction.

Question 17

Explain what each of the following tell us in two-way between-participants ANOVA:
- A significant F test for the main effect of Factor A
- A significant F test for the main effect of Factor B
- A significant F test for the A x B interaction

Answer 17

A significant F test for the main effect of factor A tells us that there is a difference somewhere among factor A. A significant F test for the main effect of factor B tells us that there is a difference somewhere among factor B. A significant F test for the A x B interaction tells us that there is a difference somewhere among the interaction.

Question 18

When the F ratio is calculated for each omnibus test in two-way between-participants ANOVA, specify which MS terms are used in the numerator and denominator in each case

Answer 18

MSA/MSerror, MSB/MSerror, MSAB/MSerror

Question 19

List the main assumptions of between-participants ANOVA

Answer 19

The main assumptions of between-participants ANOVA are homogeneity of variance, normality, independence of observations, and a continuous DV.

Question 20

Explain what an omnibus test is, and list the tests in two-way ANOVA that can be categorised as omnibus tests

Answer 20

An omnibus test is any test resulting from the preliminary partitioning of variance in ANOVA - it is designed to detect any significant deviation from a specific null hypothesis. There are three omnibus tests in two-way factorial ANOVA: main effect of factor A, main effect of factor B, and A x B interaction effect.

Question 21

Explain the difference between omnibus tests and follow-up tests

Answer 21

Omnibus tests only show that there is an effect somewhere within the factor/interaction, whereas follow-up tests establish the direction/nature of the effect.

Question 22

Specify the circumstances in which the omnibus tests would require follow-up tests in two-way ANOVA

Answer 22

In two-way ANOVA, omnibus tests would require follow up tests if it is found that there is a significant F (when we reject the null hypothesis).

Question 23

Explain the difference between linear contrasts and pairwise comparisons

Answer 23

Linear contrasts are where you compare one mean or set of means to another mean or set of means, whereas pairwise comparisons are where you compare two means at a time. While linear contrasts use t-tests, pairwise comparisons use protected t-tests (protected against unnecessary type 1 error inflation, as they are conducted after a significant F is found for the main effect).

Question 24

Specify the types of comparisons used to follow up a significant main effect in two-way ANOVA

Answer 24

Pairwise comparisons and linear contrasts (SPSS calculates pairwise comparison t-tests).

Question 25

Explain what a significant t test for a main effect comparison tells us, and what types of means are being compared in this test

Answer 25

A significant t-test for a main effect comparison tells us which differences are significant. The types of means being compared in this test are observed means.

Question 26

Specify the tests used to follow up a significant two-way interaction in two-way ANOVA

Answer 26

Simple effects are used to follow up a significant two-way interaction. Simple effects are the effects of one factor at each level of the other factor - they compare cell means of one factor at each level of the other factor, and tell us exactly which cell means significantly differ.

Question 27

Explain what a significant F test for a simple effect tells us

Answer 27

That there is a significant difference somewhere among the cell means for the levels of that factor

Question 28

Explain how we would determine the maximum number of simple effects that could be tested for a factor

Answer 28

The number of levels of each factor involved in the interaction needs to be considered

Question 29

Specify the circumstances in which simple effects would need to be followed up

Answer 29

Simple effects would need to be followed up if there are more than two levels of the factor which has been found significant.

Question 30

Specify the types of comparisons used to follow up a significant simple effect in two-way ANOVA

Answer 30

Simple comparisons (which test exactly which cell means of the factor are significantly different, at each level of the other factor.

Question 31

Explain what a significant t test for a simple comparison tells us, and what types of means are being compared in this test

Answer 31

A significant t-test for a simple comparisons tells us which cell means of the factor are significantly different, specifically.

Question 32

In two-way between-participants ANOVA, explain how the variance can be repartitioned when testing: - Simple effects of Factor A - Simple effects of Factor B

Answer 32

As simple effects re-partition the main effect and interaction variance, variance can be then repartitioned. When testing the simple effects of factor A, the sum of the variability explained by the simple effects of factor A at each level of factor B is equal to the sum of variability explained by the main effect of factor A and the interaction. When testing factor B, the sum of the variability explained by the simple effects of factor B at each level of factor A is equal to the sum of variability explained by the main effect of factor B and the interaction.

Question 33

Explain why significance tests are not that helpful when we want to determine the importance of findings

Answer 33

When determining the importance of findings, significance tests are not that helpful they use an arbitrary acceptance criterion, which results in a binary outcome (significant v non-significant). They also do not give any information about the practical significance of findings. Also, statistical significance of a tiny, unimportant effect may eventually slip under the acceptance criterion as the sample size increases.

Question 34

Define effect sizes and explain why they are useful

Answer 34

Effect sizes are statistical measures that indicate the magnitude or strength of a relationship between variables, or the difference between groups. They are useful because they give another way of assessing the importance of an effect.

Question 35

Describe the three types of effect size that accompany F-tests and explain the differences between them (e.g., how they are calculated, bias/conservativeness): - Eta-squared (η2) - Omega-squared (ω2) - Partial eta-squared (ηp2)

Answer 35

The three estimates differ depending on sample size, and error variance. Eta-squared is the proportion of total variance in the samples DV scores that is accounted for by the effect. It is considered a biased estimate of the true magnitude of the effect of the population, as it is less conservative. Omega-squared is the estimated proportion of total variance in the populations DV scores that is accounted for by the effect. It is considered a less biased estimate of the true magnitude of the effect in the population, as it is more conservative. Partial eta-squared is the proportion of residual variance in the samples DV scores that is accounted for by the effect. It is the least conservative, and has an inflated effect size estimate.

Question 36

Describe the following statistics that typically accompany t-tests: - Cohen’s d - 95% confidence intervals

Answer 36

Cohen's d is a standardised effect size measure that is used to quantify the difference between two group means - how many standard deviations the two group means are apart from each other. A larger d indicates a larger difference between the groups. 95% confidence intervals is a range of values that you can be sure contains the true mean of the population. CI can also be used to determine the significance of a difference - if CIs cross 0 this indicates the statistic is non-significant.

Question 37

Explain what higher-order designs are

Answer 37

Higher-order designs are factorial designs with more than 2 factors. They allow for designs with higher external validity - more factors in design, more representative of complex nature of world. However, it does make the pattern of effects and interactions more complex.

Question 38

Explain how a manipulation check is conducted:

Answer 38

A regular factorial ANOVA is conducted, where the manipulation check measure is used instead of the DV. Ideally it should be found that the main effect of the factor being checked was significant, and that the main effects of the other factor/s and interaction/s was non-significant.

Question 39

List the omnibus effects in three-way ANOVA

Answer 39

Main effect x 3 (specifies whether there is an effect of one factor - averaging over levels of the other two factors). Two-way interactions x 3 (specifies whether the effect of one factor changes depending on the level of another factor - averaging over levels of the third factor). Three-way interaction (specifies whether the two-way interaction between two factors changes depending on the level of the third factor.

Question 40

Describe the research questions that can be addressed in three-way ANOVA

Answer 40

Is there a main effect of factor A/B/C on the DV? IS there a Factor A/B x B/C interaction? Is there a factor A x B x C interaction?

Question 41

Describe how three-way interactions are graphed

Answer 41

Three-way interactions are plotted as two separate two-way interactions at each level of the third factor.

Question 42

Explain how variance is partitioned in three-way ANOVA

Answer 42

Variance is partitioned into main effect of A, main effect of B, main effect of C, A x B interaction, A x C interaction, B x C interaction, and A x B x C interaction, and error.

Question 43

In three-way ANOVA, explain what the null hypothesis represents in tests of the main effects, two-way interactions, and three-way interaction

Answer 43

In tests of the main effects, the null hypothesis represents that there is no effect on one factor. In tests of two-way interaction, the null hypothesis represents that there is no effect of one factor that changes depending on the level of another factor. In tests of three-way interaction, the null hypothesis represents that the two-way interaction between two factors does not change depending on the level of the third factor.

Question 44

In the omnibus summary table for three-way ANOVA, explain what the SS and MS values represent, and how F is calculated for each effect

Answer 44

SS values represents the sum of squares (the variability around a mean), the MS represents the mean square (an estimate of population variance. For each effect, the F is calculated by dividing the MS of the factor being tested by the MSerror, and then it assesses the statistical significance of each effect.

Question 45

In three-way ANOVA, explain what a significant F test for the main effect of Factor A tells us:

Answer 45

That there is a significant difference somewhere among the means of factor A.

Question 46

In three-way ANOVA, explain what a significant F test for the main effect of Factor B tells us:

Answer 46

That there is a significant difference somewhere among the means of factor B.

Question 47

In three-way ANOVA, explain what a significant F test for the main effect of Factor C tells us:

Answer 47

That there is a significant difference somewhere among the means of factor B.

Question 48

In three-way ANOVA, explain what a significant F test for the A x B two-way interaction tells us:

Answer 48

That effect of factor A is different at each level of factor B.

Question 49

In three-way ANOVA, explain what a significant F test for the A x C two-way interaction tells us:

Answer 49

That effect of factor A is different at each level of factor C.

Question 50

In three-way ANOVA, explain what a significant F test for the B x C two-way interaction tells us:

Answer 50

That effect of factor B is different at each level of factor C.

Question 51

In three-way ANOVA, explain what a significant F test for the A x B x C three-way interaction tells us:

Answer 51

That there is a significant three-way interaction, but follow up tests are needed to find out more/where.

Question 52

Specify the circumstances in which the following would require follow-up tests in three-way ANOVA, as well as the tests that would be used the case of an omnibus main effect

Answer 52

If any omnibus main effect is significant and has more than two factors.

Question 53

Specify the circumstances in which the following would require follow-up tests in three-way ANOVA, as well as the tests that would be used in the case of an omnibus two-way interaction

Answer 53

If any omnibus two-way interaction is significant, the simple effect is tested of one variable at each level of the other (ignoring the third). If that simple effect is significant, and the factor has more than two levels, simple comparisons need to be run.

Question 54

Specify the circumstances in which the following would require follow-up tests in three-way ANOVA, as well as the tests that would be used in the case of an omnibus three-way interaction

Answer 54

If this interaction is found significant, simple interactions need to be run, then simple simple effects if they are significant, and then simple simple comparisons (for simple simple effects of factors with more than 2 levels) if they are found significant.

Question 55

Explain what simple two-way interactions are in three-way ANOVA and how they differ from omnibus two-way interactions and describe the different sets of follow-up tests that would be conducted if we wished to follow up these two different types of two-way interaction

Answer 55

Simple two-way interactions are the first step in interpreting a three-way interaction. They break down the three-way interaction into three sets, which are dependent upon research question and hypothesis. They are different to omnibus two-way interactions as they do not ignore the third factor - instead they test them at each level of the third factor. If we wished to follow up these different types of two-way interactions we would conduct simple simple effects.

Question 56

Explain what a significant F test for a simple two-way interaction tells us (e.g., significant simple A x B interaction at first level of Factor C)

Answer 56

A significant F-test for a simple two-way interaction tells us that the A x B interaction is significant at that level of factor C. If multiple levels of factor C are found to be significant, simple simple effects need to be tested for to see how they change depending on the level.

Question 57

Specify the tests used to follow up a significant simple two-way interaction in a three-way ANOVA

Answer 57

Simple simple effects - then simple simple comparisons

Question 58

Explain what simple simple effects are in three-way ANOVA and how they differ from simple effects

Answer 58

Simple simple effects are a test used to interpret a significant simple two-way interaction. They break the interaction down into the simple simple effects of one factor, at each level of the second factor, at the level/s of the third factor in which the simple two-way interaction was significant. They are conceptually identical to simple effects, however simple effects are the effect of factor A at each level of factor B, whereas simple simple effects are the effect of factor A at each level of factor B, at each level of factor C.

Question 59

Explain what a significant F test for a simple simple effect tells us

Answer 59

That there is a significant difference among the cell means of the levels. If only one simple simple effect is found to be significant, then that shows where the difference is, however if there are multiple found significant, then further tests need to be done to tell us exactly which means significantly differ (simple simple comparisons).

Question 60

Specify the types of comparisons used to follow up a significant simple simple effect in a three-way ANOVA

Answer 60

Simple simple comparisons

Question 61

Explain what simple simple comparisons are in three-way ANOVA and how they differ from simple comparisons

Answer 61

Simple simple comparisons show which means significantly differ in regards to the cell means for a level of a factor. They are different to simple comparisons, as simple comparisons compare levels of factor A at each level of factor B, whereas simple simple comparisons compare levels of factor A at each level of factor B at each level of factor C.

Question 62

Explain what a significant t test for a simple simple comparison tells us

Answer 62

That that level of factor A at the level of factor B at the level of factor C is significantly different.

Question 63

Specify the error term that is used for all follow-up tests in a factorial between-participants ANOVA

Answer 63

Within-groups variance/residual