MIDTERM Flashcards

(224 cards)

1
Q

less focused research question, collect large amounts of relatively “unfiltered” data from a relatively small number of individuals

A

QUALITATIVE RESEARCH

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

describe their data using non-statistical techniques.

A

QUALITATIVE RESEARCH

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

help researchers to generate new and interesting research questions and hypotheses.

A

PURPOSE OF QUALITATIVE RESEARCH

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

provide rich and detailed descriptions of human behavior in the real-world contexts in which it occurs.

A

PURPOSE OF QUALITATIVE RESEARCH

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

convey a sense of what it is actually like to be a member of a particular group or in a particular situation

A

PURPOSE OF QUALITATIVE RESEARCH

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

“lived experience” of the research participants

A

PURPOSE OF QUALITATIVE RESEARCH

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

qualitative research tend to be unstructured

A

INTERVIEW

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

consisting of a small number of general questions or prompts that allow participants to talk about what is of interest to them.

A

INTERVIEW

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

researcher can follow up by asking more detailed questions about the topics that do come up.

A

INTERVIEW

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Small groups of people who participate
together in interviews focused on a
particular topic or issue.

A

FOCUS GROUP

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

The interaction among participants in a
focus group can sometimes bring out information than can be learned in a one-on- one interview.

A

FOCUS GROUP

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Researchers become active participants
in the group or situation they are
studying.

A

PARTICIPANT OBSERVATION

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

The data they collect can include interviews (usually unstructured), their own notes based on their observations and interactions, documents, photographs, and other artifacts.

A

PARTICIPANT OBSERVATION

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

start with the data and develop a theory or an interpretation that is ―grounded in‖ those data.

A

GROUNDED THEORY

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

the 3 ground theory stages

A
  1. Identify ideas that are repeated throughout the data.
  2. Organize these ideas into a smaller number of broader themes.
  3. Write a theoretical narrative—an interpretation—of the data in terms of the themes that they have identified.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Experimental research strategy establishes the existence of a cause-and- effect relationship between two variables.

A

CAUSE AND EFFECT RELATIONSHIPS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

To accomplish this goal, an experiment manipulates one variable while a second variable is measured and other variables are controlled.

A

CAUSE AND EFFECT RELATIONSHIPS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Experiment or a true experiment attempts to show that changes in one variable are directly responsible for changes in a second variable.

A

CAUSE AND EFFECT RELATIONSHIPS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

4 basic of elements

A
  • manipulation
  • measurement
  • comparison
  • control
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

one variable by changing its value to create a set of two or more treatment conditions.

A

MANIPULATION

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

second variable is measured for a group of participants to obtain a set of scores in each treatment condition.

A

MEASUREMENT

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

scores in one treatment condition are compared with the scores in another treatment condition.

A

COMPARISON

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

All other variables are controlled to be sure that they do not influence the two variables being examined.

A

CONTROL

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

All other variables are controlled to be sure that they do not influence the two variables being examined.

A

CONTROL

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
variable manipulated by the researcher.
INDEPENDENT VARIABLE
26
situation or environment characterized by one specific value of the manipulated variable.
TREATMENT CONDITION
27
different values of the independent variable selected to create and define the treatment conditions.
LEVEL
28
different values of the independent variable selected to create and define the treatment conditions.
DEPENDENT VARIABLE
29
all variables in the study other than the independent and dependent variables.
EXTRANEOUS VARIABLES
30
One problem for experimental research is that variables rarely exist in isolation.
Causation & the Third-Variable Problem
31
A study may establish that two variables are related, it does not necessarily mean that there is a direct (causal) relationship between the two variables.
Causation & the Third-Variable Problem
32
research study may establish a relationship between two variables, the existence of a relationship does not always explain the direction of the relationship.
Causation & the Directionality Problem
33
To establish a cause-and-effect relationship, an experiment must control nature
Controlling Nature
34
creating an unnatural situation wherein the two variables being examined are isolated from the influence of other variables
Controlling Nature
35
exact character of a relationship can be seen clearly.
CONTROLLING NATURE
36
Consists of identifying the specific values of the independent variable to be examined
Manipulation
37
creating a set of treatment conditions corresponding to the set of identified values.
MANIPULATION
38
Simply observing that a relationship exists does not explain the relationship
Manipulation and the Directionality Problem
39
certainly does not identify the direction of the relationship.
Manipulation and the Directionality Problem
40
help researchers control the influence of outside variables
Manipulation and the Third-Variable Problem
41
the existence of a relationship does not necessarily mean that there is a direct connection between the two variables.
Manipulation and the Third-Variable Problem
42
The particular concern is to identify and control any third variable that changes systematically along with the independent variable
Control and the Third-Variable Problem
43
potential to influence the dependent variable
Control and the Third-Variable Problem
44
housands of potentially confounding variables, however, the problem of controlling (or even monitoring) every extraneous variable appears insurmountable.
Extraneous Variables and Confounding Variables
45
standardizing the environment and procedures, most environmental variables can be held constant.
Holding a Variable Constant
46
eliminates its potential to become a confounding variable.
Holding a Variable Constant
47
Control over an extraneous variable can also be exercised by matching the levels of the variable across treatment conditions.
Matching Values across Treatment Conditions
48
use of a random process to help avoid a systematic relationship between two variables.
Randomization
49
use of a random process to assign participants to treatment conditions.
Random assignment
50
goal of an experiment is to show that the scores obtained in one treatment condition are consistently different from the scores in another treatment and that the differences are caused by the treatments.
Comparing Methods of Control
51
The two active methods of control (holding constant and matching) require some extra effort or extra measurement
Advantages & Disadvantages of Control Methods
52
typically used with only one or two specific variables identified as real threats for confounding
Advantages & Disadvantages of Control Methods
53
condition in which the treatment is administered
Experimental condition
54
condition in which the treatment is not administered.
Control condition
55
condition in which the participants do not receive the treatment being evaluated.
No -Treatment Control Conditions
56
inert or innocuous medication, a fake medical treatment
Placebo
57
absolutely no medicinal effect, but produces a positive or helpful effect simply
placebo
58
individual expects or believes it will happen.
placebo
59
positive response by a participant to an inert medication that has no real effect on the body.
PLACEBO EFFECT
60
thinks the medication is effective.
PLACEBO EFFECT
61
participants receive a placebo instead of the actual treatment.
Placebo control condition
62
measure to assess how the participants perceived and interpreted the manipulation and/or to assess the direct effect of the manipulation.
MANIPULATION CHECK
63
4 situations of manipulation check
Participant Manipulations. Subtle Manipulations. Placebo Controls. Simulations.
64
creation of conditions within an experiment that simulate or closely duplicate the natural environment in which the behaviors being examined would normally occur.
simulation
65
Research conducted in a place that the participant or subject perceives as a natural environment.
field studies
66
allow researchers to investigate behavior in more lifelike situations and, therefore, should increase the chances that the experimental results accurately reflect natural events.
advantage of simulation and field studies
67
allowing nature to intrude on an experiment means that the researcher often loses some control over the situation and risks compromising the internal validity of the experiment.
disadvantage of simulation and field studies
68
compares different groups of individuals.
characteristics of between subjects design
69
requires a separate, independent group of individuals for each treatment condition.
Between-subjects experimental design
70
between-subjects design allows only one score for each participant. Every individual score represents a separate, unique participant.
Independent Scores
71
individual score is independent from the other scores
advantage of between subject designs
72
require a relatively large number of participants.
Disadvantages of Between- Subjects Designs
73
personal characteristics that differ from one participant to another
individual differences
74
between-subjects design must also be concerned with threats to internal validity from environmental variables that can change systematically from one treatment to another
other confounding variables
75
participant characteristics that can differ from one participant to another.
confounding from individual differences
76
Environmental variables are any characteristics of the environment that may differ.
confounding from environmental variables
77
both the opportunity and the responsibility to create groups that are equivalent.
equivalent groups
78
process used to obtain participants should be as similar as possible for all of the groups.
created equally
79
Except for the treatment conditions that are deliberately varied between groups, the groups of participants should receive exactly the same experiences.
treated equally
80
characteristics of the participants in any one group should be as similar as possible to the characteristics of the participants in every other group.
Composed of equivalent individuals
81
process is used to assign participants to groups.
random assignments
82
goal is to ensure that all individuals have the same chance of being assigned to a group.
random assignments
83
group assignment process is limited to ensure predetermined characteristics (such as equal size) for the separate groups.
restricted random assignments
84
involves assigning individuals to groups so that a specific participant variable is balanced, or matched, across the groups.
matching
85
The intent is to create groups that are equivalent (or nearly equivalent) with respect to the variable matched.
matching
86
method of preventing individual differences from becoming confounding variables is simply to hold the variable constant.
range of variability
87
An alternative to holding a variable completely constant is to restrict its range of values.
range of variability
88
good because they provide evidence of differential treatment effects.
differences between treatments
89
bad because the differences that exist inside the treatment conditions determine the variance of the scores
differences within treatments
90
All participants within a group should be treated exactly the same
Standardize Procedures and Treatment Setting
91
Researchers should avoid making any changes in the treatment setting or the procedures used from one individual to another.
Standardize Procedures and Treatment Setting
92
Holding a participant variable constant or restricting its range could be effective techniques used for limiting confounding from individual differences
limit individual differences
93
4 Minimizing Variance within Treatments
1. Standardize Procedures and Treatment Setting 2. Limit Individual Differences 3. Random Assignment and Matching 4. Sample Size
94
participant withdrawal from a research study before it is completed.
attrition
95
rates from one group to another and can threaten the internal validity of a between-subjects experiment.
differential attritions
96
Whenever the participants in one treatment condition are allowed to talk with the participants in another condition, there is the potential for a variety of problems to develop
communication between groups
97
spread of the treatment from the experimental group to the control group, which tends to reduce the difference between the two conditions.
diffusion
98
Another risk is that an untreated group learns about the treatment being received by the other group and demands the same or equal treatment.
Compensatory equalization
99
untreated group works extra hard to show that they can perform just as well as the individuals receiving the special treatment.
Compensatory rivalry
100
participants in an untreated group simply give up when they learn that another group is receiving special treatment
Resentful demoralization
101
the simplest version of a between- subjects experimental design involves comparing only two groups of participants.
Single-factor two-group design (two- group design)
102
The researcher manipulates one independent variable with only two levels.
Single-factor two-group design (two- group design)
103
use this design with more than two groups to evaluate the functional relation between an independent and a dependent variable or to include several different control groups in a single study.
Single-factor multiple-group design
104
compares two or more different treatment conditions (or compares a treatment and a control) by observing or measuring the same group of individuals in all of the treatment conditions being compared.
Within-subjects experimental design or repeated- measures experimental design
105
looks for differences between treatment conditions within the same group of participants.
within subjects design
106
Environmental variables are characteristics of the environment that may change from one treatment condition to another.
Confounding from environmental variables
107
This design comes from the fact that the design often requires a series of measurements made over time.
Confounding from time-related variables.
108
environmental events other than the treatment that change over time and may affect the scores in one treatment differently than in another treatment.
history
109
When a group of individuals is being tested in a series of treatment conditions, any physiological or psychological change that occurs in participants during the study and influences the participants’ scores.
maturation
110
Changes in the measuring instrument that occurs during a research study in which participants are measured in a series of treatment conditions.
instrumentation
111
environmental events other than the treatment that change over time and may affect the scores in one treatment differently than in another treatment.
Statistical regression (regression toward the mean)
112
when the experience of being tested in one treatment condition (participating and being measured) has an influence on the participants’ scores in a later treatment condition(s).
order effects
113
(progressive decline in performance as a participant works through a series of treatment conditions)
fatigue effects
114
(progressive improvement in performance as a participant gains experience through the series of treatment condition).
practice effect
115
(progressive improvement in performance as a participant gains experience through the series of treatment condition).
carry over effects
116
subjective perception of a treatment condition is influenced by its contrast with the previous treatment.
Contrast effect
117
changes in a participant’s behavior or performance that are related to general experience in a research study but not related to a specific treatment or treatments.
Progressive error
118
produce changes in the scores from one treatment condition to another that are not caused by the treatments and can confound the results of a research study.
order effects as a confounding variables
119
Within-subjects designs can control environmental threats to internal validity using the same techniques that are used in between- subjects designs.
order effects as a confounding variables
120
order effects as a confounding variables controlled by
1. Randomization 2. holding them constant 3. matching across treatment conditions
121
one treatment condition to the next, a researcher has some control over time- related threats to internal validity.
controlling time
122
shortening the time between treatments can reduce the risk of time-related threats
controlling time
123
this technique can often increase the likelihood that order effects will influence the results.
controlling time
124
order effects are so strong and so obvious that a researcher probably would not even consider using a within-subjects design.
Switch to a Between-Subjects Design
125
A between-subjects design (with a separate group for each treatment) is available as an alternative and completely eliminates any threat of confounding from order effects.
switch to a between subjects design
126
changing the order in which treatment conditions are administered from one participant to another so that the treatment conditions are matched with respect to time.
counterbalancing
127
purpose of counterbalancing is to eliminate the potential for confounding by disrupting any systematic relationship between the order of treatments and time-related factors.
Counterbalancing: Matching Treatments with Respect to Time
128
distribute order effects evenly across the different treatment conditions. However, this process does not eliminate the order effects.
coubterbalancing and variance
129
A more serious problem is that counterbalancing adds the order effects to some of the individuals within each treatment but not to all of the individuals.
counterbalancing and variance
130
order effects are relatively large, the process of counterbalancing can undermine the potential for a successful experiment.
counterbalancing and variance
131
One treatment might produce more of an order effect than another treatment.
asymmetrical order effects
132
order effects are not symmetrical and counterbalancing the order of treatments does not balance the order effects.
asymmetrical order effects
133
The idea behind complete counterbalancing is that a particular series of treatment conditions may create its own unique order effect.
Counterbalancing and the Number of Treatments
134
One solution to this problem is to use partial counterbalancing.
Counterbalancing and the Number of Treatments
135
uses enough different orderings to ensure that each treatment condition occurs first in the sequence for one group of participants, occurs second for another group, third for another group, and so on.
partial counterbalancing
136
requires relatively few participants in comparison to between-subjects designs
Advantages of Within-Subjects Designs
137
no individual differences between groups because there is only one group of participants
Advantages of Within-Subjects Designs
138
each participant appears in every treatment condition, each individual serves as his own control or baseline.
Advantages of Within-Subjects Designs
139
In statistical terms, a within-subjects design is generally more powerful than a between- subjects design.
Advantages of Within-Subjects Designs
140
Each participant often goes through a series of treatment conditions, with each treatment administered at a different time.
Disadvantages of Within-Subjects Designs
141
individuals who start the research study may be gone before the study is completed
Participant attrition
142
Each individual in one group is matched with a participant in each of the other groups.
Matched-Subjects Designs
143
The matching is done so that the matched individuals are equivalent with respect to a variable that the researcher considers to be relevant to the study.
Matched-Subjects Designs
144
many of the same advantages and disadvantages as the two-group between-subjects design.
Two-Treatment Designs
145
Design is easy to conduct and the results are easy to understand.
positive of two treatment design
146
It is very easy to counterbalance the design to minimize the threat of confounding from time- related factors or order effects.
positive of two treatment design
147
Study with only two treatments provides only two data points.
Negative of two treatment design
148
The data are more likely to reveal the functional relationship between the two variables being studied
Multiple-Treatment Designs Advantage
149
Produces a more convincing demonstration of a cause-and-effect relationship than is provided by a two-treatment design.
Advantage of Multiple-Treatment Designs
150
If a researcher creates too many treatment conditions, the distinction between treatments may become too small to generate significant differences in behavior.
Disadvantage of multiple treatment design
151
Multiple treatments for a within-subjects design typically increase the amount of time required for each participant to complete the full series of treatments increasing the likelihood of participant attrition.
Disadvantage of multiple treatment design
152
Typically involve comparison of scores from different groups or different conditions
Nonexperimental and Quasi-Experimental Research Strategies
153
two strategies use a non-manipulated variable to define the groups or conditions being compared
Nonexperimental and Quasi-Experimental Research Strategies
154
make little or no attempt to control threats to internal validity
nonexperimental designs
155
actively attempt to limit threats to internal validity.
quasi-experimental designs
156
Between-subjects designs, also known as
nonequivalent group designs
157
Within-subjects designs, also known as
pre-post designs
158
Compares preexisting groups of individuals
Between-Subjects Designs
159
Examples of Between-Subjects Designs
o Differentialresearch o Posttest-only non-equivalent o control group design o Pretest–posttest non o Equivalent control group o Design o Cross-sectional
160
Compares two or more scores for one group of participants
Within-Subjects Designs
161
Examples of Within-Subjects Designs
o Pretest–posttest design o Time-series design o Longitudinal o Developmental design
162
A research study in which the different groups of participants are formed under circumstances that do not permit the researcher to control the assignment of individuals to groups, and the groups of participants are, therefore, considered nonequivalent.
Non Equivalent Group Design
163
the researcher cannot use random assignment to create groups of participants.
Non Equivalent Group Design
164
Individual differences between groups that individual differences create a confound whenever the assignment procedure produces groups that have different participant characteristics.
Threats to Internal Validity for Nonequivalent Group Designs
165
A research study that simply compares preexisting groups.
The Differential Research Design
166
A differential study uses a participant characteristic such as gender, race, or personality to automatically assign participants to groups.
The Differential Research Design
167
classified as a nonexperimental research design.
Differential Research
168
compares two non-equivalent groups
Pretest–Posttest Nonequivalent Control Group Design
169
One group is measured twice, once before a treatment is administered and once after. The other group is measured at the same two times but does not receive any treatment.
Pretest–Posttest Nonequivalent Control Group Design
170
classified as quasi-experimental.
Pretest–Posttest Nonequivalent Control Group Design
171
research study in which a series of observations is made over time for one group of participants.
Pre–post design
172
Threats to internal validity for pre-post designs
1. History 2. Instrumentation 3. Order effects 4. Maturation 5. Statistical regression
173
used to examine changes in behavior related to age.
Developmental research designs
174
The different groups are measured at one point in time and then compared.
Cross-Sectional Developmental Research Design
175
Uses different groups of individuals, each group representing a different age.
Cross-Sectional Developmental Research Design
176
individuals who were born at roughly the same time and grew up under similar circumstances.
cohort
177
differences between age groups (or cohorts) caused by unique characteristics or experiences other than age.
cohort effects and generation effects
178
Examines development by observing or measuring a group of cohorts over time.
Longitudinal Developmental Research Design
179
The absence of cohort effects because the researcher examines one group of people over time rather than comparing groups that represent different ages and come from different generations.
Strengths of the Longitudinal Developmental Design
180
researcher can discuss how a single individual’s behavior changes with age.
Strengths of the Longitudinal Developmental Design
181
Extremely time-consuming, both for the participants and the researcher
Weakness of the Longitudinal Developmental Design
182
designs are very expensive to conduct because researchers need to track people down and persuade them
Weakness of the Longitudinal Developmental Design
183
high dropout rates of participants
Weakness of the Longitudinal Developmental Design
184
data consist of numerical scores, and then the appropriate statistical analysis is a two- factor, mixed design analysis of variance (the pre–post factor is within-subjects and the group factor is between-subjects).
The Pretest–Posttest Nonequivalent Control Group Design
185
used to differentiate the groups of participants or the groups of scores being compared
Quasi-independent variable
186
The variable that is measured to obtain the scores within each group.
Dependent variable
187
independent variable in an experiment, especially those that include two or more independent variables.
factor
188
research design that includes two or more factors.
factorial design
189
notation system that identifies both the number of factors and the number of values or levels that exist for each factor
factorial design
190
mean differences among the levels of one factor
main effect
191
research study is represented as a matrix with one factor defining the rows and the second factor defining the columns
main effect
192
Occurs whenever two factors, acting together, produce mean differences that are not explained by the main effects of the two factors.
Interaction between factors (interaction)
193
exists between the factors when the effects of one factor depend on the different levels of a second factor.
interactions
194
When the results of a two-factor study are graphed, the existence of nonparallel lines (lines that cross or converge) is an indication of an interaction between the two factors.
interaction
195
data matrix, you must compare the mean in any individual row (or column) with the mean differences in other rows or columns.
identifying interactions
196
data are evaluated by a hypothesis test, be cautious about interpreting any results from a two-factor study.
Interpreting main effects and interactions
197
presence of an interaction can obscure or distort the main effects of either factor.
Interpreting main effects and interactions
198
Whenever a statistical analysis produces a significant interaction, you should take a close look at the data before giving any credibility to the main effects.
Interpreting main effects and interactions
199
Two-factor study allows researchers to evaluate
Independence of Main Effects and Interactions
200
They are best suited to situations in which a lot of participants are available, individual differences are relatively small, and order effects are likely
Between subject designs
201
A study in which there is a separate group of participants for each of the treatment conditions.
Between subject designs
202
A single group of individuals participates in all of the separate treatment conditions.
Within subject designs
203
They are best suited for situations in which individual differences are relatively large, and there is little reason to expect order effects to be large and disruptive
Within subject designs
204
factorial study that combines two different research designs.
mixed design
205
factorial study with one between-subjects factor and one within-subjects factor
mixed design
206
uses two different research strategies in the same factorial design.
Combined Strategies study
207
One factor is a true independent variable (experimental strategy)
experimental strategy
208
one factor is a quasi- independent variable
nonexperimental or quasi- experimental strategy
209
measured before and after receiving a treatment.
one group - treatment group
210
measured twice (pretest and posttest) but does not receive any treatment between the two measurements.
second group - control group
211
researcher has control over assignment of participants to groups and can create equivalent groups.
random assignment
212
basic concepts of a two-factor research design can be extended to more complex designs involving three or more factors
Higher order factorial designs
213
statistical evaluation of the results from a factorial study depends in part on whether the factors are between-subjects, within-subjects, or some mixture of between-subjects and within- subjects.
Statistical analysis of factorial designs
214
Conducts three separate hypothesis tests: one each to evaluate the two main effects and one to evaluate the interaction.
Two factor ANOVA
215
Usually conducted using a statistical computer program such as SPSS.
Two factor ANOVA
216
the correct choice is an independent- measures two-factor ANOVA
two between subject factors
217
must specify which it is and use a mixed-design two-factor ANOVA
one of the two factors is between subjects
218
use a repeated-measures two-factor ANOVA.
Two within-subjects factors
219
Factorial designs are developed when researchers plan studies that are intended to build on previous research results.
Expanding and replicating a previous study
220
current research tends to build on past research, factorial designs are fairly common and very useful.
Expanding and replicating a previous study
221
simple fact that differences between participants can result in large variance for the scores within a treatment condition. Large variance can make it difficult to establish any significant differences between treatment conditions.
Reducing variance in between subjects designs
222
tempting to eliminate or reduce the influence of the specific characteristic by holding it constant or by restricting its range.
Reducing variance in between subjects designs
223
order effects can alter and distort the true effects of a treatment condition, they are generally considered a confounding variable that should be eliminated from the study
Evaluating order effect in within subjects designs
224
It is possible to create a research design that actually measures the order effects and separates them from the rest of the data.
Evaluating order effect in within subjects designs