section 5-experimental design Flashcards
1
Q
experimental control
functional relations analysis
control
A
- when a PREDICTABLE CHANGE (DV) can be RELIABLY produced by SYSTEMATIC MANIPULATION of some aspect of the individual’s ENVIRONMENT (IV)
- analysis dimension of the 7 dimensions of ABA (BATCAGE)
Behavioral (observable & measurable)
Applied (socially significant Bx)
Technological (replicable)
Conceptually systematic (tie to basic principle)
Analytical (functional relationship/believability)
Generality (time/setting/behaviours)
Effective (practical)
2
Q
4 important elements of B
A
- individual
* group of ppl do NOT behave*
- a person’s interaction with the environment
- ABA experimental strategy is based on SINGLE-subject methods - continuous
- CHANGE over time
- requires continuous measurement over time - determined
- the occurrence of any event is determined by the FUNCTIONAL RELATIONS it holds to other events
- B is NATURAL phenomenon & subject to the same natural laws as other natural phenomena - extrinsic to the organism
- variability (change in B) is the result of the environment: IV, some uncontrolled aspect of the experiment, uncontrolled factors outside experiment (e.g. weather)
3
Q
what to do when seeing variability in data
A
- should attempt to manipulate factors suspected of causing the variability in the data to look for the causal factors
- seek treatment variable robust enough to overcome variability
4
Q
6 components of experiments in ABA
A
- at least 1 subject
- at least 1 B (DV)
- at least 1 setting
- at least 1 treatment (IV)
- a measurement system & ONGOING analysis of data
- 1 experimental design
5
Q
experimental question
A
- ALL well planned experiments begin with the experimental question
- brief & specific statement of what researchers want to learn from conducting the experiment
- in question / statement form
6
Q
- at least 1 subject
SINGLE-case designs
within-subject designs
intra-subject designs
A
- ABA uses SINGLE-subject design, does NOT use group comparison*
- the subject acts as one’s own CONTROL
- does NOT mean there’s only 1 subject, usually involves more than 1 subject (commonly 4-8)
- REPEATED measure of the subject’s B during each phase of the study –> provide the basis for comparing EXPERIMENTAL VARIABLES (IVs) –> present / withdraw the IV in subsequent conditions
- the individual is exposed to EACH condition SEVERAL times over the study
- each subject’s data are graphed SEPARATELY
7
Q
- at least 1 B (DV)
A
some studies measure more than 1 DVs:
- to provide data patterns that can serve as controls for EVALUATING & REPLICATING the effects of an IV
- assess if any COLLATERAL EFFECTS: when the IV affects Bs other than the targeted B
- to determine whether changes in the B of a person other than the subject occur during the experiment & if such changes can explain changes in the subject’s B
8
Q
- at least 1 setting
A
control 2 sets of environmental variables to demonstrate experimental control:
- IV (present, withdraw, varied values)
- extraneous variables: prevent UNPLANNED environmental variation
- when unplanned variations occur, you MUST try to wait them out/incorporate them into the design.
- REPEATED measures tell whether unplanned environmental changes are of concerns
9
Q
- at least 1 treatment (IV) / experimental varibale
A
- the particular aspect of the ENVIRONMENT that is MANIPULATED to find out whether it affects the subject’s B
10
Q
- a measurement system & ONGOING analysis of data
A
- conduct observation & recording in a standardized manner (every aspect of the measurement: define B, schedule of observations)
- detect changes in LEVEL, TREND, VARIABILITY
11
Q
- 1 experimental design
A
- particular arrangement of conditions in the study to have a meaningful comparison of effects of IV: present, absent, varied values
- change ONLY 1 variable/1 treatment package/1 behavioral package at a time
e. g. entire package: a token economy + praise + time-out - select & combine designs best fit the research
- NONparametric analysis: IV either present or absent
- parametric analysis: manipulated IV value to discover the DIFFERENTIAL effects of a range of values
12
Q
component analysis
A
- looks at the effect of each part of a treatment package/behavioral package
- determine the effective components, keep the effective components & get rid of ineffective parts
13
Q
steady state responding
stable state responding
A
- a pattern of RESPONDING that exhibits very little variation in its measured dimensional quantities over a period of time
- provides the basis for BASELINE LOGIC
14
Q
baseline logic
A
- 3 elements: prediction, verification, replication
- each element depends on an overall experimental approach called steady state strategy
15
Q
steady state strategy
A
REPEATED expose a given subject to a given CONDITION
–> try to eliminate the EXTRANEOUS influence on B & obtain a STABLE pattern of responding before introducing the NEXT condition
16
Q
function of baseline data
A
- serves as a control condition
- NOT imply the absence of INTERVENTION, can be the absence of a specific IV
17
Q
benefits of baseline data
A
- use the subject’s performance in the absence of the IV as an objective basis for detecting change
- obtain descriptions of ABC correlations for the planning of an effective treatment
- guide to set the INITIAL CRITERIA for R
- to see if the B targeted for change really warrants intervention
18
Q
4 patterns of baseline data
A
- descending
- ascending
- variable
- stable
19
Q
- descending baseline
A
- shows the B is already changing
- generally, one should NOT implement the IV when the baseline is descending
- if the descending baseline is due to a behavior you want to decrease, you should wait coz the B is already improving
- implement IV if you try to increase sth & the descending trend shows it’s worsening
20
Q
- ascending baseline
A
- shows the B is already changing
- generally, one should NOT implement the IV when the baseline is ascending
- if the ascending baseline is due to a behavior you want to increase, you should wait coz the B is already improving
- implement IV if you try to decrease sth & the ascending trend shows it’s worsening
21
Q
- variable baseline
A
- NO clear trend
- wait it out & do NOT introduce IV
- assumed to be due to environmental variables that are UNCONTROLLED
- if introduce IV now, will NOT be able to tell if it changed the B or not
- should try to control UNCONTROLLED sources of variability
22
Q
- stable baseline
A
- NO evidence of ascending / descending trend
- all DV values fall in a SMALL range
- BEST way to look at the effects of IV on DV
- can introduce IV NOW
23
Q
3 parts of baseline logic
A
in successive order:
- prediction
- verification
- replication
24
Q
baseline logic: prediction
A
- anticipate outcome of a presently unknown measurement
- data should be collected until STABILITY is CLEAR
- the more data points, the better predictive power
- are data stable enough to serve as the basis for experimental comparison?*
25
baseline logic: affirmation of the consequent
inductive logic:
- if IV not applied, the B won't change as indicated by baseline
- experimenters predicts IV will change B
- if IV is controlling DV (A), then data path with the presence of IV will show DV changes (B)
- when IV presents, the data should DV changes: B is true
- thus, the IV is controlling the DV: A is true
26
baseline logic: verification
- (reverse design) terminate/withdraw the treatment variable to verify a previously predicted level of baseline responding
27
baseline logic: replication
- replication is the essence of BELIEVABILITY
- shows RELIABILITY of behavior change: can make it happen again
- reintroduce the IV to achieve replication
28
5 main experimental design
MC RAW
1. multiple baseline
2. changing criterion
3. reversal
4. alternating treatments
5. withdrawal
29
5 main experimental design
1. multiple baseline
- most WIDELY used design
- highly flexible
- staggered implement the intervention in a step-wise fashion across B, SETTINGS, SUBJECTS
- NO withdrawal/reverse session
* *ethical**: when reverse conditions are unethical/impractical, when the B is irreversible, use multiple baseline design
- demo FUNCTIONAL RELATIONS: requires a change in B with the onset of the intervention:
apply IV to B1 when you confidently PREDICT that the B would remain the SAME in constant condition (stable responding in the baseline)
--> if B2 & B3 remain unchanged after applying IV to B1, this VERIFY the predication
--> if the IV changes B2 like it did to B1, the effect of the IV has been REPLICATED
--> the more replications, the more convincing the demonstration.
- most commonly 3-5 tiers
30
a. multiple-baseline across Bx
- 2 or more Bx of the SAME subject
- each subject serves as one's own CONTROL
- after steady STABLE BASELINE responding, the IV is applied to the 1st B while other Bx are kept in baseline
- when steady STABLE responding is reached for the 1st B, then the IV is applied to the next B
31
b. multiple-baseline across SETTINGS
- a SINGLE B is targeted in 2 or more different settings / conditions
- after steady STABLE BASELINE responding, the IV is applied to the 1st setting while other settings are kept in baseline
- when steady STABLE responding is reached for the 1st setting, then the IV is applied to the next setting
32
c. multiple-baseline across SUBJECTS
| * most widely used multiple baseline design*
- a SINGLE B is targeted for 2 or more subjects in the SAME setting
- after steady STABLE BASELINE responding, the IV is applied to the 1st subject while other subjects are kept in baseline
- when steady STABLE responding is reached for the 1st subject, then the IV is applied to the next subjects
33
2 weaker variations of multiple baseline design
1. multiple PROBE design
2. DELAYED multiple baseline design
- both variations weaker than traditional multiple baselines
- used when EXTENDED BASELINE measurement is UNnecessary, impractical, too costly, unavailable
34
1. multiple PROBE design
- analyze relation between the IV & acquisition of skill SEQUENCES
- probes provide the basis for determining if B change has occurred PRIOR to intervention
- NO continuously measured baseline
- B1 (probe 3 times): probe 3 times before implementing IV to B1-->no B change occurred
- B2 (probe 2 times): probe 1 time at the beginning of the experiment, probe 1 more time before implementing IV to B2
- B3 (probe 3 times): probe 1 time at the beginning of the experiment, probe 1 more time before implementing IV to B2, probe 1 more time before implementing IV to B3.
35
2. DELAYED multiple baseline design
- initial baseline & intervention begin & subsequent baseline are added in a delayed/staggered fashion
- effective when:
a. reversal design is impossible
b. limited resources preclude a full-scale design
c. when a new B/subject/setting becomes available
- limitations: shorter baselines do NOT show interdependent of DVs
36
guideline for multiple baseline design
1. select independent & functionally similar baselines
* Bx are FUNCTIONALLY independent of one another
* Bx share enough SIMILARITY --> they will change with the application of the SAME IV
* Bx should be DIFFERENT response classes (i.e. same function within 1 response class) --> INDEPENDENT
2. select concurrent & plausibly related multiple baselines
* Bx must be measured concurrently
* all relevant variables that influence 1 B must have the opportunity to influence other Bx
3. do NOT apply the IV to the next B too soon
4. vary significantly the LENGTH of multiple baselines
* the more baselines differ in length, the stronger the design
5. intervene on the MOST stable baseline first
37
advantages of multiple baseline design
- successful intervention do NOT have to be removed (no reverse)
- evaluates generalization
- easy to implement
38
disadvantages of multiple baseline design
- functional relationship is NOT DIRECTLY shown in this design (coz no reverse)
- effectiveness of the IV is demonstrated but not information regarding the FUNCTION of the target B
- IV may be delayed for certain B, settings, subjects
- takes resources to implement properly
39
5 main experimental design:
| 2. changing criterion design
- baseline phase is followed by a series of treatment phases consisting of SUCCESSIVE & GRADUALLY changing criteria for R / P
- only 1 B in the design
- B has to already in the subject's REPERTOIRE
- evaluates treatment is applied in a GRADUATED/STEP-WISE fashion
- technically, it's a variation of the multiple baseline design
e.g. use it to assess how a person's B changes when the researcher provides the person with R contingent upon 10 responses per minute, then 20 responses per minute, then 30 responses per minute, and so on
40
demonstrate functional relations in changing criterion design
- the criterion lines should have a LARGE SEPARATION to show a functional relationship
- experimental control: the EXTENT that the level of responding changes to conform 遵从 to each new criterion
- If data points do NOT fall around the criterion lines --> it shows there is very LITTLE experimental control
- the greater the VERTICAL distance between the criterion lines, the MORE experimental control
41
guideline for changing criterion design
1. length of phases
* each phase must be long enough to achieve STABLE responding
* target Bx that are SLOWER to change require longer phases
* VALIDITY of the design is INCREASED when you VARY the LENGTH OF each phase
2. magnitude of criterion changes
* the size of the changes between each criterion should VARY to prove strong functional relations
* changes in size must be LARGE enough to be detectable, but not so large as to be unachievable
* changes in size can be smaller if dealing with STABLE data
3. number of criterion changes
* the MORE criterion changes, the BETTER proof of experimental control
42
advantage of changing criterion design
- NOT require reversal of improved behavior
| - enable experimental analysis within the context of a gradually improving B
43
disadvantage of changing criterion design
- target B must already be in the person's repertoire
- NOT appropriate for analyzing the effects of a shaping program (shaping is used to develop novel B)
- NOT a comparison design
44
changing criterion design vs. shaping
shaping:
- a B changing strategy but NOT experimental design
- used to teach NOVEL Bx: reinforcing responses that meet a gradually changing criterion (successive approximations) towards the terminal B
- changing response criterion: TOPOGRAPHICAL in nature, require different FORMS of B at each new level
changing criterion design:
- is an experimental design that results in B change
- can NOT use with NOVEl B
- best for evaluating the effects of instructional techniques on step-wise changes in RATE, ACCURACY, DURATION, LATENCY of a single target B
45
5 main experimental design:
| 3. reversal design
- any experimental design that the researcher REVERSES responding to a level obtained in a PREVIOUS condition
- IV is withdrawn (e.g. ABAB) or reversed in its focus (e.g. DRI incompatible B/DRA alternative B)
- alternating between baseline & a particular intervention
- each reversal strengthens experimental control & functional relation: switch from 1 condition to the other with a corresponding change in trend & level
- for a reversal, B must approximate the initial baseline level
- at least 3 consecutive phases: ABA
- ABAB preferred over ABA as stronger design
- most powerful WITHIN-SUBJECT design for demon functions
**ethics**
if client is displaying severe & dangerous Bx (e.g. SID, elopement)
--> NOT spend time just taking baseline from the start
--> ethical responsibility to get in & immediately provide treatment for health & safety of the client
--> can use BAB reversal
46
demo functional relations in reversval designs
- prediction-verificaiton-replication
| - if repetition of baseline & treatment phases approximate the original phases --> IV is responsible for B change
47
5 variations of reversal design
1. repeated reversal
2. BAB reversal
3. multiple treatment design
4. NCR reversal technique
5. DRO/DRA/DRI reversal technique
48
5 variations of reversal design:
| 1. repeated reversal
- simple extension of ABAB, e.g. ABABABAB
- more reversal, stronger evidence of control
- redundancy may be concerns
49
5 variations of reversal design:
| 2. BAB reversal
(B) IV implemented --> (A) IV removed --> (B) IV reintroduced
- weaker than ABA design coz does not enable assessment of the effects of IV during baseline
- best design when client displays severe & dangerous B
- appropriate when IV is already in place & you have limited time
- disadvantage: SEQUENCE EFFECTS coz the level of B in condition A may be influenced by the IV before it
* sequence effects/alternation effects/carryover effects*
- effects on a subject's B in a given condition that are the result of the subject's experience with a PRIOR condition
50
5 variations of reversal design:
| 3. multiple treatment design
- compares 2 or more IVs to baseline and/or to one another
e. g. ABACABAC, ABCDACAD
- disadvantage: SEQUENCE EFFECTS
51
5 variations of reversal design:
| 4. non-contingent R (NCR) reversal design
- shows the effect of R by using NCR as a CONTROL condition instead of baseline with no R
- allows examining contingent R
- the reinforcer is presented in fixed/variable time schedule INDEPENDENT of the subject's B
e.g. baseline-->contingent R-->NCR-->contingent R-->NCR
52
5 variations of reversal design:
| 5. DRO/DRI/DRA reversal technique
- shows the effects of R by using DRO, DRI, DRA as CONTROL condition instead of baseline with no R
- allows examining contingent R
- DRO: R any B other than the target B
- DRI: R B that is physically INCOMPATIBLE with the target B
- DRA: R an alternative B other than the target B
53
advantage of reversal design
- clear demo of the existence / absence of a functional relations between IV & DV
- enable us to count the amount of B change
- return to baseline tells: we need to program for maintenance
54
disadvantage of reversal design
- irreversibility: the level of B observed in an earlier phase can NOT be reproduced even though experimental conditions are the same as the earlier phase
**ethics**
remove an effective IV can cause ethical, social, educational issues
55
5 main experimental design:
4. alternating treatments design
simultaneous treatments design
concurrent schedules design
multi-element (baseline) design
multiple schedules design
- 2 or more conditions are presented in RAPIDLY alternating succession INDEPENDENT of the level of responding & the differential effects on the target B
- compare 2 or more IVs to see which IV is best
- based on stimulus discrimination: each IV has an obvious SD signaling which IV is in effect at a given time
- data for each IV are plotted SEPARATELY on the SAME graph
- IVs maybe alternated across daily sessions/given in sessions occurring the same day/implement during each portion of the same session
56
demo functional relations in alternating treatments design
on graph:
- visual inspecting the differences between / among the data paths produced by each treatment
- functional relations demo when: 1 data path is CONSISTENTLY higher than the other & NO OVERLAPPING
- the degree of differential effects produced by 2 treatment is determined by the VERTICAL distance between the data paths
- prediction/replication/verification is NOT identified in separate phases.
- each successive data point plays all 3 roles
57
3 variations of alternating treatment design
1. single phase without baseline: no initial baseline
2. with baseline: with initial baseline
3. with baseline & final best treatment phase: most WIDELY used:
initial baseline --> alternating treatments (e.g. T1, T2, T1>T2) --> T1
58
3 problems avoided by alternating treatments design
1. irreversibility
2. sequence effects
3. unstable data
59
advantage of alternating treatments design
- NOT require treatment withdrawal
- speedy comparison
- min. irreversibility problem & sequence effects
- can be used with unstable data
- can be used to assess generalization of effects
- intervention can begin IMMEDIATELY without baseline data
60
disadvantage of alternating treatments design
- multiple treatment interference as multiple treatments are going on at the same time
- unnatural nature of rapidly alternating treatments
- limited capacity of the design: max. 4 conditions
61
5 main experimental design:
| 5. withdrawal design
- aka: ABAB design
- describe experiment that an effective treatment is sequentially / partially withdraw to promote the MAINTAINANENCE of B changes
62
ethical considerations in single-case experimental designs to demo treatment effectiveness
*1st goal is to CLEARLY show IV changes the target B & nothing else*
3 concerns:
1. baseline trends
- increase/decrease trends in baseline do NOT allow for clearly demo IV cuauses the change in B
- continue observe for longer time
- try to reverse the trend, e.g. DRO
- select designs that do NOT require stable baseline
- use statistical technique to take initial trend into account
2. excessive variability in data
- variability in data can OBSCURE intervention effects
- block consecutive data points, plot blocked AVERAGE rather than day-to-day performance
- search for causes of the variability / the situation, e.g. environmental stimuli
3. duration of phases
- duration of each phase can involve problems related to trend & variability
- no rigid rules abt how many data points need for each phase
- objective criteria: use this to decide when to shift phases, help to reduce subjectivity
- single subject designs best for treatment package evaluation
- the generality of the research results: use replication of IV across subjects to assess the generality
63
why ABA not use group approach to research
- group data not representative of individual performance
- group data masks variability
- absence of intrasubject replication
64
2 types of validity in experimental design
1. internal validity
| 2. external validity
65
1. internal validity
- the extent to which an experiment shows convincingly that changes in B are a function of the IV & not the result of uncontrolled/unknown variable
- an internally validity study involves 1 IV at a time --> avoid confounding
- high internal validity = designs show strong experimental control
66
4 confounding threats to internal validity
1. measurement confounds
- # & the intricacy/complexity of targeted Bx, e.g. target numerous complicated Bx
- observer drift: when observers unknowingly alter the way they apply a measurement system
- reactivity: B of clients changing when observed
* maintain baseline long enough to reduce reactivity
- observer bias/expectations:
* keep observer naive to expected outcomes of a study
2. IV confounds
- IVs are complicated & given tgt in a treatment package
- reduce: placebo control/double blind control
3. subject confounds
- maturation 发酵，成熟: changes in subject over course of study
- REPEATED measurement detects uncontrolled variables
4. setting confounds
- studies in natural settings are more prone to confounding variables
- should hold all possible aspects of the study CONSTANT until repeated measurements again reveal stable responding
- BOOTLEG R may occur in the natural environment: secretive R that is not part of your B plan
67
confounding vs. extraneous variables
- aka for uncontrolled influence on a research study
- should be reduced / eliminated as much as possible to demo experimental control
extraneous variable: any aspects of the ENVIRONMENT that must be held CONSTANT to prevent unplanned environmental variation
e.g. light, space, the temperature of the room
confounding variable: any uncontrolled factor known / suspected to EXERT influence on DV
68
external validity
generalizable to the external world
- the degree that a study's results are generalizable to other subjects, settings, Bx
- the degree that a functional relation discovered in a study will hold under different conditions
- external validity is range from a little to a lot
- REPLICATION establishes external validity
1. direct replication: exactly duplicated
intrasubject or intersubject
2. systematic replication: purposefully varies 1 or more aspects of an earlier experiment
- demo RELIABILITY & external validity: showing same effect occur under different conditions
- generally used in ABA
69
treatment integrity
procedure fidelity
fidelity of implementation
program integrity
- the extent that the IV is implemented/carried out as planned
- low treatment integrity: very difficult to interpret experiment results
- TREATMENT DRIFT: when the application of the IV in later phrases differs from the original application
* ensure high treatment integrity*
- precise operational definition of treatment procedures
- simplify, standardize, automate: simple treatment is more likely to be consistently delivered; simple & easy to implement techniques are more likely to be used & socially validated
- training & practice: detailed script, verbal instructions etc.
* assess treatment integrity*
- observation & calibration: ongoing retraining & practice to ensure high treatment integrity
- reduce, eliminate, identify the influence of potential confounding variables
70
2 types of errors in evaluation ABA research
1. type I error/false positive
- statistical analysis tends to lead to more type I error
2. type II error/false negative
- visual analysis in ABA tend to lead to more type II error
