Section 5: Experimental Design Flashcards Preview

BCBA Exam Prep > Section 5: Experimental Design > Flashcards

Flashcards in Section 5: Experimental Design Deck (61):
1

Experimental Control 

(Functioanl Relations; Analysis; Control

  • When a predictable change in behavior [dependent variable-DV] can be reliably produced by the systematic manipulation of some aspect of the individual's environment [independent variable-IV]
  • The analysis dimension of the 7 dimensions of ABA 

2

4 Important Elements of Behavior

DICE

  • Behavior is DETERMINED
    • the occurrence of any event is determined by the functional relations it holds to other events
    • behavior is a natural phenonmenon and subject to the same natural lawas as other natural phenomena 
  • Behavior is INDIVIDUAL
    • ​behavior is defined as a person's interaction with the environment 
    • Groups of people do not behave 
    • Experimental strategy of ABA is based on single subject methods of analysis 
  • ​Behavior is CONTINUOUS 
    • behavior changes over time 
    • behavior requires continuous measurement over time 
  • Behavior variability is EXTRINSIC to the organism 
    • variability (change in behavior) is the result of environemnt:
      • the IV under investigation 
      • some uncotrolled aspect of the experiment (e.g. the behavior of another child in the classroom) 
      • uncontrolled factor outside of experiment (e.g. weather changes) 

3

When we see variability in our data, what should we do?

  • we should attempt to experimentally manipulate factors suspected of causing the variuability in the data to look for causal factors
  • seek treatment variables robust enough to overcome variability 

4

6 Components of Experiments in ABA 

MS. BEST

  1. a  measurement system and ongoing analysis of data 
  2. at least one subject
  3. at least one behavior (DV) 
  4. an experimental design 
  5. at least one setting 
  6. at least one treatment (IV) 

 

 

 

 

A image thumb
5

Experimental Question

  • a brief but specific statment of what the researcher wants to learn from conducting the experiment
  • ex. what are the effects of the IV on the DV for what population and in what setting?
  • ex. the purpose of the study was to see the effects of the IV on the DV. 

6

At Least One Subject (Single-Subject Designs) 

(Single-Case Designs; Within-Subject Designs; Intra-Subject Designs) 

(one of the 6 components of ABA experiments)

  • ABA uses single-subject designs but thats does not mean there's only 1 subject in the research 
    • it's called single subject because the subject as as their own control 
      • ​repeated measures of the subject's behavior during each phase of the study provide the basis for comparing experimental variables as they are presented or withdrawn in subsequent conditions (ie the presence and absence of the IV) 
      • the individual is exposed to each condition several times over the course of a study 
  • studies usually involve more than 1 subject-- usually 4 to 8 
  • Each subject's data is graphed seperately 
  • ABA does not use group comparison designs (traditionally used in psychology) because group designs mask individual progress 

7

At Least One Behavior 

(Dependent Variable)

(one of the 6 componenets of ABA experiments)

  • The behavior is the dependent variable. 
  • DEPENDENT VARIABLE is the variable in the experiment measured to determine if it changes as a result of manipulations of the independent variable. In ABA, the DV must be a socially signficant behavior. 
  • some studies measure more than 1 DV. Reasons:
    • provide data patterns that can serve as controls for evaluating and replicating the effects of an IV 
    • Assess if any COLLATERAL EFFECTS occured (a phenomenon in which the IV effects behaviors other than the targeted behavior) 
    • determine whether changes in the behavior of a person other than the subject occur during the course of an experiment and if such changes can explain changes in the subject's behavior 
      • Ex. you are implementing an intervention for your client and the behavior of his brother changes as well 

8

Collateral Effect

A phenomenon in which the IV effects behaviors other than the targeted behavior 

9

At Least One Setting

(one of the 6 components of ABA experiments)

  • Control 2 sets of enviornmental variables to demonstrate experimental control:
    • IV (present, withdraw, or vary its value) 
    • Extraneous Variables (prevent unplanned environmental variation) 
  • In labs, we can control enviornments better
  • When unplanned variations take place, you can wait them out or incorporate them into the design 
    • repeated measures of behavior tell us whether unplanned environmental changes are of concern 

10

At Least One Treatment

(Independent Variable; Intervention; Experimental Variable)

(one of the 6 components of ABA experiments)

  • The IV is the variable that is systematically manipulated by the researcher in an experiment to see whether changes in the independent variable produce reliable changes in the dependent variable. 
  • .... a particular aspect of the environment that the experimenter manipulates to find out whether it affects the subject's behavior 

 

11

A Measurement System and Ongoing Analysis of Data

(one of the 6 components of ABA experiments)

  • observation and recording procedures must be conducted in a standardized manner 
  • standardization involves every aspect of the measurement system (e.g. from the behavior definition to scheduling of observations) 
  • behaviorists must detect changes in level, trend, and variability 

12

Experimental Design 

  • the particular arrangement of conditions in a study so that meaningful comparisons of the effects of the presence, absence, or different values of the IV can be made 
  • Important Rules of Experimental design 
    • change only one variable at a time 
      • if you are examining a treatment package (behavioral package)ensure the entire package is presented or whithin drawn at the same time 
    • select and combine designs that best fit the research question 

13

Two Types of Experimental Designs

Para = Pyramid of Values 

Nonparametric Analysis- the IV is either present or absent during the study (ex. medication is either given or not given in a drug study) 

 

Parametric Analysis - The value of the IV is manipulated. This seeks to discover the differential effects of a range of values. (ex. various doses of medication are given throughout the study) 

 

14

Treatment Package

(Behavioral Package)

  • when multiple IVs are bundled together into one program (e.g. a token economy with praise and time out procedures) 

15

Component Analysis 

  • a process that looks at the effect of each part of a treatment package 
  • used to determine the effective components of a treatment package 

16

Steady State Responding 

(Stable State Responding) 

  • a pattern of responding that exhibits very little variation in its measured dimensional quantities over a period of time 
  • Provides the basis for baseline logic 

17

Baseline Logic 

  • refers to the experimental reasoning inherent in single subject experimental designs
  • 3 elements (PAVR)
    • Prediction 
    • Affirmation of the Consequent 
    • Verification 
    • Replication 
  • each of these elements depends on an overall experimental approach called steady state strategy 

18

Steady State Strategy 

repeated exposure of a given subject to a given condition while trying to eliminate extraneous influcences on behavior and obtaining a stable pattern of responding before introducing the next condition 

19

Function of Baseline Data

It serves as a control condition but does NOT imply the absence of intervention--it can be the absence of a specific IV. 

20

Benefits of Baseline Data 

  • to use the subject's performance in the absence of the IV as an objective basis for detecting change 
  • to obstain descriptions of ABC correlations for the planning of an effective treatment 
  • to guide us in setting the initial criteria for reinforcement 
  • to see if the bheavior targeted for change really warrants intervention 

21

4 Patterns of Baseline Data 

DAVS

  1. Descending Baseline - don't implement the IV unless its a behavior (functional skill that is being lost) that you need to increase 
  2. Ascending Baseline - don't implement the IV unless it's a behavior you need to decrease (challenging behavior) 
  3. Variable Baseline - don't implement the IV until you've controlled all of the environmental variables that may be causing the variability 
  4. Stable Baseline - implement the IV when all of the values of the DV small in a fall range of values 

22

Prediction 

(the first part of baseline logic) 

  • the anticipated outcome of a presently unknown measurement 
  • data should be collected until stability is clear--the more data the better the predictive power 
  • main question to ask yourself: are data stable enough to serve as the basis for experiental comparison? 

A image thumb
23

Affirmation of the Consequent 

PREDICT THEN AFFIRM YOUR PREDICTION

(the 2nd part of baseline logic) 

  • Inductive Logic 
    • If the IV were not applied, the behavior (as indicated by baseline data) would not change 
    • the experimenter predicts the IV will change the behavior 
    • if the IV is controlling the DV, the data path in the presence of the IV will show that the DV has changed 
    • when the IV is present, data show DV has changed 
    • thus the IV is controlling the DV

A image thumb
24

Verification 

VERIFY THE EFFECTS OF THE IV

(the third part of baseline logic) 

  • Verification of a previously predicted level of baseline responding by termination or withdrawal of the treatment variable 

A image thumb
25

Replication

 

(the fourth part of baseline logic) 

  • replication is the essence of believability
  • shows reliability of behavior change 
  • replication is accomplished by reintroducing the IV 

A image thumb
26

Ethical Issues regarding experimental designs and research

  • CHECK THE CODE

27

5 Main Experimental Designs 

MC RAW

  1. Multiple Baseline
  2. Changing Criterion 
  3. Reversal 
  4. Alternating Treatments 
  5. Withdrawal 

28

Multiple Baseline Design 

  • most widely used design 
  • highly flexible 
  • staggered implentation of the intervention in a step wise fashion across behaviors, settings, and subjects 
  • do NOT have to withdraw a treatment variable in the design 
  • ETHICS WARNING- when it is unethical or impractical to reverse conditions or when the behavior is irreversible, use this design instead of a reversal design 
  • 3 Main Types 
    • Multiple Baseline Across Behaviors 
    • Multiple Baseline Across Settings 
    • Multiple Baseline Across Subjects  
      • Two Weaker Variations- used when extended baseline measurement is unnecessary, impractical, too costly, or unabailable 
        • Multiple Probe Design 
        • Delayed Multiple Baseline Design 

A image thumb
29

Prediction, Verification, and Replication in the Multiple Baseline Design 

  • a functional relation requires a change in behavior with the onset of the intervention 
  • apply the IV to behavior 1 when you can confidiently predict that the behavior would remain the same in constant conditions 
  • if behaviors 2 and 3 remain unchanged after the application of the IV to behavior 1, this verifies the prediction 
  • if the IV changes behavior 2 like it did behavior 1, the effect of the IV has been replicated 
    • the more replications, the more convincing the demonstration 
    • most mutliple baseline designs have 3 to 5 tiers 

A image thumb
30

Multiple Baseline Across Behaviors 

  • two or more different behaviors of the SAME SUBJECT
  • each subject serves as their own control 
  • after steady state baseline responding, the IV is applied to the first behavior while other behaviors are kept in baseline 
  • when steady state responding is reached for the first behavior, then the IV is applied to the next behavior 

A image thumb
31

Multiple Baseline Across Settings

  • a single behavior is targeted in two or more settings or conditions 
  • after steady state baseline responding, the IV is applied to the first setting while the other settings are kept in baseline 
  • when steady state responding is reached for the first setting, then the IV is applied to the next setting 

A image thumb
32

Multiple Baseline Across Subjects 

  • one target behavior for two or more subjects in the same setting
  • after steady state baseline responding is reached, the IV is applied to the first subject, while other subjects are kept in baseline 
  • when steady state responding is reached for the first subject, then the IV is applied to the next subject 
  • most widely used multiple baseline design 

A image thumb
33

Multiple Probe Design 

  • analyzes relation between the IV and acquistion of skill sequences 
  • instead of stimultaneious baselines, probes provide the basis for determining if behavior change has occured prior to intervention 

A image thumb
34

Delayed Multiple Baseline Design 

  • initial baseline and intervention begin and subsequent baselines are added in a delayed or staggered fashion 
  • effective when 
    • reversal design is not possible 
    • limited resources preclude a full scale design 
    • when a new behavior, subject, or setting becomes available 
  • limitations- shorter baselines do not show interdependence of DVs 

A image thumb
35

Guidelines for Multiple Baseline Design 

  • Select independent, yet functionally similar baselines 
    • behaviors are functionally independent of one another 
    • behaviors share enough similarity that they will change with the application of the same IV 
    • behaviors should be of different response classes
  • select concurrent and plausibly related multiple baselines 
    • behaviors must be measured concurrently 
    • all relevant variables that influence one behavir must have the opportunity to influence other behaviors 
  • Do not apply the IV to the next behavior too soon 
  • Significantly vary the lengths of multiple baselines 
    • the more the baselines differ in length, the stronger the design 
  • Intervene on the most stable baseline first  

36

Advantages and Disadvantages of Multiple Baseline Design 

Advantages 

  • successful intervention does not have to be removed 
  • evaluates generalization 
  • easy to implement 

Disadvantages 

  • functional relation is not directly shown in this design (as compared to reversal) 
  • effectiveness of the IV is demonstrated but not information regarding the function of the target behavior 
  • IV may be delayed for certain behaviors, settings, or subjects 
  • takes resources to implement properly 

37

Changing Criterion Design 

  • Experimental design in which an initial baseline phase is followed by a series of treatment phases consisting of successive and gradually changing criteria for reinforcement or punishment 
  • only one behavior in this design 
  • behavior must be in the subjects repertoire already 
  • evaluates treatment that is applied in a graduated or step wise fashion -- can be changes in rate, accuracy, duration, or latency of a single target behavior 
  • technically, it is a variation of the multiple baseline design 

A image thumb
38

Prediction, Verification, and Replication in the Changing Criterion Design 

  • the criterion lines should have a large separation to show a functional relationship 
  • experimental control is evidenced by the extent that the level of responding changes to conform to each new criterion 
  • if data points do not fall around the criterion lines, that shows us that there is very little experimental control 
  • the greater the vertical distance between the criterion lines, the more experimental control 

39

Guidelines for Changing Criterion Designs 

  1. Length of Phases 
    • each phase must be long enough to achieve stable responding 
    • target behaviors that are slower to change require longer phases 
    • validity of the design is increased when you vary the length of each phase 
  2. Magnitude of Criterion Changes 
    • the size of the changes between each criterion should vary to prove strong functional relations 
    • changes in size must be large enough to be detectable but not so large as to be unachievable 
    • changes in size can be smaller if you are dealing with stable data 
  3. Number of Criterion Changes 
    • the more criterion changes the better proof of experimental control 

40

Advantages and Disadvantages of Changing Criterion Designs 

  • Advantages 
    • does not require reversal of improved behavior 
    • enables an experimental analysis within the context of a gradually improving behavior -- you don't have to wait for a stable baseline 
  • Disadvantages 
    • the target behavior must already be in the person's repertoire 
    • not a comparison design 
    • not appropriate for analyzing the effects of a shaping program-- WHY?
    • shaping is a behavior change strategy--not an experimental design 
    • shaping is used to teach novel behaviors 

41

Reversal Design 

(A-B-A; B-A-B Design)

  • any experimental design in which the researcher reverses responding to a level obtained in a previous condition 
  • the most powerful within subject design for demonstrating function 
  • each reversal strengthens experimental control 
  • eivdence of a functional relation is strengthened with each reversal switch from one condition to the other with a corresponding change in trend and level 
  • for a reversal to occur, the behavior must approximate the initial baseline levels 
  • Requires at least 3 consecutive phases 
    • Initial Baseline (A)
    • Intervention (B)
    • Return to Baseline (A)
  • 5 Reversal Designs 
    • Repeated Reversals 
    • BAB
      • ETHICS WARNING- if your client is displaying sever and dangerous behaviors, then do NOT spend time just taking baseline data. it is your ethic responsiblity to start treatment immediately. So, use the BAB design 
    • Multiple Treatment Design 
    • NCR Reversal Technique 
    • DRO/DRA/DRI reversal technique 

A image thumb
42

Prediction, Verification, and Replication in the Reversal Design 

  • the IV is responsible for behavior change if repetition of baseline and treatment phases approximate the original phases 

A image thumb
43

5 Variations of the Reversal Design 

  1. Repeated Reversal 
    • simple extension of A-B-A-B
    • the more reversals, the stronger your evidence of control, but redundancy may be a concern 
  2. BAB Reversal 
    • weaker than an A-B-A design because it does not enable assessment of th effects of the IV during baseline 
    • also subject to SEQUENCE EFFECTS because baseline may have been influenced by starting with the IV
    • best used when dealing with challenging behavior that needs immediate intervention or when an IV is already in place 
  3. Multiple Treatment Design 
    • compares two or more IVs to baseline and/or to one another 
    • ABACABAC
    • ABCDACAD
    • disadvantage- sequence effects 
  4. NCR Reversal Technique 
    • uses NCR as a control condition instead of a baseline condition in which no reinforcement is provided 
    • allows you to examine contingent reinforcement 
  5. DRO/DRI/DRA Reversal Technique 
    • uses DRO, DRA, or DRI as a control condition instead of a baseline condition in which no reinforcement is provided 
    • allows you to examine contingent reinforcement 

44

Sequence Effects 

(Carryover Effects; Alteration Effects) 

  • effects on a subject's behavior in a given condition that are the result of the subject's experience with a prior condition 
  • ex. if you are using an ABCBC design, you cannot anlayze the effects of C on its own because it is always preceded by B. If you want to analyze C, you need to add ACAC 

45

Advantages and Disadvantages of Reversal Design 

Advantages 

  • clear demonstration of the existence or absence of a functional relation between IV and DV
  • enables us to count the amount of behavior change 
  • return to baseline tells us we need to program for maintenance 

Disadvantages 

  • Irreversibility 
  • Sequence Effects
  • ETHICS WARNING- ethical issues as well as social and educational issues arise when you remove an effective IV

46

Irreversibility 

  • the level of behavior observed in an earlier phase cannot be reproduced even though experimental conditions are the same as they were during the earlier phase 
  • when irreversibility is a problem, use DRO/DRI/DRA conditions as control techniques or multple baseline designs 
  • ex. learning to ride a bike is irreversible 

47

Alternating Treatments Design 

(SCAMM: 

Simultaneous Treatments Design; Concurrent Schedules Design; Alternating Treatments Design; Mutli Element Baseline Design; Mutli Element Design; Multiple Schedules Design) 

  • an experimental design in which two or more conditions are presented in rapidly alternating succession independent of th elevel of responding and the differential effects on the target behavior are noted 
  • compares two or more iVS to one another to see which IV would be best to utilize with a client 
  • based on stimulus discrimination - each IV has an obvious SD which is in effect 
  • For each IV, data are plotted separately on the same graph 
  • IVs may be 
    • alternated across daily sessions 
    • given in sessions occuring the same day 
    • implemented during each portion of the same session 

A image thumb
48

Prediction, Verification, and Replication in the Alternating Treatments Design 

On Graphs

  • visual inspection of the differences between or among the data paths produced by each treatment 
  • functional relation is shown when
    • one data path is consistently higher than the other 
    • no overlapping data paths 
  • the degree of differential effects produced by two different treatments is determined by the vertical distance between the respective data paths 
  • each successive data point in treatment plays all three roles- prediction, replication, verification 

 

49

3 Variations of Alternating Treatments Design 

  1. Alternating Treatment Designs Single Phase Without Baseline 
    • does not require an initial baseline 
  2. Alternating Treatment Designs With Baseline
    • whenever possible, you should use a baseline 
  3. Alternating Treatment Designs With Baseline and Final Best Treatment Phase 
    • most widely used 

A image thumb
50

Advantages and Disadvantages of Alternating Treatments Design 

Advantages

  • three problems avoided:
    1. Irreversibility 
    2. Sequence Effects 
    3. Unstable Data 
  • does not require treatment withdrawal 
  • speedy comparison  
  • can be used to assess generalization effects 
  • intervention can begin immediately without baselin data 

Disadvantages 

  • multiple treatment interference- two or more IVs in effect at the same time 
  • unnatural nature of rapdily alternating treatments 
  • limited capacity--maximum comparison of 4 conditions 
  • selection of treatments- the effects should be significantly different from one another 

51

Withdrawal Design 

  • Withdrawal design is a synonym for ABAB or experiments when an effective treatment is sequentially or partially withdrawn to promote the maintenace of behavior changes 
  • some researchers use "withdrawal" to describe experiments based on ABAB analysis and reserve the term "reversal design" for studies in which the behavioral focus of the treatment variable is reversed (or switched to another behavior) as in the DRO/DRI/DRA reversal techniques 
    • those literally "reverse" treatments but doesn't actually completely "withdraw" all treatment 
  • it's most common in the literature to see "reversal design" encompassing "withdrawal designs" since both attempt to demonstrate behavioral reversibility 

52

How to Identify Practical and Ethical Considerations in Single Case Experimental Design to Demonstrate Treatment Effectiveness 

  1. Baseline Trends 
    • increasing or decreasing trends in your data during baseline do NOT allow you to clearly demonstrate that your IV causes the change in behavior 
      • Address by:
        • continue observations for a longer period of time 
        • try to reverse the trend (e.g. by using a DRO) 
        • select designs that do not require a stable baseline 
        • use statistical techniques that take initial trends into account 
  2. Excessive Variability in Data
    • Address by:
      • block consectuvie data points and plot blocked averages rather than day to day performance
      • search for the causes of variability 
  3. Duration of the Phases 
    • the duration of each phase in your design can involve problems related to rends and variability in the data 
    • no rigid rules for how many data points you need in each phase, but you want steady responding 
    • use objective criteria for deciding when to shift phases to reduce subjectvity 
  4. Other Concerns
    • the range of questions about intervention effects that can be addressed with these designs--they are best to evaluate treatment packages 
    • the generality of the research results 
      • is the finding generalized beyond the subject in the design?
      • to assess generality, use replication of your IV across subjects, etc. 

53

Why ABA has problems with traditional psychology research?

  • group data is not representative of individual performance 
  • group data masks variability 
    • hides variability that occurs within and between subjects 
    • statistical control is not a substitute for experimental control 
    • to control effects of any variable you must either hold it constant or manipulate it as an IV
  • absence of intrasubject replication 

54

Two Types of Validity in Experimental Designs 

Internal Validity 

External Validity 

A image thumb
55

Internal Validity 

IV (internal validity) = IV (independent variable)

  • the extent to which an experiment shows convincingly that changes in behavior are a function of the IV and not the result of uncontrolled or unknown variables 
  • an internally valid study involves only one IV at a time. Multiple IVs are not confounded (presented at the same time) 
  • High Internal Validity = Designs showing strong experimental control 

56

4 Confounding Threats to Internal Validity 

MISS

  1. Measurement Confounds- refers to the number and intricacy of the behaviors you are targeting. If you are targeting too many complicated behaviors, your internal validity may be affected
    1. Observer Drift- when observers unknownly alter the way they apply a measurement system 
    2. Reactivity- this can refer to the bheavior of clients changing when they are being observed (to reduce, run a longer baseline) 
    3. Observer bias/expectations- the observer's expectations that changes follow a certain direction (to reduce, have naive observers) 
  2. IV Confounds- when IVs are complicated and implemented together in a treatment package 
    • ex. giving money as a reinforcer is also giving the person attention so you can't be sure if the maintaining reinforcer is the actual money or the attention 
    • to reduce- use a placebo control or double blind control procedure in which the subject is not aware if the IV is present or not 
  3. Subject Confounds 
    • Maturation - changes in subjects over the course of the study 
    • to reduce- repeat measurements 
  4. Setting Confounds 
    • studies in natural settings will have more confounds than in labs 
    • hold all possible aspects of the study constant until repeated measurements again reveal stable responding 
    • Bootleg Reinforcement (secretive reinforcement that is not part of your behavior plan) may also occur in the natural enviornment 

57

Confounding Variables 

(Extraneous variables; Unrelated Variables)

  • generally confounding and extraneous variables are AKAs that refer to variables that exert an uncontrolled influence on a study 
    • Extraneous = an aspect of the ENVIRONMENT that must be held constant 
      • ex. lighting, space, temperature in the room 
    • Confounding = any uncontrolled factor known or suspected to exert influence on the dependent variable 

58

External Validity 

External Validity = Generalizable to the External World

  • the degree to which a study's results are generalizable to other subjects, settings, and/or behaviors
  • the degree to which a functional relation discovered in a study will hold under different conditions 
  • replication establishes external validity 
    • Direct Replication 
      • exactly duplicate the study 
      • Intrasubject direct replication - same subject is used
      • Intersubject direct replication- different subjects used 
    • Systematic Replication 
      • researcher purposefully varies one or more aspects of an earlier experiment 
      • deomonstrates reliability and external validity by showing the same effect can occur under different conditions 
      • ABA uses systematic replication 

59

Treatment Integrity 

(Procedural Fidelity; Fidelity of Implementation; Program Integrity) 

  • extent to which the IV is implmented or carried out as planned 
  • low treatment integrity - very difficult to interpret experimental results 
  • treatment drift- when application of the IV in later phases differs from the original application 

60

How to Assess and Ensure a High Level of Treatment Integrity 

  • Ensure high level of treatment integrity by:
    • use precise operational defintion of treatment procedures 
    • simplify, standardize, and automate treatments--more likely to be consistently delivered and also socially validated 
    • training and practice for individuals who will conduct the experimental sessions (e.g. detailed scripts and instructions) 
  • Assessing:
    • collect treatment integrity data to measure how the actual implementation of the conditions matches the written methods 
    • observation and calibration give the researcher the ongoing ability to use retraining and practice to ensure high treatment integrity 
    • reduce, eliminate, or identify the influence of potential confounding variables 

A image thumb
61

2 Types of Errors in Evaluating ABA Research 

  • Type I Error (False Positive)
    • assuming the IV affected the DV when it did NOT actually do so 
    • Statistical Analysis tends to lead to more Type 1 errors 
  • Type 2 Error (False Negative) 
    • assuming the IV did NOT affect the DV when it actually DID
    • Visual analysis in ABA tends to lead to more Type II errors