Research Design Flashcards Preview

Research Design, Stats, and Test Construction > Research Design > Flashcards

Flashcards in Research Design Deck (57):

True experimental design

at least one IV Is manipulated and subjects are randomly assigned


Quasi-experimental design

at least one IV is manipulated, but there is non-random assignment of subjects, typically because subjects are already in pre-existing groups


Observational, passive, or non-experimental

no intervention or manipulation


Between groups design

only compares groups that are independent


Within subjects design

groups contrasted are correlated or related (e.g. matched or repeatedly measured)



Because of possible carryover effects
Latin square is most sophisticated form of counterbalancing


Mixed design

involve groups that are both independent or correlated
e.g. between and within subjects design


Single subjects designs (five types)

one or very few subjects are studied intensively and each subject is measured many times
AB, ABAB, multiple baseline, simultaneous treatment, changing criterion



effect of measuring the same person repeatedly, which results in highly correlated data
significant problem associated with any single subject design


AB Design

single subjects design
baseline condition (A) followed by treatment condition (B)
most significant problem is the threat of history - hard to determine whether it was intervention that caused change or some other event


ABAB Design

single subjects design
baseline (A), treatment (B), baseline (A), treatment (B)
protects against threat of history
two problems: failure of DV to return to baseline and issue of ethics regarding removing effective treatment


multiple baseline design

single subjects design
treatment applied sequentially or consecutively across subjects, situations, or behaviors
problems: more time consuming and expensive


simultaneous (alternating) treatment design

single subjects design
involves two or more interventions implemented concurrently during the treatment phase
treatments are balanced and varied across time of day
allows for comparison of the relative effectiveness of two or more interventions for an individual


changing criterion design

single subject design
attempt made to change behavior in increments to match a changing criterion


time sampling

type of behavioral measurement that is useful when a behavior is not discrete, and thus has no distinct beginning and end
two types: momentary time sampling and whole-interval sampling


momentary time sampling

observer recording whether target behavior is present or absent at the moment the time interval ends


whole-interval sample

aka interval sampling or interval recording
involves scoring target behavior positively only if it is exhibited for a full duration of the time interval


event recording

involves tallying the number of times the target behavior occurred


analogue research

evaluates treatment under conditions that only resemble or approximate clinical situations
typically study less severe problems
treatments tend to be highly standardized
e.g. participants are college students and volunteers and therapists are graduate students
problem - limited generalizability


clinical trials

outcome investigations conducted in clinical settings


cross-sequential research

aka cohort-sequential research
takes several cross sections (e.g. of age groups) and follows them over briefer periods of time (e.g. 5-10 years instead of lifespan)


simple random sampling

every member of population has an equal chance of being randomly selected


stratified random sampling

population first divided into strata (e.g. age levels, income levels, ethnic groups), and then random sample of equal size from each stratum is selected


proportional sampling

individuals randomly selected in proportion to their representation in the general population


systematic sampling

involves selecting every kth element after a random start


cluster sampling

involves identifying naturally occurring groups of subjects (clusters) and randomly selecting certain clusters (e.g. classes or departments at a university); once cluster has been selected, all the subjects within the cluster are usually surveyed


internal validity

degree of validity of statements made about whether X causes Y


threats to internal validity (8)

history, maturation, testing or test practice, instrumentation, statistical regression, selection bias, attrition or experimental mortality, diffusion


threat to internal validity - history

specific incidents that intervene between measuring points, either in or outside of the experimental situation


threat to internal validity - maturation

factors that affect the subjects' performance because of the passing of time (fatigue, maturing)


threat to internal validity - testing or test practice

occurs when familiarity with testing affects scores on repeated testing


Solomon Four-Group Design

controls for testing threat to internal validity
divides subjects into four groups
Group 1: pre-test, intervention, post-test
Group 2: pre- and post-test
Group 3: intervention and post-test
Group 4: post-test only


threat to internal validity - instrumentation

refers to changes in observers of calibration of equipment


threat to internal validity - selection bias

caused by non-random assignment


threat to internal validity - attrition or experimental mortality

when there is a differential loss of subjects from the groups


threat to internal validity - diffusion

occurs when the no-treatment group actually gets some of the treatment


construct validity

refers to factors other than the desired specifics of the intervention that result in differences
these factors often lumped under threats to external validity


threats to construct validity (4)

attention and contact with clients
experimenter expectancies
demand characteristics
John Henry effect


threat to construct validity - experimenter expectancies

aka Rosenthal effect
refers to cues or clues transmitted to subjects by the experimenter
eg. smiling inadvertently nodding encouragingly only to those receiving treatment


threats to construct validity - demand characteristics

factors in the procedures that suggest how subject should behave
e.g. subjects given medication told it might cause side effects, those with placebo told there won't be side effects
to reduce demand characteristics, subjects should be blind to treatment condition


threats to construct validity - John Henry Effect

aka compensatory rivalry
occurs when persons in a control group try harder than usual in spirit of competition with experimental group
JH was legendary railroad steel driver who swung his hammer in competition with steam drill and died from overexertion
to control for JOhn HEnry effect, experimental and control groups should not know each other


external validity



threats to external validity (3)

sample characteristics
stimulus characteristics
contextual characteristics


threats to external validity- sample characteristics

refers to differences between sample and population


threats to external validity - stimulus characteristics

such as artificial research arrangements
features of the study with which the intervention is associated


threats to external validity - contextual characteristics

refer to conditions in which the intervention is imbedded
reactivity - when subjects behave in a certain way just because they are participating in research and being observed
Hawthorne effect - frequently described example of reactivity


threats to statistical conclusion validity (4)

low power
unreliability of measures
variability in procedures
subject heterogeneity


threats to statistical conclusion validity - low power

low power is the diminished ability to find significant results
contributing factors - sample size, inadequate interventions (e.g. tx only lasting two sessions)


threats to statistical conclusion validity - unreliability of measures

even if intervention is effective, significant differences may not be found if the outcome measure sured is unreliable


threats to statistical conclusion validity - variability in procedures

refers to inconsistency in treatment procedures, which can obscure significant treatment findings
especially a concern in psychotherapy outcome research


threats to statistical conclusion validity - subject heterogeneity

makes it more difficult to find significant differences between groups


Face validity

What test appears superficially to measure


Experimentwise Error Rate

Probability of making a Type I error



term used to describe single subject approaches



term used to describe group approaches


normative data

data that can be compared both within and across subjects


ipsative data

results from forced choice format
can only describe relative strengths or interests wihin a subject and cannot be used for comparisons across subjects