6: Foundations of Design

Question 1

Non-experimental and experimental designs are classified as what two things each?

Answer 1

Non-experimental: descriptive; correlational.

Experimental: non-randomised (quasi-experimental); randomised.

Question 2

Other term for base rate, regarding disorders?

Answer 2

Prevalence.

Question 3

Studies that look at relationships between variables, but do not experimentally manipulate anything, are what?

Answer 3

Correlational designs.

Question 4

What two correlational designs can you take?

Answer 4

Cross-Sectional: all observations made only once at a single time.

Longitudinal: measurements made at 2 or more time points.

Question 5

What four elements do you need to infer causality from correlational research? Define each.

Answer 5

Covariation: two variables must occur together.

Precedence: hypothesized causal variable must reliably precede the outcome variable.

Exclusion of alternative explanations: other explanations for observed covariation must be reasonably excluded.

Logical mechanism: must be plausible account (i.e., THEORY) for the hypothesized causal relation.

Question 6

What are two cautions that must be taken when interpreting correlational research?

Answer 6

Bidirectionality: two-way causality.

Spurious association: not genuine; 3rd variable problem.

Question 7

A cross-sectional correlation coefficient CANNOT determine _____, but can partially deal with the _____ if control variables are used.

Answer 7

Directionality; 3rd variable problem.

Question 8

What are two primary 3rd variable problems?

Answer 8

Mediation and moderation.

Question 9

What are two quasi-experimental designs?

Answer 9

One-group posttest-only design.

One group pretest-posttest design.

Question 10

What are two experimental designs? Under what conditions are they quasi-experimental?

Answer 10

Posttest only; pretest-posttest.

Quasi-experimental if groups are not randomized.

Question 11

Both experimental designs have at least two what?

Answer 11

Groups.

Question 12

Which trial is the gold standard in clinical psychology?

Answer 12

Randomised Controlled/Clinical Trial.

Question 13

What are the five types of control group?

Answer 13

No-treatment controls.

Wait-list controls: delay in treatment.

Placebo controls: credible but inert treatment.

Comparative treatment groups: alternative treatment which is also effective.

Dismantling studies: break treatment into components, use each component in isolation as a group.

Question 14

List eight good experimental design features.

Answer 14

Patient homogeneity.

Randomised assignment.

Specific interventions.

Appropriate control groups.

Groups treated equivalently except for intervention.

Low attrition.

Patients, clinicians and raters blind.

Independent replication.

Question 15

Define external and internal validity.

Answer 15

External: to what extent can study results be generalized to other samples with different characteristics than the study sample?

Internal: degree to which causality can be inferred from a study.

Question 16

Internal validity measures the extent to which what factors and not others account for study results?

Answer 16

Intervention, manipulation and/or experimental procedures.

Question 17

List three threats to internal validity.

Answer 17

Spontaneous remission: recovery from disorder without any apparent external reason or intervention.

Interfering events (individual) and Secular Drift (long-term social changes): should be applicable to most of sample.

Maturational trends: relevant to treatment outcome studies and children.

Question 18

What is regression to mean? When is it problematic?

Answer 18

Tendency for extreme scores to revert, or regress, toward mean of distribution when measurement is re-administered.

When groups are initially selected on the basis of extreme scores.

Question 19

What is attrition? When is it a problem for internal validity?

Answer 19

Loss of participants over time.

Different rates of attrition between conditions; different reasons for attrition between conditions.

Question 20

What two disorders are associated with the highest rates of attrition?

Answer 20

Depressive disorder.

Substance use disorder.

Question 21

Some participants in a ‘control’ group may receive aspects of the intervention provided to the ‘experimental’ condition. What is this called?

Answer 21

Diffusion/imitation of treatment.

Question 22

How is special treatment/reaction of controls a potential threat to internal validity?

Answer 22

‘Control’ subjects may be aware that they are not receiving treatment and be motivated to ‘out-perform’ experimental group.

May feel demoralized or lose interest/motivation in participation if no active treatment is given.

Question 23

Relating to internal validity - if one doesn’t attend all therapy sessions or doesn’t take medication as prescribed, they have what?

Answer 23

Poor adherence to treatment protocol.

Question 24

Larco et al. (2002) found that adherence to taking anti-psychotic medication among people with schizophrenia was around 50%. What three elements worsened adherence?

Answer 24

Poor insight, negative feelings towards medication, poor therapeutic alliance.

Question 25

What are three threats to construct validity of experimental designs?

Answer 25

Confounded manipulation: sometimes accidentally manipulate more than one thing at a time.

Expectancy effects: sometimes people get better simply by thinking the therapy will work.

Hawthorne effect: sometimes people appear to get better simply because they’re being observed.

Question 26

Define statistical conclusion validity.

Answer 26

Extent to which the analyses performed enables one to draw correct inferences about the phenomena of interest.

Question 27

List five threats to statistical conclusion validity.

Answer 27

Low power: small N, small effect size.

Improper data analysis: e.g., analyzing everything and reporting the “best” results.

Confounding variables.

Poor reliability of measures.

Poor validity of measures.

Question 28

In statistical analyses, as reliability decreases, _____ increases.

Answer 28

Type II error.

Question 29

The Dead Salmon Study brought light to what problem?

Answer 29

Multiple comparisons problem: if you do many different statistical tests, some of them will give interesting results by chance.

Question 30

The most prestigious journals favor what two things?

Answer 30

Novel results.

Statistically significant results (p < .05). Null findings suppressed, leading to “file drawer effect.”