Chapter 6: Quantitative Study Design - Experimental Flashcards
Research Details and Design
What are the three criteria that must be met for causal inference in general research?
- Association - the IV and DV must covary with one another or be associated in some way. A change in one will result in a change in the other.
- The cause must precede the effect.
- Nonspuriousness - the covariation between the two variables cannot be explained as the result of influence of some other variable.
What is selection bias?
This happens when the groups in your study are not comparable.
What can contribute to selection bias?
History - An event can occur in society that can increase awareness about the topic you are studying
Testing -repeat testing can improve test taking even if it doesn’t improve knowledge on a material
Maturation - the effect of time may produce short or long-term changes in the participants of your study. These changes can impact your results
Attrition - if participants drop out before the study completion, this may affect study outcome.
What are some threats to internal validity?
Statistical Regression - as a threat to internal validity has to do with “room for improvement” When scores for assessment are particularly high or low, there is a tendency for these scores to move toward the mean or average score
I.E. People who are mostly sedentary will likely show improvement in a walking intervention because they have such a low activity to begin with. If more sedentary people are in the intervention group, they will likely show greater improvement.
Instrumentation Changes - internal validity may be compromised if the instrument or measuring device changes over time.
Compensatory Rivalry -studies using more than one group. Participants aware of the group design may become competitive.
Compensatory Demoralization - people are assigned to the control group instead of the intervention group, they may feel resentful or disadvantaged.
What is an experimental study design?
Experimental study designs have features providing the highest level of control for threats to internal validity, while meeting the three criteria for inferring causality.
Participants are randomly assigned to either an experimental group or a control group. Each group is assessed, the intervention group receives treatment and then both groups are reassessed. The results of the outcomes of interest compared between the two groups.
What is solomon four-group study design?
This combines the classic pretest -posttest control group design with the posttest-only control group design.
All 4 groups are randomly assigned, but two get the intervention; one intervention group with pretest, and one intervention group without pretest. Two control groups, one control group with pretest and one control group without pretest.
If testing influence change the outcome, you would find differences when comparing the experimental groups with each other as well as comparing the results between the two control groups.
What is the pretest-posttest control group design?
The pretest-posttest control group design is the classical experimental study design, participants are randomly assigned to an intervention or control group. Baseline assessments are taken of both groups, and the intervention group receives treatment. Assessments are taken of both groups and results are compared to see differences between groups
What is the posttest-only control group design?
The posttest-only control group design eliminates the impact of testing effect. This design is similar to pretest-posttest control group design but without the pretest.
What are dismantling study designs?
Dismantling study designs look at the whole intervention, but also break down the intervention components. For interventions having multiple components, researchers may be interested in studying the effectiveness of each component separately, or different combinations of components.
What should you consider when deciding on which experimental design?
What is the purpose of my research?
Which design will provide results to answer my research question?
What threats to internal validity do I anticipate?
What lessons learned or recommendations can be found in the literature about my topic of study?
What study design is feasible and can be implemented with my available resources?
What are some key features of experiments? (Clinical Trials)
There is a manipulation of some variable while all other factors are maintained the same between groups.
You need at least one comparison group, not necessarily a control group depending on the ethics involved with the research question.
Clinical trials are not just for drug treatment and are often used in behavioral intervention studies and can also be applied to other settings. Ex: Testing which public health message makes a greater impact?
Random assignment which makes group statistically similar (computerized algorithm), matching can also e used, good - can make groups identical on key features; bad - difficult to carry out and requires a large sample pool
Time, this affects our internal validity, how much time will you need to conduct your study? Amount of time between time points, a well as the number of time points
Experiments must have the manipulation before the measurement of the dependent variable. The cause must precede the effect.
Debriefing may need to occur
Reactive measurement effects & self report - social desirability
Investigator effects
What are some unique data analytic techniques?
Equivalency Testing - test are two treatments the same
Superiority Trial - testing if one treatment is better
Inferiority Trial - testing if one treatment is worse
What are some of the pro’s about experiments?
Lab experiments are higher on internal validity and lower on external validity
Field experiments are lower on internal validity and higher on external validity
They can provide the strongest evidence for causation
In-person assessments can increase rapport
Flexible design with level of control or manipulation over the intervention
Experiments include a control/comparison/usual care group
Experiments often have standardized procedures (especially in the biological sciences)
What are some of the cons of experiments?
Not ethical or possible for some conditions of interest
Must have enough a priori information (previous work and theoretical background) to formulate good experimental designs
Can be very expensive to conduct.
Often difficult to maintain blinding (some patients want one treatment over another; preventing a clinician from knowing the patients treatment is difficult to carry out)
Can be more difficult to generalize to larger populations.
