Unit 2 Flashcards
1
Q
What meaning does the symbol “O” have in notation in research design?
A
a formal observation or measurement
2
Q
What meaning does the symbol “X” have in notation in research design?
A
Exposure to the Experimental Treatment
3
Q
What meaning does the symbol “EG” have in notation in research design?
A
Experimental Group: Test units exposed to a treatment
4
Q
What meaning does the symbol “CG” have in notation in research design?
A
Control Group: Test units not exposed to a treatment
5
Q
What meaning does the symbol “R” have in notation in research design?
A
Random assignment of test units to the EG and CG
6
Q
What meaning does the symbol “M” have in notation in research design?
A
Assignment to a EG or CG are balanced to assure that the EG and CG are roughly equal
7
Q
What is quasi-experimental design?
A
A research design that utilizes pre-existing groups in a natural context, often used in social research, education, and program evaluation
8
Q
How does quasi-experimental design compare to experimental design in terms of validity?
A
It has more external validity but less internal validity compared to experimental designs
9
Q
Why do quasi-experimental designs have lower internal validity?
A
they use pre-existing groups rather than random assignment
-> harder to control for confounding variables
10
Q
What does it mean if an experiment as more external validity than the experimental, but less internal?
A
- results are more generalizable to real-world settings
- may be more confoundings variables or threats to internal validity
- more difficult to establish causal relationships
11
Q
What is not part of quasi-experimental design?
A
CONtrola secondary systematic variance
-> no random assignment
12
Q
What logic does quasi-experimental research share?
A
the logic of experimental research:
to establish causal relationships, certain conditions must be met
13
Q
Which conditions must be met to establish causal relationships?
A
- the IV must precede the DV
- there must be covariation between both variables
- alternative explanations must be able to be ruled out
14
Q
What do we use Quasi-experimental designs for?
A
- the evaluation of psychological and or social interventions
- the evaluation of educational programs, health programs and social services, etc.
15
Q
When do we call an experiment quasi-experimental?
A
- our groups aren’t equivalent
- we don’t know the rules of assignment
- no randomization
- no random assignment
16
Q
What are the classifications of the pre-experimental design?
A
- one-shot case study
- one group pretest-posttest
- static group
17
Q
What 3 parts can we divide the quasi-experimental design to?
A
- with control group
- with nonequivalent CG, pretest/protest, cohorts, regression discontinuity - without control group
- repeated treatment, pre and post test treatment - Interrupted time series
- simple, with nonequivalent CG
18
Q
What is the main difference between Quasi-experimental design and Pre-experimental design?
A
Quasi-experimental: causal inference
Pre-experimental: no possibility of causal inference
19
Q
Why are pre-experimental designs despite their low internal validity relevant?
A
because they serve as the foundation for quasi-experimental designs
20
Q
What does the pre-experimental design typically involve?
A
a single group or one-shot observation
-> lacks random assignment, control groups and other features of rigorous experimental designs
21
Q
When do we often use pre-experimental designs?
A
when experimental controls are not feasible or ethical or when researchers are primarily interested in exploring relationships rather than establishing causality
22
Q
in simple words: what does the researcher do in a pre-experimental design?
A
only measures outcome variable before and after the intervention, without comparing it to a CG or accounting for other factors that could influence the result
23
Q
Pre-experimental design
What are the characteristics of a case study research design in a single session?
A
- only one dependent group or variable is investigated
- post-test
- X O
24
Q
Pre-expeirmental design
What are the characteristics of One-group Pre- and post-testing?
A
- measuring single group both before and after treatment or intervention
- assess any changes in the outcome
- O1 X O2
25
# Pre-experimental design
What are the characteristics of Comparison of static groups?
- two groups are compared, one receives treatment, other doesn't
- without any pretest measurements
- --- XO --- O
26
What are Group Control Designs in Quasi-experimental design?
1. Non-equivalent control group
- pretest-posttest
- cohorts
2. regression discontinuity
27
What are Without Control Group Designs in Quasi-experimental design?
1. pretest and posttest treatment
2. repeated treatment design
28
# Quasi-experimental design
What is Pretest-Posttest Design with Non-equialent Control Group?
Similar to the experimental design of two groups with pretest and posttest
-> assignment to groups is NOT random
-> most used in social sciences
29
What is the Notation in Pretest-Posttest Design with Non-equialent Control Group?
O1 X O2
---------
O1 O2
-> groups are already formed
-> attempt to make groups as similar as possible in some relevant variables (e.g. charachteristics, background, etc.)
30
What does Pretest-Posttest Design with Non-equialent Control Group consist of?
a treatment group and another control group that does not
-> both groups undergo pretest and posttest measurements
31
# Pretest-Posttest Design with Non-equialent Control Group
What influence does it have that the groups are already formed?
we cannot guarantee the initial equivalence that is achieved with randomization
32
# Pretest-Posttest Design with Non-equialent Control Group?
Who controls treatment assignment?
the experimenter
33
# Pretest-Posttest Design with Non-equialent Control Group?
What can the non-random assignment of participants to groups introduce?
bias and it can limit the internal validity of the study
-> selection by maturation: EG and CG come from different backgrounds, may result in differen maturation patterns
-> selection by history: external events or circumstances occuring during study may affect EG and CG differently
34
Which Data analysis do we use for Pretest-Posttest Design with Non-equialent Control Group?
The simplest: Analysis of Variance (ANOVA) of post-treatment scores
-> main drawback: differences existing in the pre-test are not taken into account
SO: Analysis of Covariance (ANCOVA)
35
What is ANOVA in Data analysis?
statistical method used to compare the means of two or more groups to determine if there are significant differences among them
-> tests wether the means of the groups are equal by analyzing the variance between groups and within groups
36
What is ANCOVA in Data analysis?
statistical technique that combines elements of both analysis of variance (ANOVA) and regression analysis
-> used to compare the means of two or more groups while statistically controlling for the effects of one or more covariates
37
What is Cohort?
a group of individuals who belong to some type of formal or informal institution who are subjected to the same circumstances over a period and who move from one level to another within these institutions
38
What type of study is a cohort study?
a type of longitudinal observatiional study
-> follows a group of individuals (cohort) over a period to examine how various factors or exposures influence the occurrence of outcomes or events of interest
39
What does a cohort study examines?
how a certain event affects a group, experimental cohort (e.g. education reform) and compares it with another group that did not experience that event (control cohort)
40
What is an advantage of the cohort design?
groups usually belong to institutions
-> it is easier to have information about participants characteristics
41
What is the primary objective of a cohort study?
to assess the association between exposure to certain factors and the development of specific outcomes over time
42
What role do participants play in a cohort study?
they are initially free of the outcome being studied and they are categorized based on exposure status or other characteristics
43
# Cohort design
What are the GPA scores of the experimental cohort compared to?
to those of the control cohort using statistical methods such as t-tests or ANOVA to determine if there are significant differences in academic performance between the two groups
44
# Cohort design
What is a mixed analysis of variance with one between-subjects variable and one within-subjects variable in data analysis?
repeated measures analysis of variance (SPSS)
45
What is the regression discontinuity design (RDD)?
a research design used to evaluate the impact of an intervention or treatment by exploiting naturally occurring thresholds of cutoff points in a continuous IV
46
What does the Regression Discontinuity Design (RDD) allow researchers to?
to estimate the causal effect of the treatment by comparing outcomes for individuals just above and just below the threshold
C = cutoff
C O1 X O2
----------
C O1 O2
47
What are some Threats of the Regression Discontinuity Design (RDD)?
it does not prevent the interaction between treatment and maturation
48
What are reasons for the Quasiexperimental design not having a control group?
- time limit
- resource limit
- ethical reasons
49
What difference does a quasiexperimental design without control group has to the within-group experimental design (a single group to which all conditions are applied)?
in quasi-experimental research the control techniques usedin within-group experimental designs are not employed
-> e.g. no counterbalancing
50
What are 3 consequences of the quasi-experimental design without control group?
- reduced causal inference capability
- lower internal validity
- different variants (removed treatment design with pretest and posttest, repeated treatment design)
51
What is a removed treatent design with pretest and posttest?
A quasi-experimental design where a treatment is introduced, maintained, and then removed to measure its effects
52
what does the researcher aims to simulate in the removed treatment design with pretest and posttest?
to simulate control group conditions during during the withdrawal phase to determine if the treatment had a lasting effect
53
What are Advantages and Disadvantages of the removed treatment design with pretest and posttest
- Advantages: ability to assess changes over time and evaluate the persistence of treatment effects
- Disadvantages: requires large sample sizes, reliable measurements and careful consideration of ethical issues
54
What is the notation of removed treatment design with pretest and posttest?
O1 x O2 O3 Xmitstrich O4
55
What is the repeated treatment design?
a research design where participants receive the same treatment multiple times over the course of the study
-> allows researchers to observe how participants respond to the treatment across multiple occasions
56
What is the Notation of repeated treatment designs?
O1 X O2 Xmitstrich O3 x O4
-> participants serve as their own controls
57
What are some Threats of the repeated treatment design?
1. History
2. Maturation
58
What are 4 data analysis in quasi-experimental design without control group?
- Analysis of variance
- Analysis of variance with blocking technique
- Analysis of variance with difference or gain scores
- Analysis of variance with standardized difference scores
59
What is the interrupted time series design (ITSD)?
"Time series" = several observations of a variable are taken over time
60
# Interrupted time series design (ITSD)
What is Level Change?
Change in the magnitude of posttest records in relation to pretest records
61
# Interrupted time series design (ITSD)
What is Tendency change?
implies change in level
-> change in the relationship between posttest records in relation to pretest records
-> change in the slope of the line
62
What are the two designs in Interrupted time series design (ITSD)?
1. Simple interrupted time series design
2. Interrupted time series design with non-equivalent control group
63
What is a simple interrupted time series design?
a research design used in various fields to evaluate the impact of an intervention or treatment by measuring outcomes over time
64
What are the characteristics of the simple interrupted time series design?
- multiple measurements of the DV are collected before and after the treatment
- requires only a single group
- similar to the pretest and posttest design but involves collecting several measurements before and after introducing the treatment
65
What is the Notation of Simple Interrupted Time Series Design?
O1 O2 O3 X O4 O5 O6
66
# Simple Interrupted Time Series Design
How can threats to internal validity be avoided?
with a good interrupted time series design:
- maturation
- cyclical changes
- statistical regression
- reactivity
67
# Simple Interrupted Time Series Design
What does cyclical changes mean?
spontaneous behavior may vary depending on the season of the year
68
# Simple Interrupted Time Series Design
What does Statistical regression mean?
by studying the trend in the pretest, it can be observed whether there is regression to the mean and prevent this effect from being confused with the treatment
69
which kind of data analysis do we use for Simple Interrupted Time Series Designs?
- Autoregressive Integrated Moving Average (ARIMA) model
- requires at least 10 measurements before and after treatment
70
What does the interrupted time-series design with nonequivalent groups involve?
taking a set of measurements at intervals over a period both before and after an intervention of interest in two or more nonequivalent groups
71
# Interrupted time series design with non-equivalent control group
What is the main challenge of this design?
difficult to find a proper control group
72
What is Interrupted time series design with non-equivalent control group similar to?
experimental validity
73
Why does Interrupted time series design with non-equivalent control group allows to control some threats?
thanks to the inclusion of a control group
74
What are threats to internal validity in Interrupted time series design with non-equivalent control group that are difficult to control?
Local history (selection x history)
75
# Overview
So: What is a quasi-experimental research study?
A study where groups aren't equivalent, there is no random assignment, and the rules of assignment are unknown. They are implemented when an ideal experimental design isn't possible
76
# Overview
What are the quasi-experimental research types?
- Quasi-experimental design with control group
- Quasi-experimental design without control group
- Interrupted time series designs
- Pre-experimental designs
77
# Overview
Is quasi-experimental research qualitative or quantitative?
Quantitative - it analyzes numerical data to determine effects
78
# Overview
What is an example of a longitudinal cohort study
Studying if a mindfulness meditation program in high school improves GPA compared to a control group. The experimental cohort would participate in the meditation program, while the control cohort would not. GPA would be measured for both groups
79
# Overview
What are the Pros and cons of quasi-experimental designs?
- Pros: More generalisable to real-world settings (higher external validity). Easier to get information about participants' characteristics because the groups usually belong to institutions.
- Cons: More confounding variables and threats to internal validity due to lack of random assignment or control over extraneous variables. It is more difficult to establish causal relationships
80
# Overview
What do "maturation" and "history" mean?
- maturation: Participants in the EG and the CG come from different backgrounds, which may result in different maturation patterns in both groups
- history: External events that occur during the study affect the experimental and control groups differently
