Unit 3 Flashcards
(124 cards)
1
Q
what is validity
A
degree to which the study establishes relationships it claims to establish
Trust
2
Q
By determining quality of the study design you assess validity of study. why
A
because you trust results of studies with higher quality study design than studies with lower quality design
3
Q
We tend to trust the findings from a study if:
A
the study design is of high-quality & is valid
4
Q
The degree to which the study establishes the relationship that it claims to establish is more formally referred to as:
A
study design validity
5
Q
The three primary types of validity that make up the overall validity of a study design include:
A
statistical validity
internal validity
external validity
6
Q
what is statistical validity
A
concerned w/ whether appropriate statistical procedures were used to determine sample size & analyze data
7
Q
Using Inappropriate Statistical procedures for data analysis leads to
A
Invalid conclusions about data & relationship bw IV & DV
8
Q
In order to use a specific statistical test for analysis, the study design and data must meet very specific requirements and assumptions.
A
t
9
Q
The misuse of a statistical test can misrepresent the relationship between the independent variable and dependent variables, leading to poor statistical validity.
A
t
10
Q
what is internal validity
A
degree to which the change in the DV is caused by the manipulation of the IV
whether the IV and only the IV caused a differential change in the DV across groups
11
Q
One group pre-test post-test design
A
follow 1 group of PTs over 2 time points
12
Q
what are two threats to IV
A
maturation & hx effects
13
Q
Potential explanations for change in DV
A
threats to IV
14
Q
waht are maturation effects
A
natural changes over time in DV regardless of intervention
15
Q
what are hx effects
A
confounding factors (extraneous variables) that are introduced between pre and post-test measurements
16
Q
loss of participants during the study
A
attrition threats
17
Q
when participants are tested multiple times & leads to unclear results if the change is due to an intervention or them being tested so many times
A
testing threats
18
Q
changes in the measurement tools or procedures influence the outcome and/or measurement of DV
A
instrumentation threats
19
Q
statistical phenomenon where is something is measured more the once the scores on subsequent measurements tend to be closer to the mean than initial measurement
If participants are selected for a study based on extreme scores (e.g., very low or very high), their scores on subsequent measurements might move closer to the mean not because of an intervention, but simply due to this statistical phenomenon.
A
regression to the mean threats
20
Q
systemic differences between or within groups at the ouset of the study
Occurs when participants are not randomly assigned to groups
Ex: one group is older, healthier and more motivated than the other
Differences could be due to initial differences in the groups rather than intervention
A
selection threats
21
Q
Refers to natural change over time of a variable
A
maturation
22
Q
maturation
A
Refers to natural change over time of a variable
23
Q
Confounding factors that are introduced to the study between measurement time points
A
hx effects
24
Q
hx effects
A
Confounding factors that are introduced to the study between measurement time points
25
Term used to describe alternative explanations for changes in the DV
threats to IV
26
Change in the DV is due to the manipulation of the IV
internal validity
27
describe internal validity
Change in the DV is due to the manipulation of the IV
28
what is external validity
degree to which the results of study can be generalized to the larger target population
Concerned w/ usefulness of the results in the real clinical world
Can we replicate study findings in our own routine practice
29
what are the threats to external validity
do the participants look like PTs we care for? And does the study setting look like the PT care setting?
30
what are efficacy studies
studies occurring in well-controlled environments with optimal treatment conditions
31
studies occurring in well-controlled environments with optimal treatment conditions
efficacy studies
32
studies occurring during routine patient care and under typical treatment conditions
effectivenesss studies
33
what are effectiveness studies
studies occurring during routine patient care and under typical treatment conditions
34
When assessing the external validity of a study, you must consider the interaction between treatment and selection. This threat is most concerned about:
the study sample
35
In general, as you increase internal validity, external validity will:
decrease
36
When critically reviewing a study determine first
the research design that was used & extraneous variables
37
what are extraneous variables
factors not directly related to the purpose of the study but affect results of the study (DV)
often confounding and bias
38
factors not directly related to the purpose of the study but affect results of the study (DV)
often confounding and bias
extraneous variables
39
what are the two types of extraneous variables
instrinsic & extrinsic factors
40
what are intrinsic factors
variables representing personal characteristics or attributes of the participant (maturation, healing, aging)
41
variables representing personal characteristics or attributes of the participant (maturation, healing, aging)
intrinsic factors
42
what are extrinsic factors
variables from the environment and situation (changes in rules & regs and societal attitudes)
43
variables from the environment and situation (changes in rules & regs and societal attitudes)
extrinsic factors
44
Actively do these when reviewing articles
Identify the extraneous variables that can impact results & determine if they have appropriately controlled/accounted for extraneous variables
45
These provide best demonstration of cause and effect relationships bw IV & DVs
experimental designs
46
types of experimental designs
Randomized controlled trials - aka true experimental designs
Quasi-experimental designs
Single-subject designs (or n-of-1 designs)
47
Factors associated with personal characteristics or attributes of a participant
intrinsic factors
48
Factors not directly related ot the purpose of the study that may impact study results
extraneous variables
49
Factors that exist in the environment or situation
extrinsic factors
50
Extraneous factors are often the source of
bias
confounding
51
Different research designs possess characteristics that will control for or introduce extraneous factors into the study
true
52
In general, experimental designs provide the best opportunity to demonstrate a ________ relationship between the independent and dependent variables.
cause and effect
53
Experimental study designs include which of the following?
n-of-1 designs
Quasi-experimental designs
Randomized controlled trials
54
factors associated with the environment or situation
extrinsic factors
55
factors associated with personal characteristics of the participant (
intrinsic factors
56
True experimental design =
randomized controlled trial (RCT)
57
the gold standard design
randomized controlled trial (RCT)
58
what is the point of the gold standard
protects against threats to internal validity and controls extraneous variables
59
hallmarkk features of RCT
Control group - comparison
Random assignment
60
What is a control group
comparison group that doesn’t receive intervention of interest (active intervention)
61
Single most effective way accounting for threats to internal validity (maturation & hx effects)
control group
62
Types of RCT designs
Parallel arm design
Crossover design
Randomized block trial
Randomized block, mixed design
63
most common, participants are randomized into different levels of the IV
parallel arm design
64
parallel arm design
most common, participants are randomized into different levels of the IV
65
between group design
parallel arm design
66
participants in one group are compared to those in another group
between-group design
67
assign participants to a “block” first and then randomly assign into treatment groups
randomized block trial
68
what are randomized block trial
assign participants to a “block” first and then randomly assign into treatment groups
69
participants are randomly assigned into a sequence and each sequence includes both treatments but in a different order
cross over design
70
what is a crossover design
participants are randomly assigned into a sequence and each sequence includes both treatments but in a different order
71
what is a washout period
time between treatment protocols to allow the effects of the first treatment to diminish before starting the second treatment
72
combo of random block & crossover; participants are blocked upfront before being randomized into a sequence of treatments
randomized block mixed design
73
what is randomized block mixed design
combo of random block & crossover; participants are blocked upfront before being randomized into a sequence of treatments
74
when should you use randomized block mixed design
Used when potential of creating an imbalance between groups is based on a prognostic factor like sex or there is an potential of order effect based on sequencing of treatments
75
includes both between and within subject comparisons
mixed design
76
parallel arm
participants are randomly assigned to one of two or more groups, with each group receiving a different treatment (or a placebo/control)
helps to directly compare the effects of different interventions
77
block design
ensures that each treatment group has an equal number of participants by assigning them in "blocks" or small groups, which helps maintain balance, particularly in smaller trials
78
cross over
each participant receiving multiple treatments in a sequential manner, with a washout period in between to eliminate carry-over effects of the previous treatment
resource-efficient since each participant serves as their own control, but it's not suitable for treatments with long-lasting effects or conditions that could change significantly over time of their own accord
79
mixed design
incorporate elements from both crossover and parallel-arm designs, some participants undergoing a crossover design while others are in parallel groups
hybrid approach can offer a balance between the efficiency of crossover designs and the straightforward comparisons possible in parallel-arm designs
80
The MOST effective way to control for extraneous variables within a research study is to include:
control group
81
Which of the following are benefits is associated with random assignment? (check all that apply)
Will balance potentially prognostic indicators across groups
Allocates participants into study groups in a unbiased manner
Will create unequal groups
Helps to optimize internal validity
Helps to optimize internal validity
Will balance potentially prognostic indicators across groups
Allocates participants into study groups in a unbiased manner
82
In intervention studies, it is best practice to use which of the following for your control group?
Placebo
Standard of care
No treatment
Rest
standard of care
83
The RCT design most associated with a washout period is called a
crossover design
84
what are quasi-experimental studies
those that are missing a control group and/or random assignments
85
examples of quasi studies
Non-equivalent pretest-postest design
Single-group designs
86
Non-equivalent pretest-postest design
multiple group design including a control/comparison group
Does NOT have random assignment like in RCT
Used due to practical and or ethical coniderations that offers valuable information
Posess limitations & biases
87
one group designs
no control group and includes only one group
No random assignment but it measures at multiple time points
Valuable in assessing change over time controlling for temporal effects & provide richer context for interpreting intervention’s impact
88
what is the IV in single-group designs
time - pre and post
89
what are threats to IV in one group designs
Maturation effects
hx effects
90
how do you control for maturation effects in one group designs
through adding measurements at multiple timepoints before and/or after intervention
91
3 types of one group designs
one group pretest-posttest design
One-way repeated measure design
Time series designs
92
One group pre-test post-test design
follow 1 group of PTs over 2 time points
93
pre-intervention & post-intervention with one group receiving the same treatment
One group pre-test post-test design
94
focuses more on how effects manifest over time & includes measurements at multiple time points AFTER intervention of the DV
One-Way Repeated measures design
95
includes measurements at multiple time points BEFORE and AFTER intervention
Time series design
96
A quasi-experimental design is missing which of the following features? (check all that apply)
An independent variable
Random assignment
A control group
A target population
control & random assignment
97
Which of the following quasi-experimental study designs generally provides the highest level of internal validity?
One group pretest-posttest design
Two group pretest-posttest design
One-way repeated measures design
Time series
Two group pretest-posttest design
98
consists of one measurement before and after intervention
one group pre post design
99
Consists of multiple measurements before and after intervention
time series
100
Consists of one measurement before and multiple measurements after
One way repeated measures design
101
A quasi-experimental design is missing which of the following features? (check all that apply)
An independent variable
Random assignment
A control group
A target population
Random assignment
A control group
102
Which of the following quasi-experimental study designs generally provides the highest level of internal validity?
One group pretest-posttest design
Two group pretest-posttest design
One-way repeated measures design
Time series
Two group pretest-posttest design
103
Consists of one measurement before and after intervention
One group pretest postest design
104
Consists of multiple measurements before and after intervention
Time series
105
Consists of one measurement before and multiple measurements after
One way repeated measures design
106
Group design limitations
structure/requirements might not be practical or possible
Limited measurement points
Group means/averages can mask individual differences
107
single-subject design
study of 1 PT under controlled conditions
108
benefits of single subhect design
More feasible, weak external validity (cannot generalize from one subject), high internal validity,
109
what is iV & DV in single subect design
IV = time, DV = target behavior
110
a phase
no intervention/baseline
helps to understand natrual change of DV over time
111
what has to happen before b phase starts
Establish stability of variable (then implement B)
112
b phase
intervetnion given
113
this increases in single subject as phases are added
Internal validity
114
Ways to increase internal validity in signle subject designs
Phases to be equal (measurements)
Extending # of measures in each phase until stability is established
Adding second A and B phases to see changes to DV when intervention is taken away and implemented
115
When developing a single-subject design, the phases: (check all that apply)
Should be relatively equal in terms of length and the number of measurement points
Can consist of as little as two measurement points
Should be extended out until the stability of the variable is established
Should be extended out until the stability of the variable is established
Should be relatively equal in terms of length and the number of measurement points
116
Which of the following is true about single-subject designs?
External validity will always be low
External validity will always be high
Internal validity will always be low
Internal validity will always be high
External validity will always be low
117
Rank order the following experimental designs by their internal validity, starting with the study that generally offers the highest degree of internal validity.
AB single design
time series
crossover
one group pretest posttest design
1 - crossover design
2 - time series design
3 - one group-pretest post-test design
4 - AB Single subject design
118
based on RCT what design features would you expect to be included in this investigation?
A control (or comparison) group
Random assignment of participants into study groups
119
RCT design is primarily associated with a high degree of:
internal validity
120
As a reader/reviewer of a journal article, you would expect the exclusion criteria to identify the major factors or characteristics that would:
Likely confound the study results
Distinguish between the study groups
Describe the patient population
Likely confound the study results
121
As a reader/reviewer of a journal article, how would you determine if the authors have operationalized the independent variable sufficiently?
There are enough details to support that the groups are similar as the start of the study.
There are enough details for a clinician or researcher to replicate the treatment procedures.
There are enough details for the reader to understand there is a clear difference between the levels of the independent variable.
There are enough details to identify the tools used to measure the study outcomes.
There are enough details for a clinician or researcher to replicate the treatment procedures.
There are enough details for the reader to understand there is a clear difference between the levels of the independent variable.
122
The inclusion of a control group primarily improves _________ by controlling for __________.
External validity; history and maturation effects.
External validity; confounding variables.
Internal validity; history and maturation effects.
Statistical validity; confounding variables.
Internal validity; history and maturation effects.
123
Single-Subject A-B Design
inherent limitation of the study design used with this design
It will always suffer from low external validity
124
If the researchers wanted to enhance the internal validity of their investigation in signle subject designs which of the following could they do?
Increase the number of measurements within each phase.
Add more phases in the study.
Ensure they include males and females in the study.
Include more dependent variables.
Increase the number of measurements within each phase.
Add more phases in the study.
