Hoofdstuk 10: Rigor and validity

Question 1

What is study validity and give four elements to support validity ?

Answer 1

Extent to wich appropriate inferences can be made (cause ==> effect)

1. Statistical conclusion validity (empirical evidence)
2. Internal validity (relationship of indep-, dep var and confounders)
3. Construct validity (Instruments,designs,…)
4. External Validity (other settings or groups –> generelizations)

Question 2

Ho do you counter confounding participant characteristics ?

Answer 2

• Randomization: equal groups
• Crossover: part. serve as own control
• Homogenity: sample is homogenous with respect to confounder
• Stratification/blocking: randomized blok designs
• Matching: pair match to creat comparable groups
• Statistical control: remove effect of confounders by including in analysis

Question 3

Explain statistical conclusion validity

Answer 3

= Demostrate an empirical relation (can they be detected ?)

Threats:
1.low stat. power:
Solution: -Large sample size
-Maximize group diffrences on the independant variable to obtain clear results
-Maximize precision (accurate measuring tools, controlling confounders, stat. methods,…)
2.Restriction of range of values on the outcome:
-Limited variability –> attenuated (verzwakt) relationship
3.Unreliable implementation of treatment
-Treatment fidelity: standardize intevrention/protocols
-Manipulation check: intervention was recieved as intended ?
-Treatment adherence: Document dose of treatment for all participants

Question 4

Explain internal validity (design and analysis)

Answer 4

=extent of wich you can say for sure if the independant var is the cause of variation in outcome
Threats:
-Temporal ambiguity: cause must precede effect
-Selection: Differences between groups may explain effect
-History: occurance of external events together with indep var.
-Maturation: process accuring in participants as function of time
-Martality/attrition: non-random dropouts from study
-Testing and instrumentation: effects pre test on post test (gettong used to it), effects of changing in measurement instrument

! efforts te enhance int. val. dont end with design

- selection bias:compare pretest measures and key background var
- attrition(slijtage) bias:compare those who completed vs dropouts
    - -> intent-to-treat analysis (iedereen meenemen in analyse)
- ordering bias: compare orderings in crossover design (history?)

Question 5

Explain construct validity

Answer 5

=inferences from study particulars (treatment - setting - outcome - population) to the higher order underlying constructs they intend to represent

enhance:
1. theoretical conceptualization and explication of contructs of intrest
2. select instances that match those constructs:operalization
3. Asses the match

Threats:

• Reactivity to study sit. : Hawthorne
• researcher expectancies: comunccation of desired outcomeinfluences respones of part. –> blinding
• novelty effects: staff or family compensate for control group not recieving experimental treatment
• treatment diffusion or contamination: blurring of treatment conditions

Question 6

Explain external validity

Answer 6

= extent to wich outcomes can be generalized to other populations

enhances:

• representativess of study pop & setting
• replication: multisite studies
• use real-world circumstances (not artificial ie RCT)

Threats:

• Interaction between relationship and people (certain types of people who are exlcluded)
• Interaction between causal effects and treatment variation (ie somebody ethousiastic –> better outcome ?)

Question 7

What are trade offs in study validity ?

Answer 7

Adressing one sacrifices the other in a way (!strong in internal and external val.)

solutions:
- phased series:
1. efficacy: tightly controlled –> internal validity)
2. effictiveness studies: large sample, multiple sites, less restrictove –> external validity
3. compromise: balace between internal and external (practical or pragmatic clinical trials)