Chapter 11 - Threats to Internal Validity and Null Effects Flashcards
what are the 6 threats to internal validity for pre/post test designs?
- maturation
- history
- regression to the mean
- attrition
- testing
- instrumentation
define maturation
a change in behaviuor that emerges naturally overtime, changes could have emergered spontaneously over time
- include a no treatment comparison group to rule out
(example being changes in depression over time from therapy, it could have gotten better just with time)
define history
a specific event (unrelated to study) that takes place between the pre and post test and affects everyone in the group
- include not treatment group so you can see how this event affect everyone on a general basis without looking at the treatment (neutral group)
define regression to the mean
if perfromance is extreme at pretest, performance at post test is likely to be less extreme; an extreme finding is likely to be closer to its own mean te next time it is measurement, due to chance factors that made the finding extreme are not present anymore
- include a comparison group that is equally extreme at pretest (want to have the same starting baseline)
define attrition
when attrition between pre and post phases and systemic, not random, disproportionality extreme cases; systemic type of participants drops out of study before its finished
- remove participants who drop out and see if the same patterns shows
- compare drop outs fro people who completed it all and see whether the attrition is selective or just random
define testing
an example of order effect, scores changing overtime because participants complete test more than once, practise or fatigue effects
- use only post test
- change order of study, providing different terms to rule out order effects
- using a comparison group to help rule out
define instrumentation
reliabilty and validity of instrument changed over the course of the study (example is observer is changed throughout the study)
- use post test only
- calibrate forms to be comparable
- establish reliability and validity at both pre and post tests
- counterbalance different forms for both tests
what are the 3 threats to internal validity for any experiment?
- observer bias
- demand cahracteristics
- placebo effect
define observer bias
researchers expectations influence how the results are interpreted
define demand charactertics
particpatns guess the studys hypothesis and change their behaviour accordingly to be good participants
- use double blind design
define placebo effect
people recive a treatment and really improve but only because they believe they are receiving the proper treatment
- double blind placebo control study (should have placebo, treatment and non-placebo groups to compare)
define null effects
when there is not a significant relationship between your variables
this can mean:
- there actually isn’t a relationship in the real world
- some experimental design feature obscured a true effect
*can never know if a relationship doesn’t exist, can never be certain
what design features could lead to a null effect?
- weak manipulation of IV
- insensitive measure of DV (instrument)
- ceiling/floor effect on IV or DV
what is the solution for null effect results
manipulation check: a test used to determine the effectiveness of a manipulation in an experimental design.
what can cause lare within group variability?
measurement error: this can be reduce when the reliability of a measure goes up and when precise systems are used and when large sample is used
pre-existing individual difference: impact the response and sensitivity to treatment
situation noise: variability in external situation, want to think about controls that you can manage throughout your study
