Flashcards in Measurement, reliability + Validity Deck (23)
Why is it important for psychologist to quantify everything?
- is a defining principle of science = measurement
-- can be objectively obtained
-- publicly available
-- potentially checkable by others
What are the requirements for measurements?
- reliable = can be reproduced
- valid = actually measuring what we think it is measuring
What are the different levels of measurements?
Describe nominal level of measurement
- categorical. frq data
- when numbers are used as names (eg footballer numbers)
- All you can do is count how often each number occurs
Describe ordinal level of measurement
- numbers used as ranks
eg: order of finishing a race
Describe interval level of measurement
-scale with equal interval between points on a scale
- no true zero point
Describe ration: level of measurement
- measurements made on a scale with equal interval between points on the scale
- scale has TRUE 0 point
Eg: reaction time, error scores
What is reliability?
- does it give consistent/ reproducible results?
Error in reliability may be due to....?
- natural phenomena variation
--> within/ between individuals
- imprecision in defining + measuring psychological constructs
--> what exactly is aggression?
What are different measures of reliability?
1. Test-retest: time to time
2. Alternate forms: version to version
3. Split-half: item to item
4. Inter-scorer: person to person
What factors affect reliability?
- phenomena itself
--> trait vs state?
- Precision of measurements
- Sample size
- Variability in performance
--> High > low
- Format of test:
--> MCQ of 5Q = 25% correct by chance > than ....
--> True/ False = 50% correct by chance
What is the replication crisis?
- many well-known findings aren't that reliable
--> small sample size
--> Exact replication are rare
--> "file drawer problem"
What is the "file drawer problem"?
- hard to get replication failures published
What are some solution to the replication crisis?
- Converging operations = tackling problem with different methods, by different research groups
What are the 10 frq cited findings that have failed to replicate as found by Jarrett in 2016?
2. Smiling = happier
3. exposure to age related words = slower walk (x social priming)
4. Babies are born with the power to imitate adults
What did Nosek et al in 2015 find when he replicated 100 experimental + correlational studies?
1. og 97% significant results (p<0.5)
2. replication = 36% significant results
3. 47% replication significance size similar to og
= 68% studies w/ statistically significant effects
What does validity mean?
- are you measuring what you think you are measuring?
- a measure can be reliable without being valid (since you can replicate it i guess)
What link did Paul broca poorly attempt to make?
- anthropometric measurements + intelligence
- eg brain size saying there was a 14% difference between F +M but no regards for age/ body size given
What did Gall + Spurzheim which was scientific + testable but had poor vlaidilty?
- different parts of brain responsible for different functions
- highly developed regions = larger = reflected by bumps on skull
- craniometry measuring tool
What are the different types of validity?
- intuitively looks plausible
- relevant material covered??
- predictive or concurrent
- does performance correlate well with known measures of the phenomena?
5. Ecological/ external
- relevance to rl
What does the predictive or concurrent criterion mean?
p: Does performance on test correlate w/ later performance?
C: does performance on my test correlate w/ other tests of current performance?
What factors affect the validity of experiments?
- good or bad experimental design
-method of difference: only one thing needs to differ between comparing conditions