Extreme Psychometrics 1 Flashcards
Under what circumstances would we need to use Signal Detection Theory?
Whenever a task involves discriminating between two stimuli (commonly used to determine sensory thresholds)
In a recognition memory task, what are the four possible outcomes?
Correct hit (correctly said yes); false positive/false alarm (incorrectly said yes); correct miss (correctly said no); false negative (incorrectly said no)
If we did not use signal detection theory, how could someone cheat a recognition memory test?
By saying that they recognise every word (yes to everything)
What are sensitivity and response bias, in the context of signal detection theory?
Sensitivity is the ability to discriminate between words you previously heard and those you didn’t; response bias is the criterion for saying yes (if you just look at correct hits then sensitivity is contaminated by response bias)
How does signal detection theory de-confound sensitivity and response bias?
By looking at false positives as well as correct hits
What is d prime (d’)?
A measure of sensitivity independent of response bias; the distance in standard deviations between the signal (words from the original list) and the noise (words not on the original list) distributions
What do different values of d’ indicate?
If d’ is positive then the person is recognising the word from the original list to some degree; if 0 then the person is guessing (can’t distinguish old from new); if negative then they’re recognising words they didn’t see and not those they did see
List five examples of signal detection theory
- Detection tasks in psychophysics experiments
- Diagnosing illness (mental or physical)
- Jury decision making
- Industrial inspection (e.g., detecting unacceptable items in a factory)
- Collision anticipation tasks
Item Response Theory is arguably a superior alternative to Classical Test Theory, but much more complex. What does it involve?
The score from a test using this theory is called Theta, that’s a function of the examinee’s response interacting with the characteristics of the items; it takes the characteristics of the items into account, weighted by level of difficulty
What is Latent Trait Theory?
Another name for Item Response Theory
What are Item Characteristic Curves, and what do they have to do with Item Difficulty Indices?
They’re a plot of ability (level on some trait); they’re plotted against the item difficulty index – the probability of getting a particular question right (% of people who got it right)
What’s the link between item response theory and item characteristic curves?
While we want higher ability people to be more likely to get the item correct, the specific shape of the curve gives useful additional information; different items are “sensitive” across different ranges of Theta; we can use item characteristic curves to spot items that may have potential group biases
Detail the steps you would go through to find the best equation to represent your item characteristic curve
- Make an educated guess as to which equation will fit the best (e.g. s-shaped curves can be created using equations known as logistic functions, i.e. Rasch model)
- Use software that estimates the parameters for the equation to get a curve as close as possible to the actual curve
- Do a goodness of fit test to see how well the equation actually fits your data
- If it doesn’t fit, try again with another equation
What are the parameters in the three-parameter model?
a) Item discrimination (point where the slope is steepest)
b) Item difficulty (level of ability needed to get the item right 50% of the time)
c) Level of guessing
Once we’ve modeled an item characteristic curve then what can we use it to do?
We can define each item in terms of sophisticated non-linear parameters, with a focus at the item level (i.e. can map each item individually onto ability level – unlike classical test theory)
