Week 12 - Signal Detection Theory Flashcards
(14 cards)
SDT lie task
Sentence is either truth or lie and receiver must decide which
If sender tells a lie:
- Receiver says lie - HIT
- Receiver says truth - MISS
If sender tells the truth:
- Receiver says lie - FALSE ALARM
- Receiver says truth - CORRECT REJECTION
Two types of correct answer (H and CR), and two incorrect (M and FA)
Introduction
SDT is a framework for characterising performance when decisions are made with uncertainty (discrimination, identification, classification)
Originally formulated for engineering (e.g. radar)
Applied to overcome limitations in Classical Threshold Theory
SDT - procedure
Most simple type - ‘yes-no’ procedure
- Typically, stimulus only shown on certain trials
- Observer must respond with whether they detected it or not
Classical Threshold Theory
Based on concept of ‘fixed threshold’
- Stimulus intensity above threshold > sensation
- Stimulus intensity below threshold > no sensation
Represented by psychometric function (proportion of ‘yes’ vs intensity)
- perfect formulation is a step function, however random noise changes function to smooth/ sigmoidal function
- external noise (stimulus intensity varies), internal noise (observer sensitivity)
Estimate of detection threshold (reciprocal of sensitivity) - when stimulus is detected 50% of time
Problems - threshold estimates affected by non-sensory variables (S willingness to say ‘yes’, prevalence effect)
- These lead to a location shift of the psychometric function (different threshold measured)
SDT - types of data and data collection
2 kinds of stimulus and 2 kinds of data (shown in 2x2 table)
- whether stimulus is present (yes/no), whether it is detected (yes/no)
- Hit, Miss, False Alarm, Correct Rejection
Raw data are then turned into response rates (proportion of responses in SN trials and N trials must separately sum to 1)
SDT basic concepts - ‘signal’ and ‘noise’
Signal (stimulus)
- can be anything (light, noise, etc.)
- stimulus assumed to increase ‘internal response’
Noise (variations in response)
- always present
- main properties - variations are random, both internal (sensory) and external (other factors)
Represented graphically by two probability distributions (on average, internal response is higher for SN)
- N - noise alone
- SN - signal + noise
SDT - criterion
S sets a criterion in mind, decision is based on whether internal response is above/below criterion
On N trials - FA is area of distribution to the right of criterion, CR is area of distribution to the left of criterion (total area = 1)
On SN trials - Hits come from area of distribution to right of criterion, Misses come from area of distribution to left of criterion
SDT - sensitivity
Sensitivity (d’) determined by degree of overlap between distributions
Affected by - difference between means, SD of each distribution (separation and spread)
To calculate d’ - divide difference between means by SD of N distribution (i.e Z score) OR subtract z-scores for FAs from z-scores from Hs (d’ = z(H) – z(FA))
If hit and false alarm rate are known, easy to mathematically recover criterion, distance between distributions etc.
SDT - advantages
Sensitivity/discriminability - invariant when factors other than sensitivity change (such as observer bias changing criterion)
Provides estimate of observer criterion location (c) - observer bias is tendency to favour one response over another, c can be calculated from H and FA rates (c = -1/2 [z(H) + z(FA)])
SDT - criterion influence
Observer’s criterion can be manipulated by experimenter
- Reward - is S is rewarded for every H, they will set a low criterion (many FAs with no consequence)
- Punishment - if observer gets punished for every FA, they will set a high criterion (many Ms with no consequence)
D’ remains unchanged throughout all this, so no way to get rid of error entirely (only trading one type for another)
ROC curves
Different criteria give different patterns of Hs and FAs, considering several different criteria provides a Receiver Operating Characteristic Curve (fully characterises performance)
ROC curves plot Hs vs FAs
- when performance is pure chance, ROC curve is diagonal, when performance increases curve shifts to upper left
- If ROC is below diagonal, then FA > H (worse than chance)
Significance - ROC curves give easy way to check if analysis is correct (problem exists if you can’t obtain one)
- Also allows inference of person’s inner decision-making process
ROC is sometimes not symmetrical, which means that SDs are not the same
ROC data collection
Must change observer’s criterion several times to get H/FA pairs
Can offer incentive/punishment or ask how sure they are (Likert scale > E then changes what counts as ‘true’ or ‘false’ on Likert scale - changing criterion for them)
SDT - other procedures
Many different possible designs (yes-no most simple)
Suitable for wide variety of tasks/procedures
Mathematical procedures must be adapted to reflect increased complexity
Why do we still measure thresholds?
Thresholds are intuitive, while d’ is abstract
Concerns about bias in thresholds can be addressed by good design (removing criterion shift possibility)
SDT has encouraged ‘hybrid’ designs such as 2AFC for threshold measurement
