Midterm 2

Question 1

where is the signal detection theory stemmed from?

Answer 1

radar activation (world war II)

Question 2

what does the signal detection theory state?

Answer 2

that nearly ALL reasoning and decision making takes place in the presence of some uncertainty

Question 3

what is the purpose of signal detection theory?

Answer 3

provides a precise language and graphic notation for analyzing decision making in the presence of uncertainty

Question 4

what are some direct applications of signal detection theory?

Answer 4

• sensory experiments –> ex: outside events detected by nervous system
• medicine & diagnosis –> ex: have disease or not
• electronics & telecommunication –> ex: radars
• legal settings –> ex: someone guilty or innocent
• alarm management –> ex: emergency occurring or not
• inferential statistics & hypothesis testing –> ex: assume uncertainty has an effect but still form hypothesis and conduct experiment, just need to see if results are significant or not

Question 5

what is the signal detection theory framework?

Answer 5

• how to look at the situation in question
• table –> 4 squares total
  - vertical = ground truth (present or absent) –> not known to us
  - horizontal = decision (yes or no) –> known through physical evidence

Question 6

what is the purpose of the signal detection theory framework?

Answer 6

to maximize correct decisions (hit and CR) as much as possible while still acknowledging that errors may be present

Question 7

describe the 4 outcomes of signal detection theory framework?

Answer 7

• hit –> ground truth = present; decision = yes (correct)
• miss –> ground truth = present; decision = no (incorrect)
• false alarm –> ground truth = absent; decision = yes (incorrect)
• correct reject –> ground truth = absent; decision = no (correct)

Question 8

what was the signal detection theory framework example explained in class?

Answer 8

radiologist examining a CT scan

• presence or absence of a tumor
• radiologist say yes or no for tumor

Question 9

what are the 2 main components of the decision-making process?

Answer 9

1) information acquisition

2) criterion

Question 10

what is information acquisition?

Answer 10

• capturing all relevant empirical evident (aka information) to try to make the best decision possible
• uniformly leads to more correct decisions (hit & CR)

Question 11

what will improve information acquisition

Answer 11

more evidence = less noise = better –> each decision is based on information from empirical evidence, but there can still be an attempt to get more or better evidence

Question 12

ways to improve information acquisition and real life examples

Answer 12

• using better measurement devices –> ex: CT scans show shape, brightness, and texture of healthy tissue vs. tumors
• more training and practice to learn more information –> ex: doctors will know how to read a CT scan so they can use that tool to help them make their decision
• running another test –> ex: can try a better resolution test (MRI compared to CT) or can try test from a different angle

Question 13

what is another benefit (that relates to midterm 1 content) of acquiring more information?

Answer 13

more information = more accuracy

Question 14

what is criterion

Answer 14

• same information, same expertise, same test BUT different decisions –> based on individual’s criterion, not type of information given
• leads to a trade-off bw correct decisions (hits & CR)

Question 15

what is an example of criterion

Answer 15

doctor isn’t sure if there is a tumor or not –> some are more likely to say yes (resulting in more false alarms), or some are more likely to say no (resulting in more misses)

Question 16

criterion also includes when decisions are made based on… ?

Answer 16

what kind of error is perceived as more or less acceptable –> ex: some doctors may think its acceptable to have more false alarms compared to misses, or vice versa

Question 17

impact of criterion: criterion shift results in more “yes”

Answer 17

more hits (less misses) but also more false alarms (less correct rejects)

Question 18

impact of criterion: criterion shift results in more “no”

Answer 18

more correct reject (less false alarms) but also more misses (less hits)

Question 19

what do the 2 components of the decision process (information acquisition and criterion) impact in terms of decisions?

Answer 19

accuracy of decisions

Question 20

___ signal and ___ information uniformly leads to more correct decisions (hits or correct rejects)

Answer 20

greater; better

Question 21

what is the trade off of criterion changes?

Answer 21

• they have opposing influence on the 2 incorrect decisions –> decreasing misses increases false alarms, decreasing false alarms increases misses
• more evidence not always better

Question 22

what is internal response?

Answer 22

the variable that forms the basis of the observer’s decision

Question 23

what contributed to internal response?

Answer 23

internal response = signal + noise

Question 24

internal response takes on values that ___ from one occasion to another for the ___ same stimulus (aka “signal”)

Answer 24

vary; same

Question 25

what are frequency of occurrence curves?

Answer 25

internal response (x-axis), frequency (y-axis) --> all possible realizations about a topic and how often they occur

Question 26

what is the central tendency of a frequency of occurrence curve?

Answer 26

value that is the most likely possibility for a realization --> spread of the curve indicates that error is present

Question 27

what is the criterion of a frequency of occurrence curve?

Answer 27

indicates in what situations to say "yes" (right from criterion line) and "no" (left from criterion line)

Question 28

what is optical criterion?

Answer 28

the criterion that leads to the optimal amount of correct decisions based on the situation in question

Question 29

criterion shift

Answer 29

moving the criterion line along the frequency occurrence curve(s) --> moving right = more conservative (less yes's); moving left = less conservative (more yes's)

Question 30

hit + miss = ___

Answer 30

100%

Question 31

false alarm + correct reject = ___

Answer 31

100%

Question 32

what should you consider when choosing criterion?

Answer 32

which criterion optimizes accuracy

Question 33

what does a criterion shift do?

Answer 33

changes the percentage of hits and false alarms of the decision --> moving right = less hits, even less false alarms; moving left = more hits, more false (but not as many as hits)

Question 34

what is accuracy defined as in class in terms of frequency of occurrence curves?

Answer 34

how many correct answers gotten

Question 35

how do you change percentage to frequency?

Answer 35

percentage (%) / 100 = frequency --> usually a decimal

Question 36

what information do you need in order to calculate the accuracy of a situation?

Answer 36

- frequency (often percentage) of hits and false alarms in the population - probability of the situation being present or absent in the population

Question 37

how can the same criterion under the same noise and signal distributions results in different total accuracy?

Answer 37

accuracy itself depends on base rates (how the cases for the situation is distributed) --> criterion in terms of accuracy isn't optimized until you know the base rates for that event - base rates >50/50 (very variable) --> change criterion to be very conservative - base rates ~50/50 --> different criterion would be represented at particular accuracy level - base rates <50/50 --> different accuracy represented overall

Question 38

how can you increase your calculated accuracy?

Answer 38

by shifting criterion (more or less conservative)

Question 39

are some criterion values better than others?

Answer 39

yes, some criterion values can be better depending on... - the outcome you are optimizing (better to maximize accuracy or minimize cost, etc) - proportion of "signal present" and "signal absent" cases

Question 40

choice of criterion: sensory experiment

Answer 40

- observer views a single interval (noise or signal embedded in noise) --> each event equally likely - yes/no task --> whether or not they think signal happened

Question 41

what does "populate the table" mean

Answer 41

fill our the percentages of hit/miss & FA/CR in the framework table

Question 42

calculation of accuracy

Answer 42

accuracy = [ (number-of-cases-of-signal-present)*(frequency-of-hits-in-decimals) + (number-of-cases-of-signal-absent)*(frequency-of-correct-rejects) ] / [total-number-of-cases]

Question 43

conservative

Answer 43

need more evidence to be able to say "yes"

Question 44

liberal

Answer 44

say "yes", even with just a little evidence present

Question 45

what is the accuracy if you randomly tossed a coin to make your "yes/no" decision?

Answer 45

the accuracy you would get if you didn't even consider empirical evidence --> not very good

Question 46

how to determine optical criterion of a situation?

Answer 46

- sketch graph --> x axis = criterion (liberal to conservative); y axis = what trying to maximize - find the peak of the graph (is optimal criterion)

Question 47

choice of criterion: a special case

Answer 47

in the special case where signal absent (N) and signal present (S+N) are equally likely (base rates are 50/50), the optimal criterion that maximizes accuracy is the point where the two internal response curves cross (exact middle of the 2 distributions)

Question 48

choice of criterion: signal present and absent not equally likely

Answer 48

- usually more representative of what happens in real life - everything (curves, % of hits/CR, etc) all same --> just base rate diff) - calculate accuracy the same way as before (just different number of present and absent cases --> not 50/50) - determine optimal criterion in same way as with 50/50 base rates

Question 49

why does the optimal criterion change for base rates that aren't 50/50?

Answer 49

event less (or more) likely --> criterion needs to be more (or less) conservative

Question 50

why is the accuracy peak higher in base rates that aren't 50/50?

Answer 50

have a piece of info that creates a "pedestal" at the beginning --> results in increase accuracy

Question 51

choice of criterion: optimize different parameter

Answer 51

- same thing to determine optimal criterion - populate table - calculate what trying to maximize (ex: cost) - sketch graph - determine peak of graph

Question 52

calculation for total cost (ex of optimizing different parameter)

Answer 52

total cost = [ (number-of-cases-present)*(percent-of-outcome-in-question-in-decimals)*(cost-of-outcome-in-question-in-decimals) + (number-of-cases-absent)*(percent-of-other-outcome-in-question-in-decimals)*(cost-of-outcome-in-question-in-decimals) ]

Question 53

what was the optimal criterion for cost in the example in class?

Answer 53

between C2 and C3 --> where the costs were equal

Question 54

what does optimal criterion depend on?

Answer 54

- base rates (likelihood of proportion of "signal present" and "signal absent" cases) - what outcome trying to maximize --> ex: accuracy, cost

Question 55

discriminability

Answer 55

error can be minimized by reducing overlap bw the two curves

Question 56

why is there always error (based on distribution curves)

Answer 56

because of the overlap bw the two curves

Question 57

how to reduce overlap between the two curves?

Answer 57

- increase separation bw the curves --> get more info/evidence - reduce the spread of the curves

Question 58

discriminability index

Answer 58

the signal is highly discriminable from the noise when there is a large separation and a small spread

Question 59

what does the discriminability index d-prime (d') capture?

Answer 59

both separation and spread

Question 60

calculating d'?

Answer 60

d' = separation / spread