Neurobiology of decision making

Question 1

decisions as links between memory and future actions

Answer 1

• decision making is not an isolated process
• decisions link between past experiences that help guide future actions
• you need to activate your past experiences in memory to inform your decision

Question 2

how does longterm memory influence decisions?

Answer 2

• experiences have been stored in your long-term memory system, so you access your long-term memory to make decisions
• You will use the information from long-term memory to form predictions about the outcome of your decision

Question 3

what is the prediction-choice-outcome loop?

Answer 3

1. We start out with a goal that we we’d like to achieve, that ultimately requires us to make a decision
2. If there are many ways how to achieve this goal, we make predictions about the outcome of different options
3. We then form a decision and make appropriate actions that should get us closer to our goal
4. The result of these actions will be an action outcome that we observe
5. This outcome will be subjected to internal monitoring processes where we evaluate, if our decision and the corresponding actions have actually achieved our goal (or at least brought us closer to it). If we haven’t reached our goal, our brain seems to generate a prediction error, i.e. a signal indicating how large the discrepancy is between what we had predicted originally and what the actual outcome was.
6. The prediction error, can be used to update our memory based on the experience we have just made. This updated memory content can then be used to make more precise predictions the next time when we face the same or a similar choice.

Question 4

what are the aims of decision making?

Answer 4

• avoid harm
• minimise time we spend of effortful tasks
• minimise the amount of effort
• avoid missed opportunities
• maximise reward

Question 5

What are some factors to consider before making a decision?

Answer 5

• difficulty of the action
• probability of success and failure
• how valuable is the possible reward at this moment
• any missed opportunities

Question 5

how can biases affect decision making?

Answer 5

one bias is to stick with the same option you have chosen before, or sticking with your default option

Question 6

how have biases been shown in gambling?

Answer 6

• People show a tendency to choose certain gains over gambles (if the gambles means that you could also loose).
• Individuals also show a tendency to prefer gambles over certain losses (if the gamble means that you could potentially also win).

Question 7

what is temporal discounting?

Answer 7

choosing immediate rewards over future rewards unless benefits are made explicit

Question 8

what are the two levels of decision-making?

Answer 8

1. simple decisions
2. more complex decisions

Question 9

simple perceptual decision task, Hanks & Summerfield (2017)

Answer 9

• Monkey sat infront of a screen and focuses on a fixation
• stimulus then appears on screen: random dots
• some of the dots move coherently in one direction and some dots move randomly
• of there is a higher % of coherently moving dots on one side of the screen it is assumed to be easier to process so this is where the monkey will focus
• tracked using eye movement trackers
• suggesting the we focus on simple perceptual things

Question 10

accumulating evidence in perceptual decision, . Shadlen et al. (1996)

Answer 10

• When the stimulus, consisting of random-moving dots (and some coherently moving dots), is presented, neurons that are tuned to detect a specific motion direction will start to fire.
• Neurons in sensory brain areas are tuned to a preferred feature. For instance, some neurons respond most strongly to leftward moving stimuli while other neurons respond most strongly to upward moving stimuli, etc.
• The more dots are moving in a given direction coherently, the stronger these corresponding motion detector neurons will fire.
• The firing rate will increase as more evidence for a given motion direction is being accumulated.
• The more dots move coherently in one direction, the stronger the evidence for this direction.
• This would be reflected in the evidence accumulation curve being steeper and reaching the threshold faster.

Question 11

what are the three stages of perceptual decision making?

Answer 11

1. Detection of sensory evidence; What are the alternatives that can be detected (left/right, red/blue, etc.)?
2. Integration of evidence over time, because evidence is noisy
3. Checking if threshold has been reached, if so elicit appropriate action

Question 12

where does evidence for accumulation take place?

Answer 12

• Brain areas responsible for encoding the relevant feature, e.g. area MT/V5 if motion is relevant for decision
• Parietal and dorsal prefrontal cortex
• Recent evidence: sensorimotor areas representing possible actions, accumulate evidence as well

Question 13

Mental maps in decision-making

Answer 13

hypothesis:
- Decision making processes rely on internal models of the current task
- we have an internal representation stored in our memory that provides us with some guidance about the different options that are available in a choice situation
- The internal model helps us to predict the different outcomes of the available options based on our experiences

experiences need to be organized in internal models or mental maps

Question 14

Rats experiencing a spatial maze, Experiments by Edward Tolman. Kaplan et al. (2017

Answer 14

• Tolman had rats experiencing a spatial maze. The shortest path to the goal was blocked so the rat had to go all the way round.
• If then the path was unblocked later, the rats quickly realized that they could take a shortcut and go directly from A to B to reach the goal faster.
• the rats seem to have positions A and B encoded as being close together in space, although they had never experienced going straight from A to B.
• means that the rats had encoded transitive relations of the different locations in the maze and had used this information to build a mental map of the maze
• This mental map informed their decision at the first intersection to go straight ahead instead of taking the path they had been trained on.

Question 15

how is the hippocampus involved in decision making?

Answer 15

Retrieval of long-term memory content is associated with hippocampus

Question 15

Mental maps in decision- making - non-spatial tasks

Answer 15

• e.g. coding an experiment in Psychopy
• You might find complicated solution initially, but then find a shortcut later with more experience.

Question 16

Upcoming strategy switch in DM, Schuck et al. (2015)

Answer 16

• fMRI study where ps were told to pay attention to a pattern which was either green or red (colour wasn’t mentioned in instructions)
• some of the patterns were shifted off centre and ps had to identify when the pattern was upper left/lower right had to press the button, when the pattern was upper/lower left had to press the left button
• task was set up in a way so that you can figure it out. Red pattern = always right button. Green pattern = always left button
• some participants found a shortcut in their cognitive map of this task based on a learned association

Question 17

results from Schuck et al. (2015)

Answer 17

• Some aspects relevant for decision-making are represented in MFC
• MFC activity seems to be linked to memory formation in a decision-making task
• Activity in MFC predicts a shift in strategy based on a learned association

Question 18

Representation of hidden states - State space representation, Kaplan et al. (2017)

Answer 18

• state space: cognitive map specific for a given task, grid-like representation
• state spaces represented in orbito- and medial frontal cortex
• hidden state: our position within the current task
• mental exploration: evaluation of potential outcomes for different choices
• state spaces help structure new experiences

Question 19

what areas of the brain are associated with subjective value of decision making?

Answer 19

• medial frontal cortex
• orbitofrontal cortex (associated with goals)
• striatum and ventormedial cortex (track value depending on goals)
• thalamus

Question 20

Frontal lobe lesions and decision making, Kalat (2015)

Answer 20

• shown that after ventromedial prefrontal lesions, people show inconsistent preferences which will certainly affect their decisions.
• But also a deficient sense of guilt has been reported which could affect their decisions as well, e.g. to avoid harm when deciding for or against certain options.

Question 21

Frontal lobe lesions and decision making, (Henri-Bhargava et al., 2012)

Answer 21

• Lesion in OFC-vmPFC disrupts value-based decision-making, even very simple preference judgments

Question 21

Frontal lobe lesions and DM: the Trust Game, Kalat (2015)

Answer 21

• Person A will be given a certain amount of money with the instruction that they might get to keep whatever they have left at the end of the game
• They will then be told that they can decide how much of this money they want to give to Person B, a trustee
• Whatever amount Person A gives to Person B will be tripled by the experimenter. So the more Person A gives to B, the more money B will receive from the experimenter.
• Person A is informed that Person B can then decide how much of this money to give back to Person A.
• Obviously, a rational strategy from A’s perspective would be to give as much money as possible to Person B and trust that they share whatever they have, thus optimizing the reward for both of them.
• If patients with lesions in their ventromedial prefrontal cortex play the game as person A, they will give less to person B compared to a healthy control group
• So they would not trust the other player as much. If these patients play this game in the role of person B, they will keep almost all the money instead of returning it.

Question 22

Other brain areas associated with decision making

Answer 22

• In several fMRI studies, the lateral prefrontal cortex was activated in DM tasks across a range of different paradigms, but it does NOT seem to be involved in value-based choices.
• But active in many decision paradigms

Question 23

what is reward rate maximisation?

Answer 23

Carland et al. (2019) suggests that individuals optimise their decisions to maximise their reward rate, so they have introduced the principle of reward rate maximisation to decision making

Question 24

Formula for reward rate maximisation

Answer 24

• . The reward value (what they call utility) is weighted by the ‘reward or success probability’.
• Then, the ‘cost’ or ‘subjective effort’ is subtracted, because the cost diminished the reward value
• This whole expression is then divided by the temporal discounting factor to yield the reward rate for a given choice.
• Utility: refers to the payoff of an outcome -> that is: What can I gain when I choose this option? How valuable is the result for me? As just said, this is equivalent to what has been described as reward value earlier.
• Success Probability: means, How likely is it that I will actually get the intended outcome? How risky is this option? In other papers, this has been called reward probability.
• Cost means: What effort (both physical and cognitive) do I have to put in to get the intended result? How difficult will this be for me?
• The Deliberation Time: This refers to how long it does take me to make the decision? Obviously, this depends on whether this is an easy or difficult decision
• Handling Time: For how long do I have to do something before I get the reward
• ITI (stands for Inter-Trial Interval): This means: How long do I have to wait before I can have another go?

(utility X success) - cost/ deliberation time + handling time + ITI
= Reward rate

Question 25

Urgency signal

Answer 25

• if the decision is difficult the evidence accumulation is noisy or too many options
• deliberation time can’t last forever
• In interest of reward rate maximization, evidence accumulation signal needs to be pushed over threshold to take action.
• it makes sense to just go with one option instead of taking even more time to decide

Question 26

what brain areas are involved in the urgency signal?

Answer 26

• controlled by projections from the basal ganglia (BG) to cognitive and sensorimotor areas
• grows during deliberation time and helps to optimize it
• is modulated by task context
• is different in different individuals