Decision Making Flashcards
Decisions in everyday life
happens unconsciously most of the time
Small decisions during your day:
Decisions can be seen as a link between memory and future actions
A lot of decisions are based on experience from past memory
These decisions will influence future actions
Memory is updated/influenced after experiencing consequences of your actions; “was the experience better/ as you expected?”
Decision-making is not an isolated process.
How does it relate to LTM?
You can think of decisions as a link between memory for past experiences that help to guide future actions.
Need to activate your past experiences in memory to inform your decision. Usually, you make a decision in order to do something. And in most cases, your decisions will be informed by what you have experienced in the past.
When making the decision, you might think of pleasant memories when you experienced a really good mood.
These memories then drive your decision to go to the Peak District.
Experiences = stored/ accessed in LTM
You will use the information from long-term memory to form predictions about the outcome of your decision. You will predict that your experience this time will be very similar to last time or very different, depending on what your last experience was.
What is the Prediction-Choice-Outcome Loop?
It explains the relationship between predictions, choices, and outcomes
Form of a loop: because the outcome of past choices influences our decisions in the future
We make:
decisions based on our goals
predictions of what we are expecting, after weighing all the pros and cons of different options
form a decision and take appropriate actions that should get us closer to our goal
Internally compare experiences
This outcome will be subjected to internal monitoring processes if our decision and the corresponding actions have achieved our goal (or at least brought us closer to it).
If we haven’t reached our goal?
Brain generates a prediction error:
a signal indicating how large the discrepancy is between what we had predicted originally and what the actual outcome was.
- Use these prediction errors to adjust future expectations
- Update our memory to more precise decisions the next time
when we face the same or a similar choice.
For example, your goal could be to have a tasty meal at a restaurant where you haven’t been before. When looking at the menu, you make predictions on how much you would like each option based on your experience in other restaurants. You then decide to order something (-> this is the action) that seems to be very similar to a dish that you enjoyed a lot elsewhere. When your meal arrives, you realize that you don’t like it as much as the version you had in mind. This difference between your anticipated meal (= that it would be delicious) and the meal that was served (= i.e. not as good) reflects the prediction error. You can use this prediction error to update your memory (you do not order this meal in this specific restaurant again).
This will help you to make better decisions next time.
There is a circular relationship between predictions (derived from memory content), decisions and outcomes which will be used to guide future predictions.
Biases in decision-making might have been developed to cut down the time it takes to make a decision (which can be a cost in itself).
Name some examples of biases in decision-making:
Various biases in DM: from economic games
1- Stick with same option you have chosen before
[stick with default]
eg. same food choice on a long menu, reducing decision time
2- Choosing certain gains over gambling situations
3- Choosing gambles over certain losses
4- Temporal discounting
choosing immediate rewards!
over future rewards (unless benefits are made explicit)
Which term s used to describe a signal indicating how large the discrepancy is between what we had predicted originally and what the actual outcome was?
Prediction errors
Prediction-Choice-Outcome Loop
What are the general features/ aims of decision making?
(We rely on memory content to make predictions.)
Also need to make predictions of possible decision outcomes to optimize our decision-making process.
What are the factors to consider before making a decision?
-Difficulty of the action (effort)
Need to execute in order get some form of reward or avoid a punishment (long exercise/ high cognitive effort to solve prob)
-Probability of success and failure
Is it likely that I will succeed? Or is there a high risk of failure?
(Risk evaluation)
-Value of reward/ choice
- Value of reward might change depending on the context/ current goals a given = reward may be more/ less appealing
(eg. a chance to win a pizza as price for a quiz might be of really high value to you if you are hungry. But not if you just ate.)
-Missed opportunities
- We decide for one option, we often decide against other options.
Often cannot go for several options
(travel by train= miss out on travelling by plane excluding the other option)
Name the 2 different levels of decision-making:
Simple perceptual decisions
-perceptual decisions
you have to decide which colour a stimulus has had or what sound
you have heard
More complex decision
- Taking several factors into account to make a decision
a wider range of factors that will influence your decision
compared to the simple decisions
(consider costs, colours, time consumption)
Levels of decision-making:
Which level consists of several factors being taken into account?
More complex decisions
Levels of decision-making:
Which level consists of perceptual decisions?
Simple decisions
Researchers can also experimentally control noise levels in these
tasks. This is important because in everyday life, we also do not always have very clear information available. On a perceptual level, our view on an object might be partly obscured, or more abstract information that we require for a decision might be uncertain.
Evidence accumulation in
Simple perceptual decisions:
Perceptual decision task
Explain the noisy sensory signal
A noisy signal, that needs to be evaluated over a certain period of time, which is converted into a discrete motor action
(move left or right) when the monkey makes this decision.
The decision process is quite noisy so u need time to evaluate and come up with a decision
- time needed to accumulate the evidence, then monkey makes the decision which is transferred into a discrete motor act
This is not just the case in a random-dot lab task, but in real life the sensory input is also often not clear immediately, e.g. when our view on objects is partly obstructed or symbols or keys, that we need to process for our decision, look very similar.
Accumulating evidence in perceptual decisions:
they start to fire when they detect more evidence
Neuronal recordings in monkeys suggest that this is what is happening in the brain during such a perceptual DM task:
When the stimulus, consisting of random-moving dots (and some coherently moving dots), is presented, neurons that are tuned to detect a specific motion direction will start to fire. As you might know from other lectures, neurons in sensory brain areas are tuned to a preferred feature. For instance, some neurons respond most strongly to leftward moving stimuli while other neurons respond most strongly to upward moving stimuli, etc. The more dots are moving in a given direction coherently, the stronger these corresponding motion detector neurons will fire. The firing rate will increase as more evidence for a given motion direction is being accumulated. You can see this in the illustration on the right side. Evidence accumulation for leftward motion is shown at the top, evidence accumulation for rightward motion is shown at the bottom. The evidence for a left motion direction increases faster than the evidence accumulation for a right motion direction. The curves indicating the firing rates/evidence accumulation are not straight, but go up and down a bit around a general upward trend. This reflects the noise in the stimulus. The more dots move coherently in one direction, the stronger the evidence for this direction. This would be reflected in the evidence accumulation curve being steeper and reaching the threshold faster.
Studies suggest that evidence accumulation always increases up to a certain threshold. When this threshold is reached a decision will be made in line with the evidence leading to a corresponding action, i.e. if the evidence accumulation curve for the left motion direction reaches this threshold first, the monkey will decide to make a left-sided response.
If the evidence accumulation curve for a right motion direction would reach this threshold first, the monkey would decide to make a right-sided response.
If the stimulus is quite noisy, that is, there is more distracting information present and the relevant information is not as clear, the evidence accumulation curves would reach the decision threshold closer in time and sometimes would lead to incorrect decisions, if the accumulation process for the less dominant direction meets the decision threshold first.
Simple perceptual decisions:
Which brain areas does evidence accumulation take place in?
Brain areas responsible for encoding the relevant feature,
e.g. area MT/V5 if motion is relevant for decision
You need to specify with the motor system is responsible for the decision making:
Ie. if the task is colour detection then colour coding areas will be involved in perceptual decision-making tasks
But recent evidence: sensorimotor areas (Parietal and dorsal prefrontal cortex) are representing possible actions, accumulating evidence as well
What are the 3 stages of perceptual DM?
Detection of sensory evidence; What are the alternatives that can be detected (left/right, red/blue, etc.)?
Integration of evidence over time
-> because evidence is noisy
Checking if threshold has been reached
-> if so, elicit appropriate action
-> if not, accumulate more evidence
There are 3 stages in perceptual DM:
The first is: Detection of sensory evidence; Here individuals identify what kind of sensory evidence can be detected (e.g. all possible colours or motion directions) and start to accumulate the evidence
Then, the evidence is integrated over time to detect the relevant signals among the noise.
As a third stage, a mechanism is assumed that regularly checks if a threshold has been reached. When this is the case, an appropriate action will be elicited. If the threshold has not been reached yet, more evidence will be accumulated.
Subjective values of options:
Areas associated with subjective value of decision options
(Fellows, 2018):
Reward value is reflected in activity strength in relevant brain areas: ↑ activity = ↑ expected reward value
Damage to the striatum may disrupt some aspects of reward learning.
The more activity in these areas, the higher the expected reward value of a given option.
Suggest that goals are represented in the medial OFC.
Suggest that the vmPFC and ventral striatum track the expected value in line with the current goals
(i.e. the value representation will change for a given option depending on the current context)
Name a brain area associated with decision making:
Lateral prefrontal cortex (PFC):
But active in many decision paradigms
(NOT involved in value-based choices)
What is Evidence accumulation?
How the brain combines the info stored in memory with new incoming info
Memory must both be durable and flexible:
2 competing models of how this can be implemented in the brain
Name the 2 models of how evidence accumulation can be implemented in the brain:
A) Homogeneous model
B) Heterogeneous model
Evidence accumulation:
Memory must both be durable and flexible:
2 competing models of how this can be implemented in the brain
Homogeneous model
Homogeneous model:
All relevant neurons (to encode evidence) become active at the same time. The more evidence presented, the greater the activity shown in cells.
Sensory events make a wave of activity
To illustrate where these hypotheses (mental maps) in decision-making came from:
Tolman had rats experiencing a spatial maze
The shortest path to the goal was blocked so the rat had to go all the way round.
‘A’ and ‘B’ here just indicate spatial locations in the maze for illustration purposes, but are not actually indicated in any way to the rat.
Findings showed?
Rats created a mental map representation of the maze
Points a and b were closer together taking the shortcut
-encoding the relation between points a and b
If then the path was unblocked later, the rats quickly realized that they could take a shortcut and go directly from A to B to reach the goal faster.
Thus, the rats seem to have positions A and B encoded as being close together in space, although they had never experienced going straight from A to B.
This means that the rats had encoded transitive relations of the different locations in the maze and had used this information to build a mental map of the maze.
This mental map informed their decision at the first intersection to go straight ahead instead of taking the path they had been trained on.
Mental maps in decision making:
Problems can be described as series of decisions
Mental map guides you through different steps.
Applies to problem solving processes as well.
Problems can be described as a series of decisions.
This sequence of steps could be represented as a mental map.
But this means that you will need to activate your memory content to make these decisions.
Mental map represented in cognitive space
You can swap steps/ take short cuts
But sometimes order does matter so u learn from experience about what order makes the most sense
Mental maps in decision-making
Retrieval of long-term memory content is associated with which brain region?
The hippocampus
New experiences will require neuronal activity in the hippocampus in order to be stored in LTM. Later, when you need to make decisions that are related to these experiences, you will re-activate these memories. Thus brain areas that are involved in various aspects of decision-making need to interact with the hippocampus.
But decision-making can also bias what is being stored in memory.
What did Schuck et al. (2015) show on decision making?
They showed that some aspects that are relevant for decision-making are represented in the medial frontal cortex.
From activity in MFC: group membership could be derived even before strategy shift happened.
And this MFC activity seems to be linked to memory formation in a decision-making task.
And a shift in strategy,
(i.e. how to make decisions, based on learned associations)
could be predicted even before this shift occurred in observable behaviour from activity patterns in the MFC.
State spaces could also help us to impose a structure on new experiences.
Explain the representation of hidden states:
State spaces:
- Cognitive map specific for a given task
- Grid-like representations of all the decision points from the
original problem
- Represented in orbito- and medial frontal cortex
- Help structure new experiences
Hidden state:
- Our position within the current task
Mental exploration:
- evaluation of potential outcomes for different choices
Frontal lobe lesions and decision making:
Patient studies:
Patients with lesions in OFC-vmPFC areas are impaired in simple preference judgments in value-based DM tasks.
After ventromedial prefrontal lesions people show?
inconsistent preferences
deficient sense of guilt (might affect DM factors)
Indirectly affecting decision-making processes
e.g. to avoid harm when deciding for or against certain options.
If you have no guilt you cannot avoid harm
When we want to explore new options which brain region is activated?
Frontal pole:
exploratory behaviour
What is the Reward rate maximization formula?
How likely it is I will succeed when I make this action and subtract the cost
The reward value ( ‘reward or success probability’ )
The ‘cost’ or ‘subjective effort’ is subtracted
( because the cost diminished the reward value )
Divided by
the temporal discounting factor
= reward rate for a given choice
Utility: refers to the payoff of an outcome ->
What can I gain/ value when I choose this option?
Success Probability:
How likely is it that I will actually get the intended outcome? How risky is this option?
Cost means: What effort (both physical and cognitive) do I have to put in to get the intended result? How difficult will this be for me?
And then the temporal discounting factor
(that is, the time to-be-invested) is important for the overall reward rate. It consists of:
The Deliberation Time
Handling Time
ITI (stands for Inter-Trial Interval)
All of these factors together will determine the reward rate of a given option.
But Carland and colleagues suggested an additional mechanism that serves to optimize the reward rate: the urgency signal.
Explain The urgency signal:
In interest of reward rate maximization, evidence accumulation signal needs to be pushed over threshold to take action as
- helps to maximize the reward rate by ensuring that the deliberation time does not get overly long in uncertain situations and that there is the right balance between evidence accumulation and time spent on a decision.
What is this describing:
A mechanism that serves to maximize the reward rate by pushing evidence accumulation closer to the threshold
is controlled by projections from the basal ganglia (BG) to cognitive and sensorimotor areas
The Urgency signal
-grows during deliberation time and helps to optimize it
-is modulated by task context
eg. very simple decisions without major consequences, the urgency signal is stronger or sets off earlier, not for others
-shows inter-individual differences: different in different individuals
eg. some people seem to have longer deliberation times overall, some people tend to decide very quickly in general.
