Lecture 27 - Reinforcement Learning and Motor Sequences

Question 1

What is classical conditioning? What is the famous example?

Answer 1

a learned (reinforced) reflex/ response that is evoked by stimulus

Pavlov’s Dog

Question 2

What is reinforcement?

Answer 2

increase behaviour

Question 3

What is punishment?

Answer 3

decrease behaviour

Question 4

Give an example of positive reinforcement.

Answer 4

Sweet treat

Question 5

Give an example of negative reinforcement

Answer 5

take away homework

Question 6

Give an example of positive punishment.

Answer 6

writing lines

Question 7

Give an example of negative punishment

Answer 7

take away recess

Question 8

What does the process of reinforcement learning involve?

Answer 8

Learning to link reward with specific actions (and their outcomes) so they become repeated

Question 9

What is binary reward feedback?

Answer 9

Action is rewarded or not

Question 10

What is a scalar quantity eward feedback?

Answer 10

relative to the utility of action/reward outcomes

Question 11

What is the goal of reinforcement learning?

Answer 11

to maximize reward and minimize loss

Question 12

What is cummulative reward?

Answer 12

it might be better to sacrifice immediate reward for long-term reward

Question 13

What are some examples of cummulative rewards?

Answer 13

• chess
• investements

Question 14

What happens to actions that are associated with reward?

Answer 14

they become strengthened/repeated

Question 15

processes of reinforcement learning:

What is exploration?

Answer 15

the trial and error process of aquiring more information about the envrionemnt by searching possibilities

searchhing many action possibilites to determin which action tends to maximize reward

Question 16

processes of reinforcement learning:

What is exploitation?

Answer 16

capitalize on known information to maximize reward

actions associated with past history of reward tend to be repeated to maximize future reward

Question 17

What is the tradeoff between action exploration and exploitation?

Answer 17

shift emphasis from exploring to exploitation to maximize reward

Question 18

What is a hockey example of the exploration, exploitation trade-off?

Answer 18

exploration - you find out goal tender is weak low
exploitation - shoot low to score goals (maximize reward)

Question 19

What is the basal ganglia?

Answer 19

Collection of subcortical structures in the brain

Question 20

Where is dopamine produced?

Answer 20

In the substantia nigra

Question 21

What is dopamine?

Answer 21

a neurotransmitter that is part of the brains intrinsic reward system

Question 22

basal ganglia:

What are the 2 parts of the striatum?

Answer 22

• caudate nucleus
• putamen

Question 23

What is dopamine input into the striatum critical for?

Answer 23

learning from reward and strengtheing the representation of specific actions

Question 24

Learning the piano example:

What happens to serial actions with learning?

Answer 24

can produce faster sequences with less errors. Key presses become smoother and linked together

Question 25

What can practice do in terms of sequential actions?

Answer 25

Can link sequential actions into a single movement pattern

Question 26

# Chuncking of individual elements in a motor sequence What happens in early practice?

Answer 26

shifting gears requires several dinstinct movements

Question 27

# Chuncking of individual elements in a motor sequence What happens in middle practice?

Answer 27

independant actions are 'chunked' into a larger subunit of movement sequences

Question 28

# Chuncking of individual elements in a motor sequence What happens in late practice?

Answer 28

actions become 'chunked' together into a cohesive movement sequence where successivce actions are 'coarticulated'

Question 29

What is chunking?

Answer 29

fusing a series of individual elements into a larger subunit of a movement sequence

Question 30

What is co-articulation?

Answer 30

adjacent movement elements influence each other

Question 31

What does co-articulation cause?

Answer 31

a blending of discrete movements within a sequence and is associated with an ncrease in speed and smoothness of movement sequences

Question 32

What does sequence learning alter?

Answer 32

processing in several sensory and motor brain areas

Question 33

# Early stages of learning: What is the role of the dorsolateral prefrontal cortext?

Answer 33

- working memory, attention, action selection - conscious planning and processing of movements - likely involved in intial stages of chunking motor sequences

Question 34

# Early stages of learning: What is the role of the inferior parietal cortex?

Answer 34

processing visual and somatosensory information

Question 35

# Early stages of learning: What is the role of rostral premotor areas?

Answer 35

- planning motor actions - processing visual and somatosensory feedback

Question 36

# Early stages of learning: What is the role of basal ganglia?

Answer 36

processing reward during action

Question 37

# Later stages of learning: What is the role of supplementary motor area?

Answer 37

- selecting and planning previously learned motor responses - likely involved in later stages of motor chunking and coarticulation

Question 38

# Later stages of learning: What is the role of the dorsal premotor area?

Answer 38

- processing visual and somatosensory information - involved in selecting learned actions

Question 39

# Later stages of learning: What is the primary motor cortext involved in

Answer 39

producing motor actions

Question 40

# Early stages of learning: What is the cerebellum involved in?

Answer 40

detecting and correcting errors

Question 41

How does brain activity evolve during motor sequence learning?

Answer 41

Early stages require high cognitive demand and conscious prcoessing. Later stages require low cognitive demand and automatic processing.