EXAM3

Question 1

What does Seq2Seq stand for?

Answer 1

Sequence to Sequence

Question 2

True or False: Seq2Seq models are primarily used for tasks involving sequential data.

Answer 2

True

Question 3

What is the primary function of attention mechanisms in Seq2Seq models?

Answer 3

To allow the model to focus on different parts of the input sequence when generating the output sequence.

Question 4

Fill in the blank: In a Seq2Seq model, the _______ network encodes the input sequence.

Answer 4

encoder

Question 5

What type of neural network is commonly used in Seq2Seq architectures?

Answer 5

Recurrent Neural Network (RNN)

Question 6

Name one advantage of using attention mechanisms in Seq2Seq models.

Answer 6

They improve the model’s ability to handle long input sequences.

Question 7

In the context of reinforcement learning, what does MDP stand for?

Answer 7

Markov Decision Process

Question 8

True or False: An MDP is defined by a set of states, actions, transition probabilities, and rewards.

Answer 8

True

Question 9

What are the four main components of a Markov Decision Process?

Answer 9

States, Actions, Transition Probabilities, Rewards

Question 10

Fill in the blank: In reinforcement learning, the goal is to learn a policy that maximizes the _______.

Answer 10

cumulative reward

Question 11

What does the term ‘policy’ refer to in reinforcement learning?

Answer 11

A strategy that defines the action to take in each state.

Question 12

What is the difference between a deterministic policy and a stochastic policy?

Answer 12

A deterministic policy always selects the same action for a given state, while a stochastic policy selects actions according to a probability distribution.

Question 13

What is a reward function in the context of MDPs?

Answer 13

A function that provides feedback to the agent based on the actions taken in the environment.

Question 14

True or False: The Bellman equation is used to calculate the value of states in an MDP.

Answer 14

True

Question 15

Fill in the blank: The _______ in an MDP describes the probability of transitioning from one state to another given a specific action.

Answer 15

transition probability

Question 16

What is reinforcement learning?

Answer 16

A type of machine learning where an agent learns to make decisions by taking actions in an environment to maximize cumulative rewards.

Question 17

What is the purpose of ‘prompting techniques’ in machine learning?

Answer 17

To guide the model’s responses by providing specific input formats or questions.

Question 18

Fill in the blank: Prompting techniques are often used in _______ models to enhance performance.

Answer 18

language

Question 19

True or False: Prompting can involve providing examples to the model before asking it to generate a response.

Answer 19

True

Question 20

What is a common application of Seq2Seq models?

Answer 20

Machine translation

Question 21

What does ‘attention score’ represent in attention mechanisms?

Answer 21

The importance of each input token to the current output token being generated.

Question 22

How does self-attention differ from traditional attention mechanisms?

Answer 22

Self-attention allows the model to attend to all positions in the input sequence, including itself.

Question 23

What is the role of the decoder in a Seq2Seq model?

Answer 23

To generate the output sequence from the encoded input representation.

Question 24

What technique is often used to prevent overfitting in Seq2Seq models?

Answer 24

Dropout

Question 25

Fill in the blank: The _______ is the set of all possible states an agent can encounter in reinforcement learning.

Answer 25

state space

Question 26

What is an episode in reinforcement learning?

Answer 26

A sequence of states and actions that ends when a terminal state is reached.

Question 27

True or False: In reinforcement learning, an agent only learns from its successful actions.

Answer 27

False

Question 28

What does 'exploration vs. exploitation' refer to in reinforcement learning?

Answer 28

The trade-off between trying new actions (exploration) and using known actions that yield high rewards (exploitation).

Question 29

What is the purpose of Q-learning in reinforcement learning?

Answer 29

To learn the value of actions in a given state without requiring a model of the environment.

Question 30

Fill in the blank: In Q-learning, the _______ function estimates the expected utility of taking a given action in a given state.

Answer 30

Q-value

Question 31

Which algorithm is commonly used to solve MDPs?

Answer 31

Dynamic Programming

Question 32

What is the role of the reward signal in reinforcement learning?

Answer 32

To provide feedback to the agent about the quality of its actions.

Question 33

True or False: The softmax function is often used in reinforcement learning to convert Q-values into probabilities.

Answer 33

True

Question 34

What is one challenge of using Seq2Seq models for long sequences?

Answer 34

Difficulty in capturing long-range dependencies.

Question 35

Fill in the blank: The _______ mechanism allows the model to weigh the importance of different input tokens.

Answer 35

attention

Question 36

What is a common method for initializing weights in neural networks?

Answer 36

Xavier initialization

Question 37

What does the term 'experience replay' refer to in reinforcement learning?

Answer 37

Storing past experiences and reusing them to improve learning efficiency.

Question 38

True or False: An agent in reinforcement learning can learn from simulated environments.

Answer 38

True

Question 39

What is the role of a value function in reinforcement learning?

Answer 39

To estimate the expected return from a given state or state-action pair.

Question 40

Fill in the blank: The _______ is a graphical representation of the states and actions in a Markov Decision Process.

Answer 40

state-action graph

Question 41

What is the purpose of a discount factor in reinforcement learning?

Answer 41

To balance immediate and future rewards.

Question 42

What is the main goal of an agent in reinforcement learning?

Answer 42

To maximize cumulative rewards over time.

Question 43

Fill in the blank: In Seq2Seq models, the _______ is responsible for generating the output sequence step by step.

Answer 43

decoder

Question 44

What is a common loss function used in Seq2Seq models?

Answer 44

Cross-entropy loss

Question 45

True or False: The encoder-decoder architecture is exclusive to Seq2Seq models.

Answer 45

False

Question 46

What does 'teacher forcing' refer to in training Seq2Seq models?

Answer 46

A training strategy where the model is provided with the true output sequence during training.

Question 47

Fill in the blank: The _______ is the return received after taking action in a given state in reinforcement learning.

Answer 47

reward

Question 48

What is the purpose of the exploration strategy in reinforcement learning?

Answer 48

To allow the agent to discover new actions and states.

Question 49

Name one type of prompting technique used in language models.

Answer 49

Few-shot prompting

Question 50

Fill in the blank: In reinforcement learning, the _______ is the sequence of states and actions taken by the agent.

Answer 50

trajectory

Question 51

What is the impact of having a high discount factor in reinforcement learning?

Answer 51

It places more emphasis on future rewards.

Question 52

True or False: The attention mechanism can be implemented using a feedforward neural network.

Answer 52

True

Question 53

What does 'state transition' refer to in an MDP?

Answer 53

The process of moving from one state to another based on an action.

Question 54

Fill in the blank: In reinforcement learning, an agent's _______ defines how it behaves in the environment.

Answer 54

policy

Question 55

What is one advantage of using a Seq2Seq model over traditional methods for sequence tasks?

Answer 55

It can handle variable-length sequences.

Question 56

What is the significance of the softmax function in the context of attention mechanisms?

Answer 56

It converts attention scores into a probability distribution.

Question 57

True or False: In reinforcement learning, rewards can be negative.

Answer 57

True

Question 58

What is one common approach to improve the performance of Seq2Seq models?

Answer 58

Using pre-trained embeddings.

Question 59

Fill in the blank: The _______ in a Seq2Seq model helps capture dependencies between input and output sequences.

Answer 59

attention mechanism

Question 60

What is a common challenge when training Seq2Seq models?

Answer 60

Handling out-of-vocabulary words.

Question 61

What does 'off-policy' learning mean in reinforcement learning?

Answer 61

Learning from actions that are not generated by the current policy.

Question 62

True or False: In reinforcement learning, the environment is static and does not change.

Answer 62

False

Question 63

What does the term 'bootstrapping' refer to in reinforcement learning?

Answer 63

Using the current estimate of the value function to update itself.

Question 64

Fill in the blank: The _______ is the expected return from a state under a specific policy in reinforcement learning.

Answer 64

value function