lecture 4

Question 1

What is the focus of probabilistic models in machine learning?

Answer 1

Using probability theory to select models for given data.

Question 2

What is conditional probability?

Answer 2

p(X | Y) is the probability of X given Y.

Question 3

What is Bayes’ theorem?

Answer 3

p(X | Y) = p(Y | X) * p(X) / p(Y)

Question 4

What is a useful probability rule?

Answer 4

p(X, Y) = p(X | Y) * p(Y)

Question 5

What is (Conditional) Independence?

Answer 5

When knowing Y provides no information about X.

Question 6

What is expectation in probability?

Answer 6

The weighted average of all possible values of a random variable.

Question 7

What are common probability distributions in ML?

Answer 7

Bernoulli, Categorical, and Normal distributions.

Question 8

What does p(Data | θ) represent?

Answer 8

The likelihood of observing data given parameters θ.

Question 9

What is the goal in probabilistic learning?

Answer 9

To infer the parameters θ given only the observed data.

Question 10

What is the frequentist view of probability?

Answer 10

The true model exists but is unknown, and we estimate its parameters.

Question 11

What is maximum likelihood estimation (MLE)?

Answer 11

Finding parameters that maximize the likelihood of observed data.

Question 12

What is the likelihood function L(θ)?

Answer 12

p(X | θ), the probability of observed data given parameters.

Question 13

What is the MLE (maximum likelihood estimation) formula?

Answer 13

θ̂ = argmax_θ p(X | θ)

Question 14

What is a simple analogy for probabilistic learning?

Answer 14

A ‘machine’ generates data, and we infer its parameters.

Question 15

What does the model space represent?

Answer 15

All possible models that could explain the observed data.

Question 16

What does maximum likelihood estimation (MLE) assume?

Answer 16

That the best model is the one that makes the observed data most likely.

Question 17

What is a practical example of MLE?

Answer 17

Estimating if a coin is fair or biased based on observed flips.

Question 18

What is the difference between frequentist and Bayesian inference?

Answer 18

Frequentists estimate a fixed parameter, while Bayesians use probability distributions over parameters.

Question 19

What is the Bayesian approach to learning?

Answer 19

Updating beliefs about parameters using observed data and prior knowledge.

Question 20

What is a prior in Bayesian inference?

Answer 20

The initial belief about a parameter before observing data.

Question 21

What is a posterior distribution?

Answer 21

The updated belief about a parameter after observing data.

Question 22

What is the likelihood in Bayesian inference?

Answer 22

The probability of observed data given a parameter.

Question 23

What is the Bayes rule formula for updating beliefs?

Answer 23

Posterior ∝ Likelihood × Prior

Question 24

What is a conjugate prior?

Answer 24

A prior that, when updated, results in a posterior with the same functional form.

Question 25

What is the benefit of using conjugate priors?

Answer 25

They simplify Bayesian inference calculations.

Question 26

What is MAP estimation?

Answer 26

Maximum a posteriori estimation, which finds the most probable parameter given both data and prior.

Question 27

What is the difference between MAP and MLE?

Answer 27

MAP considers prior knowledge, while MLE relies only on data.

Question 28

What is the purpose of probabilistic modeling in ML?

Answer 28

To incorporate uncertainty and make robust predictions.

Question 29

What is the relationship between likelihood and probability?

Answer 29

Likelihood measures how well parameters explain data, while probability represents chance.

Question 30

What is the key takeaway from probabilistic models?

Answer 30

They provide a framework for handling uncertainty in machine learning.