Probability

Question 1

What does probability help us reason about in ML?

Answer 1

Uncertainty, randomness, and incomplete information.

Question 2

In the apple bowl example, what are the two random variables?

Answer 2

The bowl selected (s) and the apple colour (y).

Question 3

What does a marginal probability represent?

Answer 3

The probability of a single event without reference to other variables.

Question 4

What is a joint probability?

Answer 4

The probability that two events occur together.

Question 5

What is a conditional probability?

Answer 5

The probability of an event given that another event is known to have occurred.

Question 6

What is the formula for conditional probability?

Answer 6

P(A | B) = P(A and B) / P(B)

Question 7

What does Bayes’ Rule let us do?

Answer 7

Update our beliefs about causes after seeing an effect.

Question 8

What is the formula for Bayes’ Rule?

Answer 8

P(s | y) = P(y | s) * P(s) / P(y)

Question 9

In Bayes’ Rule, what is the ‘prior’?

Answer 9

P(s), the probability before seeing data.

Question 10

In Bayes’ Rule, what is the ‘posterior’?

Answer 10

P(s | y), the updated belief after seeing data.

Question 11

In probability, what is the sample space?

Answer 11

The set of all possible outcomes of an experiment.

Question 12

What must the total probability over the sample space equal?

Answer 12

1

Question 13

Why can’t we assign probabilities to exact values in continuous space?

Answer 13

Because the probability at any exact point is zero.

Question 14

What does a probability density function (PDF) represent?

Answer 14

How likely a value is to occur near a specific point in continuous space.

Question 15

How do we compute the probability of a range in continuous space?

Answer 15

By integrating the PDF over that interval.

Question 16

What is a histogram in probability estimation?

Answer 16

An approximation of a PDF by counting frequencies in intervals (bins).

Question 17

What is the formula for expectation in discrete space?

Answer 17

E[x] = sum of x * P(x)

Question 18

What does expectation represent intuitively?

Answer 18

The average or central tendency of a random variable.

Question 19

What is the formula for variance?

Answer 19

Var(x) = E[(x - E[x])²] or E[x²] - (E[x])²

Question 20

What does variance measure?

Answer 20

How spread out the values are around the mean.

Question 21

What does covariance tell us?

Answer 21

Whether two variables increase or decrease together.

Question 22

What is correlation?

Answer 22

A normalized measure of linear dependence between variables, ranging from -1 to 1.

Question 23

What does the Bernoulli distribution model?

Answer 23

A binary outcome with success probability π.

Question 24

What does the Poisson distribution model?

Answer 24

The count of events in a fixed interval of time or space.

Question 25

What shape does the Gaussian distribution have?

Answer 25

A symmetric bell curve centered at the mean.

Question 26

When do we use the uniform distribution?

Answer 26

When all outcomes in an interval are equally likely.

Question 27

What does the Law of Large Numbers say?

Answer 27

The sample mean converges to the true mean as the number of samples increases.

Question 28

What does the Central Limit Theorem say?

Answer 28

The distribution of sample means becomes normal as sample size increases, regardless of original distribution.

Question 29

Why is the Central Limit Theorem useful in ML?

Answer 29

It explains why averages and many model outputs are normally distributed.