Chapter 1 Info

Question 3

What is the sample space in probability?

Answer 3

The set of all possible outcomes of an experiment, denoted by Ω.

Question 4

What is an event in probability?

Answer 4

A subset of the sample space Ω, representing a collection of outcomes.

Question 5

What are Kolmogorov’s axioms of probability?

Answer 5

1. P(A) ≥ 0 for any event A. 2. P(Ω) = 1. 3. Countable additivity: P(∪A_i) = ΣP(A_i) for mutually disjoint events A_i.

Question 6

When are two events A and B independent?

Answer 6

When P(A ∩ B) = P(A)P(B).

Question 7

What is conditional probability P(A|B)?

Answer 7

P(A|B) = P(A ∩ B) / P(B), provided P(B) > 0.

Question 8

What is the law of total probability?

Answer 8

P(A) = ΣP(A|B_j)P(B_j), where B_j are mutually exclusive and exhaustive events.

Question 9

State Bayes’ Theorem.

Answer 9

P(B_i|A) = P(A|B_i)P(B_i) / ΣP(A|B_j)P(B_j), where B_j are mutually exclusive and exhaustive.

Question 10

What is a random variable?

Answer 10

A function X: Ω → ℝ that assigns a real number to each outcome in the sample space.

Question 11

What is the cumulative distribution function (CDF) of a random variable X?

Answer 11

F_X(x) = P(X ≤ x). Properties: F_X(-∞) = 0, F_X(∞) = 1, and F_X is right-continuous and non-decreasing.

Question 12

What is the probability mass function (PMF) for a discrete random variable?

Answer 12

p_X(x_i) = P(X = x_i), where x_i are the possible values of X.

Question 13

What is the probability density function (PDF) for a continuous random variable?

Answer 13

The PDF is a function that describes the likelihood of a continuous random variable taking a specific value or falling within a range. It must satisfy:

1. f(x) >= 0 for all x (non-negative).
2. The total area under the curve is 1:
   Integral of f(x) from -ve infinity to infinity is 1
3. The probability of the variable lying between two points a and b is integral from b to a of f(x) dx

Question 14

What is the expectation of a discrete random variable X?

Answer 14

E[X] = Σx_i p_X(x_i). For continuous X: E[X] = ∫_{-∞}^∞ x f_X(x) dx.

Question 15

What is the variance of a random variable X?

Answer 15

Var(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2.

Question 16

What is the covariance between two random variables X and Y?

Answer 16

Cov(X, Y) = E[(X - E[X])(Y - E[Y])] = E[XY] - E[X]E[Y].

Question 17

What is the correlation between X and Y?

Answer 17

Corr(X, Y) = Cov(X, Y) / √(Var(X)Var(Y)), with -1 ≤ Corr(X, Y) ≤ 1.

Question 18

What is the joint CDF of two random variables X and Y?

Answer 18

F_{X,Y}(x, y) = P(X ≤ x, Y ≤ y).

Question 19

What is the joint PMF for discrete X and Y?

Answer 19

p_{X,Y}(x_i, y_j) = P(X = x_i, Y = y_j).

Question 20

What is the joint PDF for continuous X and Y?

Answer 20

f_{X,Y}(x, y) such that F_{X,Y}(x, y) = ∫{-∞}^x ∫{-∞}^y f_{X,Y}(u, v) dv du.

Question 21

When are X and Y independent?

Answer 21

When p_{X,Y}(x, y) = p_X(x)p_Y(y) (discrete) or f_{X,Y}(x, y) = f_X(x)f_Y(y) (continuous).

Question 22

What is the conditional PMF of X given Y = y_j?

Answer 22

p_{X|Y}(x_i|y_j) = p_{X,Y}(x_i, y_j) / p_Y(y_j).

Question 23

What is the conditional PDF of X given Y = y?

Answer 23

f_{X|Y}(x|y) = f_{X,Y}(x, y) / f_Y(y).

Question 24

What is the conditional expectation E[X|Y = y]?

Answer 24

For discrete X: E[X|Y = y] = Σx_i p_{X|Y}(x_i|y). For continuous X: E[X|Y = y] = ∫x f_{X|Y}(x|y) dx.

Question 25

State the law of iterated expectations.

Answer 25

E[X] = E[E[X|Y]], where the outer expectation is over Y.

Question 26

What is the multinomial distribution?

Answer 26

A generalization of the binomial distribution for m+1 categories, with PMF P(N_1 = n_1, ..., N_m = n_m) = n!/(n_1!...n_{m+1}!) p_1^{n_1}...p_{m+1}^{n_{m+1}}.

Question 27

What is the bivariate normal distribution?

Answer 27

A joint distribution for X and Y with PDF f_{X,Y}(x, y) = (1/(2πσ_Xσ_Y√(1-ρ^2))) exp[-1/(2(1-ρ^2)){(x-μ_X)^2/σ_X^2 - 2ρ(x-μ_X)(y-μ_Y)/(σ_Xσ_Y) + (y-μ_Y)^2/σ_Y^2}].

Question 28

What are the marginal distributions of a bivariate normal?

Answer 28

X ~ N(μ_X, σ_X^2) and Y ~ N(μ_Y, σ_Y^2).

Question 29

What is the conditional distribution of X given Y = y in a bivariate normal?

Answer 29

X|Y=y ~ N(μ_X + ρ(σ_X/σ_Y)(y - μ_Y), σ_X^2(1 - ρ^2)).

Question 30

What is the covariance matrix for a random vector X?

Answer 30

Σ = Cov(X) with entries Σ_{i,j} = Cov(X_i, X_j). It is symmetric and positive semi-definite.

Question 31

What is the PDF of a multivariate normal distribution?

Answer 31

f_X(x) = (1/((2π)^(k/2) |Σ|^(1/2))) exp[-1/2 (x - μ)^T Σ^{-1} (x - μ)], where k is the dimension of X.

Question 32

What is the iterated conditional variance formula?

Answer 32

Var(X) = E[Var(X|Y)] + Var(E[X|Y]).