2P7 Mathematics

Question 1

What is a sample space?

Answer 1

The set of all possible outcomes in an experiment Ω

Question 2

What are the three axioms of probability?

Answer 2

P(A) >= 0
P(Ω) =1
P(A U B) = P(A) + P(B) for A,B C Ω and A ∩ B = ∅
where ∅ is the empty set

Question 3

How is the probability A given B defined?

Answer 3

P(A|B) = P(A∩B)/P(B)

Question 4

How are probability mass functions and cumulative distribution function defined for discrete random variables?

Answer 4

Px(x) = P(X=x)
Fx(x) = P(X<=x)

Question 5

How is the joint PMF defined for two discrete random variables variables?

Answer 5

P(x,y) = P( X=x ∩ Y=y )

Question 6

What is Baye’s rule?

Answer 6

P(B|A) = P(A|B)P(B)/P(A)

Question 6

What is marginilisation for joint discrete PMFs?

Answer 6

Px(x) = sum over y (P(x,y))

Question 6

What is the law of total probability?

Answer 6

P(A) = P(A|B)P(B) + P(A|B^c) P(B^c) where B^c is the complement of B

Question 7

Definition of independence of two events.

Answer 7

P(A∩B)=P(A)P(B)

Question 7

Define expectation

Answer 7

E(g(X)) = sum(g(x)Px(X)) for all x in X, where X is the support of X
It is linear.

Question 8

What is the expectation rule for two independent random variables?

Answer 8

E(XY) = E(X)E(Y) if X and Y are independent

Question 9

Define Var[x]

Answer 9

E[(X-E[X])^2] = E[x^2] - E[x]^2

Question 10

Define entropy for probability

Answer 10

H[X] = E[-log2P(X)]

Question 11

What is the Bernoulli Distribution?

Answer 11

A trial with binary output
X∈{0,1}, p ∈ [0,1]
X ~ Ber(p)
Pk = { p if k=1, 1-p if k=0, 0 otherwise}

Question 12

What is the geometric distribution used for?

Answer 12

How many trials until I’m successful. X ~ Geo(p)

Probability of success after x trials on that final trial.

Question 12

What is the formula for the geometric distrbution?

Answer 12

P(k) = p(1-p)^(k-1)

Question 13

What does the binomial distribution describe?

Answer 13

How many times was I succesful after n trials?

X~B(n,p) where n is the numper of trials, and p is the probability of successs.

Question 14

What does the poisson distribution describe?

Answer 14

Ho wmany times was I succesful given a success rate λ

X~Pois(λ),

λ is the average number of events/time interval

Px(k) is the probability that there are k of these events in the time interval.

Question 15

How does the Bernoulli distribution relate to the binomial?

Answer 15

Xj~Ber(p)

sum over j (Xj) ~ B(n,p)

Question 16

How does the binomial distribution relate to the poisson distribution?

Answer 16

B(n, λ/n) -> Pois(λ)

Question 17

What are some properties of the cumulative distrbution function F(x)?

Answer 17

Non decreasing,
Fx(a) < Fx(b)

Limits are 0 and 1 for -∞ and +∞ respectively

Fx(b) - Fx(a) = P (a<= X <= b)

Question 18

Definition of the joint CDF?

Answer 18

F(x,y) = P (X <= x ∩ Y<=y)

Question 19

What is the exponential density function?

Answer 19

X~Exp(λ) where λ is the rate of successes, and X is the distance between two successes.

Question 20

What is the beta density function?

Answer 20

The PDF of the trial probability if we observe α−1 successes and β−1 failures.

X∼Beta(α,β)

X∈[0,1] X is also a probability and has a probability distribution.

Question 21

If X, and Y are discrete random variables and Y =g(X) how do you extract Py(y)

Answer 21

Py(y) = sum(Px(x)) the sum over x | g(x)=y

Question 22

If X, and Y are continuous random variables and Y =g(X) how do you extract fy(y)

Answer 22

given g is strictly monotonic, f(y) = fx(g-1(y))/|g'(g-1(y))|

Question 23

What is the normalised Gaussian distribution?

Answer 23

(X-µ)/σ ~ N(0,1)

Question 24

How do you find the distribution of S=X+Y for discrete?

Answer 24

Ps(s) = sum over y ( P(s-y, y) = sum over x (P(x, s-x)) if X and Y are independent: P(s) = sum over y (Px(s-y) Py(y)) [convolution]

Question 25

How do you find the distribution of S=X+Y for continuous?

Answer 25

fs(s) = integral ( fxy(s-y, y)dy) for independent fs(s) = integral ( fx(s-y) fy(y) dy ) fs = fx*fy where * is convolution

Question 26

How do expectation and variance variance change with S = X+Y?

Answer 26

E[S] = E[X] + E[y] Var[s] = Var[X] + Var[Y] + 2Cov[X,Y] where COV[X,Y] = E[XY] - E[X]E[Y] if X and Y are indepedent Cov[X,Y] = 0

Question 27

What is correlation of two random variables?

Answer 27

ρ = Cov[X,Y]/sqrt(Var[X]Var[Y])

Question 28

What is the probability generating function?

Answer 28

Gx(z) = sum(z^k Px(k)) = E(Z^x)

Question 29

What's some properties of the probability generating function for discrete random variables?

Answer 29

E[X] = G'x(1) Var[X] = G''x(1) + G'x(1) - G'x(1)^2 if X and Y are independent G[X+Y](z) = GX(z) x GY(z)

Question 30

What's the moment generating function for continuous random variables?

Answer 30

gx(s) = integral(fx(x) e^sx ds) = E ( e^sx)

Question 31

What's some properties of moment generating function?

Answer 31

E[X] = g'(0) Var[X] = g''(0) - g'(0)^2 if X and Y are independent g[X+Y](s) = gx(s) x gy(s)

Question 32

What's the central limit theorem?

Answer 32

If you have a bunch of independent random variables distribution Gaussianly with means µi and variances σi^2 then [sum(Xi) - sum(µi )]/sqrt(sum(σi^2)) ~ N(0,1)

Question 33

What is the analogy for variance in a multivariate Gaussian?

Answer 33

Σ where Σij = E[(Xi-µi)(Xj-µj)]