Distributions

Question 1

What is the Bernoulli distribution?

Answer 1

Bernoulli(p) has pmf P(X = x|p) = p^x(1-p)^(1-x) where x is (usually) 0, 1 to denote two possible outcomes (success and failure)

Question 2

What is the Binomial distribution?

Answer 2

Binomial(n, p) gives the probability of x successes in n independent Bernoulli trials
pmf: P(X = x|n, p) = nCx * p^x(1-p)^(n-x)

Question 3

What is the Discrete Uniform distribution?

Answer 3

pmf: P(X = x|N) = 1/N where x = 1, 2, …, N

Question 4

What is the Geometric distribution?

Answer 4

pmf P(X = x|p) = p(1-p)^x-1 with x = 1, 2, … gives the probability of observing the first success after x independent Bernoulli trials (x - 1 fails and then a success)

Question 5

What is the Poisson distribution?

Answer 5

pmf of Poisson(λ) is P(X = x|λ) = (e^-λλx)/x! gives the probability of x occurrences of an independent outcome with mean rate of λ

Question 6

What are the higher moments of the Normal distribution?

Answer 6

Skewness = 0
Kurtosis = 3

Question 7

What is a moment?

Answer 7

A calculated (not observed) feature of a distribution

Question 8

What is the first moment of a data set?

Answer 8

The mean or expected value

Question 9

What is the expectation for a discrete random variable?

Answer 9

E(X) = Σx_iP(X = x_i)

Question 10

What is the expectation for a continuous random variable?

Answer 10

E(X) = ∫_-inf^infxf(x)dx

Question 11

What is the difference between a sample statistic and a population moment?

Answer 11

The former is a random variable, the latter is not, e.g. X̄ is a random variable but E(X) is not

Question 12

What is the expectation of rv X with pdf f when some function g is applied to it?

Answer 12

E(g(X)) = ∫_-inf^infg(x)f(x)dx

Question 13

What are non central and central moments?

Answer 13

E(X^k) is the kth non central moment of X, denoted μ_k’
E((X - μ)^k) is the kth central moment of X, denoted μ_k

Question 14

What is the variance of a random variable X as a moment?

Answer 14

The second central moment
Var(X) = σ_X² = ∫_-inf^inf(x - μ)²f(x)dx = E((X - E(X))²)

Question 15

What is the variance of the Bernoulli distribution?

Answer 15

Var(X) = p(1 - p)

Question 16

What is the covariance of two variables?

Answer 16

Cov(X, Y) = E[(X - E(X))(Y - E(Y))] = ∫_Y∫_X(x - μ_X)(y - μ_Yf(x, y)dxdy = E(XY) - E(X)E(Y)
‘Calculating the expected value of the product of the deviations of X and Y from their means, weighted by the joint pdf’

Question 17

What is the population correlation coefficient?

Answer 17

ρ = Corr(X, Y) = Cov(X, Y) / sqrt(Var(X)Var(Y))

Question 18

What does the independence of two variables say about their covariance?

Answer 18

Independence implies a covariance of 0 but a covariance of 0 does not imply independence

Question 19

What are the rules of expectation and variance for random variables?

Answer 19

For rvs X, Y and constants a, b
E(a) = a, E(a + bX) = a + bE(X), E(aX + bY) = aE(X) + bE(Y)
Var(a) = 0, Var(a + bX) = b²Var(X), Var(aX + bY) = a²Var(X) + b²Var(Y) + 2abCov(X, Y) (last term equals 0 for independent variables)

Question 20

What does a confidence interval tell you?

Answer 20

The likelihood of a moment falling within a certain range

Question 21

What are the 90% and 95% confidence intervals for a realisation of the standard normal distribution?

Answer 21

90%: [-1.645, 1.645]
95%: [-1.96, 1.96]

Question 22

How do you go from any normal distribution to the standard normal distribution?

Answer 22

P(X < a) = P((X - μ)/σ < (a - μ)/σ) = P(Z < (a - μ)/σ)