Probability & Statistics

Question 1

Definition 1.1 (Sample Space)

Answer 1

The sample space is the set of all possible outcomes for the experiment. It is denoted by S

Question 2

Definition 1.2 (Event)

Answer 2

An event is a subset of the sample space. The even occurs if the actual outcome is an element of this subset.

Question 3

Definition 1.3 (Simple Event)

Answer 3

An event is a simple event if it consists of a single element of the sample space S

Question 4

Meaning of Disjoint events

Answer 4

We say two sets A and B are disjoint if they have no element in common, i.e, A ∩ B = {}

Question 5

De Morgan’s laws

Answer 5

(A ∪ B)c = Ac ∩ Bc
(A ∩ B)c = Ac ∪ Bc

Question 6

Definition 2.1 (Kolgromov’s axioms for probability)

Answer 6

a) For every event A we have P(A) >= 0,

b) P(S) = 1,

c) If A1, A2, …, An are n pairwise disjoint events
then
P(A1 U A2 U … U An) = P(A1) + P(A2) + … + P(An)

Question 7

Proposition 2.2

Answer 7

If A is an event then
P(Ac) = 1 - P(A)

Question 8

Corollary 2.3

Answer 8

P(∅) = 0

Question 9

Corollary 2.4

Answer 9

If A is an event then P(A) <= 1

Question 10

Proposition 2.5

Answer 10

If A and B are events and A ⊆ B then
P(A) <= P(B)

Question 11

Proposition 2.6

Answer 11

The probability of a finite set is the sum of the probabilities of the corresponding simple events.

Question 12

Proposition 2.7 (inclusion-exclusion for two events)

Answer 12

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

Question 13

Proposition 2.8 (Inclusion-exclusion for three events)

Answer 13

P(A∪B∪C) = P(A)+P(B)+P(C)−P(A∩B)−P(A∩C)−P(B∩C)+P(A∩B∩C)

Question 14

Theorem 3.1(a) Ordered with replacement (repetition allowed)

Answer 14

n^r

Question 15

Theorem 3.1 (b) Ordered without replacement (no repetition)

Answer 15

n!
(n−r)!

Question 16

Defenition 4.1 (Conditional Probability)

Answer 16

If E1 and E2 are events and P(E1) 6= 0 then the conditional probability of E2 given E1, usually denoted by P(E2|E1), is

P(E2|E1) = P(E1 ∩ E2)
P(E1)

Question 17

Theorem 3.1 (c) Unordered without replacement (no repetition)

Answer 17

nCr (n)
(r)

Question 18

Defenition 4.2(independence)

Answer 18

We say that the events E1 and E2 are (pairwise) independent if

P(E1 ∩ E2) = P(E1)P(E2)

Question 19

Definition 4.4 a). Three events E1, E2, and E3 are called pairwise independent
if:

Answer 19

P(E1 ∩ E2) = P(E1)P(E2),
P(E1 ∩ E3) = P(E1)P(E3),
P(E2 ∩ E3) = P(E2)P(E3).

Question 20

Definition 4.4 b) Three events E1, E2, and E3 are called mutually independent
if:

Answer 20

P(E1 ∩ E2 ∩ E3) = P(E1)P(E2)P(E3)

Question 21

Definition 4.6. Two events E1 and E2 are said to be conditionally independent given an event E3 if:

Answer 21

P(E1 ∩ E2|E3) = P(E1|E3)P(E2|E3)

Question 22

Defenition 5.1 (Random Variable)

Answer 22

A random variable is a function from S to R

Question 23

Definition 5.2 (Discrete Random Variables)

Answer 23

A random variable X is discrete if the set of values that X takes
is either finite or countably infinite.

Question 24

Definition 5.3 (Probability Mass Function)

Answer 24

The probability mass function (p.m.f.) of a discrete random
variable X is the function which given input x has output P(X = x)

Question 25

Proposition 5.4 If X is a discrete random variable which takes values x1, x2, x3, . . ., then

Answer 25

The sum of the Outputs must equal 1

Question 26

Definition 6.1 Expectation

Answer 26

If X is a discrete random variable which takes values x1, x2, x3, . . ., then the expectation of X (or the expected value of X) is defined by
E(X) = x1P(X = x1) + x2P(X = x2) + x3P(X = x3) + · · · .

Question 27

Proposition 6.2 If m ≤ X(ω) ≤ M for all ω ∈ S, then

Answer 27

m ≤ E(X) ≤ M

Question 28

Proposition 6.3 (Expectation of a function)

Answer 28

E( f(X) ) = f(x1)P(X = x1) + f(x2)P(X = x2) + f(x3)P(X = x3) + · · ·

Question 29

Definition 6.4 (Moments)

Answer 29

The nth moment of the random variable X is the expectation E(X^n)

Question 30

Definition 6.5 (Variance)

Answer 30

Var(X) = [x1 − E(X)]2P(X = x1) + [x2 − E(X)]2P(X = x2)
+ [x3 − E(X)]2P(X = x3) + …

Question 31

Proposition 6.6( Variance formula)

Answer 31

Var(X) = E(X^2) − [E(X)]^2

Question 32

Proposition 6.7 (Linear function of expectation)

Answer 32

E(aX + b) = aE(X) + b

Question 33

Proposition 6.8 (Linear function of variancec)

Answer 33

Var(aX + b) = a^2Var(X)

Question 34

What is a Bernoulli(p) distribution:

Answer 34

It is where a random variable X only takes values 0 and 1

Question 35

Bernoulli distribution Expectation and Variance

Answer 35

E(X) = p, Var(X) = p(1 − p)

Question 36

What is Binomial distribution:

Answer 36

A discrete random variable X has the Binomial(n, p) distribution, denoted X ∼ Bin(n, p), if its p.m.f. is :

P(x =k) = nCk x p^k x (1-p)^n-k

Question 37

Binomial distribution Expectation and Variance

Answer 37

E(X) = np, Var(X) = np(1 − p)

Question 38

What is Geometric distribution:

Answer 38

A discrete random variable X has the Geometric(p) distribution, denoted X ∼ Geom(p), if its p.m.f. is :

P(X = k) = p(1 − p)^k−1

Question 39

Geometric distribution Expectation and Variance

Answer 39

E(X) = 1/p
Var(X) = 1 − p/ p^2

Question 40

What is Hypergeometric distribution

Answer 40

A bag contains n balls, m white balls and n − m black balls. You pick l balls at
random without replacement. Let X denote the number of white balls you pick.

P(X = k) = mCk x (n-m)C(l-k) / nCl

Question 41

Hypergeometric distribution Expectation and Variance

Answer 41

E(X) = l x (m/n)
Var(X) = l x (m/n) x (n-m/n) x (n-l/n-1)

Question 42

What is Negative binomial distribution

Answer 42

Consider a sequence of independent Bernoulli(p) trials. Given a fixed integer r denote by X the random variable which counts the number of failures when we have seen in total r successes (for the first time).

P(X = k) = (k+r-1)C(r-1) x p^r x (1-p)^k

Question 43

Negative Binomial distribution Expectation and Variance

Answer 43

E(T) = (1-p)r/p
Var(T) = (1-p)r/p^2

Question 44

What is Uniform distribution

Answer 44

uniform distribution refers to a type of probability distribution in which all outcomes are equally likely

Question 45

(discrete) Uniform distribution Expectation and Variance

Answer 45

where n = b-a, and m=a
E(X) = m +(n/2)
Variance = n(n+2)/12

Question 46

What is Poisson distribution

Answer 46

Consider a random variable X which takes values k = 0, 1, 2, 3, . . .. Denote by λ > 0 a fixed positive real number.

P(X=k) = (λ^k/k!) x e^-λ

Question 47

Poisson distribution Expectation and Variance

Answer 47

E(X) = λ, Var(X) = λ

Question 48

Cumulative distribution function

Answer 48

The cumulative distribution function (c.d.f.) of a random variable X is the function which given t has output
P(X ≤ t).

Question 49

Moment Generating function

Answer 49

Let X be a discrete random variable
which takes integer values. The moment generating function (mgf ) of X is the function which given t has output E(e^(tX))

Question 50

Definition 3 (Continuous random variables)

Answer 50

We say that a random variable X is a continuous random variable if there exists a
continuous function fx from R to [0, ∞) with the following property:

P(a<= X <= b) = integral of fx(t)

Question 51

Definition 4 (Expectation and Variance of crv)

Answer 51

E(X) = integral tfx(dt)
Var(X) = E(X^2) − (E(X))^2

Question 52

(continuous) Uniform Expectation and Variance

Answer 52

E(X) = (a+b)/2
Var(X) = (b-a)^2/12

Question 53

Exponential distribution

Answer 53

P(X=t) = λe^λt

Question 54

Exponential distribution Expectation and Variance

Answer 54

E(X) = 1/λ
Var(X) = 1/λ^2

Question 55

Joint Probability mass function

Answer 55

Let X and Y be two discrete random variables defined on the same sample space and taking values x1, x2, . . . and y1, y2, . . . respectively. The
function:

(xk, yl) → P( (X = xk) ∩ (Y = yl) )

Question 56

Proposition 10.2 Marginal Probability

Answer 56

P(X=xk) = suml P(X = xk, Y = yl)

same goes the other way

the idea is that if we only care about the probability of X
taking a particular value, we need to sum over all possible values of Y

Question 57

Proposition 10.3

Answer 57

If g(X, Y ) is a real-valued function of the two discrete random variables X and Y then the expectation of g(X, Y ) is obtained as

E( g(X, Y ) ) = sumk suml g(xk, yl)P(X = xk, Y = yl)

Question 58

Theorem 10.4

Answer 58

If X and Y are discrete random variables then

E(X + Y ) = E(X) + E(Y )

Question 59

Independence for Random Variables

Answer 59

Two discrete random variables X and Y are independent if
the events “X = xk” and “Y = yl” are independent for all possible values xk, yl

Question 60

Covariance of X, Y

Answer 60

The covariance of X and Y is defined by:

Cov(X, Y ) = E( [X − E(X)][Y − E(Y )] ).

Question 61

Correlation coefficient of X and Y

Answer 61

If Var(X) > 0 and Var(Y ) > 0:

Corr(X, Y ) = Cov(X,Y) /
sqrt(Var(X)Var(Y)

Question 62

Formula for Covariance (easy)

Answer 62

Cov(X, Y ) = E(XY ) − E(X)E(Y )