MTH2006 STATISTIC MODELLING AND INFERENCE

Question 1

cumulative distribution function (cdf) of a random variable Y

Answer 1

F_Y(y) = Pr(Y < y) where y belongs to the range space of Y

Question 2

probability mass function (pmf) [if Y is discrete]

Answer 2

f_Y(Y) = Pr(Y = y) and F_Y(y) = x:x

Question 3

probability density function (pdf) [if Y is continuous]

Answer 3

f_Y(y) = d/dy F_Y(y) and F_Y(y) = integral(y -> −∞)(f_Y(x)) dx

Question 4

p-quantile of a random variable Y

Answer 4

the value y_p for which Pr(y ≤ y_p) = p

Question 5

Pr(Y > y)

Answer 5

1 − Pr(Y ≤ y)

Question 6

joint cumulative distribution function (cdf) of a vector Y1,…Yn

Answer 6

F_Y(y1, . . . , yn) = Pr(Y1 ≤ y1, . . . , Yn ≤ yn)

Question 7

If Y1, . . . , Yn are discrete then their joint pmf is defined by

Answer 7

f_Y(y1, . . . , yn) = Pr(Y1 = y1, . . . , Yn = yn)

Question 8

If Y1, . . . , Yn are continuous then their joint pdf is defined by

Answer 8

fY (y1, . . . , yn) = ∂^n/∂y_1 . . . ∂y_n F_Y (y1, . . . , yn)

Question 9

Y1, . . . , Yn are independent if

Answer 9

f_Y (y1, . . . , yn) = f_Y1

(y1). . . fY_n(yn) for all y1,…,yn

Question 10

Y1, . . . , Yn are identically distributed if

Answer 10

f_Y1(y) = . . . = f_Yn(y) for all y1,…yn

Question 11

if Y1, . . . , Yn are independent and identically distributed (iid) then their joint pdf or pmf is

Answer 11

f_Y(y1, . . . , yn) = f_Y1(y1). . . f_Y1(yn)

Question 12

explanatory variable

Answer 12

plotted on the x-axis and is the variable manipulated by the researcher

Question 13

response variable

Answer 13

plotted on the y-axis and depends on the other variables

Question 14

if Y has a poisson distribution with parameter µ, then we write Y~Poi(µ) and Y has pmf

Answer 14

f_Y(y) = µ^ye^-µ / y! for y = 0,1,2…

Question 15

if Y has a exponential distribution with parameter θ, then we write Y~Exp(θ) and Y has cdf

Answer 15

F_Y(y; θ) = 1 - e^-θy for y > 0

Question 16

if Y has a exponential distribution with parameter θ, then we write Y~Exp(θ) and Y has pdf

Answer 16

f_y(y; θ) = d/dy F_Y(y; θ) = θe^-θy

Question 17

for p-quantile cdf is

Answer 17

F_Y(y_p) = p and y_p = F_Y^-1(p)

Question 18

expectation is

Answer 18

E(g(Y)) = sum(Pr(Y=x)g(x) = sum(F(x)g(X) where F(x) is the pmf

Question 19

variance of random variable Y is

Answer 19

Var(Y) = E(Y - E(Y)^2) = E(Y^2) - E(Y)^2

Question 20

empirical probability r/n is

Answer 20

r/n = Pr(X ≤ x_r)

Question 21

simple linear model means

Answer 21

one explanatory variable

Question 22

an example of a joint distribution for two variables is the…

Answer 22

bivariate normal distribution

Question 23

if X and Y are independent

Answer 23

f(x,y) = f_x(x)f_y(y)

Question 24

covariance formula:

Answer 24

Cov(X, Y) = E[(X - E(X))(Y - E(Y))] = E(XY) = E(X)E(Y)

Question 25

if independent, covariance formula:

Answer 25

Cov(X, Y) = 0 | E(XY) = E(X)E(Y)

Question 26

covariance with correlation/variance formula:

Answer 26

Cov(p*sqrt[Var(X)Var(Y)])

Question 27

an example of a joint distribution for two variables, each with a normal distribution is called

Answer 27

the bivariate normal distribution

Question 28

the joint pdf of the bivariate normal distribution

Answer 28

f(x, y; θ) = 1 / 2πσXσY sqrt(1 − ρ^2) * exp(−1/2(1 − ρ^2)[(x − µX)^2/σX^2 + (y − µY)^2/σY^2 − 2ρ(x − µX)(y − µY )/σXσY

Question 29

a continuous random variable Y defined on | (−∞,∞) with pdf f(y; θ) has expectation denoted by E(Y) and defined as..

Answer 29

E(Y ) = integral(∞ -> −∞) y * f(y;θ) dy

Question 30

a discrete random variable with range space R and pmf f(y; θ), E(Y) is defined as...

Answer 30

E(Y ) = sum(y∈R) [y * f(y; θ)]

Question 31

for a real valued function g(Y), when continuous, E[g(Y)] is

Answer 31

integral(∞ -> −∞) g(y) * f(y; θ)dy

Question 32

for a real valued function g(Y), when discrete, E[g(Y)] is

Answer 32

sum(y ∈ R) g(y) * f(y; θ)

Question 33

α-confidence interval is ...

Answer 33

an interval | estimator that contains the true parameter value θ with probability α for every θ

Question 34

null hypothesis

Answer 34

H_0 : θ = x (this is also a simple hypothesis = completely species a probability model by specifying a specific value)

Question 35

alternative hypothesis

Answer 35

H_1 : θ NOT= x | this is also a composite hypothesis = it does not completely specify a probability model

Question 36

if H_1 : θ NOT= x; which specifies values either side of H_0 it is called

Answer 36

a two-sided alternative

Question 37

if H_1 : θ < x it is called a

Answer 37

one-sided alternative

Question 38

for a null hypothesis H_0 : θ = θ_0, the null distribution is the distribution of T(Y) when ...

Answer 38

θ = θ_0

Question 39

let f_Y(y; θ) be continuous and denote the joint pdf of Y then Pr(Y ∈ C)

Answer 39

integral(C) f_Y (y; θ), if f(y; θ) is continuous

Question 40

let f_Y(y; θ) be discrete and denote the joint pmf of Y then Pr(Y ∈ C)

Answer 40

sum(y ∈ C) f_Y (y; θ), if f(y; θ) is discrete

Question 41

the size of the test α

Answer 41

α = Pr(Y ∈ C; θ_0)

Question 42

probability of a type I error

Answer 42

is to reject H_0 when it is true

Question 43

probability of a type II error

Answer 43

is to reject H_0 when it is false

Question 44

if the alternative hypothesis is simple, then the power of the test is

Answer 44

Pr(Y ∈ C; θ_1) = the probability of not making a type II error/detecting that H_0 is false

Question 45

for a set y1, ..., yn, the sample moments are

Answer 45

m^r = 1/n sum(i = 1 -> n) y^r_i

Question 46

for a continuous or discrete random variable Y, the moment generating function (mgf) of Y is

Answer 46

M_Y(t) = E(e^(tY))

Question 47

for the kth moment of Y it is

Answer 47

E(Y^k) = m_k

Question 48

the central moments of Y are

Answer 48

E[{Y − E(Y)}^r]

Question 49

the method of moment estimate , ˆθ, is such | that ...

Answer 49

m_r(ˆθ) = ˆmr for r = 1, . . . , d

Question 50

for sample variance we will use

Answer 50

s^2 = (n − 1)^(−1) * sum(i=1(yi − yBAR)^2)

Question 51

the distribution of the different estimates is called the

Answer 51

sampling distribution

Question 52

the standard deviation of the estimated sampling distribution gives the

Answer 52

estimated standard error

Question 53

for the data yBAR = (y1,...,yn) we have an estimate

Answer 53

ˆθ(y)

Question 54

for the data yBAR = (y1,...,yn) we have an estimator

Answer 54

ˆθ(Y)

Question 55

for critical region C = {y : T(y) < c}, the p-value is

Answer 55

p = Pr[T(Y ) ≤ t; θ0]

Question 56

for critical region C = {y : T(y) > c}, the p-value is

Answer 56

p = Pr[T(Y ) ≥ t; θ0]

Question 57

the null distribution of the t-statistic is defined as

Answer 57

T(Y) = YBAR − µ0 / s/√n (which is called the t-distribution with n-1 degrees of freedom)

Question 58

the t-statistic has the cdf which is denoted as

Answer 58

Φ_n−1(y)

Question 59

the sample variance of Φ_n−1(y) is

Answer 59

s^2 = 1 / n − 1 sum(n -> i=1) (Yi − YBAR)^2 is an unbiased estimator of σ^2

Question 60

critical region

Answer 60

when {C : T > tc} where tc is the critical value

Question 61

power of the test is when the alternative hypothesis is simple and is the probability of ...

Answer 61

not making a type II error or the probability of detecting that H_0 is false

Question 62

the p-value is the probability that ...

Answer 62

the observed test statistic is no better than the value we observed with respect to H_0 when H_0 is true

Question 63

sum(n -> i = m) (cx_i) =

Answer 63

c ^(n−m+1) sum(n -> i = m) x_i | .

Question 64

sum(n -> i = m) x^c_i =

Answer 64

[sum(n -> i = m) x_i ]^c

Question 65

sum(n -> i = m) c^(x_i) =

Answer 65

c^(sum(n -> i = m) x_i

Question 66

a sample, y = (y1, . . . , yn), modelled as a realisation of independent random variables, Y = (Y1, . . . , Yn). For i = 1, . . . , n, let f_Yi (y; θ) denote the pmf or pdf of Yi, where θ is the model parameter. The joint pmf or pdf of Y evaluated at our sample y is then fY (y; θ) =

Answer 66

= fY_1(y1; θ). . . fY_n(yn; θ) = sum(n -> i = 1) f_Yi(y_i; θ) by independence

Question 67

the joint pmf or pdf as a function of θ it is referred | to as the likelihood function and is denoted ...

Answer 67

L(θ; y) = f_Y (y; θ)

Question 68

the parameter value that maximizes the likelihood is called the ...

Answer 68

maximum likelihood estimate (mle)

Question 69

it is usually simpler to maximize the logarithm of the likelihood instead - the log-likelihood is denoted ...

Answer 69

l(θ; y) = log L(θ; y) = l(θ)

Question 70

a constant will not affect the shape of the likelihood - true or false..

Answer 70

true (and therefore will affect the shape of the maximum - we can ignore multipliers when we calculate the mle)

Question 71

mean squared error

Answer 71

Let ˆθ be an estimator for θ. The mean squared error of ˆθ is mse(ˆθ) = E{(ˆθ −θ)^2} and the bias of ˆθ is Bias(ˆθ) = E(ˆθ) − θ. If the bias is zero then the estimator is unbiased.

Question 72

mean squared error can be written in terms of bias and variance:

Answer 72

mse(ˆθ) = Var(ˆθ) + Bias(ˆθ)^2

Question 73

a consistent estimator

Answer 73

The estimator ˆθ is consistent for θ if, for all e > 0, lim(n→∞)Pr(|ˆθ − θ| > e) = 0. [this is the asymptotic limit)

Question 74

Only one parameter in the model, the mle is ...

Answer 74

scalar and its approximate sampling distribution will be a univariate normal distribution

Question 75

More than one parameter in the model, the mle is ...

Answer 75

a vector and its sampling distribution will be a multivariate normal distribution

Question 76

expectation of vector of random variables with ith element Yi

Answer 76

Let Y be a vector of random variables with ith element Yi. The expectation of Y is the vector with ith element E(Yi).

Question 77

variance of vector of random variables with ith element Yi

Answer 77

The variance of Y is the matrix with (i, j)th element Cov(Yi, Yj ). This can be written as Var(Y ) = E[{Y − E(Y )}{Y − E(Y )}^T] = E(YY^T)) − E(Y)E(Y)^T.

Question 78

observed information J(θ)

Answer 78

``` Let l(θ) be a log-likelihood function. The observed information is J(θ) = −∂^2 l(θ) / ∂θ ∂θ^T. ```

Question 79

If θ is a scalar then, J(θ) is

Answer 79

-d^2 l(θ) / dθ^2

Question 80

If θ is a vector with ith element θi then, J(θ) is a matrix with (i, j)th element,

Answer 80

-∂^2 l(θ) / ∂θi∂θj

Question 81

expected information is I(θ) = E{J(θ)}, that is the matrix with (i, j)th element,

Answer 81

E{−∂^2 l(θ) / ∂θi∂θj}

Question 82

multivariate normal distribution

Answer 82

The random variable Y = (Y1, . . . , Yd) has a multivariate normal distribution with expectation E(Y) = µ and variance Var(Y) = Σ if the pdf of Y is f_Y (y; µ, Σ) = (2π)^(−d/2)det(Σ)^(−1/2)exp[−(1/2)(y − µ)^(T)Σ^(−1)(y − µ)], in which case we write Y∼N(µ,Σ).

Question 83

useful properties of multivariate normal distribution:

Answer 83

``` if two random variables are independent then they are uncorrelated because independence implies that E(Y1Y2) = E(Y1)E(Y2) and so Cov(Y1, Y2) = E(Y1Y2) − E(Y1)E(Y2) = 0. also linear transformations of multivariate normal random variables are also multivariate normal ```

Question 84

THEOREM: When n is large, the sampling distribution of the mle is approximately N(θ, I(θ)^(−1)). This is called the asymptotic distribution of the mle.

Answer 84

If n is large then the square roots of the diagonal elements of I(θ)^−1 approximate the standard errors of the mles in ˆθ. These standard errors can be estimated by replacing θ with ˆθ in I(θ)^−1.

Question 85

likelihood ratio test statistic

Answer 85

T = 2 { l(ˆθ; y) − l(θ_0; y) }