Chapter 3

Question 1

define the likelihood function L(theta) for single observation

Answer 1

For a given observation x, we call theta->L(theta;x)= p(x;theta) for x discrete and f(x;theta) for x continuous

Question 2

define the maximum likelihood estimate of theta

Answer 2

the mle is the value of theta that maximises the likelihood function L(theta;x)

Question 3

for a single observation how do we find theta hat mle

Answer 3

for x=x1 we have L(theta;x1)=p(x1;theta) or =f(x1;theta). we then maximise L(theta;x1) by differentiating and setting equal to 0. Remember theta E (0,1). Then show it is a maximum

Question 4

for multiple independent observations how do we find theta hat mle

Answer 4

set L(theta)=L(theta;x1,x2,...,xn)= Px1,x,...,xn(x1,x2,...,xn)= Px1(x1;theta)Px2(x2;theta)...Pxn(xn;theta)
then differentiate ands et equal to 0 and solve for theta

Question 5

define likelihood function for multiple observations

Answer 5

Assume the data x1,..,xn are the observed values of random variables X1,...Xn whose joint distribution depends on one or more
unknown parameters θ. The likelihood function L(θ) ≡ L(θ; x1, x2, . . . , xn) is
the joint probability mass function (discrete case) or joint probability density
function (continuous case) regarded as a function of the unknown parameter θ
for these fixed numerical values of x1, x2, . . . , xn.

Question 6

define mle for multiple observations

Answer 6

For observed values {x1, . . . , xn}, the maximum likelihood estimator
(mle) θ hat mle(x1, . . . xn) is the value of θ which maximises the likelihood
function L(θ; x1, . . . , xn).

Question 7

define log-likelihood function

Answer 7

For observed values {x1, . . . , xn} and associated likelihood
function L(θ) ≡ L(θ; x1, x2, . . . , xn), the log-likelihood function is defined as
l(θ) := log L(θ), where log is the natural logarithm (and we take log 0 = −∞)

Question 8

likelihood function for simple random sample

Answer 8

If X1, X2, . . . , Xn, is a random sample of size n from a distribution
with probability mass function p (x; θ) (or probability density function
f(x; θ)) then the Xi are i.i.d. and their joint distribution factorises into the
product of marginals. Thus for a random sample
L(θ) ≡ L(θ; x1, x2, . . . , xn) =
p(x1; θ) p(x2; θ)· · · p(xn; θ) (discrete scenario)
f(x1; θ) f(x2; θ)· · · f(xn; θ) (continuous scenario

Question 9

for observations taken froma simple random sample what does the log-likelihood function=

Answer 9

(sum
i=1-n log p(xi; θ). (discrete scenario)

sum
i=1-n log f(xi; θ). (continuous scenario)

Question 10

what is the likelihood equation

Answer 10

∂/∂θ l(θ) = sum i=1-n ∂/∂θ log f(xi; θ) = 0

Question 11

procedure of calculating theta hat mle in the random sample case

Answer 11

1. Calculate ∂/∂θlogf(x;θ)
2. compute the sum ∂/∂θlogf(x;θ)
3. set sum =0 and θhat mle is the value satisfying the likelihood equation

Question 12

the invariable property of mle

Answer 12

if the quantity of interest is a function t(θ) of θ the mle of t(θ) is the plug-in estimate t(θ)hat=t(θhat)

Question 13

procedure of calculating mle for multiple parameters alpha and beta

Answer 13

for two parameters α and β, the αhat mle and βhat mle are the simultaneous solutions to the two likelihood equations
0 = sum i=1-n ∂/∂αlog f(xi; α, β)
and
0 = sum i=1-n ∂/∂β log f(xi; α, β).

Question 14

if the density is not regular how to work out mle

Answer 14

the likelihood can be maximise at one endpoint of the interval so find Lθ and differentiate or see if it is a decreasing/increasing function.