Flashcards in Chapter 3 Deck (14):
1
define the likelihood function L(theta) for single observation
For a given observation x, we call theta->L(theta;x)= p(x;theta) for x discrete and f(x;theta) for x continuous
2
define the maximum likelihood estimate of theta
the mle is the value of theta that maximises the likelihood function L(theta;x)
3
for a single observation how do we find theta hat mle
for x=x1 we have L(theta;x1)=p(x1;theta) or =f(x1;theta). we then maximise L(theta;x1) by differentiating and setting equal to 0. Remember theta E (0,1). Then show it is a maximum
4
for multiple independent observations how do we find theta hat mle
set L(theta)=L(theta;x1,x2,...,xn)= Px1,x,...,xn(x1,x2,...,xn)= Px1(x1;theta)Px2(x2;theta)...Pxn(xn;theta)
then differentiate ands et equal to 0 and solve for theta
5
define likelihood function for multiple observations
Assume the data x1,..,xn are the observed values of random variables X1,...Xn whose joint distribution depends on one or more
unknown parameters θ. The likelihood function L(θ) ≡ L(θ; x1, x2, . . . , xn) is
the joint probability mass function (discrete case) or joint probability density
function (continuous case) regarded as a function of the unknown parameter θ
for these fixed numerical values of x1, x2, . . . , xn.
6
define mle for multiple observations
For observed values {x1, . . . , xn}, the maximum likelihood estimator
(mle) θ hat mle(x1, . . . xn) is the value of θ which maximises the likelihood
function L(θ; x1, . . . , xn).
7
define log-likelihood function
For observed values {x1, . . . , xn} and associated likelihood
function L(θ) ≡ L(θ; x1, x2, . . . , xn), the log-likelihood function is defined as
l(θ) := log L(θ), where log is the natural logarithm (and we take log 0 = −∞)
8
likelihood function for simple random sample
If X1, X2, . . . , Xn, is a random sample of size n from a distribution
with probability mass function p (x; θ) (or probability density function
f(x; θ)) then the Xi are i.i.d. and their joint distribution factorises into the
product of marginals. Thus for a random sample
L(θ) ≡ L(θ; x1, x2, . . . , xn) =
p(x1; θ) p(x2; θ)· · · p(xn; θ) (discrete scenario)
f(x1; θ) f(x2; θ)· · · f(xn; θ) (continuous scenario
9
for observations taken froma simple random sample what does the log-likelihood function=
(sum
i=1-n log p(xi; θ). (discrete scenario)
sum
i=1-n log f(xi; θ). (continuous scenario)
10
what is the likelihood equation
∂/∂θ l(θ) = sum i=1-n ∂/∂θ log f(xi; θ) = 0
11
procedure of calculating theta hat mle in the random sample case
1.Calculate ∂/∂θlogf(x;θ)
2. compute the sum ∂/∂θlogf(x;θ)
3.set sum =0 and θhat mle is the value satisfying the likelihood equation
12
the invariable property of mle
if the quantity of interest is a function t(θ) of θ the mle of t(θ) is the plug-in estimate t(θ)hat=t(θhat)
13
procedure of calculating mle for multiple parameters alpha and beta
for two parameters α and β, the αhat mle and βhat mle are the simultaneous solutions to the two likelihood equations
0 = sum i=1-n ∂/∂αlog f(xi; α, β)
and
0 = sum i=1-n ∂/∂β log f(xi; α, β).
14