MLE

Question 1

Maximum Likelihood Estimation MLE principle:

Answer 1

1: Assume the distribution of the data (yi) from the parametric family of the distribution functions. yi=h(xi, εi, β) and εi~F
2: Determine the likelihood of observing the sample as a function of the unknown parameters
3: Choose the MLEs as the values of the unknown parameters corresponding to the highest likelihood

Question 2

When is MLE identified? (Global identification)

Answer 2

E(ln(Li(θ)))<E(ln(Li(θ0))) for any θ≠θ0
Note: θ is identified if f(yi,xi;θ)≠f(yi,xi,θ0)

Question 3

When is the MLE θn^ consistent?

Answer 3

1: paramater space ϴ⊆R^p is convex and θ0∈ϴ0
2: ln(Li(θ)) is concave on ϴ for any wi, uniformly strictly at θ0
3: ln(Li(θ)) is continuous in θ for any wi
4: wi are i.i.d.
5: E(ln(Li(θ))) exists for θ∈ϴ
6: E(ln(Li(θ0)))>E(ln(Li(θ))) for θ≠θ0

Question 4

There exists asymptotic normality for MLE (√n(θn^-θ0)–>N(0,H^-1IH^-1) when:

Additionally, if E(gi(θ0)|xi)=0 holds, then I=-H and V=I^-1

Answer 4

1: the first three derivatives of ln(Li(θ))=ln(f(yi|xi;θ)) w.r.t. θ∈ϴ are continuous and smaller than infinity
2: The expectations of the first and second derivative of ln(Li(θ))=ln(f(yi|xi;θ)) exist for θ∈ϴ
3: The third derivative has an integrable majorant: ||∂^3ln(Li(θ))/∂θ^3||≤M(xi,yi)=M(wi) such that E(M(xi,yi))=E(M(wi)) exists and is finite
4: θ0 is in the interior of ϴ
5: wi are i.i.d.
6: I=Var(g(wi,θ0)) exists and is positive definite
7: H=E(H(wi,θ0))=E(∂g(wi,θ0)/∂θ’) exists and has full column rank p

Question 5

Wald test (MLE):

Answer 5

H0: Rθ0=q=0 vs. H1: Rθ0-q≠0
Wn=n(Rθn^-q)’(RVn^R’)^-1(Rθn^-q)–>χ^2J
Reject H0 if Wn≥χ^2(1-α, J)

Question 6

Likelihood-ratio test (MLE):

Answer 6

H0: Rθ0=q=0 vs. H1: Rθ0-q≠0
LRn=-2n((ln(Ln(θn~))/n)-(ln(Ln(θn^))/n)–>χ^2J
Reject if LRn≥χ^2(1-α, J)