MLE

Question 1

What is the goal of Maximum Likelihood Estimation (MLE)?

Answer 1

To estimate the parameter value that makes the observed data most probable.

Question 2

What is the PMF of a Bernoulli distribution?

Answer 2

f(x; p) = p^x * (1 - p)^(1 - x) for x ∈ {0, 1}

Question 3

What is the likelihood function for a Bernoulli sample?

Answer 3

L(p) = p^(sum x_i) * (1 - p)^(n - sum x_i)

Question 4

Why do we use the log-likelihood?

Answer 4

It simplifies the product of terms into a sum, making differentiation easier.

Question 5

What is the MLE for p in a Bernoulli distribution?

Answer 5

p̂ = (1/n) * sum X_i = X̄ (sample mean)

Question 6

What is the PDF of the Pareto distribution used in the example?

Answer 6

f(x; γ) = γ / (1 + x)^(γ + 1), for x > 0

Question 7

What is the log-likelihood function for the Pareto distribution?

Answer 7

ℓ(γ) = n * ln(γ) - (γ + 1) * sum ln(1 + x_i)

Question 8

What is the MLE for γ in the Pareto distribution?

Answer 8

γ̂ = n / sum ln(1 + X_i)

Question 9

What is the invariance property of MLE?

Answer 9

If θ̂ is the MLE for θ, then g(θ̂) is the MLE for g(θ)

Question 10

What is the difference between an estimator and an estimate?

Answer 10

An estimator is a random variable; an estimate is a specific observed value.