CHAPTER 3: Useful ideas and methods for inference Flashcards

Question

Likelihood Ratio Tests

Answer 1

(a) Likelihood Ratio Test for a simple null hypothesis Likelihood Ratio Test for the Simple Hypothesis H0 (b) Generalized Likelihood Ratio Test

Answer 2

Suppose θ∗ is a speciﬁc value and we wish to test the hypothesis H_0 : θ_0 = θ∗ against the alternative H_1: θ_0 ≠ θ∗. The relative likelihood RL(θ∗) = L(θ∗)/ L(ˆ θ) is useful for this, because: RL(θ∗) small suggests evidence against H_0 RL(θ∗) close to 1 suggests H_0 plausible Thus, since logRL(θ∗) = l(θ∗)−l(ˆ θ), l(θ∗)−l(ˆ θ) well below 0 suggests evidence against H_0 l(θ∗)−l(ˆ θ) close to 0 suggests H_0 plausible Equivalently we want to deal with positive values W = −2l(θ∗)−l(ˆ θ) well above 0 suggests evidence against H_0 W = −2l(θ∗)−l(ˆ θ) close to 0 suggests H_0 plausible

Answer 3

In the random sample case, when the true value of θ is θ∗ (ie H0 true), W = −2(l(θ∗)−l(ˆ θn)) = −2logRL(θ∗) → χ^2 _p in distribution as n →∞, where p is the dimension of θ. * for larger values, if H_0 is true: tells us that when H_0 is true, the probability of observing a value of W larger thn a particular w is p_obs = P(W ≥ w | H0) ≈ P(χ^2 _p ≥ w). eg p=2 2 state MC with parameters θ= (α,β) or normal dist with unknown mean and variance

Answer 4

Likelihood Ratio Test for the Simple Hypothesis H0, 1. From the data calculate the observed value w of the test statistic W, 2. Find (from χ2 tables or via a computer) the probability pobs = P(χ2 p ≥ w) 3. Interpret pobs as a measure of the weight of evidence in the data against H0 in the sense that the smaller pobs, the more surprising would the observed data be if H0 were true (and therefore the stronger the evidence against H0). e. g if we specify the values of alpha and beta in a vector and check ie checking a specific value of theta* * small p value means stronger evidence AGAINST H_0

Answer 5

Key Fact 2 also gives us a way to choose the constant c in likelihood regions {θ_0 : l(θ_0) bigger than l(ˆ θ)−c} If we choose 2c = χ^2_{p,0.95}, then P(l(θ_0) bigger than l(ˆ θ)−c) = P(l(ˆ θ)−l(θ0) < c) ≈ P(χ2 p < 2c) = 0.95, when θ_0 is the true parameter value, so the likelihood interval with this c is an approximate 95% conﬁdence interval. For example, when p = 1 we have χ2 1,0.95 = 3.84, so we can choose c = 1.92, sometimes approximated as c = 2

Answer 6

we might want to test whether alpha and beta are the same ie many different values might satisfy this (rather than their specific values) ie a set of values that theta might be in Suppose that θ is p-dimensional with values in a set Θ ∈ R^p and suppose that we wish to test a null hypothesis H_0 that the true value θ_0 belongs to a subspace Θ_0 of Θ, where Θ0 is q-dimensional with q < p. The alternative hypothesis H1 is that θ0 ∈ Θ\Θ_0. α=β p=2 and q=1 as if we determine β we determine the other.

Answer 7

``` Consider the statistic GLR = max θ∈Θ0 RL(θ) = maxθ∈Θ0 L(θ) /maxθ∈Θ L(θ) = L(˜ θ) /L(ˆ θ) , where ˆ θ is the usual mle (the global mle) and ˜ θ is the value of θ maximizing L within Θ0 (the restricted mle). ``` *the largest value that the relative likelihood can take for θ within Θ_0 (ie theta which satisfy our hypothesis) * denominator: unconstrained θ maximum likelihood numerator: θ which satisfy hypothesis Θ_0 eg α=β *Note that when Θ0 is a single point, −2logGLR reduces to the W used in subsubsection (a) above

Answer 8

* if H0 is true, then the global maximum of L is likely to occur close to Θ0, so ˜ θ and ˆ θ are likely to nearly coincide and GLR to take a value close to 1. * On the other hand, if H1 is true, then the maximum of L within Θ0 is likely to be considerably less than the global maximum, so GLR will tend to be considerably smaller than 1. This suggests that a test statistic for H0 could be based on GLR. −2logGLR reduces to W above in the special case of a point Θ

Answer 9

In the random sample case, when the true value of the parameter θ0 ∈ Θ0, W = −2(l(˜ θ)−l(ˆ θ)) = −2logGRL → χ2 _{p−q} in distribution, as n →∞, where p is the dimension of Θ, the full parameter space, and q is the dimension of the restricted parameter space Θ0. (p-q is the difference in dimensionality between θ and Θ_0) = degrees of freedom- ie restricting toΘ_0

Answer 10

1. From the data calculate the observed value w of the test statistic W = −2logGLR, 2. Find (from χ2 tables or via a computer) the p-value P(χ^2 _{p−q} ≥ w) 3. Interpret the p-value as a measure of the weight of evidence in the data against H0 in the sense that the smaller the p-value, the stronger the evidence against H0 * ie restriction α=β 1 degree of freedom from 2 to 1 * When Θ0 is a single point then q = 0 and the test reduces to the version in (a)

Answer 11

θ = (φ,ψ), the null hypothesis that φ = ψ can be written as θ ∈ Θ0, where Θ_0 is the line ψ = φ. ****The log likelihood given data x and y is l(φ,ψ;x,y) = −n(logφ + logψ + ¯x/φ +¯y/ψ). (independence so we can add together) ***** (putting xbar and y bar as φ and ψ ) It is easy to see that ˆ θ = (¯ x, ¯ y). Thus l(ˆ θ) = −n(log ¯ x + log ¯ y + 2). (log likelihood at max value) The restricted MLE assuming φ = ψ is ˜ θ =((¯ x+¯ y)/2 ,( ¯ x+¯ y)/ 2)* (sample means, are the same as sample sizes are for both ˜ θ, ˆ θ is individual sample means ) (now substituting φ,ψ as *) ``` so l(˜ θ) = −n(2log((¯ x + ¯ y)/2) + (2¯ x)/( ¯ x + ¯ y) + (2¯ y)/(¯ x + ¯ y)). ``` (log likelihood at restricted value) So we have W = −2(l(˜ θ)−l(ˆ θ)) = 2n(2log((¯ x + ¯ y)/ 2) −log( ¯ x)−log (¯ y)). ``` With n = 10, ¯ x = 2.3 and ¯ y = 1.7, we get W = 0.455, which is to be compared with χ2 _1. As pchisq(0.455,1) in R gives 0.5, there is no evidence in this case against H_0 ```

Answer 12

Ese = Sqrt( standard dev/n) Eg sqrt( p(1-p)/n) n=227 out of 1000?

Answer 13

The probability density function for each observation is f_{Xi}(x; θ) = {(1/θ)e^{−x/θ} x ≥ 0 {0 x < 0