Univariate exponential family regression

Question 1

Univariate exponential family density and requirements

Answer 1

f(y|θ) = exp{ (yθ-b(θ)) / φ * x + c(y, φ, w)}
Useful for likelihood quantities: (y|θ) = exp{ (yθ-b(θ))}
- Support of y must be independent of θ;
- w weights (=1 for individual data, =n_i for grouped data, =1/n_i for summed data per group);
- b(θ) must be twice differentiable and such that f(y|θ) can be normalized;
- φ=1 for Binomial and Poisson distributions.
E(y) = μ = b’(θ)
Var(y) = φb’’(θ) / w = φv(μ) / w with v(μ) variance function

Question 2

Advantages of using a canonical function in the univariate exponential family regression

Answer 2

• Every EF distribution has a unique canonical link function with θ_i = η_i = x_i‘β;
• It ensures concavity of the log-likelihood;
• It ensures the existance of a unique ML estimate;
• F(β) = H(β), so Fisher scoring = Newton-Rhapson

Question 3

Score for EF

Answer 3

y_i IID:
s(β) = Σ xi h’(η_i)/σ² (y_i-μ) = X’DΣ^-1(y-μ)
s(^β) = 0

Question 4

Fisher matrix for EF

Answer 4

y_i IID:
F(β) = Σ x_ix_i‘w_i~ = X’WX
w_i~ = (h’(η_i))²/σ_i²

Question 5

Iterative weighted least squares (IWLS) for exponential family

Answer 5

Equivalent to Fisher scoring
β^^(t+1) = (X’W^(t)X)^-1 X’ W^(t)y~^(t)
- y~_i^(t) = ^η_i^(t) + [y_i-h(^η_i^(t))] / h’(^η_i^(t))
- X’W^(t)X must be invertible;
- If we are not using the canonical link function, we need multiple starts for the algorithm.

Question 6

Asymptotical properties of β^ if using a canonical function

Answer 6

• (X’X)^-1 -> 0
• λ_min(X’X) -> +&infty; so the “actual dimension” of the matrix tends to infinity

These are sufficient for:
β^ ~. Np(β, F^-1(β^)) and weak consistency of β^

Question 7

Overdispersion estimation for EF

Answer 7

φ^ = 1/(G-p) Σ [y_i - h(η^_i)]²/[v(μ^_i)/n_i]
- We need grouped data;
- The estimator is consistent;
- Leads to a quasilikelihood approach.

Question 8

Hypothesis testing for EF

Answer 8

LR, W, u quantities defined in terms of b(θ), h(.), v(μ), ecc.
Still ~. χ²_r

Question 9

Model selection for EF

Answer 9

χ² = Σ (y_i-μ^_i)² / [v(μ^_i)/w_i]
D = -2Σ[l_i(μ^_i) - l_i(y-_i)]
Both ~. φχ²_G-p
They measure the fit quality relatively to the saturated model
AIC=-2l(β) + 2p for model comparison as always - p+1=k+2 if we have the nuisance parameter φ too