Generalized linear models

Question 1

Link and response functions

Answer 1

η = x’β = g(μ) link
μ = h(η) response

Question 2

Grouping data process and pros

Answer 2

When I have same covariate patterns, I create G group, each with n_i observations and I model E(y-_i) instead of E(y_i)
- I can estimate model fit;
- I can estimate overdispersion.

Question 3

Overdispersion estimation and resulting coefficients

Answer 3

When I can group data: Var(y-_i) = φ * theoretical Var because of unobserved heterogeneity of positive correlation between responses (coming from clusters).
- φ_P = 1/(G-p) χ²
- φ = 1/(G-p) D

It gives a quasi-likelihood approach where β^ .~ N(β , φ F^-1(β^)), affecting the significance. AIC and other likelihood quantities cannot be estimated.

Question 4

Maximum likelihood estimation requirements

Answer 4

• Conditional independence: yi independent of yj | X ∀ i, j
• Invertibility of F(β) ∀ βs, equivalent to rank(X)=p

Question 5

Fisher scoring algorithm and results

Answer 5

^β^(t+1) = ^β^(t) + F^-1(^β^(t))s(^β^(t))
^β .~ N(β, F^-1(^β)) with Var(^βj) = [F^-1(^β)]jj

Question 6

Observed and expected information matrices

Answer 6

H(β) = - d²l(β) / dβdβ’
F(β) = E(H(β)) = cov(s(β)) = E(s(β)s(β)’)
- F(β) = H(β) for binary logit model

Question 7

Hypothesis testing

Answer 7

H0: C_rxpβ_px1 = d_rx1
- Both models: LR = -2[l(^β_H0)-l(^β)] .~ χ²_r
- Only initial: W = (C^β-d)’[CF^-1(^β)C’]^-1(C^β-d) .~ χ²_r
- Only restricted: u = s(^β_H0)’F^-1(^β_H0)s(^β_H0) .~ χ²_r

Question 8

Model quality and selection

Answer 8

Comparing fitted model to saturated one:
- Pearson .~ χ²_G-p
- Deviance: χ² = -2 Σ [l_i(^μ_i) - l_i(y-_i)] .~ χ²_G-p
Comparing models (no quasi-):
- AIC = -2l(^β) + 2p (min)
- Nagelkerke’s R²