Categorical regression

Question 1

Types of categorical response variables

Answer 1

• Ordinal: sequential or not
• Nominal (not ordered)

Question 2

Multinomial distrbution

Answer 2

MN(m, π)
f(y_cx1|π) = π₁^y₁ * π₂^y₂ * … π_c^y_c * (1-Σ π_r)^m-Σy_r
- E(Y)_cx1 = mπ
- cov(Y)_cxc = m [πi(1-πi) diagonal, -πiπj out]
- Ungrouped data: m=1
- Grouped data: m=ni

Question 3

(Extended) Multinomial logit

Answer 3

For nominal variable:
π_ir = h(η_ir) = exp[x_i‘β_r + (w_ir-w_i(c+1))’γ] / [1+Σ_s=1^c exp(x_i‘β_s + (w_is-w_i(c+1))’γ)]
π_i(c+1) = 1 / [1+Σ_s=1^c exp(x_i‘β_s)]
η_ir = log(π_ir/π_i(c+1))
- Category specific: (w_ir-w_i(c+1)), β_ir;
- Category unspecific: γ, x_i;
- βr is the log(relative risk) of category r compared to the reference category (c+1) and NOT relative to other categories;
- exp(βr) is the effect on the odds;
- If β_rj > 0: ODDS_r increase relatively to reference category of exp(β_rj) with one unit increase of x_j. The probability itself is not increasing for increasing x_j!

Question 4

Latent utility formulation for categorical regression

Answer 4

u_r = ~η_r+ε_r
Y=r <=> u_r = max of all u_s
- ε_i Logistic: P(Y=r) = exp(~η_r - ~η_(c+1)) / [1 + Σ exp(~η_s - ~η_(c+1))];
- ε_i Normal(0,1): independent probit;
- ε_i MVN: multivariate probit.

Question 5

Cumulative categorical regression model

Answer 5

For ordinal non-sequential variable:
u_i = -x_i‘β + ε_i i=1,…G
Y_i = r <=> θ_(r-1) < u_i < θ_r
P(Y_i=r) = F(θ_r+x_i‘β) - F(θ_(r-1)+x_i‘β)
- NO INTERCEPT for identifiability;
- F_ε Logistic: Proportional Odds model;
- F_ε Extreme min value: Proportional Hazards model.

Question 6

Proportional odds model

Answer 6

For ordinal non-sequential variable, ε logistic:
P(Y_i <= r) = exp(θ_r +x_i‘β)/[1+exp(θ_r+x_i‘β)]
- OR = exp((x_i - ~x_i)’β) independent of category.

Question 7

Proportional hazards / grouped Cox model

Answer 7

For ordinal non-sequential variable, ε extreme min-values:
P(Y_i <= r) = 1-exp(-exp(θ_r + x_i‘β))

Question 8

ML estimation in categorical regression

Answer 8

β^ such that
s(β^) = X’DΣ^-1(y-niπ) = 0, with F(β) = X’WX
^β .~ N(β, F^-1(^β))
- d_ii = h’(η_i)
- w_ii = [h’(η_i)]²/σ²

Question 9

Xi for ordinal cumulative model

Answer 9

Since we have the c thresholds θ and k regressors (NO intercept):
X_i = cbind( diag(1, c), rep(xi, c) )

Question 10

X_i for extended multinomial model

Answer 10

We have k+1 regressors β and also γs:
X_i = cbind( diag(xi, c), c(w_i1‘-w_i(c+1)’, .. w_ic‘-w_i(c+1)’ )