L3 Logistic Regression

Question 1

Logistic Regression, model

Answer 1

• No closed form solution
• Logistic probability model
• Well suited for binary classification

Conditional Bernouilli:
p(y(i) = 1| x(i)) = 1 / (1 + exp(−w^T φ(x(i))) )

General: p(y(i) | x(i)) = 1 / (1 + exp(−y(i) w^T φ(x(i))) )

Question 2

Logistic Regression, optimization

Answer 2

Max likelihood: arg min(w) Sum_{(x(i),y(i))∈D}􏰀 log (􏰠1 + exp(−y(i) w^T φ(x(i)))􏰡 )

Use of gradient descent