Logistic Regression Flashcards
What type of ML problems to linear classifiers solve?
Classification problems
What is the shape of the decision boundary for a linear classifier?
Line (2d), hyper plane (>2d)
What function do we learn building a linear classifer?
For a linear classifer, how is the input x classified?
If F(x, w) > 0 then y = 1 else y = 0
What is the normal vector to a linear classifiers decision boundry?
w
Describe the decision boundary of a linear classifier with set builder notation
{x | wTx + w0 = 0}
Definition of logisitics function
1 / (1 + exp(-z))
What is f(0) where f is the logistics function?
0.5
What is P(y=1 | x) for logistic regression?
f(wTx) where f is the logistic function
What is P(y=0 | x) for logistic regression?
1 - f(wTx) where f is the logistic function
What is the role of the logisitics function in logistics regression?
“squash” real line into [0,1] so we have a probablistic model
What is the significance of the magnitudes of w1, w2, …, wd for logistic regression?
How certain the classifications are:
small w’s - probabilitys close to 0.5
large w’s - probabilitys close to 0/1
Whats the problem for maximizing L(w) for logistic regression?
No analytical solution, need to use numerical optimization
What is the score function for logisitc regression?
log likelihood
How is logistic regression extended more than 2 classes?
- Create weight vector wk for each class k
- Use softmax function
Softmax function
