LR

Question 1

continuous class, but a discrete learner (“classifier”)

Answer 1

discretization

Question 2

discrete class, but a continuous learner (“regressor”)

Answer 2

Build a suite of regression tasks via multi-response linear regression:
• perform one regression per discrete class, with all instances of that class set to 1, and other instances set to 0
• classify a given test instance by regressing its value relative to each class, and selecting the class with the highest value

like “one-vs-rest” multi-class SVM, 
-> too slow....

Question 3

Logistic Regression

Answer 3

a model where 1 = interesting class and 0 = uninteresting class

Question 4

Logistic Regression and NB

Answer 4

they both try to maxmize P(c|T) but in LR we estimate it directly with beta without making any assumption like NB.

Question 5

How do determine β in LR?

Answer 5

use Gradient Descent to maximize the log-likelihood

Question 6

Pros and Cons of LR

Answer 6

Pros:
• Vast improvement on Naive Bayes
• Particularly suited to frequency-based features (so, popular in NLP)

Cons:
• Slow to train
• Some feature scaling issues
• Often needs a lot of data to work well
• often need regularisation due to overfitting