05: Linear Model

Question 1

What are the linear models available for regression?

Answer 1

1. Simple linear regression
2. Multiple linear regression
3. Ridge regression
4. Lasso regression

Question 2

Simple linear regression?

Answer 2

• Models the relationship between a single independent variable x and a dependent variable y using a straight line
• y = w[0] * x[0] + b
• Use case: When there’s only one predictor.

Question 3

Multiple linear regression?

Answer 3

• Extends simple linear regression to multiple independent variables
• y = w[0] * x[0] + w[1] * x[1] + ……… + w[p] * x[p] + b
• Use case: Predicting a target using several predictors.

Question 4

Ridge regression?

Answer 4

• Adds a penalty for large coefficients to reduce overfitting
• Use case: When features are highly correlated or number of features > number of samples

Question 5

Lasso regression?

Answer 5

• Adds a penalty equal to the absolute value of the coefficents; can shrink some coefficents to zero, thus performing featrure selection.
• Use case: For sparse models with automatic feature elimination

Question 6

Linear models used in classification?

Answer 6

• Logistic regression
• Multinomial logistic regression
• Linear discriminant analysis
• Linear SVM (support vector machine)

Question 7

Linear models in multiclass classification?

Answer 7

• Multinomial logistic regression (Softmax reg)
• One-vs-Rest logistic regression (OvR)
• Linear SVM