L4 - Regression Flashcards
(8 cards)
What do Regression models do?
Attempt to predict the value of a continuous responce variable as a function of one or more predictor variables.
Eg.
Fuel Economy is a function of vehicle size, weight, shape, transmission type, engine size …etc
They can be either parametric or non-parametric
Parametric regression models assume a particular form for the relationship y = C0 + C1x
Non-parametric models apply an algorithm to the data, resulting in a model without closed-from representation
What is a Linear Regression Model?
It is a parametric regression technique
The response is modelled as some known formula given in terms of the predictor variables.
It takes a form as the sum of (predictor variable) x linear parameter.
y = a + bx
y = a + bx + cx²
FuelEcon = a + Weight⋅b+HorsePower⋅c+…etc
How do regularised linear regression models compare to standard linear regression models?
Regularised linear regression models shrink the regression coefficients by applying a penalty for large coefficient values.
This reduces the variance of the coefficients and can create models with smaller prediction error.
How are coefficients chosen in linear regression?
By minimising the mean squared error (MSE).
Error = Observed responce - prediction
MSE = SUM(Error²)/Number of samples
How do Ridge and Lasso Regression differ from Linear Regression?
A penalty term is added to the mean square error calculation
This penalty penalises large coefficients.
What is Elastic Net Regression?
Elastic Net regression adds a new parameter which defines the weights of the Ridge and Lasso MSE calculations - so it uses both!
What is a Regression Tree?
Regression Trees are an example of non-parametric model structures.
Similar to Decision Trees but rather than representing a discrete category each leaf represents a numeric value.
What is Gaussian Process Regression?
Gaussian process regression (GPR) is a nonparametric regression technique.
In addition to predicting the response value for given predictor values, GPR models optionally return the standard deviation and prediction intervals.
