Regression Models

Question 1

What is supervised learning?

Answer 1

Learning a function that maps an input and some parameters to a predicted output (label), which is then evaluated based on a ground truth value

Question 2

What is an observation in supervised learning?

Answer 2

One row of data in the dataset

Question 3

What is a feature in supervised learning?

Answer 3

One column of data in the dataset

Question 4

What are hyperparameters?

Answer 4

Parameters that we define before running the model, and are not learned directly from the data

Question 5

Models are typically trained in two phases: […] and […]

Answer 5

Training and evaluation (/prediction)

Question 6

The training phase involves…

Answer 6

Teaching the models what predictors fall into what category, learning parameters that define the relationships between the features

Question 7

The prediction/evaluation phase involves…

Answer 7

Gaining new observations and feeding them into our trained model to create a prediction

Question 8

An update rule is defined which is calculated using the value from a […].

Answer 8

Loss function

Question 9

What is a loss function?

Answer 9

A quantitative measure of error in predicted values of a model

Question 10

Problems with quantitative responses are usually […] problems, while those with qualitative responses are usually […] problems.

Answer 10

Regression, classification

Question 11

Provide an example of a regression model used for qualitative responses.

Answer 11

Logistic regression, since it estimates the probability of a choice

Question 12

What are regression models?

Answer 12

Regression models are used to model any continuous target or outcome (i.e. loss, revenue)

Question 13

How does a linear regression model create predictions?

Answer 13

Defining a straight mathematical line that attempts to go through each point, with which we can substitute x and y to find y’.

Question 14

In a linear regression model, epsilon represents…

Answer 14

A hyperparameter chosen at implementation that guides the model to learn the data

Question 15

In a linear regression model, b0 and b1 represent…

Answer 15

Parameters that represent the y-intercept and slope of the line respectively, calculated via an equation

Question 16

What is Mean Squared Error (MSE), and how can we use it for a linear regression model?

Answer 16

MSE is the sum of squared errors divided by the number of data points, which we can use to measure the error of our model and adjust coefficients accordingly

Question 17

Error metrics are useful for training a model because…

Answer 17

We can focus on easier-to-read parameters rather than the predictions themselves to generate model/data insights

Question 18

What is a black-box model?

Answer 18

A black-box is a type of learning model that does not explain its process, creating a sort of ‘black-box’ between input and output

Question 19

The best step-by-step practice for modelling involves…

Answer 19

Establishing the ideal cost function, developing multiple models with different hyperparameters, and comparing the results according to our loss function (establishing, developing, comparing)

Question 20

We can calculate the parameters of our linear regression model by…

Answer 20

Solving the linear system of equations formed by our data points for epsilon, calculating the sum of these equations, then using the derivative to solve for bn.

Question 21

The area above the mean of a linear regression line is called…

Answer 21

Explained variation

Question 22

The area below the mean of a linear regression line is called…

Answer 22

Unexplained variation

Question 23

Total variation is calculated as the sum of…

Answer 23

Explained and unexplained variation

Question 24

How can we add more curvature to our linear regression model?

Answer 24

By using a polynomial regression instead

Question 25

If a first order polynomial regression is a linear regression that generates a straight line, a second order polynomial regression generates...

Answer 25

A curve with one local minima/maxima

Question 26

The order of a polynomial refers to...

Answer 26

The largest exponent in any of the terms

Question 27

Are polynomial regression models linear?

Answer 27

Yes, the algorithm itself is still a linear combination of features

Question 28

What is the bias-variance tradeoff?

Answer 28

Model adjustments that decrease bias, often increase variance, and vice versa, therefore this tradeoff is analogous to a complexity tradeoff

Question 29

What can we use to measure the complexity of a model?

Answer 29

Bayesian Information Criterion (BIC)

Question 30

Why should we always include an unseen testing set?

Answer 30

If we train only on the training set, our model may generalise too hard to it, learning only how to replicate and not predict

Question 31

If knowledge of the test set leaks into the training set, we can encounter an issue called...

Answer 31

Data leakage

Question 32

What is cross validation?

Answer 32

Instead of using one training and testing set, you instead split the dataset into four sets of training/testing sets, then test the model on each and compare the error between them

Question 33

What is the problem with randomly sampling the training set?

Answer 33

In the case where we have data with a large bias, that bias may not be properly represented in the random samples

Question 34

What problem does stratified sampling solve and how?

Answer 34

Bias in the training set, by dividing the dataset into groups based on the different classes, and randomly selecting an amount of data equal to their proportion in the data from among each group