Regression and Classification

Question 1

Accuracy

Answer 1

Correct / Total. Good when equal consequences from false positive or false negative (e.g., manufacturing QC, weather, manufacturing stock prediction, school grades)

Question 2

AUC

Answer 2

Area under the ROC curve - represents the chances that a randomly positive will have higher ranking than random negative. Coin is .5, perfect is 1. Good general way to choose between models for a balanced dataset.

Question 3

Batch Gradient Descent

Answer 3

Graident Descent Algorithm that goes through the entire data set to calculate partial derivatives and update weight and bias

Question 4

Batch Size

Answer 4

The number of examples used in a single learning iteration (before updating weight and bias)

Question 5

Binary classification

Answer 5

Type of classification that predicts one of two mutually exclusive classes - positive and negative

Question 6

Clasification

Answer 6

Learning models that attempt to predict a defined number of categories that can be numbers or or non-numeric (e.g., dog, cat, bird)

Question 7

Class-Imbalanced

Answer 7

A dataset for a classification problem where the number of labels of each class differs significantly

Question 8

Common causes of data quality and reliability issues

Answer 8

Omitted, duplicate, value errors, label errors, bad sections

Question 9

Confusion matrix

Answer 9

Used to summarize the performance of a classification algorithm. Typically a 2x2 with actual vs predicted with actual as rows and predicted as values

Question 10

Cost function

Answer 10

A combination across all loss functions. May include some sort of penalty for complexity.

Question 11

Decision boundary

Answer 11

A surface or line that separates different classes predited by a classification algorithm. It markes the boundary of one class vs another.

Question 12

Early stopping

Answer 12

Regularization method that stops before training loss stops decreasing, stops when validation loss increases (or generalization performance worsens)

Question 13

epoch

Answer 13

A full training pass over the entire training set (every example processed once). 1 epoch = 1/batch size

Question 14

false positive rate

Answer 14

FP / (FP + TN) - rate that negative is accidentally called a positive. This is the x-axis for ROC curve.

Question 15

Feature

Answer 15

Individual model input that is an individual property or characteristic typically represented by columns (or “x” in a model formula)

Question 16

Feature engineering

Answer 16

Using intuition to design new features by transforming or combining original features (using depth and width to define “area”)

Question 17

Feature scaling

Answer 17

Scaling the range of features to improve gradient descent performance - typically done via mean normalization or z-score normalization. Goal is generally -1 < x < 1

Question 18

Gradient Descent

Answer 18

An optimization function to minimize a loss function by iteratitvely adjusting the parameters of a model based on the partial derivatives that point in the steepest direction of the loss function.

Question 19

Hyperparameters

Answer 19

The variables that you or a hyperparameter tuning service adjust between successive runs of training a model. (e.g., learning rate). This is in contast to parameters, like weight and bias that the model learns during training.

Question 20

L1 loss

Answer 20

A loss function used for regression that calculates the aboslute difference between actual labels and what a model predicts. L1 loss is less sensitive to outliars than L2 loss.

Question 21

L1 Regularization

Answer 21

A type of regularization that penalizes weights in proportion to the sum of the absolute value of the weights. L1 regularization helps drive weights of irrelevant or almost irrelevant features to exactly 0 - removing it from the model.

Question 22

L2 Loss

Answer 22

A loss function used for regression that calculates the square of the difference between actual labels and what amodel predicts. L2 loss is more sensitive to outliars than L1 loss.

Question 23

L2 Regularization

Answer 23

A type of regularization that penalizes weights in proportion to the sum of the square of the weights. L2 regularization helps drive outlier weights closer to zero (but not zero). Those with low 0 values stay in the model. L2 regularization always improves generalization in linear models.

Question 24

Label

Answer 24

A label is the target output or value that a model is trained to predict. It represents the answer or ground truth for a single data point in supervised learning. Also known as target.

Question 25

Learning Curve

Answer 25

Graph of cost (J(w,b)) vs # of iterations. Can be used to determine that gradient descent is working correctly. J(w,b) should quickly approach 0 and stop decreasing if everything is working right.

Question 26

Learning Rate

Answer 26

A key hyperparameter that tells gradient descent how strongly to adjust weights and measures on each iteration. If too low training will take too long, if too high it will have trouble reaching convergence. Typically a number between 0 and 1.

Question 27

Log Loss

Answer 27

Optimization/cost function that is used in logistic regression. Penalizes based on probability of prediction (e.g., highly penalizes a high-probability incorrect class).

Question 28

Logistic Regression

Answer 28

Logistic regression is the algorithm used for binary classification tasks. It takes a linear combination of features and maps it to a probability between 0 and 1 using a sigmoid function.

Question 29

Loss function

Answer 29

During training, a function that calculates the loss for a single example. A loss function returns a lower loss for a model that makes better predictions vs a model that makes worse predictions. The purpose of training is typically to minimize the loss that a loss function returns across all examples.

Question 30

Mean Squared Error

Answer 30

Average L2 loss per example (used for regression). Squared loss is another term for L2 loss.

Question 31

Methods of imputation

Answer 31

The process of replacing missed values in a dataset so that the examples can still be used for modeling and analysis. Common methods include simple (e.g., mean, mode, median) and fixed (e.g., -1, 0)

Question 32

Mini-batch

Answer 32

A small, randomly selected subset of a batch processed in one iteration. It's more efficient to calculate loss on a mini-batch then on all the examples in the full batch.

Question 33

Model formula

Answer 33

f(x) -> y-hat where x is the feature, f(x) is the model, and y-hat is the prediction

Question 34

Noise

Answer 34

Random or irrelevant data that obscures underlying pattern

Question 35

Overfitting

Answer 35

A model that is too complex to generalize well (e.g., 4-th order function for 2nd order function). Can be addressed with regularization. Also known as high variance (i.e., model changes a lot with a small amount change in training data). Solutions are 1) more data 2) fewer features 3) regularization.

Question 36

Polynomial regression

Answer 36

Feature engineering that includes feature values raised to powers (e.g., including x, x^2, and x^3). Can be used to find non-linear relationships.

Question 37

Precision

Answer 37

TP / (TP + FP). If we detect positive, high likelihood that it's actually positive. Examples are where false positives have high pain (e.g., spam detection, verdict)

Question 38

Quantity of data

Answer 38

Model should train on on an order of magnitude (or two) more than the number of trainable parameters. Large datasets with small numbers of features generally perform better.

Question 39

Recall

Answer 39

True positive rate. TP / (TP + FN). When detecting positives is most important and can't miss them (e.g., identifying cancer, fraud, security, churn)

Question 40

Regression

Answer 40

Learning models that attempt to predict numbers with infinte outcomes

Question 41

Regularization

Answer 41

Any mechanism to reduce overfitting without removing features. Typically done with a modified cost funtion that includes a "regularization term" and an additional penalty "lambda." Can also be thought of as a penalty on model complexity. By convention only on "w" parameters and not "b" parameter. Popular ones are L1, L2, dropout, and early stopping.

Question 42

relu

Answer 42

"Rectified linear unit" - An activation with the following behavior - if input is negative or 0, then the ouput is 0, else the value is equal to the input. (-3 = 0, 3 = 3). Enables a neural network to learn non-linear relationships between features and the label.

Question 43

ROC

Answer 43

Visual representation of a model's accuracy at all thresholds. Graphs the TPR vs FPR at all thresholds. Stands for receiver operating characteristic.

Question 44

Sigmoid

Answer 44

A mathematical finction that "squishes" the output between two values - typically 0 and 1 or -1 to +1. Has several uses including converting raw output of logistic regression to a probability and acting as an activation function in some networks.

Question 45

Supervised Learning

Answer 45

Learning where training data (inputs and output label) includes correct predictions.

Question 46

Test set

Answer 46

A portion of the data that is reserved specifically for evaluating the performance of the trained model. Helpful for identifying under or over fitting.

Question 47

Threshold

Answer 47

The specific numeric cutoff used applied to probabilities to convert them to a class label (e.g., over 50% -> class A)

Question 48

training set

Answer 48

The subset of the dataset used to train the model. Typically datasets are broken into three distinct subsets - training, testing, and validation. Ideally, any example should only belong to one of these three subsets.

Question 49

True negative

Answer 49

An example where the model correctly predicts the negative class (e.g., an email is not spam)

Question 50

True Positive

Answer 50

Same as recall rate. TP / (TP + FN). When detecting positives is most important and can't miss them (e.g., identifying cancer, fraud, security, churn)

Question 51

Unsupervised Learning

Answer 51

Find something interesting in unlabeled data (e.g., clustering, dimensionality reduction, anomoly detection)

Question 52

validation loss

Answer 52

A metric used to understand how well training worked on validation set. Discrepancies between training loss and validation loss can indicate overfitting.

Question 53

validation set

Answer 53

A slice of the data used to test paramter testing or model selection. Often used multiple times before a test set is used.

Question 54

Ways to handle incomplete data (missing features)

Answer 54

Delete examples, impute missing values

Question 55

Z-score normalization

Answer 55

Normalization such that all features have a mean of 0 and a standard deviation of 1. Each feature value is subtracted by the mean of all values for that feature and divided by the standard devision of that feature.

Question 56

Underfit

Answer 56

A model that is not complex enough to generalize well (e.g., linear model for quadratic function). Also known as high bias.