Midterm

Question 1

when to use ML?

Answer 1

to hard to hardcode
automation
problem is changing frequently

Question 2

when to not use ML?

Answer 2

algorithm already exists
not enough data
ethical concerns
requires explanations, not just predictions

Question 3

what is the free lunch theorem?

Answer 3

no model works best for every dataset

Question 4

what does regularization do?

Answer 4

lowers variance without raising bias much

Question 5

how to pick hyperparameters?

Answer 5

grid search, random select

Question 6

what is softmax regression?

Answer 6

train multiple logistic regressions
make sure probabilities all sum to 1
higher temperature means less confident

Question 7

what is logistic regression?

Answer 7

turns linear regression into classification
outputs a probability

Question 8

hard margin SVM

Answer 8

trying to perfectly separate the two classes

Question 9

soft margin SVM

Answer 9

allow some points to be misclassified, as long as the mistakes are not too large

Question 10

kernels in svm

Answer 10

allows to map to non linear relationships without transforming the data

Question 11

OvR vs OvO

Answer 11

OvR - train c models each saying if it is the class or not
OvO - train models to compare between each possible pair of classes

Question 12

MSE and MAE formulas

Answer 12

take difference, square/take absolute value, sum up, divide by n

Question 13

advantages of MSE and MAE

Answer 13

MSE - differentiable, good for learning
MAE - result is interpretable, simple, less prone to outliers

Question 14

L1 regularization

Answer 14

make least important weights as close to 0 as possible

Question 15

L2 regularization

Answer 15

have the smallest weights possible

Question 16

Elastic net

Answer 16

combines l1 and l2

Question 17

l0 regularization

Answer 17

looks to have the most amount of 0s in the weights

Question 18

what do gini impurity and entropy measure?

Answer 18

how impure the data is
a lower number means the data is more pure
we want more pure data in decision trees, means the split is better

Question 19

pros and cons of decision trees

Answer 19

pros
- not much feature engineering
- interpretable
- applies to many tasks
cons
- can overfit
- high variance

Question 20

what is bagging

Answer 20

training lots of models of the same type, each on a different subset of the data
bagging has replacement

Question 21

what is boosting

Answer 21

training lots of weak learners, trained to undo the errors of the previous models

Question 22

types of kernels

Answer 22

quadratic, polynomial, rbf

Question 23

when to use cross validation?

Answer 23

when the data set is small
when we want to do hyperparameter tuning

Question 24

why have a validation set?

Answer 24

to make sure the model is not overfitted (hyperparameters)

Question 25

what is inductive bias

Answer 25

bias from assumptions that the model makes based on its design

Question 26

type 1 error

Answer 26

false positive, predicted positive when it should have been negative

Question 27

type 2 error

Answer 27

false negative, predicted negative when it should have been positive

Question 28

accuracy

Answer 28

correct vs total

Question 29

recall

Answer 29

also called true positive rate TP / (TP + FN) how many predicted positives were correct

Question 30

false positive rate

Answer 30

FP / (FP + TN)

Question 31

precision

Answer 31

TP / (TP + FP) how many true positives were positive?

Question 32

how to set up confusion matrix

Answer 32

actual on top, predicted on the side positive positive in the top left

Question 33

why does precision recall tradeoff exist

Answer 33

precision and recall are sort of inversely related guessing more positives leads to higher recall, but lower precision

Question 34

ensemble learning for classifiers

Answer 34

hard voting - predict the class with the most votes soft voting - predict the class with the highest probability

Question 35

what is the base rate fallacy?

Answer 35

occurs when the positive label is rare most of the time, then, when we predict positive, it is a false positive