Statistics/ML

Question 1

Bonferroni

Answer 1

Alpha/m

Question 2

Bagging

Answer 2

Take n bootstrap samples, fit model to each, take average

Question 3

Random forest

Answer 3

Basically bagging with decision trees, but for each split, choose from random p/3, for example, of the covariates

Question 4

Support vector machines

Answer 4

Maximize M subject to all but some number of points being farther away than M from decision boundary, with sum of distances for those that aren’t <= C

Question 5

Newton Raphson

Answer 5

x_1 = x_0 - f(x_0)/f’(x_0)

Question 6

Gradient descent

Answer 6

x_1 = x_0 - gamma * gradient of F

Question 7

Logit

Answer 7

P(y=1) = e^(X beta)/(e^(X beta)+1)

log(p/(1-p)) = X beta

Question 8

K nearest neighbors

Answer 8

Use plurality vote for classification, or mean for regression

Question 9

Std error

Answer 9

Std deviation of a statistic’s sampling distribution , or an estimate of it, eg

(1/sqrt(n))sqrt(sum(xi - mean)^2)

Question 10

Normal density

Answer 10

(1/sigma sqrt(2 pi)) e^( - (1/2)((x- mu)^2/sigma^2))

Question 11

T test

Answer 11

Sample should be normal, but ok for large samples I believe.

Tau-hat/(se(tau-hat))

Eg difference divided by

Sigma hat * Sqrt (1/n1 + 1/n2)

Question 12

Covariance of beta hat for regression

Answer 12

Sigma^2. (X’X)^(-1)

Estimate sigma with
1/(n-p). *. Sum of (y - X beta)^2

Question 13

Law of large numbers

Answer 13

Lim as n -> inf
P(|mean(Y1,..,Yn) - mu|>= ep)
=0

Question 14

Central Limit Theorem

Answer 14

Limit as n -> infinity
P(. (1/(sigma/sqrt(n)). *
(Mean(Y1,…,Yn) - mu) <= z). =

Phi(z)

Ie

(1/(sigma/sqrt(n)) times (Ybar - mu)
converges to a unit normal

Note that Ybar has std dev
sigma * sqrt(n)

Question 15

SUTVA

Answer 15

Stable Unit Treatment Value Assumption

Response of one unit only depends on their treatment not on treatment of others

Eg if some people assigned to travel on public transportation, some in cars, then wouldn’t hold, because it affects the traffic

Question 16

Kmeans

Answer 16

Start with initial k points.
Assign points to one of these.
Calculate new centroids
Repeat

Question 17

SMOTE

Answer 17

For class imbalance:
select one of uncommon class at random,
then select neighbor in class at random,
draw line between them, choose something on line, assign uncommon value to that

You can also undersample majority class and/or oversample (i.e. repeat) minority class.

Question 18

Accuracy

Answer 18

(TP + TN)/
(TP + TN + FP + FN)

Question 19

Precision

Answer 19

TP/(TP + FP)
PREcision is TP divided by PREdicted positive

Question 20

Sensitivity

Answer 20

TP/(TP + FN)
SeNsitivity is Positive
Correct positives among all positives

Question 21

Specificity

Answer 21

TN/(TN + FP)

Also called recall

SPIN. sPecificity is Negative
Correct negatives among all negatives

Question 22

Amazon recommendation systems, Netflix

Answer 22

item-based collaborative filtering algorithms, using cosine distance.

Question 23

Type 1 and 2

Answer 23

Type 1 False Positive, ie mistaken rejection of null hypothesis
Type 2 False Negative

Question 24

softmax

Answer 24

exp(zi) / (sum exp(zi))

Question 25

imputing missing values

Answer 25

can just impute with mean or median etc. or some constant

“MICE stands for Multivariate Imputation via Chained Equations, and it’s one of the most common packages for R users. It assumes the missing values are missing at random (MAR).

The basic idea behind the algorithm is to treat each variable that has missing values
as a dependent variable in regression and treat the others as independent
(predictors).

https://www.r-bloggers.com/2023/01/imputation-in-r-top-3-ways-for-imputing-missing-data/

Question 26

gbm hyperparameters

Answer 26

shrinkage or learning rate
add on only small multiple of new tree at each step

bag fraction
“fraction of independent training observations selected to create the next tree in the
expansion. Introduces randomness in the model fit; if bag_fraction < 1 then running the
same model twice will result in similar but different fits.”

num_features: “number of random features/columns to use in training model. “

interaction depth: max depth of each tree (although different in gbm3)

some of this from gbm3 documentation

Question 27

boxplot definitions

Answer 27

` boxplot quantile 25 to 75, median, outer lines at most 1.5*IQR, outliers