Questions #2

Question 1

True or false : Putting too many parameters into a model results in overfitting the model

Answer 1

True

Question 2

True or false : The best model should provide the most information

Answer 2

True

Question 3

Can you tell me the definition of Information

Answer 3

The information of an outcome is defined as the decrease in uncertainty from observing the outcome

Question 4

What are the 3 properties for a measure of uncertainty?

Answer 4

1. Continuity : Should be a continuous function of the parameters of the distribution
2. Additivity
3. Monotonicity

Question 5

What are the unique measure of uncertainty that satisfies the 3 properties ?

Answer 5

Information entropy

Question 6

What is the definition of cross-entropy?

Answer 6

Cross-entropy is a measure of uncertainty of using a different distribution with event probabilities q, to estimate a distribution with the same events with probabilities p

Question 7

True or false : Cross-entropy is symmetric

Answer 7

False

Question 8

True or false : Using a low-entropy distribution to predict a high entropy distribution is worst than the opposite

Answer 8

True

Question 9

True or false : The Kullback-Leibler Divergence grows as the esitmate moves away from the true distribution

Answer 9

True

Question 10

True or false : The Kullback-Leibler Divergence is symmetric

Answer 10

False

Question 11

Define the Deviance formula

Answer 11

-2 times the loglikelihood

Question 12

True or false : The lower the deviance, the better

Answer 12

True

Question 13

Tell me the steps to calculate LPPD

Answer 13

1. At each point, take the average of the sample
2. Log the average
3. Sum all the logs over all points

Question 14

True or false : Deviance is measure of predictive accurary, not of truth

Answer 14

True

Question 15

True or false : When doing cross-validation, to truly use deviance or lppd as measure of accuracy, it should be calculated on the test data

Answer 15

True, because the deviance will be lower on the training data when we add parameters, even if they are not relevant

Question 16

Complete : One way to avoid overfitting is to use ..

Answer 16

regularizing prior

Question 17

What is the regularizing prior?

Answer 17

A regularizing prior is one that contains information. The more informationit has, the stronger the regularization

Question 18

True or false : A regularizing prior is skeptical of the information; the stronger the regularization, the more data is needed to overwhelm it

Answer 18

True

Question 19

True or false : LOOCV does not require lots of computer runs

Answer 19

False. It requires a lot of computer runs

Question 20

Can you tell me an alternative to LOOCV

Answer 20

Pareto-Smoothed importance sampling cross-validation (PSIS)

Question 21

Can you tell me an alternative to LOOCV which is not PSIS

Answer 21

Information criteria

Question 22

Define the formula of the AIC with the deviance

Answer 22

Use the D of the training data

Question 23

When we are calculating AIC, we are using the deviance of the training data. AIC can be calculated with the deviance of the test if : (3)

Answer 23

1. The prior is flat
2. The posterior is approximately multivariate normal
3. The size of the same is much greater than the number of parameters

Question 24

True or false : WAIC and PSIS are similar for ordinary linear models

Answer 24

TRUE

Question 25

Cross-validation and PSIS have higher variance as estimators of divergence, but WAIC has higher bias

Answer 25

True

Question 26

Large differences between WAIC and PSIS imply one of them is unreliable

Answer 26

True

Question 27

WAIC identifies highly influential observations, unlike PSIS

Answer 27

False, c'est le contraire

Question 28

True or false : When doing the information criteria, it is important that each model has the same number of observations

Answer 28

True

Question 29

True or false : Normal distribution has a heavier tail than the student distribution

Answer 29

False. Student has a heavier tail