Assessing Algorithms

Question 1

KNN Pro/Con

Answer 1

P - Doesn’t overfit model

C - Can’t extrapolate

Question 2

Parametric Pro/Con

Answer 2

P - Ability extrapolate

C -

Question 3

RMS Error

Answer 3

sqrt ( (sum(actual - predicted)^2) / n))

Question 4

out of sample error

Answer 4

RMSE on test data

Error for data predicted outside of training model data

Question 5

Cross validation

Answer 5

When there isn’t enough training data, split existing data into chunks.

Then, use different combinations of chunks to train/test, eg round1: test(1-4), train(5), round2: test(2-5), train(1)

Question 6

Cross validation and financial data

Answer 6

Because it can “peek” at future values, it does not fit finance data well

Question 7

Roll forward cross validation

Answer 7

Cross Validation where Training data is ALWAYS before test data

Question 8

Correlation

Answer 8

For a regression algorithm:
Look at relationship between predicted and actual values

Scatter plot actual and predicted values, and calc correlation coefficient

Correlation != slope

Question 9

Overfitting

Answer 9

When in sample error decreases and out of sample error increases