Model Evaluation, Hyperparameter Tuning, Classification & Regression Metrics

Question 1

What is the solution to model evaluation problems?

Answer 1

Split the data into training, validation, and test sets.

Question 2

What is a Training Set?

Answer 2

Used to train the model.

Question 3

What is a Validation Set?

Answer 3

Used during training to tune hyperparameters.

Question 4

What is a Test Set?

Answer 4

Used after training to check final performance.

Question 5

What is the Holdout method in model validation?

Answer 5

One-time split: Train (e.g. 60%), Val (20%), Test (20%).

Question 6

When is the Holdout method best used?

Answer 6

For large datasets.

Question 7

What is K-Fold Cross Validation (KCV)?

Answer 7

Split data into k parts, rotate training/testing.

Question 8

When is K-Fold Cross Validation best used?

Answer 8

Best for small data, better accuracy.

Question 9

What is Overfitting?

Answer 9

Too good on training, bad on new data.

Question 10

What is Underfitting?

Answer 10

Bad on both training and new data.

Question 11

What are the characteristics of Overfitting?

Answer 11

High variance, memorizing.

Question 12

What are the characteristics of Underfitting?

Answer 12

High bias, guessing.

Question 13

What is Early Stopping?

Answer 13

Stop training when validation loss goes up.

Question 14

What is L2 Regularization?

Answer 14

Penalizes large weights to keep the model simple.

Question 15

What is the L2 Regularization formula?

Answer 15

λ * Σ(weights²) → encourages smaller weights.

Question 16

What are Hyperparameters?

Answer 16

Settings you pick before training (not learned from data).

Question 17

Give examples of Hyperparameters.

Answer 17

• Learning rate
• Batch size
• Number of layers
• Activation functions

Question 18

What is Grid Search?

Answer 18

Try every combo of settings.

Question 19

When is Grid Search effective?

Answer 19

Good for small search spaces.

Question 20

What is a disadvantage of Grid Search?

Answer 20

Super slow if too many options.

Question 21

What is Random Search?

Answer 21

Pick random combos.

Question 22

When is Random Search better?

Answer 22

Better for large/continuous spaces.

Question 23

What are Classification Models used for?

Answer 23

To assign a class (label) to data.

Question 24

What is an example of a Classification Model?

Answer 24

Is this email spam? Is the tumor benign or malignant?

Question 25

What is Accuracy in model evaluation?

Answer 25

% of correct predictions.

Question 26

What is a limitation of Accuracy?

Answer 26

Doesn't work well when classes are imbalanced.

Question 27

What is a Confusion Matrix?

Answer 27

A table used to describe the performance of a classification model.

Question 28

What does TP stand for in a Confusion Matrix?

Answer 28

True Positives.

Question 29

What does FN stand for in a Confusion Matrix?

Answer 29

False Negatives.

Question 30

What does FP stand for in a Confusion Matrix?

Answer 30

False Positives.

Question 31

What does TN stand for in a Confusion Matrix?

Answer 31

True Negatives.

Question 32

What is Precision?

Answer 32

TP / (TP + FP) → How many predicted positives were correct?

Question 33

What is Recall?

Answer 33

TP / (TP + FN) → How many actual positives were found?

Question 34

What is the F1 Score?

Answer 34

Harmonic mean of Precision & Recall → 2 * (P * R) / (P + R).

Question 35

What is AUC?

Answer 35

Area Under Curve measures model’s ability to distinguish classes.

Question 36

When should you use F1 or AUC?

Answer 36

When data is imbalanced or missing a positive is worse than a few false alarms.

Question 37

What are Regression Models used for?

Answer 37

When predicting a number (not a category).

Question 38

Give examples of Regression Models.

Answer 38

* House prices * Stock market trends

Question 39

What is MAE?

Answer 39

Mean Absolute Error – average of errors, less sensitive to outliers.

Question 40

What is MSE?

Answer 40

Mean Squared Error – squares errors, punishes big mistakes more.

Question 41

What are Clustering Models used for?

Answer 41

When you don’t have labels – model tries to find natural groupings.

Question 42

Give an example of a Clustering Model use case.

Answer 42

Segmenting customers into behavior types.

Question 43

What metric is used for evaluating clustering?

Answer 43

Silhouette Coefficient.

Question 44

What does a higher Silhouette Coefficient indicate?

Answer 44

Better clustering.