8.5 Evaluating Predictive Models – Accuracy & Beyond Flashcards
Why can accuracy be misleading in machine learning?
Because it doesn’t account for imbalanced data or the type of errors being made.
When might a model with 99.9% accuracy be considered bad?
When predicting rare events like fraud, since predicting “not fraud” all the time can still give very high accuracy but miss actual fraud cases.
In critical applications like medical diagnosis, what matters more than overall accuracy?
- The type of error
- e.g., missing a sick patient (false negative) can be more dangerous than giving treatment to a healthy person (false positive).
What is a True Positive (TP) in binary classification?
A positive case correctly predicted as positive.
What is a True Negative (TN)?
A negative case correctly predicted as negative.
What is a False Positive (FP)?
A negative case incorrectly predicted as positive.
What is a False Negative (FN)?
A positive case incorrectly predicted as negative.
What does a confusion matrix show?
It shows how predicted classes relate to actual classes, helping to identify false positives and false negatives.
Why is the confusion matrix useful?
it helps us inspect the types of errors the model is making, not just the number.
What is Accuracy as a metric?
The percentage of all predictions that were correct.
What is Precision?
The percentage of positive predictions that were actually positive.
E.g., “Of the customers we said would buy, how many actually did?”
What is Recall?
The percentage of actual positives that were correctly predicted.
E.g., “Of all customers who actually bought, how many did we correctly identify?”
