Decision Trees Flashcards
(20 cards)
What is a decision tree?
A flowchart-like model that makes predictions by asking binary questions on features.
What are decision nodes in a decision tree?
Points where the tree asks a question about a feature.
What are leaf nodes in a decision tree?
Endpoints of the tree that assign a class label or prediction.
What is the role of edges in a decision tree?
They represent the outcome (True/False) of a decision node’s question.
What does entropy measure in decision trees?
The level of disorder or impurity in the data.
What is the entropy when all data points belong to one class?
Zero — the data is perfectly pure.
What is the entropy when classes are evenly split?
One — the data is maximally impure.
What is the formula for entropy?
E(x) = -Σ P(i) log₂ P(i), where P(i) is the probability of class i.
What is information gain in decision trees?
The reduction in entropy after splitting on a feature.
What is the formula for information gain?
G(F) = E(S) - E(F)
What feature should a decision tree split on first?
The one with the highest information gain.
What does a high information gain mean?
The feature splits the data into more homogeneous groups.
How do decision trees handle continuous features?
By testing split points between sorted values using midpoint thresholds.
What is the first step to split a continuous feature?
Sort the values in ascending order.
How are candidate split points chosen for continuous features?
By calculating midpoints between adjacent values.
What happens at each candidate split for continuous features?
The data is divided into ≤ and > groups, and information gain is calculated.
What is a limitation of decision trees shown in the flood example?
They can memorize training data and fail to generalize to physical reality.
Why are decision trees considered high variance models?
They are sensitive to small data changes and prone to overfitting.
What are some advantages of decision trees?
Easy to interpret, handles both numerical and categorical data, no need for scaling.
What are some disadvantages of decision trees?
Prone to overfitting, sensitive to noisy data, biased toward features with many levels.
