lecture 9 Flashcards
(29 cards)
What are decision trees used for?
Classifying data by recursively splitting it based on feature values.
What is an internal node in a decision tree?
A decision point that tests a feature to split data.
What is a leaf node in a decision tree?
A terminal node that assigns a class label.
Why are decision trees prone to overfitting?
They can learn noise and irrelevant patterns in training data.
What is the primary advantage of decision trees?
They are easy to interpret and visualize.
What is a common method to regularize decision trees?
Limiting tree depth or pruning unnecessary branches.
What is the standard algorithm for training decision trees?
The ID3 (Iterative Dichotomiser 3) algorithm.
What is the greedy strategy in decision tree learning?
Choosing the best split at each step without backtracking.
How do decision trees handle missing values?
They can assign the most common value or split on other features.
What is entropy in decision trees?
A measure of uncertainty in a dataset.
What is information gain?
The reduction in entropy after splitting on a feature.
What is the Gini index?
A measure of how pure a split is in decision trees.
What is overfitting in decision trees?
When a tree is too complex and memorizes training data instead of generalizing.
What is pruning in decision trees?
Removing branches that do not improve generalization.
What is an ensemble model?
A combination of multiple models to improve performance.
What is bagging in ensemble learning?
Training multiple models on different subsets of the data and averaging predictions.
What is the key idea behind random forests?
Building multiple decision trees with random feature selection to improve generalization.
What is boosting in ensemble learning?
A method that trains models sequentially, giving more weight to misclassified instances.
What is a weak learner in boosting?
A model that performs slightly better than random chance.
What is AdaBoost?
A boosting algorithm that combines weak learners to create a strong classifier.
What is gradient boosting?
A boosting method that optimizes a loss function by adding models sequentially.
How does boosting differ from bagging?
Boosting trains models sequentially, while bagging trains models independently.
What is the role of decision stumps in boosting?
They serve as simple weak learners in boosting algorithms.
Why are ensemble models often better than single models?
They reduce variance and improve generalization.
