Random Forest

Question 1

What is a random forest?

Answer 1

An ensemble of decision trees that votes to make predictions.

Question 2

Why use multiple decision trees in a random forest?

Answer 2

To reduce overfitting and improve generalisation.

Question 3

What type of model is a random forest?

Answer 3

An ensemble learning method.

Question 4

How does a random forest make a prediction?

Answer 4

By taking a majority vote across all decision trees.

Question 5

What problem does a random forest solve that a single decision tree can’t?

Answer 5

Overfitting and poor generalisation.

Question 6

What is bootstrapping in random forests?

Answer 6

Sampling training data with replacement to create varied datasets for each tree.

Question 7

Why is sampling done with replacement in bootstrapping?

Answer 7

To allow some examples to appear multiple times and some to be left out, increasing model diversity.

Question 8

What is the size of each bootstrapped sample?

Answer 8

The same as the original dataset size, but with duplicates and omissions.

Question 9

What percentage of data is typically included in each bootstrapped set?

Answer 9

Around 63% of the original training data.

Question 10

What are out-of-bag (OOB) samples?

Answer 10

The ~37% of data not included in a tree’s bootstrapped training set.

Question 11

What is the purpose of out-of-bag samples?

Answer 11

To estimate model error without needing a separate validation set.

Question 12

How is OOB error calculated?

Answer 12

By testing each OOB sample on trees that did not train on it, then averaging the results.

Question 13

What does a low OOB error indicate?

Answer 13

That the model is generalising well to unseen data.

Question 14

What evaluation method can be used if OOB is not suitable?

Answer 14

A traditional train/validation/test split or cross-validation.

Question 15

How does each tree in a random forest avoid using the same features at each split?

Answer 15

By randomly selecting a subset of features at each split (feature bagging).

Question 16

What is feature bagging?

Answer 16

Randomly selecting a subset of features at each decision node to promote tree diversity.

Question 17

Why is feature bagging important in random forests?

Answer 17

It reduces correlation between trees, increasing the strength of the ensemble.

Question 18

What is the final prediction in a random forest classification task?

Answer 18

The class with the majority vote from all trees.

Question 19

What kind of learning algorithm is each tree in a random forest?

Answer 19

A weak learner (specifically, a high-variance model).

Question 20

What kind of data can random forests handle?

Answer 20

Both categorical and continuous features.

Question 21

Do random forests require feature scaling?

Answer 21

No, they are unaffected by feature magnitude.

Question 22

Are random forests robust to outliers?

Answer 22

Yes, because decision trees are not sensitive to extreme values.

Question 23

What is a key advantage of random forests over single trees?

Answer 23

They reduce variance and overfitting through averaging.

Question 24

What happens if too few trees are used in a random forest?

Answer 24

The model may underperform due to insufficient averaging.

Question 25

Why is a random forest less transparent than a decision tree?

Answer 25

You can't easily trace how a prediction was made across many trees.

Question 26

What is a major strength of random forests?

Answer 26

They reduce overfitting compared to single decision trees.

Question 27

Why are random forests robust to noisy data?

Answer 27

Because individual errors are averaged out across many trees.

Question 28

Why are random forests considered flexible?

Answer 28

They work well with both classification and regression problems.

Question 29

Can random forests handle missing data?

Answer 29

Yes, they can tolerate some missing values during training or prediction.

Question 30

How do random forests support feature importance analysis?

Answer 30

They estimate how much each feature contributes to prediction accuracy.

Question 31

Why are random forests good for high-dimensional data?

Answer 31

Because they can model complex interactions and ignore irrelevant features.

Question 32

What kind of model variance do random forests have compared to single trees?

Answer 32

Lower variance due to ensemble averaging.

Question 33

Can random forests capture non-linear patterns?

Answer 33

Yes, the ensemble of trees can model complex decision boundaries.

Question 34

Why are random forests often used as baselines in ML tasks?

Answer 34

Because they tend to perform well out-of-the-box without much tuning.

Question 35

What is a key limitation of random forests in terms of interpretability?

Answer 35

They are difficult to interpret compared to individual decision trees.

Question 36

Why are random forests computationally expensive?

Answer 36

They require training and storing many trees.

Question 37

How does prediction speed compare between decision trees and random forests?

Answer 37

Random forests are slower due to combining many tree outputs.

Question 38

What is a memory drawback of random forests?

Answer 38

They consume more memory because all trees must be stored.

Question 39

Can random forests extrapolate beyond the training range?

Answer 39

No, like decision trees, they cannot extrapolate well for continuous values.

Question 40

What kind of overfitting risk still exists in random forests?

Answer 40

If the number of trees is too small or trees are too deep, overfitting may still occur.

Question 41

Are random forests sensitive to class imbalance?

Answer 41

Yes, they may favor the majority class unless sampling or weighting is adjusted.

Question 42

What is a drawback when using random forests with very large datasets?

Answer 42

Training can be slow and require parallelization or optimization.

Question 43

Why is explaining individual predictions difficult in random forests?

Answer 43

Because the final decision results from many complex tree paths.

Question 44

How do random forests handle categorical variables with many levels?

Answer 44

They may overfit unless levels are grouped or encoded properly.

Question 45

What is ensemble learning?

Answer 45

Combining multiple models to produce a stronger overall model.