Week 9

Question 1

Motivate ensemble methods

Answer 1

Question 2

Idea of ensemble methods

Answer 2

Question 3

What is Bagging

Answer 3

Bootstrap aggregating

Question 4

What is boosting?

Answer 4

Question 5

Bagging - process

Answer 5

Question 6

Goal of boosting

Answer 6

Question 7

Boosting procedure

Answer 7

Question 8

AdaBoost

Answer 8

Question 9

AdaBoost advantages/disadvantages

Answer 9

Question 10

AdaBoost algo

Answer 10

Question 11

Stacked generalisation

Answer 11

Question 12

Parallel structure for ensemble classifiers

Answer 12

Question 13

Serial structure for ensemble classifiers

Answer 13

Question 14

Hierarchical structure for ensemble classifiers

Answer 14

Question 15

Random forest

Answer 15

(Random forest is ensemble of BAGGED decision trees, with randomised feature selection)

Question 16

Training decision tree

Answer 16

Question 17

Notes on decision trees

Answer 17

Decisions at nodes could be more complex
Prediction performance can be poor (tends to overfit)
Unstable (small changes to training set cause large changes in classification accuracy. although bagging improves stability)

Question 18

Constructing binary decision tree algorithm

Answer 18

Question 19

Randomised tree learning procedures algo

Answer 19

Question 20

Advantages of random forests

Answer 20

Fast (scalable)
Accurate
Simple to implement
Popular