Attribute Selection and Imbalanced Classes

Question 1

What are the two approaches that can be used to select attributes?

Answer 1

Filter and Wrapper

Question 2

What is the Filter approach?

Answer 2

Attribute selection is INDEPENDANT of classification algorithm to be applied later.
- Evaluates the quality of candidate attribute
subset without use the target classification
algorithm

Question 3

What is the Wrapper approach?

Answer 3

Attribute selection is TAILORED to the classification to be applied later.
- Quality of candidate attribute is evaluated
based on the classification accuracy (on
training data) of the target classification
algorithm run with attribute subset.

Question 4

What are the main components of most attribute selection methods?

Answer 4

A search method

An evaluation function

Question 5

What is the pros and cons of Sequential Attribute Selection Methods?

Answer 5

Pros:
- Forwards and backwards are simple to understand and implement.
- Forwards sequential is fast.

Cons:
- Backwards selection is quite slow
- Both are heuristic methods - no guarantee of finding the optimal solution
- Both are greedy (do not cope well with interactions)

Question 6

What is the approach that can be used to deal with imbalanced classes?

Answer 6

Re-sampling techniques

Question 7

What are the different Re-sampling techniques?

Answer 7

Under-sampling the majority class
Over-sampling the minority class
Hybrid approaches

Question 8

How does Under-sampling work?

Answer 8

Remove randomly chosen instances from majority class. This:
- Throws away a lot of relevant information.
- Reduces the time taken to run the
classification algorithm.

Question 9

How does Over-sampling work?

Answer 9

Duplicate minority class instances chosen at random or create new synthetic minority class instances. This:
- Avoids the loss of instances associated with
Under-sampling.
- Introduces redundancy or new new
potentially noisy class labels.
- Increases the time taken to run the
classification algorithm.

Question 10

What is SMOTE?

Answer 10

Synthetic Minority Oversampling TEchnique

Question 11

What does SMOTE do?

Answer 11

Creates synthetic instances along the line joining a minority class instances and some or all of its Nearest Neighbours of the same minority class.