How to Use RFE for Feature Selection

Question 1

WHAT ARE THE TWO IMPORTANT CONFIGURATION OPTIONS WHEN USING RFE? P192

Answer 1

The choice in the number of features to select (n_features_to_select)
The choice of the algorithm used to help choose features. (estimator)

Question 2

IS THE PERFORMANCE OF THE RFE STRONGLY DEPENDENT ON n_features_to_select & estimator HYPERPARAMETERS BEING TUNED? P192

Answer 2

No

Question 3

WHAT DOES “RFE IS A WRAPPER-TYPE FEATURE SELECTION ALGORITHM” MEAN? P193

Answer 3

This means that a different machine learning algorithm is given and used in the core of the method, is wrapped by RFE, and used to help select features

Question 4

HOW DOES RFE WORK? P193

Answer 4

1-Fitting the given ML algorithm used in the core of the model
2-Ranking features by importance
3-discarding the least important features
4-re-fitting the model and repeating, till the desired number of features remain

Question 5

HOW CAN WE FIND OUT WHICH FEATURES WERE CHOSEN BY RFE? P194

Answer 5

“support_” attribute. It provides True/False for each feature

Question 6

HOW CAN WE SYNTHESIZE A CLASSIFICATION PROBLEM? CODE P194

Answer 6

X, y = make_classification (n_samples=1000, n_features=10, n_informative=5, n_redundant=5, random_state=1)

Question 7

HOW CAN WE AUTOMATICALLY SELECT THE BEST NUMBER OF FEATURES FOR RFE? P200

Answer 7

Using RFECV

Question 8

USING WHICH PARAMETER OF RFECV, CAN WE CHOOSE THE MINIMUM NUMBER OF CHOSEN FEATURES? WHAT IS ITS DEFAULT VALUE? P200

Answer 8

min_features_to_select, default=1

Question 9

USING WHICH ATTRIBUTE OF RFE AND RFECV CLASSES CAN WE FIND OUT THE RANKING OF FEATURES? P201

Answer 9

“ranking_” attribute