Feature Selection Flashcards
(15 cards)
What is feature selection
the process of identifying and selecting the most relevant features (variables) from a dataset to improve model performance, reduce overfitting, and enhance interpretability, by removing irrelevant or redundant features
Why does feature selection improve the model performance?
By focusing on the most predictive features, models can learn more accurately and generalize better to unseen data
Why does feature selection reduce overfitting?
Overfitting occurs when a model learns the training data too well, including noise and irrelevant details, leading to poor performance on new data. Feature selection helps prevent this by focusing on the most relevant information.
Why does feature selection enhance interpretability?
Simpler models with fewer features are easier to understand and explain, which can be crucial for decision-making and trust in the model’s predictions.
Why does feature selection reduce computational cost?
Using fewer features can significantly speed up model training and reduce the memory required to store and process the data.
Why does feature selection reduce data?
Feature selection helps to reduce the dimensionality of the dataset, which can be beneficial for both storage and processing
What are the filter methods?
These methods evaluate the relevance of features independently of any machine learning algorithm, using statistical measures like correlation, chi-square, or information gain
What are the wrapper methods
These methods use a machine learning algorithm to evaluate different subsets of features, selecting the subset that yields the best performance
What are embedded methods?
These methods integrate feature selection into the model training process, allowing the model to learn which features are most important during training
What is recursive feature elimination? RFE
This wrapper method iteratively removes features based on their importance scores, starting with all features and gradually reducing the set until the desired number of features is reached
What is sequential feature selection (SFS)
This method builds a feature subset by sequentially adding or removing features based on their impact on model performance
What is principal component analysis? PCA
This technique transforms the data into a new set of uncorrelated variables (principal components), allowing for dimensionality reduction and feature selection.
What is the chi-squared test?
A filter method, the most common, that calculates a chi-square score for every input variable against the target and the variables with the highest score are selected as the input set of features
What is the Pearson correlation?
Pearson correlation coefficient is calculated for each input against each other when several independent variables are correlated. One among each highly correlated variable is removed.
What are decision tree algorithms?
Features are selected such that a node condition should be able to split the data so that similar values in the target variable end up in the same split.
