Class Imbalance and Dimensionality Reduction

Question 1

What is the advantage of Leaky ReLU?

Answer 1

Allows small negative values to avoid ‘dead neurons’

Leaky ReLU modifies the traditional ReLU to allow a small, non-zero gradient when the unit is not active.

Question 2

What does the sigmoid function do?

Answer 2

Maps to (0, 1); good for binary classification

The sigmoid function is often used in logistic regression and binary classification tasks.

Question 3

What is the range of the tanh function?

Answer 3

Maps to (-1, 1)

The tanh function is symmetric around the origin, which can be beneficial for learning.

Question 4

What is the purpose of optimisers in training?

Answer 4

Update weights to minimise loss

Optimisers adjust the weights of the model to improve performance based on the loss function.

Question 5

What does SGD stand for?

Answer 5

Simple gradient descent

SGD is a common optimisation algorithm used in machine learning.

Question 6

What does Adam optimiser combine?

Answer 6

Momentum + RMSprop

Adam is an adaptive learning rate optimisation algorithm that is popular due to its efficiency.

Question 7

Why is visualising CNNs important?

Answer 7

See what filters are learning, Debug issues, Understand model behaviour

Visualisation helps in interpreting the features learned by convolutional neural networks.

Question 8

What is one technique for visualising CNNs?

Answer 8

Feature map visualisation

This technique helps in understanding which features are being activated by certain inputs.

Question 9

Fill in the blank: Data augmentation helps fight overfitting by training on ‘______’ versions of your data.

Answer 9

new

Question 10

What is class imbalance?

Answer 10

Class imbalance = When one class has way more examples than another.

This can lead to biased model predictions.

Question 11

What is an example of class imbalance?

Answer 11

Negative samples: 998, Positive samples: 2.

Such a scenario can heavily skew model performance.

Question 12

What accuracy could a model achieve by always predicting ‘Negative’ in a class imbalance scenario?

Answer 12

99.8% accuracy.

This illustrates how misleading accuracy can be in imbalanced datasets.

Question 13

Why is class imbalance a problem?

Answer 13

Model ignores the minority class.

This can lead to poor performance in predicting the minority class.

Question 14

What is the impact of class imbalance on model predictions?

Answer 14

Biased boundaries = Bad predictions.

Class imbalance can result in a model that is biased towards the majority class.

Question 15

In which areas is class imbalance especially problematic?

Answer 15

Medical diagnoses, Fraud detection, Rare event prediction

These domains often involve critical decisions based on minority classes.

Question 16

What is Binary Cross-Entropy Loss (BCE)?

Answer 16

L_BCE = - y_i * log(y_i) - (1 - y_i) * log(1 - y_i).

BCE is commonly used for binary classification tasks.

Question 17

How does class imbalance affect the Binary Cross-Entropy Loss?

Answer 17

Majority class dominates the loss function.

This can lead to suboptimal learning for the minority class.

Question 18

What metric does the model optimize for in the presence of class imbalance?

Answer 18

Overall accuracy, not fair balance.

This can result in a model that performs well overall but poorly on minority classes.

Question 19

What is a solution to class imbalance in model training?

Answer 19

Weighted Loss Functions.

These functions adjust the loss to give more importance to the minority class.

Question 20

What is Weighted Binary Cross Entropy?

Answer 20

Assign higher importance (weight) to the minority class.

This helps to mitigate the effects of class imbalance.

Question 21

Provide a Keras example for setting class weights.

Answer 21

class_weights = {0: 1.0, 1: 5.0}.

This example assigns a weight of 5 to the minority class.

Question 22

What is Weighted Categorical Cross Entropy used for?

Answer 22

For multi-class problems.

This is an extension of weighted binary cross-entropy for multiple classes.

Question 23

What is one strategy for fixing imbalanced data?

Answer 23

Collect more data.

Increasing the number of examples for the minority class can help balance the dataset.

Question 24

What is oversampling?

Answer 24

Duplicate minority class samples.

This can help to balance the dataset by increasing the representation of the minority class.

Question 25

What is undersampling?

Answer 25

Remove majority class samples. ## Footnote This technique reduces the number of examples in the majority class to balance the dataset.

Question 26

What is data augmentation?

Answer 26

Make more diverse samples for the minority. ## Footnote This technique generates new training examples by modifying existing ones.

Question 27

What does SMOTE stand for?

Answer 27

Synthetic Minority Over-sampling Technique. ## Footnote SMOTE is a popular method for generating synthetic samples in the minority class.

Question 28

What are the steps involved in SMOTE?

Answer 28

Pick a minority sample, Find its k-nearest neighbors, Interpolate a new sample between it and a neighbor. ## Footnote This process helps to create new, synthetic examples of the minority class.

Question 29

What is an analogy for the SMOTE process?

Answer 29

Draw a line between two known dots and place a new dot somewhere along it. ## Footnote This visualization helps to understand how SMOTE generates new samples.

Question 30

What is the limitation of SMOTE?

Answer 30

Great for structured datasets – not ideal for raw images. ## Footnote SMOTE's effectiveness can vary based on the type of data being used.

Question 31

What is the purpose of data augmentation for images?

Answer 31

Trick your model into seeing new examples by tweaking real ones. ## Footnote This technique helps improve model robustness and generalization.

Question 32

Provide a Keras example for data augmentation.

Answer 32

datagen = ImageDataGenerator(rotation_range=40, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode='nearest'). ## Footnote This example shows how to create a data generator for augmenting image data.

Question 33

What is a warning regarding data augmentation and SMOTE?

Answer 33

Don’t augment or SMOTE your whole dataset before splitting into train/test. ## Footnote This can lead to data leakage and biased evaluations.

Question 34

What set should you apply balancing/augmentation to?

Answer 34

Only on the training set. ## Footnote This ensures that the model is trained on augmented data without contaminating the test set.

Question 35

Why is accuracy misleading in the presence of class imbalance?

Answer 35

Accuracy does not reflect the performance on the minority class. ## Footnote High accuracy can be achieved by favoring the majority class.

Question 36

What are better metrics to use when class imbalance exists?

Answer 36

Recall, Precision, Accuracy ## Footnote These metrics provide a more nuanced view of model performance on imbalanced datasets.

Question 37

What does Recall measure?

Answer 37

TP / (TP + FN). ## Footnote Recall indicates the ability of a model to find all relevant cases.

Question 38

What does Precision measure?

Answer 38

TP / (TP + FP). ## Footnote Precision reflects the accuracy of positive predictions made by the model.

Question 39

What does Accuracy measure?

Answer 39

(TP + TN) / total. ## Footnote Accuracy gives the overall proportion of correct predictions.

Question 40

What is the ideal combination of metrics for a model?

Answer 40

High recall + high precision. ## Footnote This combination indicates that the model is effectively identifying positive cases while minimizing false positives.