preventing overfitting Flashcards
(25 cards)
What is the goal of generalisation in machine learning?
To perform well on unseen data, not just the training set.
What is early stopping?
A method that stops training when validation loss begins to increase.
Why is early stopping useful?
It prevents overfitting and reduces unnecessary computation.
What does data augmentation do?
Expands the training dataset by modifying existing samples.
What are common image augmentation techniques?
Rotation, flipping, noise addition, and brightness variation.
What is a common audio augmentation technique?
Pitch shifting or adding background noise.
What is a text augmentation method?
Synonym replacement or sentence shuffling.
What is a drawback of data augmentation?
It can introduce irrelevant distortions or noise.
What does dropout do in neural networks?
Randomly disables neurons during training to reduce overfitting.
Why does dropout help generalisation?
It prevents reliance on specific neurons and encourages redundancy.
What happens to dropout during inference?
It is disabled and full connectivity is restored.
What is a potential downside of dropout?
Slower training convergence and less reproducibility.
What is explicit regularisation?
Adding a penalty to the loss function to control model complexity.
What does L1 regularisation encourage?
Sparsity by driving some weights to zero.
What is the formula for L1 regularisation penalty?
λ × sum of absolute values of weights.
What does L2 regularisation encourage?
Smooth shrinking of all weights.
What is the formula for L2 regularisation penalty?
λ × sum of squared weights.
What is the elastic net?
A regularisation method combining L1 and L2 penalties.
What does the alpha parameter control in elastic net?
The mix between L1 and L2 regularisation.
What is one benefit of L1 regularisation?
It performs feature selection by zeroing out irrelevant weights.
What is one benefit of L2 regularisation?
It encourages small, distributed weights to prevent overfitting.
What type of validation is needed for early stopping?
A separate validation set monitored during training.
What kind of models benefit most from dropout?
Deep neural networks with many parameters.
How does regularisation affect model weights?
It discourages overly large or unnecessary weights.
