Optimization, Loss Functions, and Regularization Flashcards
What is the purpose of a loss function?
It measures model error.
Loss functions are crucial for training models as they provide a metric to minimize.
What type of loss function is Log Loss?
Cross-Entropy Loss.
Log Loss is commonly used in binary classification problems.
What loss function is used when there are only two classes?
Binary Cross-Entropy.
This loss function calculates the error for binary classification tasks.
What is the formula for Cross-Entropy Loss?
L = - (y log(y_pred) + (1 - y) log(1 - y_pred))
What is backpropagation used for?
To adjust model weights to reduce error.
Define Gradient Descent.
Step-by-Step Learning.
What is Stochastic Gradient Descent (SGD)?
Updates weights after each training sample.
What is Mini-Batch Gradient Descent?
Updates weights using small batches.
What is the learning rate’s role in gradient descent?
Controls how big the update steps are.
What does momentum do in gradient descent?
Helps avoid local minima by adding past weight updates to the current one.
What is dropout in the context of model training?
Randomly removes neurons during training.
What does regularization do?
Penalizes overly complex models to encourage generalization.
What is early stopping?
Stops training when validation loss stops improving.
What are epochs in machine learning?
Number of times the entire dataset is passed through the model.
What is the batch size in gradient descent?
Number of samples used per gradient update.
Name a common optimizer used in machine learning.
Adam.
What is the main takeaway regarding activation functions?
They enable deep learning.
True or False: Dropout and regularization help prevent overfitting.
True.
What are the two main types of supervised machine learning?
Classification and Regression.
What does regression in AI predict?
Continuous values based on input data.
How does regression work in AI?
AI learns a function y = f(x) to predict Y.
What is Mean Squared Error (MSE)?
A loss function that measures how far off AI’s predictions are.
What does a lower MSE indicate?
A better AI model.
What is a Simple Artificial Neural Network (ANN) for regression composed of?
Layers of neurons making decisions.
