Week 7: Neural Networks & Deep Learning Flashcards
What is Deep Learning (DL)?
A subfield of machine learning that uses deep neural networks to model complex patterns in data.
Why has DL become prominent recently?
Due to better algorithms, computing power (GPUs), large labeled datasets, and open-source tools.
What is an Artificial Neural Network (ANN)?
A network of interconnected artificial neurons that process information in layers.
What is a perceptron?
The simplest type of neural network unit performing binary classification using weighted inputs and a step function.
What is the activation function in a perceptron?
A step function that outputs 1 if the weighted sum exceeds a threshold, else 0.
What is a sigmoid neuron?
A neuron that outputs a smooth value between 0 and 1 using the sigmoid function, enabling learning.
Why use sigmoid neurons over perceptrons?
Because small changes in weights lead to small changes in output, allowing gradient-based learning.
What is the sigmoid function formula?
σ(z) = 1 / (1 + e^(-z))
What is the purpose of an activation function?
To introduce non-linearity into the network, enabling the learning of complex patterns.
List common activation functions.
Sigmoid, ReLU, tanH, linear, step, Gaussian.
What is forward propagation?
The process of computing outputs from input by passing data through the network layers.
What is backpropagation?
A learning algorithm for neural networks that computes gradients to update weights and biases.
What cost function is often used in binary classification?
Binary cross-entropy or log loss.
What is the softmax function used for?
To convert logits to probabilities in multiclass classification problems.
What is the mathematical form of softmax?
softmax(z_i) = e^(z_i) / sum(e^(z_j)) for all j.
What is the purpose of a cost function?
To measure how well the neural network predictions match the true outputs.
What is stochastic gradient descent (SGD)?
An optimization algorithm that updates weights using one or a few training examples at a time.
What does a neuron compute?
A weighted sum of its inputs followed by an activation function.
What is meant by deep in deep learning?
The use of multiple hidden layers in a neural network.
What are input, hidden, and output layers?
Input: receives data; Hidden: performs computation; Output: provides final prediction.
What is a bias term in a neuron?
An additional parameter that allows the activation function to shift left or right.
What is overfitting in neural networks?
When a model learns training data too well, failing to generalize to new data.
What is regularization in DL?
Techniques like L1/L2 penalties or dropout used to prevent overfitting.
What is an epoch?
One complete pass through the entire training dataset during training.
