L5 Flashcards
(31 cards)
What is a linear classifier?
A model that classifies inputs based on linear combinations of features
- Many decisions involve non-linear functions of the input
What is a neural network?
A model made up of ‘artificial neurons’ connected in layers
- Each neuron receives inputs, applies weights, sums them up, and passes the result through an activation function (like sigmoid, tanh, or ReLU).
- The network can learn complex, non-linear relationships.
What do neurons in a neural network do?
Receive inputs, apply weights, sum them up, and pass the result through an activation function
What is the purpose of the input layer in a neural network?
Receives the raw data/features (independent variable)
What do hidden layers in a neural network do?
Perform complex transformations and learn more abstract features
Do the “thinking” — each layer learns a more complex transformation.
What does the output layer of a neural network provide?
Gives the final answer (dependent variable) (e.g., probability, class)
What is a single-layer perceptron?
A neural network with one layer of weights, equivalent to logistic regression
What is a multi-layer perceptron (MLP)?
A neural network with two or more layers
What is the representational power of a neural network with a single hidden layer?
Can approximate any function with enough units
More layers and units = more powerful but also harder to train
What are weights (W) and biases (b) in a neural network?
Parameters that the model learns
What is the role of activation functions in a neural network?
Make the network non-linear
What is the output range of the sigmoid activation function?
Between 0 and 1
What does the tanh activation function output?
Between -1 and 1
What is the ReLU activation function?
Outputs 0 if input is negative, otherwise outputs the input itself
What are the two main steps in training a neural network?
- Feedforward
- Backpropagation
- repeated over and over (each round is called an “epoch”)
What occurs during the feedforward step of training?
Pass the input through the network to get the prediction
What occurs during the backpropagation step of training?
Compare prediction to actual label, compute error, and adjust weights and biases to reduce the error (using gradients) / performs learning
What is the key idea behind backpropagation?
Use calculus (the chain rule) to compute weight adjustments so the prediction gets better
- For appropriate amnt to adjust → Compute the derivative of the loss function with respect to weights and biases → derivative of func = slope of func
What defines a deep neural network?
Having many hidden layers
- Learn hierarchical features (each layer learns more abstract concepts).
- Power deep learning in speech, image, and text recognition.
What do convolutional neural networks (CNNs) specialize in?
Used mostly for images
What is the function of convolutional layers in CNNs?
Detect patterns like edges and shapes
- Connect each hidden unit to a small input patch and share the weight across space
What is the purpose of pooling layers in CNNs?
Downsample and keep only the most important information
What does a max pooling layer do?
Finds the maximum locally, indicating the maximum response from the previous layer
What is the role of fully connected layers in a CNN?
Do the final decision-making
