Neural Networks and Deep Learning Foundations

Question 1

What is the main difference between Traditional Machine Learning and Deep Learning?

Answer 1

Traditional Machine Learning involves manually selecting features, while Deep Learning learns features automatically from raw data.

Question 2

What is the purpose of Gradient Descent in machine learning?

Answer 2

To reduce errors in predictions and improve the model’s accuracy.

Question 3

What are the steps involved in the Backpropagation process?

Answer 3

• Forward Pass
• Compute Loss
• Backpropagation
• Gradient Descent

Question 4

True or False: Neural Networks are inspired by the structure of the human brain.

Answer 4

True

Question 5

What are the three types of layers in a Neural Network?

Answer 5

• Input Layer
• Hidden Layers
• Output Layer

Question 6

Fill in the blank: The equation for a prediction in a neural network is Prediction = Input × Weight → _______.

Answer 6

[Activation Function]

Question 7

What are the two types of Loss Functions mentioned?

Answer 7

• Binary Cross-Entropy
• Categorical Cross-Entropy

Question 8

What is the role of an Optimizer in deep learning?

Answer 8

To help the AI learn faster by adjusting learning rates and strategies.

Question 9

What dataset is used in the example of Handwritten Digit Recognition?

Answer 9

MNIST Dataset

Question 10

What is the first step in building a simple neural network using Keras?

Answer 10

Import Libraries

Question 11

What is the purpose of normalizing pixel values in the MNIST dataset?

Answer 11

To improve training efficiency by scaling values from 0-255 to 0-1.

Question 12

What is One-Hot Encoding used for in the context of neural networks?

Answer 12

To convert labels into a format the neural network can understand.

Question 13

What activation function is used in the first layer of the example neural network?

Answer 13

ReLU (Rectified Linear Unit)

Question 14

What metric is used to evaluate the performance of the model on test data?

Answer 14

Accuracy

Question 15

True or False: Keras simplifies the process of building neural networks.

Answer 15

True

Question 16

What is the goal of training a neural network on the MNIST dataset?

Answer 16

To correctly predict handwritten digits (0-9).

Question 17

What is the goal of Backpropagation and Gradient Descent?

Answer 17

Minimize the error between predicted and actual outputs.

Question 18

What do activation functions prevent in neural networks?

Answer 18

They prevent neural networks from behaving like linear regression models and allow them to learn complex relationships.

Question 19

What is the equation without activation functions?

Answer 19

output = dot(W, input) + b

Question 20

What is the equation with activation functions?

Answer 20

output = ReLU(dot(W, input) + b)

Question 21

What is a Linear Activation Function?

Answer 21

Output = Input (Straight line)

Question 22

What is the main issue with the Sigmoid activation function?

Answer 22

Vanishing Gradient – When values go beyond ±3, the gradient becomes tiny, and learning slows down.

Question 23

What does the Softmax activation function do?

Answer 23

Converts values into probabilities that sum to 1.

Question 24

What is an example of Softmax output?

Answer 24

• Cat: 70%
• Dog: 20%
• Bird: 10%

Question 25

What are the benefits of the Tanh activation function?

Answer 25

Maps inputs between -1 and 1, allowing negative values.

Question 26

What is a key problem with Tanh?

Answer 26

Still suffers from vanishing gradient for large values.

Question 27

What distinguishes ReLU from other activation functions?

Answer 27

Only activates for positive values; negative inputs become 0.

Question 28

What is a problem associated with ReLU?

Answer 28

Dead Neurons – if a neuron only gets negative values, it stops learning.

Question 29

How does Leaky ReLU address the dead neuron problem?

Answer 29

Gives negative values a small value instead of zeroing them out.

Question 30

What is the best use case for the Sigmoid activation function?

Answer 30

Probability outputs (binary classification).

Question 31

What problems are associated with Softmax?

Answer 31

Can be computationally expensive.

Question 32

When is Tanh best utilized?

Answer 32

In situations where negative values are needed.

Question 33

What are the common uses for ReLU?

Answer 33

Deep learning models (default choice).

Question 34

What is the main advantage of Leaky ReLU?

Answer 34

Prevents dead neurons and improves training.

Question 35

Fill in the blank: Activation functions allow _______ models to learn complex data.

Answer 35

[deep learning]

Question 36

What is the most commonly used activation function for hidden layers?

Answer 36

ReLU

Question 37

What activation function is best for multi-class classification?

Answer 37

Softmax

Question 38

What is the purpose of activation functions?

Answer 38

They enable deep networks to learn complex data relationships.

Question 39

What is a key limitation of linear activation functions?

Answer 39

They can't handle complex patterns.

Question 40

Name a common activation function used for probabilities.

Answer 40

Sigmoid

Question 41

What is the main drawback of the sigmoid activation function?

Answer 41

It suffers from vanishing gradients.

Question 42

Which activation function is used in multi-class classification?

Answer 42

Softmax

Question 43

What range does the Tanh activation function map inputs to?

Answer 43

-1 and 1.

Question 44

What is the most commonly used activation function?

Answer 44

ReLU

Question 45

What problem does Leaky ReLU address?

Answer 45

The 'dying neuron' problem.