L6 - Introduction to Deep Learning

Question 1

What is the major benefit of Deep Learning over Machine Learning?

Answer 1

Automatic Feature Learning

Question 2

What 3 advancements have made Deep Learning feasible/popular?

Answer 2

1. Big Data - Easier collection, storage and availability.
2. Hardware - better GPUs
3. Software - Improved techniques

Question 3

Explain the Perceptron

Answer 3

The Output is the result of an activation function which takes the sum of the input values multiplied by thier input weights.
A Bias term may be added as an extra ‘constant’ input.

Question 4

What is the purpose of the Activation function in the Perceptron theory?

Answer 4

The Activation function introduces non-linearity.

Sigmoid Function is commonly used.

Question 5

What are the steps to optimise a Loss in a Neural Network?

Answer 5

1. Initialise weights randomly
2. Compute the Loss Gradient
3. Update the weights to reduce the loss
4. Repeat from step 2 until convergence.

Question 6

What is the learning rate of Loss Optimisation?
What makes choosing it difficult?

Answer 6

The size of the steps taken when updating the weights in the optimisation process.
Too large or too small and the true global optimum could be missed.

The solution is to use an adaptive learning rate which changes depending on various factors.

Question 7

What is overfitting ?

In the context of neural networks specifically

Answer 7

Wwhen a model learns the details and noise in the training dataset to the extent that it negatively impacts its performance on new data. This means the model memorizes specific examples from the training data rather than generalizing from them, resulting in high accuracy on the training set but poor performance on validation and test sets.

Question 8

What is Dropout Regularisation?

Answer 8

A technique that involves randomly setting a fraction of neurons to zero during each forward and backward pass, preventing the model from becoming overly reliant on any specific neurons.