Lecture 6: Deep Learning

Question 2

Drawbacks of ANN

Answer 2

1. standard backproop with signmoi doesn’t scale well with multiple layers - weights change too slow
2. Overfitting - lots of parameters, more capable of learning by heart
3. Require lots of labelled ata

Question 3

Deal with poor backpropagation

Answer 3

• Vanishing gradient -> shrink expon.
• Exploding gradient -> grow expon.
  Use other activation functions or set bounds on the gradients

Question 4

Solutions to overfitting

Answer 4

Regularization - modify error function to penalize alrge weights
dropout - keep neuron active with some probability p of setting it to 0

Question 5

utility of pooling layers

Answer 5

Used to:

To reduce the size of the activation maps

So that we reduce the number of parameters of the network and hence avoid overfitting.