Lecture 5: Artificial Neural Networks

Question 1

What is the behaviour of a neuron?

Answer 1

• A neuron receives inputs from its neighbors
• If enough inputs are received at the same time
  —the neuron is activated
  —and fires an output to its neighbors.
• Repeated firings across a synapse increases its sensitibity and the future likelihood of its firing.

Question 2

What is a perceptron?

Answer 2

A SINGLE coputational neuron

Question 3

What are the inputs of a perceptron?

Answer 3

Input signals Xi, weights Wi for each feature xi (strenght of connection)

Question 4

What are the output of the perceptron?

Answer 4

Output:
 if sum of input weights >= some threshold, neuron fires (output=1)
 otherwise output = 0
 If (w1 x1 + … + wn xn) >= threshold
 Then output = 1
 Else output = 0

Question 5

How is a perceptron trained?

Answer 5

1. Step 1: Set weights to random values
2. Step 2: Feed perceptron with a set of inputs
3. Step 3: Compute the network outputs
4. Step 4: Adjust the weights
5. if output correct → weights stay the same
6. if output = 0 but it should be 1 →
7. increase weights on active connections (i.e. input xi=1)
8. if output = 1 but should be 0 →
9. decrease weights on active connections (i.e. input xi=1)
10. Step 5: Repeat steps 2 to 4 a large number of times until the network
    converges to the right results for the given training examples

Question 6

What is a bias?

Answer 6

• Useful to avoid figuring out the threshold by using a “bias”
• A bias is equivalent to a weight on an extra input feature that has always the value of 1.
• Is added at the end of weighted calculation

Question 7

Limit of perceptrons early on

Answer 7

Only linearly separable functions
can be represented by a
perceptron

Question 8

What are Multilayer Neural networks?

Answer 8

1. to learn more complex
   functions (more complex
   decision boundaries), have
   hidden nodes
2. and for non-binary decisions,
   have multiple output nodes
3. use a non-linear activation
   function

Question 9

What is “feed forward”

Answer 9

Feed-forward:
 Input from the features is fed forward in the network
from input layer towards the output layer

Question 10

What is backpropagation?

Answer 10

Error rate flows backwards from the output layer to the input layer (to adjust the weights in order to minimize output error)

• Essentailly, this is for computing the errors in hidden layers