Feedforward networks

Question 1

What are other ways to call a neuron?

Answer 1

Perceptron, elementary computing unit

Question 2

How does a neuron work?

Or in other words, what is it composed of?

Answer 2

• Parametric linear transform
• followed by a fixed scalar non linear function

Each neuron computes one single thing

Question 3

What are some activation functions?

Answer 3

• Linear: g(a)=a, unbounted but limited to linear relation
• Sigmoid: g(a)=sigma(a)= 1/(1+exp(-a)), bounded btw 0 and 1, squashes large/small values
• ReLU (Rectified linear activation): g(a) = max(0, a), lower bounded, negatives become 0

Question 4

What is a feedforward NN?

Answer 4

Multilayer perceptron.
There is an input layer (features, followed by at least one hidden layer, and an output layer.
Each neuron from a layer takes as input all the neurons of the previous layer