Perceptron Learning Algorithm (pseudocode)

Question 1

Describe the Perceptron learning rule

Answer 1

Automatic algorithm that works efficiently even for high dimensional problems

Learns the parameters, weights and biases directly

Question 2

What is the purpose of the trainable parameters

Answer 2

To give importance to the features that contribute more towards the learning

Question 3

Explain the steps to an artificial neural network

Answer 3

Inputs passed in with corresponding weights

Σ : Linear transformation (g(x)) combines multiple inputs into one output value

f(z): Activation function is applied to g(x) the activation function

y hat: Forward pass - a calculation is predicted.

Error: Backward pass - adjust the model parameters according to the error

Question 4

What is the bias term b

Answer 4

The bias b is a learnable scalar that shifts the decision boundary off the origin

Question 5

How do we merge the bias b with the weight vector

Answer 5

Add the bias b to the end of the weight vector. add a constant 1 to the end of the input vector as this will be multiplied by the bias when dotted together

Question 6

Write the perceptron Learning algorithm in pseudocode

Answer 6

P and N are inputs with class labels (1,-1)

Learning rate is a hyperparameter

InitialiseW: randomly, is initialising the weights and biases

LINE: while Not converge do:

Convergence refers to the training stoppping criteria which could be:
No more errors
Model stop making improvement
Max number of iteration

LINE: Do forwards pass
Activation function

LINE: if X’ e P and W^T * X’ < 0
It is saying for each sample in the positive class, check if it is negative, if it is then append the negative of the sample to the error class (delta)

Next if statement:
If sample is in negative class and is positive, add the sample to the error class delta

After all if statements

New set of weights =

Weights - learning rate * sum of all errors
Increase t (iterations)

Question 7

How to tell which class is positive and negative

Answer 7

Check the first two vectors, take these as co-ordinates and will point in the direction of the positive class e.g. (1,1) shows the top right quadrant is positive so the class where the decision boundary has the majority of the top right quadrant will be positive

Question 8

Answer 8

Sum of errors = (-0.6,0.7,0) and then next step ish shown in image

Question 9

We are using boolean logic OR as the classifier and seeing if the activation function classifies correctly

What is X’ for the training row 1

Answer 9

X’ = x1,x2,1

(1 is for the bias)

Question 10

The result of the activation function is the parameters dotted with X’ as shown by g(x), which row will misclassify and not give the same answer as OR

Answer 10

Row 1: =-0.5 so gives 0. Correct
Row 2: =0.1 so gives 1. Correct
Row 3:=-0.1 so gives 0. Incorrect
Row 4:=0.5 so gives 1. Correct

Question 11

What is the sum of errors (delta)

Answer 11

Sum of errors is given by sum of X’
(X’) becomes negative if it was wrongly classified as positive

SO:

Row_3’s X’ = [1,0,1]

so sum of errors (delta) = [-1,0,-1]^t

Question 12

Formula for new set of weights

Answer 12

New set of weights =

Weights - learning rate * sum of all errors
Increase t (iterations)

Question 13

Formula for sum of errors (delta)

Answer 13

sum of X’ when it was misclassified, you add a -X’ if it was misclassified as positive

Question 14

Answer 14

[1.1,0.6,0.2]