20. Artificial Neural Networks 1

Question 1

Describe the McCulloch-Pitts neuron model.

Answer 1

Spikes=potentials, synaptic strength=weights, excitation=positive weights, inhibition=negative weights, activation occurs when sum exceeds threshold

Question 2

What are the steps in the Perceptron learning algorithm?

Answer 2

1. Random weight initialization 2. Present input pattern 3. Compute potential 4. Compute output 5. Calculate error 6. Update weights 7. Repeat

Question 3

What are the three key elements of Artificial Neural Networks?

Answer 3

1. Highly interconnected processing elements (neurons) 2. Configurable topology 3. Learning via synaptic adjustments

Question 4

How does information flow in biological neurons?

Answer 4

1. Dendrites receive signals 2. Soma integrates inputs 3. Axon transmits outputs 4. Synapses pass signals chemically

Question 5

What are the five key steps when working with ANNs?

Answer 5

1. Data preparation 2. Architecture design 3. Neuron structure 4. Parameter initialization 5. Learning algorithm

Question 6

What are the key differences between biological and artificial neurons?

Answer 6

Biological: Analog, parallel, self-organizing. Artificial: Digital, often serial, designed architecture

Question 7

What is the weight update rule in Perceptrons?

Answer 7

w_i(t+1) = w_i(t) + ηx_{s,i}[d_s - o_s] where η=learning rate, d=desired output, o=actual output

Question 8

What is the XOR problem and why is it significant?

Answer 8

A non-linear classification problem that revealed limitations of single-layer Perceptrons

Question 9

What is the chain rule used for in backpropagation?

Answer 9

To calculate gradients through nested functions: ∂L/∂w_{j,k} = -2(d_k-o_k)o’(p_k)o_j

Question 10

What is content-addressable memory in neural networks?

Answer 10

Memory recalled by content rather than address - similar to human associative memory

Question 11

What is the weight update rule in backpropagation?

Answer 11

w_{ij}(t+1) = w_{ij}(t) + ηo_iδ_j where η=learning rate, o=neuron output, δ=error

Question 12

What are three limitations of backpropagation?

Answer 12

1. Vanishing gradient problem 2. High computational cost 3. Black-box nature

Question 13

What are three advantages of backpropagation?

Answer 13

1. Can learn complex functions 2. Handles noisy data 3. Good generalization

Question 14

What are the main components of a biological neuron?

Answer 14

Dendrites (input), Soma (cell body), Axon (output), Synapses (connections)

Question 15

How is error calculated in output layer neurons during backpropagation?

Answer 15

δ_i = o_i’(p_i)[d_i - o_i] where o’ is derivative of activation function

Question 16

What limitation did Minsky and Papert discover about Perceptrons?

Answer 16

They can only solve linearly separable problems (cannot solve XOR)

Question 17

What are the three phases of backpropagation?

Answer 17

1. Forward pass 2. Error calculation 3. Backward weight update

Question 18

How is error calculated in hidden layer neurons during backpropagation?

Answer 18

δ_i = o_i’(p_i)∑w_{ij}δ_j - weighted sum of downstream errors

Question 19

What is the sigmoid activation function formula?

Answer 19

O(p_i) = 1/(1 + e^{-p_i}) - outputs values between 0 and 1

Question 20

What is the Perceptron convergence theorem?

Answer 20

Perceptrons converge only for linearly separable functions