Week 3

Question 1

Neurons in brain of human, cat, insects

Answer 1

Question 2

Neuron consists of

Answer 2

Question 3

What is a synapse

Answer 3

Connects between axons of 1 neuron and dendrites of another

Question 4

Explain how neurotransmitters work

Answer 4

Activity in pre synaptic neuron causes release of neurotransmitters from synaptic vesicles

Neurotransmitters diffuse across gap to receptors on post synaptic neuron and cause activity there

100 different neurotransmitters eg dopamine, serotonin, acetylcholine

Question 5

Information flow through neuron

Answer 5

Question 6

What is an action potential

Answer 6

An electrical impulse
Neuron is an electrical device with a voltage (or potential)
Dendrites and axons act like (highly nonlinear) wires

Question 7

Spiking threshold

Answer 7

Reaching requires either
Repetitive stimulation of the synapse (temporal summation)
Simultaneous stimulation of a large number of synapses (spatial summation)

Or (most typically) both

Question 8

2 types of synapse and effects (with example)

Answer 8

Excitatory
Tend to cause spiking in the postsynaptic neurone
Glutamate

Inhibitory
Tend to prevent spiking in post synaptic neuron
GABA

Question 9

Weights of synapses

Answer 9

Different synapses may have stronger or weaker affects in postsynaptic neuron
Synapses have different weights

Question 10

Synaptic plasticity & its mechanisms

Answer 10

Plasticity permits nervous system to adapt to its environment

1) creation of new synaptic connections between neutrons

2) modification of existing synapses

Question 11

Define ANN

Answer 11

A network of simple processing units which communicate by sending signals to each other over weighted connections

Processing units
Analogous to neurons

Weighted connections
Analogous to synapses

Question 12

ANN is AKA

Answer 12

Parallel Distributed Processing (PDP)
Connectionist models

Question 13

Why study ANNs?

Answer 13

ANNs are pwoerful computational devices and Turing complete universal computers

Any continuous function from input to output can be implemented in a 3 layer ANN

Question 14

Universal property of ANNs

Answer 14

Question 15

2 main types of layers

Answer 15

Hidden - only recieve inputs and send outputs to other processing units

Visible - can revived inputs from or send outputs to external environment

Question 16

2 types of visible layers

Answer 16

Input
Receive signal from environment

Output
Send signal to environment

Question 17

Input layers are typically

Answer 17

Linear - they don’t perform any processing

Question 18

Response function

Answer 18

Split into
Transfer function - determines how inputs are integrated
Activation function - determines the output the neuron produces

Question 19

ways to determine weights in NN

Answer 19

Training
Setting manually (given prior knowledge)
Optimising connectivity to achieve some objective (eg using genetic algorithm or gradient descent)

Question 20

Linear threshold unit

Answer 20

AKA perceptron

When Weights and activations are binary, this is known as Threshold Logic Unit or McCulloch Pitts neuron

Question 21

Linear threshold unit as equation

Answer 21

Question 22

Linear threshold unit as vectors

Answer 22

Question 23

Decision boundary for LTU

Answer 23

w and θ define a hyperplane that
Divides the input space into 2 Parts

Question 24

Linear separability and logical functions

Answer 24

AND and OR are linearly separable functions

Not all logical functions are (eg XOR)

All can be represented by a multi layer network of perceptrons though

Question 25

Linear discriminant function

Answer 25

Question 26

LTU equivalence to linear discriminant function

Answer 26

Question 27

Producing non linear decision boundaries with LTU

Answer 27

Question 28

Delta learning rule

Answer 28

Question 29

Delta learning rule - 2 types of update

Answer 29

Question 30

Delta learning rule for gradient descent

Answer 30

Question 31

Sequential delta learning algorithm

Answer 31

Question 32

Hebbian learning rule

Answer 32

Question 33

Competitive learning networks

Answer 33

Output units compete for the right to respond to the input Using inhibitory lateral weights requires outputs to be determined iteratively Using a selection process is simpler and more stable

Question 34

Winner takes all

Answer 34

Simplest selection process for competitive learning networks Neuron with largest response is winner All other neurons have their response set to 0 WTA is an activation function Can be used with Hebbian learning to perform clustering

Question 35

K winners take all

Answer 35

K neurons with largest response remain active All other neurons have response set to 0 kWTA is an activation function

Question 36

Soft Max selection process

Answer 36

Question 37

Negative feedback networks

Answer 37

Activation Tries to minimise response from input units (eg minimise reconstruction error) Tries to find the y values that accurately reconstruct the input Learning Adjust weights to accurately reconstruct the input

Question 38

What is an Auto encoder network?

Answer 38

Question 39

Hidden units in auto encoder networks

Answer 39

They impose an information bottleneck Limit number of hidden nodes Hidden units learn a useful representation of the data

Question 40

Encoding in auto encoder networks

Answer 40

Question 41

De noising auto encoders

Answer 41

To avoid overfitting, they add noise to inputs used for learning Encoding performed with corrupted input Decoding compared to uncorrupted input