Lecture 08 - Artificial neural networks

Question 1

Artificial neural networks

What is the name of this function?

Answer 1

Step function

Question 2

Artificial neural networks

What is the name of this function?

Answer 2

Sigmoid

Question 3

Artificial neural networks

What is the name of this function?

Answer 3

Hyperbolic tangent (tanh)

Question 4

Artificial neural networks

What is the name of this simple network?

Answer 4

Perceptron

Question 5

Artificial neural networks

What is the name of this function?

Answer 5

Rectified Linear Unit (ReLU)

Question 6

Artificial neural networks

What is the name of this function?

Answer 6

Leaky ReLU

Question 7

Artificial neural networks

Why are nonlinear activation functions used?

Answer 7

Without nonlinear activation functions, the neural network would only be able to model linear functions

Question 8

Artificial neural networks

What is a feed-forward neural network?

Answer 8

A network in which each neuron is passed information only one time.

Question 9

Artificial neural networks

What is a fully connected layer?

Answer 9

A layer where each neuron is connected to all neurons in the next layer.

Question 10

Artificial neural networks

What is a layer where each neuron is connected to all neurons in the next layer called?

Answer 10

A fully connected layer.

Question 11

Artificial neural networks

What is deep learning?

Answer 11

Deep learning is the research field that works with deep neural networks

Question 12

Artificial neural networks

What is a deep neural network?

Answer 12

Deep neural networks have three or more hidden layers

Question 13

Artificial neural networks

What is the name for a neural network with 3+ layers?

Answer 13

Deep neural network

Question 14

Artificial neural networks

What is a recurrent neural network (RNN)?

Answer 14

A network where information can pass more than one time to the same neuron.

Question 15

Artificial neural networks

What is a network where information can pass more than one time to the same neuron called?

Answer 15

Recurrent neural network (RNN)

Question 16

Artificial neural networks

What does it mean to unfold a recurrent neural network?

Answer 16

Diagrams of RNNs are often written in compressed form.

Unfolding means untangling the diagram to show each time step being processed.

Question 17

Artificial neural networks

What is the image an example of?

Answer 17

Unfolding a recurrent neural net (RNN)

Question 18

Artificial neural networks

What is a bidirectional RNN?

Answer 18

An RNN with an additional layer for input in the reverse order

Question 19

Artificial neural networks

What’s depicted in the image?

Answer 19

A bidirectional RNN.

Question 20

Artificial neural networks

What types of data are RNNs good at processing? (SSTV)

Answer 20

• Serial data
• Sequential data
• Time-series data
• Variable-length data

Question 21

Artificial neural networks

What are some common applications for RNNS? (TVNS)

Answer 21

• Time-series prediction
• Video analysis
• Natural Language Processing (NLP)
• Speech recognition

Question 22

Artificial neural networks

What is this image an example of?

Answer 22

A convolutional layer

Question 23

Artificial neural networks

What is a kernel in an convolutional layer?

Answer 23

Tensors that slide across input data, performing calculations on patches of the inputs.

E.g. a 3x3 kernel slides across an image to blur it.

Question 24

Artificial neural networks

What is this called (see image)?

Answer 24

Convolution

Question 25

# Artificial neural networks What is this called (see image)?

Answer 25

Pooling (Max-pooling)

Question 26

# Artificial neural networks What is max-pooling?

Answer 26

Reducing the size of an input tensor by selecting the max value within a window.

Question 27

# Artificial neural networks What are some common applications of convolutional neural networks (CNNs)? (IVG)

Answer 27

- Image classification - Video analysis - Image generation

Question 28

# Artificial neural networks What is CNN short for?

Answer 28

Convolutional neural network

Question 29

# Artificial neural networks What is RNN short for?

Answer 29

Recurrent neural network

Question 30

# Artificial neural networks What is an autoencoder?

Answer 30

A network with a "bottleneck"

Question 31

# Artificial neural networks What is this an example of?

Answer 31

An autoencoder

Question 32

# Artificial neural networks How does an autoencoder work?

Answer 32

Autoencoder has a bottleneck that downscales (compresses) an input to its most important representation, then tries to upscale that representation to recreate the original input.

Question 33

# Artificial neural networks What are some common applications for autoencoders? (DCOGS)

Answer 33

- Image denoising - Data compression - Object detection - Image generation - Super-resolution

Question 34

# Artificial neural networks What is GAN short for?

Answer 34

Generative adversarial network

Question 35

# Artificial neural networks What is the image an example of?

Answer 35

A generative adversarial network (GAN)

Question 36

# Artificial neural networks What parts does a GAN consist of? (2)

Answer 36

- The generator - The discriminator

Question 37

# Artificial neural networks In a GAN, what does a generator do?

Answer 37

It creates data that resembles the real ones

Question 38

# Artificial neural networks In a GAN, what does a discriminator do?

Answer 38

It's a binary classifier which identifies if the data is real or created by the Generator

Question 39

# Artificial neural networks What are some applications of GANs? (1)

Answer 39

- Image generation

Question 40

# Artificial neural networks What is a LIF short for?

Answer 40

Leaky-integrate-and-fire

Question 41

# Artificial neural networks What does a leaky integrate-and-fire model do?

Answer 41

- It takes input and integrates (sums) them. - The output is usually a 0. - If the model has summed past a threshold, the sum is reset (or reduced by some fixed amount) and a 1 is sent as output. - The leakiness means the value decays over time.

Question 42

# Artificial neural networks What is the image an example of?

Answer 42

A leaky-integrate-and-fire (LIF) neuron.

Question 43

# Artificial neural networks What purpose does the weight in a LIF model have?

Answer 43

Weight or synaptic strength defines how much an incoming spike affects the membrane potential or neuron state of the postsynaptic neuron

Question 44

# Artificial neural networks Which two types of neurotransmitters can spiking neurons release?

Answer 44

Excitatory and Inhibitory

Question 45

# Artificial neural networks What do Excitatory neurotransmitters do? (2)

Answer 45

- Increases the membrane potential. - increases the probability of producing a spike.

Question 46

# Artificial neural networks What do Inhibitory neurotransmitters do? (2)

Answer 46

- Reduces the membrane potential. - Reduces the probability of producing a spike.

Question 47

# Artificial neural networks What is a central pattern generator?

Answer 47

A generator that produces rhythmic outputs in the absence of rhythmic inputs.

Question 48

# Artificial neural networks What is this image an example of?

Answer 48

Outputs from a central pattern generator

Question 49

# Artificial neural networks Who proposed sound localization with spike-coincidence neurons?

Answer 49

Jeffress

Question 50

# Artificial neural networks When was sound localization with spike-coincidence neurons proposed?

Answer 50

1948

Question 51

What is Sound localization with spike-coincidence neurons?

Answer 51

Detecting the location (direction) of incoming sound by producing output spikes when signals arrive at neurons close in time.