Lecture 5

Question 1

What is supervised learning in machine learning?

Answer 1

Given a set of inputs and outputs, we build a model to predict the output for a new input.

Question 2

What are we interested in with unsupervised learning?

Answer 2

Only providing inputs to find patterns in the data.

There is not a known correct answer.

An unsupervised algorithm also outputs a model
- Clustering - grouping together data points that are similar to each other
- Detecting anomalies - finding data points that do not appear to be from the same probability distribution eg fraud detection
- Association - finding relationships between variables

Question 3

What is a key use of unsupervised learning?

Answer 3

To infer probability distributions

Question 4

What do we assume about the data in unsupervised learning?

What do we do for parametric methods?

Answer 4

We assume this data was drawn from a probability density p(x).

For parametric methods, we choose a functional form for the distribution eg p(x) proportional to exp(-(x-u)T sum-1 (x-u))

We want to find the parameters (eg maximum likelihood). Finding which parameters are most likely to give you the data you see.

Question 5

What are the disadvantages of parametric methods?

Answer 5

Not very flexible.

Requires us to make some assumptions about the data.

Question 6

What parametric method might you choose if you don’t know much about the data?

Answer 6

The central limit theorem
- Assumes a large amount of data that behaves normally
- It is a very limited functional form
- There are a small number of parameters (mean and covariance) which reduces flexibility

Question 7

In comparison to parametric methods, what do unsupervised learning give us access to?

Answer 7

Unsupervised learning gives us access to non-parametric methods that do not make assumptions.

NB: non-parametric may be misleading as there are still parameters

Question 8

Describe an example of unsupervised learning giving access to non-parametric methods that do not make assumptions?

Answer 8

K-nearest neighbours (KNN).

We have N points and a parameter K set in advance. To find p(x) we sit at a point x and draw (hyper)spheres around this point until we find K points.

If V is the volume of the final hypersphere, p(x) ~= K / NV

Rather than a rigid functional form, the based probability distribution is based on how much data you have. The probability distribution is highest when the points are most dense.

This is an estimate of the probability distribution that is data driven.

Question 9

What function does a supervised method often rely on?

Answer 9

Basis functions - we represent data by taking functions of the data.

We pick a set of functions to represent the data eg a + bx + cx^2 for linear regression.

Question 10

The data we put into the neural network might be a function, not a set of numbers eg a histogram p(x,y). What is the problem with this?

Answer 10

There is a lot of redundant information. If we used the values of a histogram on a grid, we would have lots of information we don’t need (many of the numbers are zero).

We want to design better features to go into the model - ie representing this as a set of numbers.

Question 11

What could we use as features instead of using the values of histogram on a grid?

Answer 11

The projections of our histogram on some basis functions, rather than looking at everything.

Question 12

How do we pick the set of basis functions?

Answer 12

• Sometimes we can choose them by physical intuition
• But often we don’t know which best represent the data

Discussion of K-means clustering, with a predetermined K. We randomly assign each data point into one of the sets. Calculate the mean of each set and then re-assign data points to the set whose means they are closest to. We repeat until there are no changes.

Question 13

What are two examples of how deep learning can be used in an unsupervised way?

Answer 13

• Autoencoders
• Generative adversarial networks (GANs)

Question 14

Briefly, what do auto encoders do?

Answer 14

These can encode data by projecting it onto a lower-dimensional representation. This projection is learned by training a network whose outputs are as close as possible to its inputs.

Non-linear combinations of data of lower dimensionality.

Projecting information through non-linearity to get fewer dimensions.

Question 15

Briefly, what are GANs?

Answer 15

Generative adversarial networks - these are used to train generative networks that can produce data points from an appropriate distribution. Two networks compete with each other, one to generate possible examples and one to detect false examples. (They compete in a zero sum game).

Question 16

What is the difference between GANs and CNNs?

Answer 16

GANs are generative models that can generate new examples from a given training set, while convolutional neural networks (CNN) are primarily used for classification and recognition tasks.

Question 17

What is the need for dimensionality reduction?

Answer 17

Often, we use more features to represent our data points than we need to (redundancies and dependencies of features). Dimensionality reduction aims to eliminate this redundancy, going from a high dimensional feature space to a lower-dimensional representation.

Question 18

Briefly, discuss auto encoders and dimensionality reduction.

Answer 18

Autoencoders use deep learning to very efficiently reduce the dimensionality of a problem. (In a non-linear way, making it powerful).

Question 19

What is the network architecture of an auto encoder?

Answer 19

The network architecture is deep, but becomes narrow in the middle. Effectively it is like a deep neural network with a bottle neck region, where are fewer nodes than inputs (and outputs).

The central layer is the code - this is the bottleneck region, here the number of features is smaller than the input.

There are two steps in forward propagation: encoding and decoding.

Question 20

What are the two steps in forward propagation in an auto encoder?

Answer 20

• Encoding
• Decoding

Question 21

Briefly describe the encoding part of forward propagation in an auto encoder.

Answer 21

Features propagate through the network until you reach the code region, where the number of nodes is smaller than the number of inputs.

When we apply the encoding step, we are compressing the data, however we are losing information - this is lossy compression.

Question 22

Briefly describe the decoding part of forward propagation in an auto encoder.

Answer 22

After the input data has been encoded into a lower-dimensional representation, it is decoded it back to the original input space.

Question 23

What is the target output of an auto encoder?

Answer 23

We want to get back something very similar to what we put in.

It will not be exactly the same as we have information loss due to lower-dimensional space. Recall that a single-layer neural network acts as a universal approximation, but only if we have lots of nodes.

Question 24

Why are the inputs and outputs of an auto encoder not exactly the same?

Answer 24

Information loss due to lower-dimensional space.

Question 25

Why can we describe the encoding step as lossy information?

Answer 25

When we apply the encoding step, we are compressing the data, however we are losing information - this is lossy compression.

Question 26

What are autoencoders trained to recognise? What is the implication of this?

Answer 26

Autoencoders are trained to recognise the key features in data. Because the input is projected onto these features, it is generally quite resistant to noise. We can train autoencoders specifically for removing noise from the input - working out what the input should have been without noise.

Question 27

How can we train autoencoders specifically for removing noise from the input?

Answer 27

We add some noise to the input before it goes into the auto encoder. The network learns how to remove noise. We apply the ampount of noise we expect the auto encoder to be produced with. The output is a non-noisy input but not identical to the original input version. [See diagram]

Question 28

How do PCA and out encoder reconstructions compare for functions with increasing powers of x?

Answer 28

PCA reconstruction (feature projected to one principal component) and auto encoders (features project to one feature) are both able to reconstruct a linear function well - both show a straight line. PCAs are only capable of reproducing this linear relationship with a single feature, however autoencoders are able to achieve non-linearalities.

Question 29

What other method involves extraction of important features? How does the auto encoder compare?

Answer 29

Principal component analysis (PCA) does a similar thing (it is linear, rather than network-based). An auto encoder could be thought of as a nonlinear version of PCA. The non-linearity combination of features is brought about by the coding region. This nonlinearity can mean that autoencoders may perform better when features are related nonlinearly. In turn, auto encoders might require fewer features than PCA to give the same level of error. Eg PCA - 10 features, AE - 1 or 2 features. Training an autoencoder involves a neural network, so we may incur problems (eg overfitting) that we may not have with PCA. PCA involves only one line of code (eg diagonalising a matrix), whereas an autoeconder involves setting up a neural network and weights etc. You can interpret PCA better, the auto encoder is harder to understand / interpret as you have less intuition.

Question 30

What is a use of autoencoders, similar to PCAs?

Answer 30

Representation learning. Sometimes (eg when we know something about the symmetry of a problem) we can choose the right features from our data. Quite often, the problem is too complex to do so - but we still might feel we are using more data than we need to. Autoencoders project the input (nonlinear) into fewer dimensions, finding a smaller number of features that still represent the data very well. There are similar advantages to PCA, but with added non-linearity.

Question 31

How do we train an auto encoder?

Answer 31

We train it as we do a standard (deep) network. In the dataset D, the inputs and outputs (x1, t1) can be represented as (x1, x1) as the inputs and outputs should be the same.

Question 32

What is the error function for autoencoders?

Answer 32

Reconstruction error.

Question 33

When writing the error function eg for regression, what is one subtlety between previous error functions?

Answer 33

Use of | | instead of ( ) because we have vectors.

Question 34

How do we optimise the errors for autoencoders?

Answer 34

We have the error function (reconstruction error). We can find the derivatives of the error with respect to the weights using back-propagation, and optimise the errors as usual.

Question 35

What is the "bare minimum" basic architecture for an auto encoder?

Answer 35

The basic auto encoder architecture includes a bottle neck. An auto encoder can already be produced with a single hidden layer in the encoder and decoder sections (three layers total). Deeper networks are preferred for neural networks. There are several advantages to using deep networks including the amount of training data needed. For models involving images, convolutional layers are also likely needed in order to "pre process" the data.

Question 36

What are other possible architectures for an auto encoder?

Answer 36

Sparse auto encoders The code layer no longer needs to have fewer nodes than inputs (no bottle neck). Instead, the error function has an extra term that penalises activations - this means that not every node is activated. Some information will get to the code layer and no further, because large activations are penalised. Contractive auto encoders. Here the error function also has an extra term - this time, to penalise large derivatives of activations with respect to the data ie situations where a small change in input gives a large change in output. - Variational autoencoders. These are similar to a standard autoencoder, but the code is now a probability distribution, rather than a set of values. We can draw from this probability distribution and send the result through the decoder, to give an output that isn't in the training set, but should be from the same probability distribution. After training, we decouple the encoding and decoding regions.

Question 37

What applications are there for autoencoders?

Answer 37

- Removing noise from images - Compression of inputs - Detection of anomalies - data points that are not likely to be from the same probability distribution as the training set. Putting these points through the autoencoder, they will be poorly reconstructed. Eg hand drawn letters, passing in :), the auto encoder will not know to deal with this if it wasn't seen in the training set. You will get back nonsense. - Inference of missing values - similar to noise removal, we can train the network by removing features at random from the input data (to simulate missing data) and then the final network should be able to reconstruct the original point. - Generative models - for these, we could consider variational autoencoders.

Question 38

How are anomalies detected using auto encoder?

Answer 38

Putting these points through the autoencoder, they will be poorly reconstructed.

Question 39

What is a generative model used for?

Answer 39

Used to produce examples that are similar to its training set. Often re-cast in a probabilisitic way - we model the probability distribution that produced the training data and we generate new points from that distribution. We want to consider complex problems, such as training a model and showing it a variety of pictures, then having it generate new, unseen convincing pictures of these types.

Question 40

What is a discriminative model?

Answer 40

A discriminative model takes in an input x and predicts an output z (classification or regression). More properly, in statistical terms it models P(z|x). That is, given an input x, what is the probability of output z? The final output of the model is then the most likely from this distribution.

Question 41

What is a generative model?

Answer 41

A generative model is a model for the joint distribution P(x,z), from which we draw inputs and outputs. P(x,z) = P(z|x) P(x)

Question 42

What is a generative adversarial network (GAN)?

Answer 42

A generative adversarial network (GAN) is a deep learning architecture. It trains two neural networks to compete against each other to generate more authentic new data from a given training dataset. Made up of two networks: - Generative model - Discriminative model The two models play a zero-sum game. The generative model aims to fool the discriminative, and the discriminative aims not to be fooled (they have opposite goals).

Question 43

What does the generative model of a GAN do?

Answer 43

Produces examples of whatever we want to generate eg generating images.

Question 44

What does the discriminative model of a GAN do?

Answer 44

Decides whether or not the generative model's examples are "real" examples from the correct distribution. ie works to correctly classify the images produced, deciding if it is real or fake.

Question 45

What kind of data does the discriminative model take?

Answer 45

The discriminative model takes in a data point, x, which may be from the generative model (t=0) or a "real" example (t=1) It then classifies this model either as real (z=1) or fake (z=0)

Question 46

Discuss the error function of a GAN?

Answer 46

There are two error functions, one for each model. These are combined to make 1 error function, so there are two terms. Whenever the discriminator makes a prediction, both error functions are updated based on whether the generator was able to "fool" it. Eg if the generator produces an image, and the discriminator assigns "fake", the generator error is updated. If the discriminator assigns this as "real", the discriminator error is updated. eg if a real image is produced and the discriminator assigns "real", the discriminator error is not updated. However, if it is assigned "fake", the discriminator error is updated. The weights are updated using gradients obtained from the discriminator loss function (and back-propagation).

Question 47

How does updating the error influence the model?

Answer 47

When we start, what comes out of the generator will be garbage and the discriminator identifies this. But the generator will get better over time and the discriminator gets worse (it will make mistakes). The discriminator is just a model for classifying inputs (C1 = "real" and C2 ="fake") and can have any architecture.

Question 48

Which model, the generative or discriminative, is harder to train? Why?

Answer 48

A generative model is more difficult to train than a discriminative one - the two tasks are quite asymmetric. Consider the two tasks: - Model 1 decides whether or not a picture contains a human face - eg easier to recognise a face than it is to construct one yourself - Model 2 has to put together a convincing human face - this is a much harder job

Question 49

Describe the generative network, with aid of the diagram.

Answer 49

Starts off with a random input - it doesn't matter what goes into the generative model, it matters what goes out. This is transformed into a sample data point (an image), which goes into the discriminator. This classifies the data point, and the generator error is updated. We then want to use the generator error to update the weights. This error depends on the weights of both generator and discriminator, but we want to only update the generator now. Back-propagate through the discriminator to the generator, and update the weights only of the generator.

Question 50

Discuss training the GAN.

Answer 50

We don't want both networks to change at the same time. The generator should adapt to fool the discriminator as it currently is. The discriminator should adapt to avoid being fooled by the current generator. We alternate between training each of the two models, with the other one kept constant. Each time we train, we send a number of examples either from the true distribution or the generative model through the discriminative model, and accumulate both errors. We then only update one of the models.

Question 51

We alternate training the two networks - when do we stop?

Answer 51

When the generator is as good as it can get, the discriminator is no longer able to make predictions about whether a data point is real. That means it will make a completely random guess. ie the discriminator will get things right 50% of the time. At this point, we can stop the training and decouple the two networks.

Question 52

What error (loss) function do we use for the GAN?

Answer 52

Classification - real or fake, therefore based on cross entropy error. The discriminator wants to maximise the probability that it labels a real or fake image correctly. The generator wants to maximise the probability that the discriminator labels a fake image as real. We use a minimax function which has two sums [see flashcard]

Question 53

Discuss the minimax function.

Answer 53

Minimax function [see flashcard] - is positive, but if the equation is negative, this will be the opposite. The discriminative network wants to maximise this. The generative network wants to minimise this. The two are competing against each other - they have opposite goals for this error function.

Question 54

Discuss a potential problems with GANs.

Answer 54

Mode collapse The generator may learn only to generate one type of output (eg on particular digit). Eg if we build a generative model to generate pictures of animals (you have a repository of real images), it might get very good at producing one kind of output. The generator stops updating, but none of the other animal types are very good, so it stays in a local minimum.Any different types of digits are caught by the discriminator, so the generator stays in a local minimum. We need to be careful about where we stop training. When the generator is "good enough", the responses of the discriminator are random. The generator will then start to train on this random feedback.

Question 55

How are conditional GANs trained?

Answer 55

Using labelled data, and a label is supplied to the generator, to give a particular type of output.

Question 56

What are other examples of generative models?

Answer 56

- Large language models (LLM) - the underlying ideas are familiar, but there are some new things too. In particular, reinforcement learning.

Question 57

What is reinforcement learning?

Answer 57

Reinforcement Learning (RL) is a branch of machine learning focused on making decisions to maximise cumulative rewards in a given situation. It is distinct from supervised / unsupervised learning. Unlike supervised learning, which relies on a training dataset with predefined answers, RL involves learning through experience. In RL, an agent learns to achieve a goal in an uncertain, potentially complex environment by performing actions and receiving feedback through rewards or penalties. ie the model takes actions and is "rewarded" or "punished" for them. There is no right or wrong answer, the model learns for itself how to perform a task.