Tut 7

Question 1

List 5 methods that can be used to perform feature extraction

Answer 1

PCA
ICA independent component analysis
whitening
Random projections
LDA linear discriminant analysis
Sparse coding

Question 2

How to find the proportion of variance explained by specific principal components

Answer 2

Eigen Values for PCs/(Eigen values for ALL)

Question 3

Oja’s rule

Answer 3

w <- w + η( y(x^T - yw))

Question 4

What does whitening do

Answer 4

After PCA
Makes covariance matrix in new space equal to identity matrix
Each dimension then has same variance

Question 5

Drawing PCA and LDA

Answer 5

For PCA ignore class labels and look for longest straight line running through points in the cloud (highest variance line)

For LDA, don’t ignore class labels, look for the line which could be the axis of separation (using only values on that line could separate the class well)

Question 6

L0 norm

Answer 6

Count number of non zero elements in a vector

Question 7

Asked for lowest sparsity

Answer 7

Cmpare L0 norm
||y||_0
if equal, compare reconstruction error ||x - V^Ty||_2