Data pre-processing

Question 1

Possible data pre-processing procedures

Answer 1

Input preprocessing:

1) input centering
2) input normalization
3) input whitening
4) data cleaning

PCA can also be considered a data pre-processing procedure.

Question 2

Why is data pre-processing used?

Answer 2

Data should be pre-processed to make the suited for Learning, and to achieve better results.
Pre-processing operations must be done without snooping the data!

Question 3

Input preprocessing: general procedure

Answer 3

Given the input data matrix X app R^N*d
input pre-processing consists in finding a standardization transform Φ, that gives
zn = Φ(xn) for any n = 1,..,N

The final hypothesis will be
h(x) = h(Φ(x))

Question 4

Input centering

Answer 4

Goal: to remove any bias from the input.
zn = xn - x_

x_ = 1/N sum(n=1,N) xn (mean of the data)

Question 5

Input normalization

Answer 5

Goal: to scale the input wrt its variance.
Assuming it is centered:
zn = [zn1 … znd]' = [xn1/σ1 … xnd/σd]
where
σi^2 = 1/N sum(n=1,N) xni^2 , i=1,...d
(variance of the features)

Question 6

Input whitening

Answer 6

Goal: to decorrelate input samples, if it is known that they are decorrelated.
zn = A^-1/2 xn
where A is the covariance matrix of the x’s

Question 7

Data cleaning

Answer 7

Goal: to remove outliers

• use simple models
• compute leverage score of validation
• look at the data

Question 8

Causes of outliers data

Answer 8

• stochastic output noise

- System complexity not modelled (deterministic noise)