2. Data Pre-processing

Question 1

Types/Techniques of Data Preprocessing

Answer 1

• Aggregation
• Sampling (Simple random, with replacement and stratified)
• Dimensionality reduction
• Discretization + Binarization
• Attribute Transformation (normalization, standardization)
• Attribute Subset Selection
• Attribute Creation

( Data visualization for relationship visualization )

Question 2

Aggregation purpose

Answer 2

• change of scale
• data reduction
• data variability reduction

Question 3

Sampling purpose

Answer 3

• Reduce time and cost associated with re-sampling the full data set
  == data reduction

Question 4

Sampling techniques

Answer 4

• Simple random sampling
• Replacement
• Stratified (split into partitions)

Question 5

Discretization purpose

Answer 5

Change of scale

Question 6

Binarization purpose

Answer 6

Change of scale

Question 7

Attribute transformation purpose

Answer 7

Change of scale

Mapping of complete values
e.g. simple functions: log(x), |x|, e^x, x^k, normalization, standardization

Question 8

Dimensionality reduction purpose

Answer 8

• Reduce time and memory costs for data mining algorithms
• Remove irrelevant features (noise)
• Makes visualization easier

Example: PCA

Question 9

Dimensionality reduction techniques

Answer 9

• Principle Component Analysis (PCA)
• Attribute Subset Selection (if data set contains irrelevant or redundant/duplicate information)

Question 10

Attribute creation purpose

Answer 10

to Capture important information better

( Map attributes on newly created ones )

Question 11

Data visualization techniques

Answer 11

• Box plot: Percentiles + outliers
• Scatter plot: relationship analysis (2D 3D)
• Mean graph: relationship analysis (categorical)
• Matrix plot: relationship analysis (often normalized to prevent domination)
• Parallel Coordinates (each object is a line): relationship analysis (=> higher D)