L11 Flashcards
(20 cards)
What are the three types of learning in machine learning?
- Supervised Learning
- Unsupervised Learning
- Reinforcement Learning
Supervised learning involves labeled data, unsupervised learning does not, and reinforcement learning involves learning through interaction with the environment.
What is the objective of supervised learning?
Learn function F: Xk→YkX_k
ightarrow Y_k
The aim is to map inputs to outputs based on labeled data.
What is the objective of unsupervised learning?
Discover patterns, cluster/group similar items
Examples include topic clustering on Twitter or detecting fraud through outlier detection.
What is the curse of dimensionality?
Exponential increase in feature space volume with more features
This leads to difficulties in generalization, increased risk of overfitting, and sparsity of data.
What is dimensionality reduction?
Process of reducing the number of features in a dataset
It speeds up computation, improves generalization, enhances visualization, and eliminates redundant features.
What are the two main approaches to dimensionality reduction?
- Feature Selection
- Dimensionality Reduction
Feature selection keeps original variables while dimensionality reduction creates new features as combinations.
What does Principal Component Analysis (PCA) do?
Finds new axes (principal components) by rotating the coordinate system
The first component captures the direction of maximum variance, while the second is orthogonal to the first.
What is the first step in performing PCA?
Standardize data (mean 0, variance 1)
Standardization ensures that each feature contributes equally to the analysis.
What are the properties of PCA?
- Orthogonal components
- Linear combinations of original variables
- Each additional component explains less variance
PCA is commonly used for data visualization and feature extraction.
How many principal components should be kept in PCA?
Enough to reach ≥90% explained variance
The number of components depends on training data size and classifier complexity.
True or False: PCA is color-blind and ignores class labels.
True
This means PCA does not consider class information when reducing dimensions.
What is Non-negative Matrix Factorization (NMF)?
Unsupervised method for non-negative data that factorizes a matrix into non-negative latent components
NMF is useful for text mining and image analysis.
What are the strengths of NMF?
- Interpretability
- Parts-based representations
NMF is particularly useful for analyzing topics in text and features in images.
What is a downside of NMF?
Requires positive data and is sensitive to initialization
NMF may also struggle with non-convex optimization.
What does t-SNE do?
Projects high-dimensional data into 2D or 3D while keeping local neighborhoods close
It is particularly effective for visualizing clusters in data.
What are some hyperparameters for t-SNE?
- n_components (usually 2)
- perplexity (typically 5–50)
- early_exaggeration
- learning_rate (10–1000)
- n_iter (≥250, typically 1,000)
These hyperparameters control various aspects of how t-SNE processes data.
What is a key limitation of t-SNE?
Emphasizes local over global structure
It is non-parametric and cannot easily project new points.
What are the strengths and weaknesses of PCA?
- Strengths: Fast, interpretable
- Weaknesses: May lose class info
PCA is a linear method that is efficient but may overlook important class distinctions.
What are the strengths and weaknesses of NMF?
- Strengths: Interpretability, text/image analysis
- Weaknesses: Non-convex, needs positive data
NMF is useful but has specific data requirements and optimization challenges.
What are the strengths and weaknesses of t-SNE?
- Strengths: Great for visualization
- Weaknesses: Not useful for modeling
t-SNE excels at visualizing data but is not suitable for predictive modeling tasks.
