gmm Flashcards
(28 cards)
What is the key limitation of K-means that GMMs address?
K-means makes hard assignments and can’t handle overlapping clusters.
What type of clustering does a GMM perform?
Soft clustering using probabilistic assignments.
What is the basic assumption of a Gaussian Mixture Model?
Data is generated from a mixture of several Gaussian distributions.
What does the mixture weight πₖ represent in GMMs?
The prior probability of cluster k.
What does the term p(x | c, θ) represent in a GMM?
The likelihood of x given that it came from cluster c.
What is the full expression for the probability of x in a GMM?
p(x) = Σₖ πₖ · N(x | μₖ, Σₖ)
What kind of distribution does each component in a GMM represent?
A multivariate Gaussian distribution.
What is the role of the covariance matrix in a GMM?
It controls the shape and orientation of each Gaussian component.
What is the posterior probability p(c | x, θ) used for in GMMs?
It represents the responsibility or soft assignment of x to cluster c.
What does the EM algorithm optimize in GMMs?
The log-likelihood of the observed data under the model.
What is computed in the E-step of EM for GMMs?
Responsibilities: the posterior probabilities of each cluster for each point.
What is updated in the M-step of EM?
The means, covariances, and mixing proportions of each Gaussian component.
What is the formula for updating μₖ in the M-step?
μₖ = Σ qₙₖ xₙ / Σ qₙₖ
What is the formula for updating Σₖ in the M-step?
Σₖ = Σ qₙₖ (xₙ - μₖ)(xₙ - μₖ)ᵀ / Σ qₙₖ
What is the formula for updating πₖ in the M-step?
πₖ = (1/N) Σ qₙₖ
What does the EM algorithm guarantee?
That the data log-likelihood will increase with each iteration.
Does EM always find the global maximum of the log-likelihood?
No, EM finds a local maximum and is sensitive to initialization.
What does a GMM reduce to when covariances become isotropic and identical?
It becomes equivalent to K-means clustering.
How does K-means relate to GMMs conceptually?
K-means is a special case of GMM with hard assignments and fixed variances.
What does GMM use instead of distance for assignment?
Probability density functions based on Gaussian distributions.
What type of clustering method is a GMM?
A generative, probabilistic clustering model.
What kind of outputs does GMM provide for each data point?
Probabilities of membership in each cluster.
What is the main advantage of GMM over K-means?
It can model elliptical clusters and overlapping data regions.
Why is GMM considered more flexible than K-means?
Because it learns full covariance matrices and uses soft assignments.
