11.1 Mixture models

Question 1

Which of these produces strictly non-overlapping clusters?

Answer 1

• K-means

K-means produces non-overlapping clusters. Soft k-means and Gaussian mixtures are probabilistic clustering methods. In hierarchical clustering, some clusters are contained within other clusters.

Question 2

Suppose you input a pair of values [a b] to softmax, b > a. Which of these is a possible output (after rounding to the nearest float)?

Answer 2

• [0.0 1.0]

Softmax outputs a probability distribution – values can only be in the range 0-1 and they sum to 1. The rank order of values in the output is the same as in the input, so if b>a, the second value of the output must be higher than the first.

Question 3

Model likelihood is guaranteed to increase after each iteration of the EM algorithm.

Answer 3

• Likelihood of the instances given the current model parameters

The expectation step computes the likelihood of the instances given the current model parameters. In the maximization step, the model parameters are updated to maximize the likelihood.

Question 4

Which of these is computed during the expectation step of the EM algorithm?

Answer 4

• Likelihood of the instances given the current model parameters

The expectation step computes the likelihood of the instances given the current model parameters. In the maximization step, the model parameters are updated to maximize the likelihood.

Question 5

When using the supervised method to evaluate a clustering result, a good cluster model should have ___ purity and ___ entropy.

Answer 5

• high, low

Both measures reflect the consistency of labels within clusters. If the labels are very consistent within clusters, the purity will be high and the entropy will be low.