(a) Illustration of how singularities can arise in the likelihood function of GMMs. Here $K=2$, but the first mixture component is a narrow spike (with $\sigma _1 \approx 0$) centered on a single data point $x_1$. Adapted from Figure 9.7 of \citep{BishopBook}. Generated by mix_gauss_singularity.ipynb . (b) Illustration of the benefit of MAP estimation vs ML estimation when fitting a Gaussian mixture model. We plot the fraction of times (out of 5 random trials) each method encounters numerical problems vs the dimensionality of the problem, for $N=100$ samples. Solid red (upper curve): MLE. Dotted black (lower curve): MAP. Generated by mix_gauss_mle_vs_map.ipynb .