Abstract
We investigate the landscape of the negative log-likelihood function of Gaussian Mixture Models (GMMs) with a general number of components in the population limit. As the objective function is non-convex, there can exist multiple spurious local minima that are not globally optimal, even for well-separated mixture models. Our study reveals that all local minima share a common structure that partially identifies the cluster centers (i.e., means of the Gaussian components) of the true location mixture. Specifically, each local minimum can be represented as a non-overlapping combination of two types of sub-configurations: 1) fitting a single mean estimate to multiple Gaussian components; or 2) fitting multiple estimates to a single true component. These results apply to settings where the true mixture components satisfy a certain separation condition, and are valid even when the number of components is over- or under-specified. We also present a more fine-grained analysis for the setting of one-dimensional GMMs with three components, which provide sharper approximation error bounds with improved dependence on the separation parameter.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 4218-4257 |
| Number of pages | 40 |
| Journal | IEEE Transactions on Information Theory |
| Volume | 70 |
| Issue number | 6 |
| DOIs | |
| State | Published - Jun 1 2024 |
| Externally published | Yes |
ASJC Scopus subject areas
- Information Systems
- Computer Science Applications
- Library and Information Sciences
Keywords
- Gaussian mixture models
- local minimizers
- maximum likelihood estimation
- nonconvex landscape
Fingerprint
Dive into the research topics of 'Local Minima Structures in Gaussian Mixture Models'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver