Skip to main navigation Skip to search Skip to main content

Local Minima Structures in Gaussian Mixture Models

Research output: Contribution to journal › Article › peer-review

Abstract

We investigate the landscape of the negative log-likelihood function of Gaussian Mixture Models (GMMs) with a general number of components in the population limit. As the objective function is non-convex, there can exist multiple spurious local minima that are not globally optimal, even for well-separated mixture models. Our study reveals that all local minima share a common structure that partially identifies the cluster centers (i.e., means of the Gaussian components) of the true location mixture. Specifically, each local minimum can be represented as a non-overlapping combination of two types of sub-configurations: 1) fitting a single mean estimate to multiple Gaussian components; or 2) fitting multiple estimates to a single true component. These results apply to settings where the true mixture components satisfy a certain separation condition, and are valid even when the number of components is over- or under-specified. We also present a more fine-grained analysis for the setting of one-dimensional GMMs with three components, which provide sharper approximation error bounds with improved dependence on the separation parameter.

Original languageEnglish (US)
Pages (from-to)4218-4257
Number of pages40
JournalIEEE Transactions on Information Theory
Volume70
Issue number6
DOIs
StatePublished - Jun 1 2024
Externally publishedYes

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Keywords

  • Gaussian mixture models
  • local minimizers
  • maximum likelihood estimation
  • nonconvex landscape

Fingerprint

Dive into the research topics of 'Local Minima Structures in Gaussian Mixture Models'. Together they form a unique fingerprint.

Cite this