Estimating the Number of Components in Finite Mixture Models via Variational Approximation

Music City Center 

This work introduces a new method for selecting the number of components in finite mixture models (FMMs) using variational Bayes, inspired by the large-sample properties of the Evidence Lower Bound (ELBO) derived from mean-field (MF) variational approximation. Specifically, we establish matching upper and lower bounds for the ELBO without assuming conjugate priors, suggesting the consistency of model selection for FMMs based on maximizing the ELBO. As a by-product of our proof, we show that the MF approximation inherits the stable behavior of the posterior distribution, which benefits from model singularity and tends to eliminate the extra components under model overspecification. This stable behavior also leads to the $n^{-1/2}$ convergence rate for parameter estimation, up to a logarithmic factor, under model overspecification. Empirical experiments validate our theoretical findings and compare them with other advanced methods for selecting the number of components in FMMs.

Finite mixture models

Model selection

Evidence lower bound

Mean-field approximation

Singular models 

International Society for Bayesian Analysis (ISBA)