Composite Transportation Divergence and Finite Mixture Models

Music City Center 

When data from a statistical population is large and distributed across multiple locations,
initial estimates of the population distribution are often computed on local machines. These local estimators are then transmitted to a central machine for aggregation. For parametric models, simple aggregation via arithmetic means typically achieves optimal convergence rates. However, in finite mixture models, where the parameter space is non-Euclidean, proper aggregation demands more nuanced approaches, considering both computational and statistical challenges. To address the computational burden, we propose using the composite transportation divergence to aggregate mixture distributions. This divergence-based approach identifies an aggregated estimator that is optimal under the defined criteria. We introduce an MM algorithm guaranteed to converge to at least a local optimum
after a finite number of iterations. Our method is further applicable to Gaussian mixture reduction, where a high-order Gaussian mixture is approximated by one of lower order. Under slightly stronger assumptions, the aggregated estimator retains the optimal convergence rate and can be made tolerant to Byzantine failures.
This work is based on joint research with Qiong Zhang and Gong Archer Zhang.