A Multistage Approach to Posterior Sampling for Bayesian Models

Music City Center 

The surge in access to computing resources has attracted attention toward the development of algorithms that can run efficiently on multi-core processing units or in distributed computing environments. In the context of Bayesian inference, MCMC still remains the most reliable and widely applicable algorithm to characterize posterior distributions, however its Markovian nature imposes challenges when it comes to parallelization. Running independent chains is inefficient due to the need to discard a fixed number of observations as burn in, while parallelizing a single chain leads to additional communication costs at every iteration. To circumvent both of these issues, multistage approaches have been proposed, where a sample from a computationally convenient and parallel friendly approximation of the posterior distribution is obtained and later corrected using an importance sampling or Metropolis-Hastings post-processing step. In this work, propose an extension of pre-existing multistage approaches, showcasing the effectiveness of the resulting algorithm considering both simulated experiments and real data.

Bayesian inference

embarrassingly parallel MCMC

Gaussian process