The Paradox of Exact and Differentially Private Bayesian Inference

Oregon Convention Center 

While several results in the literature (e.g., Dimitrakakis et al., 2017; Zhang and Zhang, 2023) demonstrate that Bayesian inference approximated by MCMC output can achieve differential privacy with zero or limited impact on the ensuing posterior, we reassess this perspective via an alternate "exact" MCMC perturbation inspired from Nicholls et al. (2012) within a federated learning setting. Our conclusion is that the ensuing privacy is mostly related to a slowing-down of MCMC convergence rather than a generic gain in protecting data privacy.