Bayesian methods, meaningful parameters, and the importance of calibration

Music City Center 

Parametric Bayesian models are specified by a prior distribution over the parameter. In simple models, the parameter vector is low-dimensional and the posterior concentrates around the "truth" at an appropriate rate--provided the model is exactly right for the data. However, the models behave differently when the stream of data arises from a distribution that lies outside the parametric family under consideration. In this case, analyses typically show mixed asymptotic performance: although the Bayes estimator may be consistent for the parameter of interest, Bayes estimators for nuisance parameters are inconsistent. As a consequence, credible intervals do not cover the parameter of interest at the nominal rate, even asymptotically. This phenomenon is well known for Bayesian versions of quantile regression, an important exemplar of the generalized Bayes technology.

This talk examines the phenomenon of miscalibration of misspecified models. We advocate the use of meaningful parameters, construct families of robust models that are indexed by these parameters, discuss the relationship between prior distribution and sensitivity analysis, and suggest methods for handling calibration.

Bayes

misspecified model

sensitivity analysis

generalized Bayes

robust model 

Section on Bayesian Statistical Science