Inference Without Exact Likelihoods

Bayesian inference for doubly intractable distributions is challenging because they include intractable terms, which are functions of parameters of interest. Although several alternatives have been developed for such models, they are computationally intensive due to repeated auxiliary variable simulations. We propose a novel Monte Carlo Stein variational gradient descent (MC-SVGD) approach for inference for doubly intractable distributions. Through an efficient gradient approximation, our MC-SVGD approach rapidly transforms an arbitrary reference distribution to approximate the posterior distribution of interest, without necessitating any predefined variational distribution class for the posterior. Such a transport map is obtained by minimizing Kullback-Leibler divergence between the transformed and posterior distributions in a reproducing kernel Hilbert space (RKHS). We also investigate the convergence rate of the proposed method. We illustrate the application of the method to challenging examples, including a Potts model, an exponential random graph model, and a Conway--Maxwell--Poisson regression model. The proposed method achieves substantial computational gains over existing algorithms, while providing comparable inferential performance for the posterior distributions. 

Models with intractable normalizing functions arise in a wide variety of areas, for instance network models, spatial models for lattices and point processes, flexible models for count data and gene expression, and models for permutations. Some of the most practical algorithms for these problems do not have rigorous theoretical justifications so it is difficult to determine how to tune them or assess their quality. I will discuss new diagnostics that are useful for assessing the quality of samples from these algorithms. These diagnostics are also useful for tuning the algorithms and they provide general insights about some popular intractable likelihood algorithms.
 

Complex simulators of physical processes are used increasingly often for scientific applications. However, the parameters governing these physical processes can be highly uncertain so reducing this uncertainty is important for improving the accuracy of simulator predictions. To this end, a growing amount of work focuses on constraining these physical parameters by comparing the simulator outputs to corresponding real-world observations. However, these simulators can often be systematically misspecified which requires accounting for this model discrepancy when inferring model parameters. Existing work does this by incorporating a data-driven additive model discrepancy error but this treatment does not necessarily represent how we expect many simulators to be misspecified. In particular, for simulators that output spatial fields, we hypothesize that misspecification could lead to spatial warping errors between the simulation and observations where the simulated structure of a spatial feature may be correct overall but it is displaced in space or distorted in shape. To address this, we propose a novel spatiotemporal modeling framework and estimation procedure that models the spatial warping errors as random transport maps that capture these spatial distortions. Our method generates plausible transport maps using convex Gaussian processes that preserve the spatial structure of the simulation. However, using this shape-constrained process results in a challenging likelihood-free inference problem. We demonstrate how inference is nevertheless possible using sampling-based surrogate likelihoods which we estimate and maximize using exact Hamiltonian Monte Carlo sampling and Neural Likelihood techniques. As a concrete application of our methodology, we consider the UKESM1 climate model and remotely sensed aerosol observations where we expect misspecification in atmospheric dynamics or meteorology to lead to spatial warping errors. Our results show that modeling spatial warping errors yields significantly higher likelihoods and stronger parameter-constraining capabilities compared to models that rely solely on additive errors. 

Approximately valid inference is available with and without likelihoods, e.g., using the asymptotic normality of maximum likelihood estimators and M-estimators more generally. In certain applications, however, there is a need for methods which are exactly (not just asymptotically) valid in an anytime sense; that is, methods that offer frequentist error rate guarantees even if the sample size depends on the observed data. Likelihood-based methods are now available that achieve this goal of anytime validity, but what about cases when the data analyst is not willing to specify a likelihood? In such cases, the relevant unknowns often take the form of risk (expected loss) minimizers. In this talk, I'll start by describing the growth-rate optimal e-process that offers anytime valid and efficient inference on risk minimizers. This optimal e-process depends on certain features that are typically unknown, so a data-driven construction is needed. For this, I'll develop a generalized universal inference framework that mimics the optimal e-process and show some efficiency results and numerical illustrations. To date, generalized universal inference has only been demonstrated to achieve anytime validity empirically in simulations, and I'll highlight the challenges in proving this theoretically. 

Across scientific disciplines, researchers increasingly use generative AI to approach ``inverse problems'' (inferring hidden parameters from observed data). Although these methods bypass intractable likelihoods and reduce computational costs, they can produce misleading conclusions through biased and/or overconfident parameter estimates. We present Frequentist-Bayes (FreB), a protocol that transforms AI-generated probability distributions into rigorous confidence regions that consistently include true parameters with the expected probability while remaining precise when training and target data align. We demonstrate FreB's effectiveness by tackling diverse case studies in the physical sciences: identifying unknown sources under domain shift, reconciling competing theoretical models, and mitigating selection effects in observational studies. By providing validity guarantees without sacrificing efficiency, FreB enables trustworthy scientific inference and uncertainty quantification across fields where direct likelihood evaluation remains impossible or prohibitively expensive.