Tuesday, Aug 5: 2:00 PM - 3:50 PM
4133
Contributed Papers
Music City Center
Room: CC-102B
Main Sponsor
Section on Bayesian Statistical Science
Presentations
Laplace approximation is arguably the simplest approach for uncertainty quantification using intractable posteriors associated with deep neural networks. While the Laplace approximation based methods are widely studied, they are not computationally feasible due to the involved cost of inverting a (large) Hessian matrix. This has led to an emerging line of work which develops lower dimensional or sparse approximations for the Hessian. We build upon this work by proposing two novel sparse approximations of the Hessian: (1) greedy subset selection, and (2) gradient based thresholding. We show via simulations that these methods perform well when compared to current benchmarks over a broad range of experimental settings.
Keywords
Laplace Approximation
Uncertainty Quantification
Posterior predictive distribution
Hessian matrix
Subset selection
Discovering graph structures with cycles from covariate-rich, heterogeneous datasets has been a very challenging and less studied problem. Most of the existing works predominantly centered around learning directed acyclic graphs (DAGs) or undirected graphs in such setup which may be too restrictive for practical implementations. In this article, we propose Bayesian covariate dependent Directed Cyclic Graph model (DCGx) which uses structural equation modelling. Our method allows to learn the underlying cyclic graphs which smoothly vary with covariates by utilizing covariate dependent partition model. For posterior inference we develop parallel tempered Markov Chain Monte Carlo (MCMC) sampler which ensures the eigen condition is satisfied for each covariate dependent cyclic graph. We demonstrate the ability of the proposed algorithm through extensive simulation studies and application to spatial transcriptomics dataset from dorsolateral prefrontal cortex (DLPFC) in human brains where the cell locations serve as the covariates.
Keywords
Directed cyclic graphs
Covariate dependent partition model
Parallel Tempering
MCMC
Covariate dependent graphs
This paper introduces a highly scalable tuning-free algorithm for variable selction in logistic regression Using Pólya-Gamma data augmentation. The proposed method is both theoretically consitent and robust to potential misspecification of the tuning parameter, achived through a hierarchical approach. Exisiting works suitable for high-dimensional settings primarily rely on t-approximation of the logistic density, which is not based on the original likelihood. The proposed method not only builds upon the exact logisitc likelihood, offering superior emperical performance, but is also more computationally efficient, particularly in cases involving highly correlated covariates, as demonstrated in a comprehensive simuation study. We apply our method to a gene expression PCR data from mice and RNA-seq dataset from asthma studies in humans. By comparing its performance to existing frequentist and Bayesian methods in variable selection, we demonstrate the competitive predictive capabilities od the Pólya-Gamma-based approach. Our results indicate that this method enhances the accuracy of variable selection and improves the robutness of predictions in complex, high-dimensional datasets.
Keywords
Logistic regression
Pólya-Gamma distribution
Hierarchical Skinny Gibbs
Spike-and-Slab prior
We introduce a transport map approach for spatial uncertainty quantification in multifidelity problems. By learning invertible transformations between target distribution and reference distribution, the method enables efficient sampling from complex high-fidelity distributions. It captures nonlinear, non-Gaussian dependencies without assuming restrictive functional forms, and scales to high-dimensional spatial settings. Exploiting spatial locality and adaptive parameterization, it achieves accurate inference at reduced cost. Applied to coastal flooding, the method more effectively represents storm surge variability than standard Gaussian process models, demonstrating the strength of transport-based solutions for multifidelity inference in environmental systems.
Keywords
Bayesian inference
EM algorithm
Transport Maps
Recently, Approximate Message Passing (AMP) has been integrated with stochastic localization (diffusion model) by providing a computationally efficient estimator of the posterior mean. Existing (rigorous) analysis typically proves the success of sampling for sufficiently small noise, but determining the exact threshold involves several challenges. In this paper, we focus on sampling from the posterior in the linear inverse problem, with an i.i.d. random design matrix, and show that the threshold for sampling coincide with that of posterior mean estimation. We give a proof for the convergence in smoothed KL divergence whenever the noise variance Δ is below Δ[/AMP], which is the computation threshold for mean estimation introduced in (Barbier et al., 2020). We also show convergence in the Wasserstein distance under the same threshold assuming a dimension-free bound on the operator norm of the posterior covariance matrix, strongly suggested by recent breakthroughs on operator norm bounds in similar replica symmetric systems. A key observation in our analysis is that phase transition does not occur along the sampling and interpolation paths assuming Δ<Δ[/AMP].
Keywords
Approximate Message Passage(AMP)
Bayesian posterior sampling
Random linear models
Diffusion models
Stochastic localization
Many Bayesian analyses depend on posterior samples obtained via Markov Chain Monte Carlo (MCMC) methods such as Metropolis, Slice, and Hamiltonian samplers. While effective, these samplers require assessing autocorrelation, burn in, and convergence, all of which can affect accuracy, efficiency, and computation time. We propose a novel Riemann sampler that discretizes the posterior kernel into equally spaced regions (rectangle in 1D, hyperrectangles in higher dimensions), weights them proportionally, and draws independent samples within each region. Unlike MCMC, this method does not generate a Markov chain. Instead, it produces an independent sample that converges to the true posterior as the region width approaches zero. This approach to sampling is fully efficient, requires no burn-in or convergence checks, is parallelizable, and can be computationally faster than traditional samplers. We compare its accuracy, efficiency, and run time to Metropolis, Slice, Hamiltonian, and No-U-Turn for low dimensional settings.
Keywords
MCMC Algorithm
Bayesian Methods
Sampling Algorithm
Prior Distribution and Likelihood
Posterior Distribution
Kernel Density Estimation