Bayesian Model Selection and Averaging with Latent Binary Bayesian Neural Networks

Artificial neural networks (ANNs) yield accurate predictions but are often over-parameterized and hard to interpret. Bayesian neural networks (BNNs) mitigate uncertainty by treating weights probabilistically, while latent binary BNNs (LBBNNs) handle structural uncertainty via weight sparsification. We extend LBBNNs by allowing covariates to skip to any subsequent layer or be excluded entirely, yielding an input-skip LBBNN (ISLaB) that learns parsimonious structures-from linear to intercept-only models-when suitable. ISLaB achieves extreme compression (over 99–99.9% weight reduction) with minimal loss in accuracy; on MNIST, it attains 97% accuracy and excellent calibration using only 935 weights. Moreover, it identifies relevant covariates, captures nonlinearity, and introduces theoretically grounded, intrinsic local and global model explanations without post hoc tools. The methods are available in the open-source R package LBBNN on CRAN.

Bayesian model averaging

Interpretable deep learning

Stochastic variational Bayes

Uncertainty in deep learning

Structure learning