In Praise of Modern Statistical and Machine Learning Methods for Digital Engineering

Complex engineered multi-physics systems tend to have many interacting materials, fields, models, and geometries. Our goal is to develop a foundation model for these systems that can be readily fine-tuned for new applications. We are beginning this process by building large transformer-based models for future-frame prediction models for shaped-charge simulations. 

Computer simulations are essential for exploring input-output relationships in engineering and science but can be computationally expensive for extensive what-if analyses. Gaussian process emulators offer a powerful statistical approach to approximating simulations, but their scalability is often hindered by the costly inversion of large correlation matrices. To overcome this challenge, we introduce new methods leveraging the random Fourier feature technique from computer science to accelerate Gaussian process emulators. Our approach enhances computational efficiency while maintaining accuracy, making it suitable for a broad range of simulations, including those with gradient information, functional outputs, and stochastic outputs. Through numerical experiments, we demonstrate that our methods outperform existing ones in speed and accuracy, with theoretical results validating these improvements. 

The Bayesian Additive Regression Tree model (BART) has received much attention in recent years as an alternative to Gaussian Process (GP) emulators, particularly in the large sample size and high input dimension settings where GPs struggle with the curse of dimensionality. However, a longstanding limitation has been the discontinuous response surface of the BART model since many emulation problems involve computer simulators which feature smooth, differentiable responses. A smooth version of BART, SBART, was previously introduced, however the approach required joint updates of terminal nodes, and calibration of the prior, particularly balancing the tradeoff between tree-implied localization versus global continuity, could be a challenge. In our work, we introduce a new approach to creating smooth BART response surfaces by using randomized paths (RP). Our proposed RP-BART model retains the conditionally independent updates of terminal node parameters while introducing a cohesive modeling structure that integrates both the localization of the tree and the continuity-inducing randomized splits in a complimentary manner. Borrowing from the GP literature, a lightly data-informed prior calibration of our model is facilitated by deriving the RP-BART semivariogram, which provides for a natural extension of the original BART prior calibration technique. We demonstrate RP-BART on a climate model application where smooth latent RP-BART weight functions are learned to combine an ensemble of climate models for predicting global mean surface temperature. 

Many real-world manufacturing applications involve experiments with vector-valued inputs, where multiple parameters must be optimized simultaneously. Traditional design of experiments (DoE) methods often struggle with such high-dimensional, structured input spaces, calling for new approaches. In this work, we introduce branched orthogonal arrays (BOAs), a novel class of experimental designs tailored for vector-valued inputs. We present theoretical constructions for both regular and non-regular BOAs, along with efficient algorithmic generation methods. The proposed designs exhibit superior space-filling properties, stratification, and flexibility compared to conventional designs. We investigate their optimality criteria, including uniformity and orthogonality, and demonstrate their advantages in practical manufacturing settings. This work bridges the gap between advanced experimental design theory and complex manufacturing applications, offering a powerful tool for engineers and researchers working with vector-valued inputs.