Wednesday, Aug 6: 8:30 AM - 10:20 AM
0632
Topic-Contributed Paper Session
Music City Center
Room: CC-202A
uncertainty quantification
kriging
surrogate
calibration
software
active learning
Applied
Yes
Main Sponsor
Uncertainty Quantification in Complex Systems Interest Group
Co Sponsors
Quality and Productivity Section
Section on Physical and Engineering Sciences
Presentations
Computer simulations are indispensable for analyzing complex systems, yet high‑fidelity models often incur prohibitive computational costs. Multi‑fidelity frameworks address this challenge by combining inexpensive low‑fidelity simulations with costly high‑fidelity simulations to improve both accuracy and efficiency. However, certain scientific problems demand even more accurate results than the highest‑fidelity simulations available. In this paper, we introduce the Diffusion Non‑Additive (DNA) emulator that (1) captures Markovian dependencies among tuning parameters and extrapolates to the exact solution corresponding to a zero-valued tuning parameter, which cannot be simulated directly. The DNA emulator further (2) models complex, non-additive relationships across fidelity levels, (3) employs nonseparable covariance kernels to capture interactions between tuning parameters and input variables, and (4) supports fully nonnested experimental designs for enhanced flexibility. We (5) derive closed‑form expressions for the posterior predictive mean and variance under the nonseparable kernel, enabling efficient inference and rigorous uncertainty quantification. Lastly, We demonstrate the efficacy of our approach on a suite of numerical studies and real-world case studies.
Keywords
surrogate model
finite-element simulation
multi-fidelity
Active learning in computer experiments aims at allocating resources in an intelligent manner based on the already observed data to satisfy certain objectives such as emulating or optimizing a computationally expensive function. There are two main ingredients for active learning: an initial experimental design, which helps to approximately learn the function, and a surrogate modeling technique, which provides a prediction of the output along with its uncertainty estimates. Space-filling designs are commonly used as initial design and Gaussian processes for surrogate modeling. This article aims at improving the active learning procedure by proposing a new type of initial design and a new correlation function for the Gaussian process. The ideas behind them are known in other fields such as in sensitivity analysis or in kernel theory, but they never seem to have been used for active learning in computer experiments. We show that they provide substantial improvement to the state-of-the-art methods for both emulation and optimization. We support our findings through theory and simulations, and a real experiment on the vapor-phase infiltration process.
Keywords
Bayesian optimization
Gaussian process
Screening design
Sequential design
Surrogate modeling
Computer simulations play a significant role in today's scientific discovery and engineering advancements. The validation of computer simulation models is a critical step in ensuring their reliability and accuracy before they are employed in practical applications. Effective validation not only builds confidence in the models' predictive capabilities but also identifies potential limitations and areas for improvement. This work focuses on statistical validation of computer models, with the goal of determining whether the computer response function differs from the physical process in a detectable manner. We formulate the validation process as a hypothesis testing problem: the computer model is invalidated if the null hypothesis is rejected. We introduce the local validation, which aims at identifying the subregions where the computer response and the physical process differ. Rigorous statistical methods are developed for global and local validation. The local tests are performed under a multiple testing framework.
Keywords
Computer experiments
Uncertainty quantification
Multiple testing
Kernel ridge regression
Speaker
Rui Tuo, Texas A&M University
Software testing is essential for the reliable and robust development of complex software systems. This is particularly critical for cyber-physical systems (CPS), which require rigorous testing prior to deployment. The complexity of these systems limits the use of formal verification methods. Furthermore, testing and fault localization can be very costly. To mitigate this cost, we outline in this work a holistic machine-learning-guided test case design and fault localization (MaLT) framework, which leverages recent probabilistic machine learning methods to accelerate the testing of complex software systems. MaLT consists of three steps: (i) the construction of a suite of test cases using a covering array for initial testing, (ii) the investigation of posterior root cause probabilities via a Bayesian fault localization procedure, then (iii) the use of such Bayesian analysis to guide selection of subsequent test cases via active learning. The proposed MaLT framework can thus facilitate efficient identification and subsequent diagnosis of software faults with limited test runs.
Keywords
Fault localization
Bayesian modeling
Combinatorial testing
Active learning
Probabilistic machine learning
Stratified sampling is one of the powerful variance reduction methods for analyzing system performance, such as reliability, with stochastic simulation. It divides the input space into disjoint subsets, called strata, to draw samples from each stratum. Partitioning the input space properly and allocating greater computational effort to crucial strata can help accurately estimate system performance with a limited computational budget. How to create strata, however, has yet to be thoroughly examined. We analytically derive the optimal stratification structure that minimizes the estimation variance for univariate problems. Further, reconciling the optimal stratification into decision trees, we devise a robust algorithm for multi-dimensional problems.
Keywords
variance reduction
uncertainty quantification
stochastic simulation
decision-tree