Wednesday, Aug 5: 10:30 AM - 12:20 PM
6041
Contributed Papers
Thomas M. Menino Convention & Exhibition Center
Room: CC-105
Main Sponsor
Business and Economic Statistics Section
Presentations
Evaluation of diagnostic tests and biomarkers using parametric ROC methods often requires that data be transformable to an approximately normal distribution. The Box-Cox transformation is commonly used but is restricted to non-negative values and therefore cannot be applied to data containing both negative and positive measurements. The Yeo-Johnson transformation is a flexible alternative that can accommodate both negative and positive observations; however, its use for ROC-based inference has not been formally developed. In this study, we explore an ROC inference framework based on the Yeo–Johnson transformation that accounts for estimation variability of the transformation parameter. These include key ROC functionals such as the AUC, the Youden index, and the sensitivity at a fixed specificity (or vice versa). We further extend this framework to paired designs, allowing comparison of correlated ROC curves when two biomarkers are measured on the same individuals. We evaluate the performance of the explored methods through simulation studies and further demonstrate them on real data involving pancreatic cancer biomarker data. Finally, we discuss an accompanying R package, rocYeoJ.
Keywords
Biomarker Evaluation
Yeo–Johnson transformation
ROC curve inference
Sensitivity
Specificity
Correlated Biomarkers
As AI systems are increasingly deployed across organizations and sectors, risk increasingly arises from correlated errors across multiple, ostensibly independent systems. Shared training data, architectures, foundation models, and alignment pipelines can induce synchronized failures, creating accumulation and systemic risks that evade standard single‑model evaluations. This paper surveys methodological approaches for quantifying behavioral similarity among machine learning models, focusing on error correlation as a key indicator of systemic risk. We introduce a risk‑oriented evaluation framework grounded in desirable statistical properties and assess the applicability of existing metrics under realistic auditing constraints. Drawing on recent empirical evidence, we identify common drivers of correlated behavior and examine their implications for downstream deployment risk. We argue that effective AI governance requires shifting from isolated model validation toward portfolio‑level auditing and dependency‑aware risk management aligned with emerging regulatory and assurance frameworks.
Keywords
Correlated Models
Systemic Risk
Safety‑critical Machine Learning
Auditing
Governance
Synthetic control methods are widely used for policy evaluation, but most existing methods rule out interference among units, compromising validity when such effects are present.
We develop a framework that accommodates contaminated donor pools and unknown interference patterns through two stages: factor-model adjustment for unobserved confounding, followed by robust regression in which direct and interference effects appear as a sparse outlier component.
When the number of units is fixed and at least half are unaffected by interference, high-breakdown robust regression yields consistent identification of valid controls and asymptotically normal inference. When the number of units diverges, we allow for sparse large and dense weak interference, with robust M-estimation remaining valid even when the post-intervention period is short.
Unlike methods requiring pre-specified valid controls or parametric modeling of interference, our framework relies only on coarse sparsity information and enables formal inference on both direct and interference effects.
Simulations and two empirical examples demonstrate the method's validity and reveal novel insights about interference effects.
Keywords
comparative case study
factor analysis
policy evaluation
interference effect
robust regression
Multivariate time series in economics and finance are often moderately to highly dimensional and may display nonlinear dynamics, limiting the effectiveness of classical vector autoregressive (VAR) models. We propose neural VAR models with built-in dimension reduction that first map the series into a low-dimensional dynamic subspace and then fit a VAR structure augmented with a feed-forward neural component, yielding parsimonious yet flexible representations. Simulation studies show that the proposed method achieves smaller estimation error and better forecast accuracy across a range of data-generating processes. An application to a system of macroeconomic indicators demonstrates that the model delivers more accurate forecasts and a more interpretable dependence structure than competing VAR models.
Keywords
Dimension reduction
Neural networks
Multivariate time series
Time series modeling has long been used for forecasting financial data, with traditional financial time series models providing interpretable and theoretically grounded results. Recent advances in deep learning have introduced flexible, data-driven approaches that often achieve high predictive accuracy but differ fundamentally in estimation and validation procedures. This study addresses the challenge of comparing these two model paradigms under a mathematically consistent framework and proposes a unified evaluation methodology for meaningful comparison in stock price forecasting. The framework is applied to daily stock prices of major U.S. technology firms, including Apple, Amazon, Meta, Microsoft, Alphabet, and Tesla. Financial time series models such as ARIMA, GARCH, GJR-GARCH, and ARIMA–GARCH ensembles are evaluated alongside RNN, LSTM, and GRU models. Model performance is assessed under a common validation structure, ensuring that observed differences reflect underlying model dynamics rather than inconsistencies in transformation or evaluation design, while highlighting trade-offs between theoretical structure and predictive flexibility within a unified comparison framework.
Keywords
Financial Time Series Models
Cross-Paradigm Validation
Forecast Evaluation
ARIMA–GARCH
Deep Learning
Stock Prices
We study the dynamic optimal portfolio allocation problem in a market responsive to macroeconomic regimes, as characterized by VIX. Unlike the traditional approach, we train an artificial neural network (ANN) to learn the optimal allocation as a feedback function. Our regime-specific strategy, benchmarked against the classical Merton portfolio, shows superior performance subject to realistic diversification constraints with borrowing/short selling excluded. A 35-year backtest (1990-2024), including 17 out-of-sample years, on a diversified portfolio of twelve assets plus cash, reveals that accounting for regime shifts improves both expected utility and average returns. Our approach can be applied to other types of regime models, as well as other models generally that strive to optimize investors' utility. It can also be used to optimize other metrics such as the Sharpe ratio or Sortino ratio, which are favored by many practitioners.
Keywords
Merton's problem
portfolio optimization
market regimes
deep learning
artificial neural network
multi-layer perceptron
The Population Stability Index (PSI) is a widely used measure for detecting distributional changes in applications such as finance and machine learning, yet its statistical properties in particular between two continuous distributions remain largely unexplored. In this work, we study inference for PSI between two absolutely continuous distributions based on independent samples. By expressing PSI as a symmetrized Kullback–Leibler divergence, we reduce the problem to the estimation of differential entropy and cross-entropy. We construct three classes of PSI estimators derived from k-nearest neighbor, histogram, and kernel density entropy estimators. For each estimator, we investigate large-sample properties, including consistency and asymptotic distribution, under suitable regularity conditions. These results enable formal hypothesis testing for distributional equality and provide a theoretical foundation for detecting concept shift. We further discuss the asymptotic relative efficiency of the proposed estimators and offer practical recommendations for their use.
Keywords
Population Stability Index
Divergence
Hypothesis Testing
Concept Drift
Model Monitoring