Sunday, Aug 3: 4:00 PM - 5:50 PM
0305
Invited Paper Session
Music City Center
Room: CC-104A
Applied
No
Main Sponsor
Section on Statistical Computing
Co Sponsors
IMS
Section on Statistical Learning and Data Science
Presentations
Generative models for vision and language have shown remarkable capacities to emulate creative processes but still lack fundamental skills that have long been recognized as essential for genuinely autonomous intelligence. Difficulties with causal reasoning and concept abstraction highlight critical gaps in current models, despite their nascent capacities for reasoning and planning. Bridging this gap requires a synthesis of deep learning's expressiveness with the powerful framework of statistical causality.
We will discuss our recent efforts towards building generative models that extract causal knowledge from data while retaining the flexibility and expressivity of deep learning. Unlike traditional causal methods that rely on predefined causal structures, we tackle the more complex problem of learning causal structure directly from data—even when the causal variables themselves are not explicitly observed. This introduces significant challenges, including ill-posedness, nonconvexity, and the exponential complexity of combinatorial search. We will outline statistical aspects of these problems and present progress towards resolving these challenges with differentiable approaches to causal discovery and representation learning.
Keywords
Greedy optimization
Misspecified Nonparametric Model Selection
Laplace approximation
Nonparametric Graphical Models
Existing approaches to differentiable structure learning of directed acyclic graphs (DAGs) rely on strong identifiability assumptions in order to guarantee that global minimizers of the acyclicity-constrained optimization problem identifies the true DAG. Moreover, it has been observed empirically that the optimizer may exploit undesirable artifacts in the loss function. We explain and remedy these issues by studying the behavior of differentiable acyclicity-constrained programs under general likelihoods with multiple global minimizers. By carefully regularizing the likelihood, it is possible to identify the sparsest model in the Markov equivalence class, even in the absence of an identifiable parametrization. We first study the Gaussian case in detail, showing how proper regularization of the likelihood defines a score that identifies the sparsest model. Assuming faithfulness, it also recovers the Markov equivalence class. These results are then generalized to general models and likelihoods, where the same claims hold. These theoretical results are validated empirically, showing how this can be done using standard gradient-based optimizers, thus paving the way for differentiable structure learning under general models and losses.
Keywords
DAG Learning
Identifiability
Sparsest model
Regularization
Estimating causal effects from observational data is inherently challenging due to the lack of observable counterfactual outcomes and even the presence of unmeasured confounding. Traditional methods often rely on restrictive, untestable assumptions or necessitate valid instrumental variables, significantly limiting their applicability and robustness. In this paper, we introduce Augmented Causal Effect Estimation (ACEE), an innovative approach that utilizes synthetic data generated by a diffusion model to enhance causal effect estimation. By fine-tuning pre-trained generative models, ACEE simulates counterfactual scenarios that are otherwise unobservable, facilitating accurate estimation of individual and average treatment effects even under unmeasured confounding. Unlike conventional methods, ACEE relaxes the stringent unconfoundedness assumption, relying instead on an empirically checkable condition. Additionally, a bias-correction mechanism is introduced to mitigate synthetic data inaccuracies. We provide theoretical guarantees demonstrating the consistency and efficiency of the ACEE estimator, alongside comprehensive empirical validation through simulation studies and benchmark datasets. Results confirm that ACEE significantly improves causal estimation accuracy, particularly in complex settings characterized by nonlinear relationships and heteroscedastic noise.
Keywords
Causal effect estimation
Data augmentation
Unmeasured confounding
Generative models
Transfer learning
We develop and analyze algorithms for instrumental variable regression by viewing the problem as a conditional stochastic optimization problem. In the context of least-squares instrumental variable regression, our algorithms neither require matrix inversions nor mini-batches and provides a fully online approach for performing instrumental variable regression with streaming data. When the true model is linear, we derive rates of convergence in expectation, that are of order O(log T/T) and O(T^{ι-1}) for any ι>0, respectively under the availability of two-sample and one-sample oracles, respectively, where T is the number of iterations. Importantly, under the availability of the two-sample oracle, our procedure avoids explicitly modeling and estimating the relationship between confounder and the instrumental variables, demonstrating the benefit of the proposed approach over recent works based on reformulating the problem as minimax optimization problems. Numerical experiments are provided to corroborate the theoretical results.
Keywords
Online Instrumental Variable Regression
Conditional Stochastic Optimization
Stochastic approximation
Causal Inference
Streaming Data
Recently, causality has garnered significant interest within the research communities of statistics, machine learning, and computer science. A central challenge in this field is uncovering the underlying causal structures and models. Traditional methods for causal structure learning often assume the absence of latent confounders. In this talk, I will highlight recent advances in causal structure learning that specifically address the challenges posed by latent confounders. I will focus on three key techniques and their associated structural or distributional constraints, which enable us to identify latent variables, determine their cardinalities, and map out the structure involving both latent and observed variables.
Keywords
Casual Discovery
Latent Confounders
Speaker
Biwei Huang, Halicioğlu Data Science Institute (HDSI), UC San Diego (UCSD)