Non-normal Distributions and Robust Tools Against Assumption Violation

Dixin Shen Chair
Gilead Sciences
 
Thursday, Aug 7: 10:30 AM - 12:20 PM
4224 
Contributed Papers 
Music City Center 
Room: CC-212 

Main Sponsor

IMS

Presentations

It’s Hard to Be Normal: The Impact of Noise on Structure Agnostic Estimation

Modern statistical and causal estimation problems often require estimating high-dimensional, complicated nuisance functions, that are well-suited for using modern machine learning (ML) techniques. Recent work introduced the structure-agnostic estimation, which only relies on black-box nuisance estimates and does not impose any structural assumptions on the nuisances, thereby allowing for flexible incorporation of ML-based methods. This paper studies the impact of noise on the minimax rates of structure-agnostic estimation. Focusing on the partial linear outcome model popular in causal inference, we first show that for Gaussian treatments, the widely adopted double/debiased machine learning (DML) is optimal even with complete distributional information, resolving an open problem from [mackey2018orthogonal]. For non-Gaussian treatment, we propose a general procedure for constructing robust estimators against nuisance errors. For homoscedastic treatment, our procedure induces hocein, a structure-agnostic estimator that achieves fully higher-order robustness, which is the first estimator of such type. Experiments demonstrate the effectiveness of our approach. 

Keywords

causal inference

causal machine learning

semiparametric estimation 

Co-Author(s)

Lester Mackey, Microsoft Research New England
Vasilis Syrgkanis, Stanford University

First Author

Jikai Jin, Stanford University

Presenting Author

Jikai Jin, Stanford University

Adversarially Robust Synthetic Control: Ensuring Robustness Against Highly Correlated Controls and Weight Shifts

The synthetic control method estimates the causal effect by comparing the outcomes of a treated unit to a weighted average of control units that closely match the pre-treatment outcomes of the treated unit. This method presumes that the relationship between potential outcomes of treated and control units remains consistent before and after treatment. However, this estimator may become unreliable when there are shifts in these relationships or when control units are highly correlated. To address these challenges, we introduce the Adversarially Robust Synthetic Control (ARSC). This framework enhances robustness by accommodating potential shifts in relationships and addressing high correlations among control units, thereby ensuring a more reliable causal estimand. When key assumptions for the classical synthetic control method hold, the ARSC method performs comparably to traditional methods. However, under assumption violations, ARSC produces a conservative estimate of the true effect. The ARSC approach employs a distributionally robust optimization by defining the causal estimand as the solution to a worst-case optimization problem, taking into account all possible distributions of the treated unit's potential outcomes that align with observed pre-treatment data. We derive a closed-form solution for the population ARSC estimand and develop a corresponding data-dependent estimator. The consistency of this estimator is established, and its practical utility is demonstrated through various case studies, including an analysis of the economic impact of terrorism in the Basque Country. 

Keywords

Causal inference

Synthetic control method

Adversarial robustness

Distributional shift 

Co-Author

Zijian Guo, Rutgers University

First Author

Taehyeon Koo, Rutgers University

Presenting Author

Taehyeon Koo, Rutgers University

Bivariate Tail Probability Approximation

We generalize saddlepoint approximations for tail areas of cumulative distribution functions of multivariate random variables by extending the univariate Lugannani and Rice saddlepoint tail approximation to multiple dimensions. The proposed approximation is derived by parts. The resulting approximation uses a multivariate Gaussian approximation to the distribution of the signed roots of the log-likelihood statistic. As in the univariate case, the next correction terms involve the difference between reciprocals of the signed root of the likelihood statistics and the analogous Wald statistics. This approximation also uses the curvature of the boundary of the tail region in terms of signed roots of likelihood ratio statistics. The separate versions of the approximation extended to lattice variables and conditional distributions are also provided. Numerical comparisons with other methodologies show better agreement with the exact distribution. We discuss the practical importance of the approximation for multiple dimensions, providing applications for sufficient statistics and statistical multivariate inference. 

Keywords

Saddlepoint approximation

Multivariate Gaussian approximation

Complex analysis

Curvature of the boundary of tail region 

Co-Author

John Kolassa, Rutgers University

First Author

Donghyun Lee, SAS - Statistics, Rutgers New Brunswick

Presenting Author

Donghyun Lee, SAS - Statistics, Rutgers New Brunswick

Broadly discrete stable distributions

The central limit theorem shows that a suitably rescaled sum of random variables with finite variance converges to a Gaussian distribution. When the variance of the summands is infinite, the limiting distribution is instead a heavy tailed stable distribution, indexed by $\alpha\in (0,2]$ where $\alpha=2$ is the Gaussian special case. All nondegenerate stable distributions are absolutely continuous, but a discrete notion of stability can be obtained by replacing scaling with binomial thinning. Under this definition, it has been shown that the strictly discrete stable distributions are Poisson mixtures with maximally skewed stable mixing distributions for $\alpha\leq 1$. In previous work we have established the validity of Poisson-stable mixtures for $\alpha\in [1,2]$ despite the mixing distribution having support on negative numbers. Here we show that this mixed Poisson family is the unique set of distributions satisfying a broader definition of discrete stability with Poisson translation playing the role of subtraction. We also provide a compound Poisson representation and discuss discrete infinite divisibility. 

Keywords

probability

stable distributions

discrete stability

mixed Poisson

count data 

First Author

Will Townes, Carnegie Mellon University

Presenting Author

Will Townes, Carnegie Mellon University

Estimation of Parameters of the Truncated Normal Distribution with Unknown Bounds

Estimators of parameters of truncated distributions, namely the truncated normal distribution, have been widely studied for a known truncation region. There is also literature for estimating the unknown bounds for known parent distributions. However, to our knowledge, there are no works that accommodate both parameter and bound estimation of the truncated normal distribution. In this work, we develop a novel algorithm under the expectation-solution (ES) framework, which is an iterative method of solving nonlinear estimating equations, to estimate both the bounds and the location and scale parameters of the parent normal distribution utilizing theory of best linear unbiased estimates from location-scale families of distribution and unbiased minimum variance estimation of truncation regions. The conditions for the algorithm to converge to the solution of the estimating equations for a fixed sample size are discussed, and the asymptotic properties of the estimators are characterized using results on M- and Z-estimation from empirical process theory. The proposed method is then compared to methods utilizing the known truncation bounds via Monte Carlo simulation. 

Keywords

Truncated Normal Distribution

Parameter Estimation

EM Algorithm

Order Statistics 

Co-Author(s)

Semhar Michael, South Dakota State University
Christopher Saunders, South Dakota State University

First Author

Dylan Borchert, South Dakota State University

Presenting Author

Dylan Borchert, South Dakota State University

On the Generalized-G Poisson Family of Distributions

In recent years, extensive work has been done into developing broader families of distributions to better capture the complexity of various data types. We will present a family of distributions, in an effort to add a more flexible family in the field of probability distributions. This family is derived from the foundation of the zero-truncated Poisson (ZTP) distribution. Some special cases of the generalized-G Poisson family will be presented to show their usefulness to model different types of real-life data sets. 

Keywords

Generalized distribution

zero truncated Poisson distribution

McDonald distribution

Beta distribution 

First Author

Gokarna Aryal, Purdue University Northwest

Presenting Author

Gokarna Aryal, Purdue University Northwest

Thresholding based robust estimation for generalized mixture model

Finite mixture regression models are versatile tools for analyzing mixed regression relationships within clustered and heterogeneous populations. However, the classical normal mixture model often falls short when dealing with nonlinear regression data, especially in the presence of severe outliers. To address this, we introduce a novel generalized robust mixture regression procedure within the finite mixture regression framework. This procedure features sparse, scale dependent mean shift parameters, facilitating outlier detection and ensuring robust parameter estimation. Our approach incorporates three key innovations. (1)A penalized likelihood approach using a combination of L0 and L2 regularization to induce sparsity among mean shift parameters.(2) A close connection to the method of trimming, including explicit outlyingness parameters for all samples, which simplifies computation, aids theoretical analysis, and reduces the need for parameter tuning.(3) High scalability, allowing the implementation to handle nonlinear regression data. A threshold-based generalized Expectation-Maximization algorithm has been developed to ensure stable and efficient computation. 

Keywords

Mixture regression models

EM algorithm

Thresholding

Outlier detection

Mean-shift

Robust procedures 

Co-Author(s)

Weixin Yao, University of California-Riverside
Zhen Zeng, Nanjing University of Finance and Economics

First Author

Xin Shen, University of California-Riverside

Presenting Author

Xin Shen, University of California-Riverside