Joint Probability Estimation of Many Binary Outcomes via Localized Adversarial Lasso

Music City Center 

We estimate the probability of many (possibly dependent) binary outcomes, which is at the core of many applications. Without further conditions, the distribution of an M-dimensional binary vector is characterized by exponentially in M coefficients -- a high-dimensional problem even without covariates. Understanding the (in)dependence structure substantially improves the estimation as it allows an effective factorization of the distribution. To estimate this distribution, we leverage a Bahadur's representation connecting the sparsity of its coefficients with independence across components. We use regularized and adversarial regularized estimators, adaptive to the dependence structure, allowing rates of convergence to depend on the intrinsic (lower) dimension. We propose a locally penalized estimator the presence of (low dimensional) covariates, and provide rates of convergence addressing several challenges in the theoretical analyses when striving for making a computationally tractable formulation. We apply our results in estimating causal effects with multiple binary treatments and show how our estimators improve the finite sample performance compared with non-adaptive estimators.

many binary classification

penalized regression

adversarial lasso 

Section on Statistical Learning and Data Science