It’s Hard to Be Normal: The Impact of Noise on Structure Agnostic Estimation

Music City Center 

Modern statistical and causal estimation problems often require estimating high-dimensional, complicated nuisance functions, that are well-suited for using modern machine learning (ML) techniques. Recent work introduced the structure-agnostic estimation, which only relies on black-box nuisance estimates and does not impose any structural assumptions on the nuisances, thereby allowing for flexible incorporation of ML-based methods. This paper studies the impact of noise on the minimax rates of structure-agnostic estimation. Focusing on the partial linear outcome model popular in causal inference, we first show that for Gaussian treatments, the widely adopted double/debiased machine learning (DML) is optimal even with complete distributional information, resolving an open problem from [mackey2018orthogonal]. For non-Gaussian treatment, we propose a general procedure for constructing robust estimators against nuisance errors. For homoscedastic treatment, our procedure induces hocein, a structure-agnostic estimator that achieves fully higher-order robustness, which is the first estimator of such type. Experiments demonstrate the effectiveness of our approach.

causal inference

causal machine learning

semiparametric estimation 

IMS