Efficient Semiparametric Inference for Distributed Data with Blockwise Missingness

Music City Center 

We consider statistical inference for a finite-dimensional parameter in a regular semiparametric model under a distributed setting with blockwise missingness, where entire blocks of variables are unavailable at certain sites. We propose a class of augmented one-step estimators that incorporate information from external sites through "transfer functions." The proposed approach has several main advantages. First, it is communication-efficient, requiring only one-round communication of summary-level statistics. Second, it satisfies a "do-no-harm" property in the sense that the augmented estimator is at least as efficient as the original one based solely on the internal data. Third, it is statistically optimal, achieving the semiparametric efficiency bound when the transfer function is appropriately estimated from data. Finally, it is scalable, remaining asymptotically normal even when the number of external sites grows with the internal sample size. Simulation studies confirm both the statistical efficiency and computational feasibility of our method in distributed settings.

Blockwise missing

Distributed inference

Semi-parametric inference 

IMS