28: Causal Imitation Learning Under Measurement Error and Distribution Shift

Thomas M. Menino Convention & Exhibition Center 

We study offline imitation learning (IL) when part of the decision-relevant state is observed only through noisy measurements and the distribution may change between training and deployment. Such settings induce spurious state--action correlations, so standard behavioral cloning (BC)---whether conditioning on raw measurements or ignoring them---can converge to systematically biased policies under distribution shift. We propose a general framework for IL under measurement error, inspired by explicitly modeling the causal relationships among the variables, yielding a target that retains a causal interpretation and is robust to distribution shift. Building on ideas from proximal causal inference, we introduce CausIL, which treats noisy state observations as proxy variables, and we provide identification conditions under which the target policy is recoverable from demonstrations without rewards or interactive expert queries. We develop estimators for both discrete and continuous state spaces; for continuous settings, we use an adversarial procedure over RKHS function classes to learn the required parameters. We evaluate CausIL on semi-simulated longitudinal data from

Causal Inference

Imitation Learning

Distribution Shift

Measurement Error 

Section on Statistical Learning and Data Science