Geometric Stochastic Interventions for Efficient Policy Learning

Thomas M. Menino Convention & Exhibition Center 

We introduce a geometric framework for constrained policy optimization using stochastic intervention analysis. Our goal is to estimate the causal effect of a policy that evolves smoothly from a behavioral baseline to an optimal target. We formulate the optimization problem as maximizing expected reward, subject to a Bregman divergence constraint and then show that the optimal solution lies on an e-geodesic, generated by a strictly convex non-affine generator G. For the expected reward along the geodesic path, an Efficient Influence function (EIF) is derived and an explicit decompositation of its variance is provided. We also develop a data-driven methodology to learn the generator G which minimizes the variance of EIF. This provides  a statistically efficient estimator, that extends beyond the limitations of exponentially family distributions.

Stochastic Interventions

Causal Inference

Constrained Policy Optimization

Efficient Influence Function 

Section on Statistical Learning and Data Science