New Methods in Causal Inference and Reinforcement Learning for Personalized Decision-Making

1201 

Invited Paper Session 

Yongyi Guo (1), Kelly Zhang (2), James Robins (3), Linbo Wang (4), Anish Agarwal (5), Raaz Dwivedi (6), Yongyi Guo (1)

(1) University of Wisconsin-Madison, Madison, WI, (2) Columbia Unviersity, New York, NY, (3) Harvard School of Public Health, Boston, MA, (4) University of Toronto, Toronto, Canada, (5) Columbia University, New York, NY, (6) Cornell Tech, New York, NY

Kelly Zhang  
Columbia Unviersity

Yongyi Guo  
University of Wisconsin-Madison

James Robins  
Harvard School of Public Health

Linbo Wang  
University of Toronto

Anish Agarwal  
Columbia University

Raaz Dwivedi  
Cornell Tech

Yongyi Guo  
University of Wisconsin-Madison

In the rapidly evolving landscape of decision-making (e.g., in healthcare), there is an increasing demand for innovative statistical and machine learning approaches to extract meaningful insights from complex data. This session is dedicated to exploring novel theories and methods aimed at achieving this goal, offering a unique perspective from the realms of statistics and machine learning. Our esteemed speakers will present exciting research that not only advances the fields of causal inference and reinforcement learning but also addresses pressing challenges in real-life decision-making applications.

Three of our speakers will delve into the development of statistically efficient causal inference methods tailored for complex, high-dimensional observational data. These methods play a crucial role in discerning the effectiveness of specific interventions-an essential step in the era of personalized healthcare. Another speaker will shed light on the intricate process of designing effective reinforcement learning (RL) techniques for personalizing interventions in digital health, that provides support to users for adopting healthier behaviors. Furthermore, a dedicated speaker will present methodologies to rigorously evaluate whether RL successfully delivered and personalized support as intended.

Collectively, these talks will foster a rich exchange of ideas, offering insights into the construction of an end-to-end pipeline for sequential decision-making. This pipeline commences with the analysis of observational data, enabling the derivation of data-driven estimates for intervention effects. Subsequently, these estimates inform the development of adaptive algorithms aimed at optimizing intervention delivery through personalization, a critical component of adaptive experiments. The resulting study data will then be subject to meticulous examination, including an assessment of the extent to which personalization was achieved by the RL algorithm and identifying opportunities for enhancing its effectiveness in subsequent studies.

In conclusion, this session encapsulates timely and profoundly relevant topics that are pivotal in addressing crucial challenges, including the efficient estimation of treatment effects, handling unmeasured confounding, navigating high-dimensional inference, and striking a balance between efficiency and personalization in the realm of online decision-making.

The format of the session will be five presentations by the speakers, followed by a panel discussion with the speakers that will enhance the exchange of ideas. Below are the speakers and their tentative presentation titles:

Speaker 1 (James M. Robins)
Title: On minimaxity and admissibility of double machine learning (DML) estimators: Implications for decision making

Speaker 2 (Linbo Wang)
Title: The Promises of Parallel Outcomes

Speaker 3 (Anish Agarwal)
Title: Causal inference with Complex Data Structures

Speaker 4 (Raaz Dwivedi)
Title: Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling

Speaker 5 (Yongyi Guo)
Title: MiWaves: RL Algorithm Design for Digital Health Interventions

ENAR ¹

IMS ³

Society for Medical Decision Making ²