Joint Modeling for Learning Decision-Making Dynamics in Behavioral Experiments

Music City Center 

Major depressive disorder (MDD), a leading cause of disability and mortality, is associated with reward-processing abnormalities and concentration issues. Motivated by the probabilistic reward task from the EMBARC study, we propose a novel framework that integrates the reinforcement learning (RL) model, hidden Markov model (HMM), and drift-diffusion model (DDM) to analyze reward-based decision-making alongside response times. To model latent state switching, we use an HMM. In the 'engaged' state, decisions follow an RL-DDM, simultaneously capturing reward processing, decision dynamics, and temporal structure. In contrast, in the 'lapse' state, decision-making is modeled using a simplified DDM, where specific parameters are fixed to approximate random guess. The method is implemented via a computationally efficient EM algorithm with forward-backward procedures. Numerical studies show superior performance over competing methods across various settings. When applied to EMBARC, our framework reveals that MDD patients engage less than healthy controls and take longer to decide when engaged. We also examine associations between brain activities and decision-making characteristics.

Brain–behavior association

Cognitive modeling

Drift-diffusion models

Mental health

Reinforcement learning

State switching 

Mental Health Statistics Section