Optimizing Navigation in Uncertain Terrain with Spatially Correlated Obstacles

Music City Center 

We propose a novel hybrid reinforcement learning (RL) framework for path-planning in an environment with uncertain, spatially correlated obstacles. Using Gaussian Random Field to capture spatial dependencies, we utilize Bayesian approach for sequential blockage probability update. Unlike prior approaches based on point estimates for value functions, we develop a distributional RL approach to model state value functions using categorical distributions, providing a comprehensive characterization of future traversal costs to improve robustness against information uncertainty and sample variation. We integrate distributional Bellman update with adaptive support refinement via Bayesian updates to ensure the true distributions are accurately reflected. In addition, we introduce a search space reduction technique to identify decision candidates, enhancing scalability. Combining distributional RL with posterior sampling of environment dynamics, experimental results show that the resulting decision-making policy effectively balances immediate traversal costs and the long-term value of information, offering a principled solution to the exploration–exploitation tradeoff in optimal navigation.

stochastic path planning

sequential decision making

Bayesian update

distributional Reinforcement Learning

network traversal

Gaussian random fields 

Section on Statistical Learning and Data Science