GS-BART: Bayesian Additive Regression Trees with Graph-split Decision Rules for Generalized Spatial Nonparametric Regressions

Music City Center 

Ensemble decision tree methods such as XGBoost, random forest, and Bayesian additive decision trees (BART) have gained enormous popularity in data science for their superior performance in machine learning regression and classification tasks. In this paper, we develop a new Bayesian graph-split-based additive decision trees method, called GS-BART, to improve the performance of BART for spatially dependent data. The new method adopts a highly flexible split rule complying with spatial structures to relax the axis-parallel split rule assumption adopted in most existing ensemble decision tree models. We consider a generalized spatial nonparametric regression model using GS-BART and design a scalable informed MCMC algorithm to sample the decision trees of GS-BART, which apply to both point referenced and areal unit data as well as Gaussian and non-Gaussian responses. The algorithm leverages a gradient-based recursive algorithm on root directed spanning trees or chains (called arborescences) The superior performance of the method over conventional ensemble tree models and Gaussian process regression models is illustrated in various spatial data analysis.

Bayesian Nonparametric Regression

Complex Domain

Decision Trees

Informed MCMC

Spatial Prediction

Spanning Tree