Hierarchical Conformal Prediction for Clustered Data with Missing Responses

Music City Center 

Existing prediction methods for clustered data often depend on strong model assumptions, making them vulnerable to model misspecification. We propose a hierarchical conformal prediction framework for predicting outcomes of new subjects at specific time points or trajectories in clustered data with missing responses, without requiring the specification of the prediction model or within-subject correlations. The idea is to establish marginal prediction for clustered data while utilizing subsampling techniques to accommodate dependency and appropriate weighting to address distribution shifts caused by missing data.
To address complex error distributions, including skewed and multimodal cases, we construct the prediction region using the highest conditional density set of the target distribution. Additionally, we propose an enhanced approach, termed localized prediction, to more effectively adapt to heterogeneous or atypical subjects. This method achieves not only marginal coverage but also local and asymptotic conditional coverage for a given subject within a subset or specific profile, while converging to optimal interval lengths under consistent estimation conditions.

Conformal prediction

Conditional coverage

Distribution shift

Marginal prediction

Missing at random

Repeated subsampling 

Section on Nonparametric Statistics