CONDITION INDEPENDENCE WITH DEEP NEURAL NETWORK BASED BINARY EXPANSION TEST (DEEPBET)

Oregon Convention Center 

This project focuses on testing conditional independence between two random variables (X and Y) given a set of high-dimensional confounding variables (Z). The high dimensionality of confounding variables poses a challenge for many existing tests, leading to either inflated type-I errors or insufficient power. To address this issue, we leverage the Deep Neural Network (DNN)'s ability to handle complex, high-dimensional data while circumventing the curse of dimensionality. We propose a novel DeepBET test procedure. First, we utilize a DNN model to estimate the conditional means of X and Y given Z using part of the data and obtain predicted errors using the other part of the data. Then, we apply a novel binary expansion statistics to construct our test statistics using predicted errors for dependence detection. Furthermore, we implement a multiple-split
procedure to enhance power, utilizing the entirety of the sample while minimizing randomness. Our results show that the proposed method adeptly controls type I error control and exhibits a significant capacity to detect alternatives, making it a robust approach for testing conditional independence.

Conditional independence

Deep Neural Network

Non-parametric Statistics

Binary Expansion Testing

Multi-split method 

IMS