Exact Statistical Testing Through Integer Programming: Leveraging An Overlooked Approach for Optimizing Power in Binomial Tests

Music City Center 

Traditional p-value methods for binomial tests can be either too liberal or overly conservative, especially in small samples. We investigate an alternative approach, first proposed in 1969, that formulates hypothesis testing as a direct decision-making problem. This method represents the test as a binary decision for each outcome pair and uses integer programming to find a decision boundary that optimizes power subject to type I error constraints. Our analysis provides new insights into this approach's properties and advantages. When optimized for average power over all possible parameter configurations under the alternative, the method exhibits remarkable robustness - performing optimally or near-optimally across specific alternatives while maintaining exactness. It can then be further customized for particular prior beliefs. We establish theoretical guarantees for controlling type I error despite discretization of the null space, and empirically quantify the power advantages compared to sophisticated p-value calculations. The findings highlight both the method's practical utility and how techniques from combinatorial optimization can enhance statistical methodology.

Integer Programming