Does great power come with great type I error rate inflation? Exact tests for power-maximizing response-adaptive designs

Music City Center 

In this talk, we consider power-maximizing response-adaptive designs (RADs). In addition to Neyman allocation and variations thereof, we consider RADs found using a recently introduced constrained Markov decision process (CMDP) approach. Such CMDP procedures target maximum Bayesian average power while constraining the type I error rate. We demonstrate that, while the Wald test shows higher power under Neyman allocation than under equal allocation, the type I error rate inflation can be substantially higher. We investigate whether, for two-arm binary outcome trials, combinations of RADs and statistical tests can be found that show a higher power than equal allocation while controlling type I errors. For the RADs, we investigate modifications to control allocations to the inferior arm. For testing, we employ a recently introduced approach to efficiently construct exact tests for RADs. Our results show that CMDP procedures that constrain both type I error and the expected allocations to the inferior arm show the best performance, can reach power gains over equal allocation, control type I errors, and tend to allocate most participants to the superior treatment in expectation.

Neyman allocation

Constrained Markov decision process

control of expected treatment failures

binary outcomes

two-arm trial