Generative models for simulating patients in clinical trials with insufficient accrual

Music City Center 

Insufficient patient accrual is a major challenge in clinical trials and can result in underpowered studies, as well as exposing study participants to toxicity and additional costs, with limited scientific benefit. Insufficient accrual can occur due to running out of resources, or a study being stopped early for other reasons. We performed a retrospective analysis using ten datasets from nine fully accrued, completed and published cancer clinical trials. For each trial we simulated insufficient accrual and generated virtual patients to compensate for that. We then replicated the published analyses on this augmented dataset to determine if the findings are the same. Replication of the published analyses utilized four metrics: decision agreement, estimate agreement, standardized difference, and confidence interval overlap. Sequential synthesis performed well on the four replication metrics for the removal of up to 40% of the last recruited patients (decision agreement: 88% to 100% across datasets, estimate agreement: 100%, cannot reject standardized difference null hypothesis: 100%, and CI overlap: 0.8 to 0.92). There was no evidence of a monotonic relationship in the estimated effect size with recruitment order across these studies. This suggests that patients recruited earlier in a trial are not systematically different than those recruited later, at least partially explaining why generative models trained on early data can effectively simulate patients recruited later in a trial.

For an oncology study with insufficient accrual with as few as 60% of target recruitment, sequential synthesis can enable the simulation of the full dataset had the study continued accruing patients, and can be an alternative to drawing conclusions from an underpowered study or even abandoning the data. These results provide evidence demonstrating the potential for generative models to rescue poorly accruing clinical trials.

These results are limited to drug trials (surgery trials, for example, may demonstrate a learning effect over time), for oncology, and did not consider safety data. Furthermore, for small trials pre-trained generative models may provide a better alternative for simulating patients.