Monday, Aug 4: 2:00 PM - 3:50 PM
0623
Topic-Contributed Paper Session
Music City Center
Room: CC-209C
The implementation of Bayesian adaptive trials in clinical research presents unique practical challenges, particularly when integrating these methodologies into software products. Commercial packages can be somewhat constraining and inflexible, and often fail to incorporate modern tools for design optimization and causal inference. By contrast, the writing of bespoke computer code for each new design is often inefficient and unacceptably work-intensive. In this session, we will hear from 4 speakers who have extensive experience in both the methodological and computational aspects of this tradeoff. Speaker 1 (Carlin) will provide an overview of recent advances in computational power to manage the complexity of the Bayesian adaptive approach, contrasting it with frequentist approaches. He will then describe a new platform whose application programming interface (API) facilitates effective communication among diverse stakeholders, and presents complex Bayesian results in an intuitive manner. Group sequential methods implemented with moderate-dimensional alpha- and beta-spending functions emerge as key to this process. Speaker 2 (Wathen) will offer an in-depth exploration of the unique challenges of adaptive platform designs, and discuss practical solutions for navigating this intricate landscape using a R package called OCTOPUS. This talk will emphasize the critical importance of simulating the exact platform trial one plans to conduct, and of accounting for the addition and removal of new treatments over time, as well as other sources of variation that may impact the performance of the platform. Speaker 3 (Pryluk) will introduce a commercial platform incorporating a novel ensemble estimation approach that leverages causal machine learning methods to enhance the detection and assessment of heterogeneity in adaptive trials. The framework uses conformal prediction to assess uncertainty in its ML estimates for finite samples, facilitating a more nuanced understanding of how different patient subgroups respond to treatments. Speaker 4 (Roychoudhury) will present experience at his firm to date with a new approach that largely eschews commercial software packages and instead embraces the free open-source software movement, driven by R and Rmarkdown. Finally, the discussant (Thompson) will summarize the presentations, offer a regulatory perspective on the commercial and open-source approaches, give her view on the future of AI in regulatory science, and suggest areas for future work. The proposed session is timely (as the cost of clinical trials continues to grow and the free software movement continues to burgeon), and should appeal to a large collection of biostatisticians and trialists hoping to use modern adaptive and causal inference tools on a routine basis, especially in rare and pediatric drug development.
adaptive trials
commercial software
open-source software
trial optimization
machine learning
causal inference
Applied
Yes
Main Sponsor
Biopharmaceutical Section
Co Sponsors
ENAR
Section on Statistical Computing
Presentations
Clinical trials are becoming increasingly complex, incorporating numerous parameters and degrees of freedom. Optimal analytic approaches for these intricate trial designs are often unavailable, necessitating extensive simulation to control the Type I error and power, and to achieve small sample size and other favorable operating characteristics. This paper proposes a general method to reduce the number of parameters using group stepwise methods and Monte Carlo simulations, significantly decreasing the number of iterations required to identify near-optimal parameters. The key idea is the use of the Hwang-Shih-DeCani family of error-spending functions, which use just two parameters (an alpha-spending parameter γ_α and a beta-spending parameter γ_β) that determine sensible stopping boundaries for efficacy and futility, respectively. The algorithm then optimally determines stopping boundaries in such a way that power is maximized and overall Type I error is strictly controlled at a predetermined α level. Our method extends classical group sequential designs, but does not rely on normality assumptions, and can accommodate complex trial designs. We illustrate in the case of a multi-arm clinical trial with one control arm and k treatment arms, and where interim analyses are performed when 30%, 50%, and 70% of the total sample size have been accumulated. Our approach delivers boundaries that offer significant average sample size reductions under both the null and alternative hypotheses.
Keywords
adaptive trials
Optimal clinical trial design
As clinical trial design evolves to encompass innovative methodologies like Bayesian adaptive designs and master protocols, the complexity and variety of design options can be overwhelming. This presentation will delve into the intricacies of designing and simulating adaptive platform trials, highlighting the pivotal role of statisticians in this new landscape.
Building on a platform trial currently under design, this talk will illustrate how the role of statisticians has evolved from traditional data analysis to key contributors in the trial design process. As design complexity increases, extensive simulations become a necessity, opening up opportunities for statisticians to guide the design process.
This presentation will provide a brief overview of adaptive platform trials, including their terminology, potential risks and benefits. It will demonstrate how simulations are conducted using OCTOPUS, combined with trial-specific additions, to accommodate various options explored for Bayesian analysis. It will present several visualizations used to illustrate trade-offs and evaluate the frequentist operating characteristics of a Bayesian design.
The focus will be on how intuitive visuals and advanced simulation tools can guide the team and stakeholders to understand the trial processes and expected outcomes under a wide range of scenarios. This approach facilitates informed decision-making and optimizes trial design to potentially save time, resources, and most importantly, enhance patient outcomes.
Keywords
platform trials
Understanding and assessing heterogeneity in clinical trials is essential for personalized medicine and optimizing treatment strategies. This talk introduces a software platform that employs novel ensemble approaches, leveraging causal machine-learning methods to enhance the detection and assessment of heterogeneity in clinical trials while maintaining statistical guarantees. The new ensemble methods integrate multiple estimators to enhance prediction stability and performance - e.g., Stacked X-Learner which uses the X-Learner with model stacking for estimating the nuisance functions, and Consensus Based Averaging (CBA), which averages only the models with highest internal agreement.
Causal machine learning excels at identifying and interpreting relationships between variables, enabling a nuanced understanding of how different patient subgroups respond to treatments. However, selecting the optimal causal ML method is challenging due to frequent disagreements among algorithms, each with unique advantages and limitations. Additionally, many causal ML methods lack well-understood uncertainty estimates, especially in the finite samples common in clinical trials. Our platform addresses these challenges by implementing a framework for estimating uncertainty in finite samples and integrating a weighted combination of different algorithms through Bayesian or Frequentist approaches that account for model uncertainty. This fusion results in robust performance across a wide range of data generation processes.
The user-friendly interface supports commercial use throughout the entire life cycle, from data ingestion and model building to result interpretation and actionable decision-making, including report generation using LLMs.
Moreover, the model output from the heterogeneity analysis can then be used in the other module of the platform to inform simulation analysis of the next clinical trial, whether it's an adaptive or fixed design.
Keywords
causal inference
The pharmaceutical industry is increasingly adopting open-source software (OSS) to drive innovation, enhance efficiency, and reduce costs across research, development, and manufacturing. A well designed and documented OSS offers numerous opportunities, including increased collaboration, transparency, and accelerated drug discovery through advanced computational tools. It enables cost-effective solutions for clinical trial design and simulation, data analytics, and regulatory compliance while fostering a culture of scientific openness. However, the integration of OSS in pharma also presents challenges within an organization. These include concerns about data security, regulatory compliance, software validation, training for associates, and long-term support. Additionally, the lack of standardized frameworks and potential intellectual property risks may hinder widespread adoption. This talk will share some of the firsthand experiences regarding the benefits and limitations of using OSS in the pharma. It'll highlighting strategies to overcome challenges and maximize its potential. By leveraging OSS effectively, pharma companies can drive innovation while maintaining compliance and ensuring data integrity in a highly regulated industry. The talk will highlight a few examples including tools developed inhouse with a pharma and tools adopted from Comprehensive R Archive Network (CRAN) in collaborations with academia.
Keywords
open-source software