Using Statistics, Data Science and AI to Enrich the Assessment of Treatment Effect Heterogeneity in Drug Development

Subgroup analyses present significant challenges in biostatistics, particularly in clinical trials. Estimation of treatment effects within subgroups in an exploratory setting is often unreliable due to limited sample sizes and multiplicity issues. Through the past decades, many efforts have been made to address this problem. Among them, Muysers et al. (2020) considered generating a graphical display that presents numerous subgroups on the same figure and could potentially illustrate homogeneity or heterogeneity. This interactive plot has outcome variable (treatment effect measure) on the y-axis and subgroup size on the x-axis. We refer to this plot as an exhaustive subgroup treatment effect plot. As the original plot avoids inferential statistics, there is still a need for a global assessment of whether the observed heterogeneity is expected or larger than expected under global homogeneity. In this presentation, we will introduce a computationally efficient method to create the exhaustive subgroup treatment effect plot and derive the confidence regions that control the familywise error rate at a pre-specified significance level, which can also serve as a global interaction test. The methodology uses the double robust learner approach (Kennedy, 2023) and employs high-dimensional integration to directly compute quantiles of the cumulative distribution function of the maximum statistics to obtain the rejection region. We also conduct a comprehensive simulation study to evaluate the validity of the approach and benchmark it with other approaches.  

This talk will compare different ways of implementing Bayesian shrinkage estimation for subgroup analysis on clinical trials data. Traditionally Bayesian shrinkage is applied to non-overlapping subgroups using hierarchical models. This implies that several models need to be fitted when several subgroup defining variables are of interest. Recently Wolbers et al (2024) propose to use a single global regression model using shrinkage priors for the used model.
We will compare the performance of different shrinkage approaches based on a real data benchmark. The evaluated approaches include no and full-shrinkage towards the overall treatment effect, Bayesian hierarchical shrinkage and more novel priors such as the global model prior R2D2 proposed by Zhang et al (2020) will be compared.
 

In regression and causal inference, controlled subgroup selection aims to identify, with inferential guarantees, a subgroup (defined as a subset of the covariate space) on which the average response or treatment effect is above a given threshold. E.g., in a clinical trial, it may be of interest to find a subgroup with a positive average treatment effect. However, existing methods either lack inferential guarantees, heavily restrict the search for the subgroup, or sacrifice efficiency by naive data splitting. We propose a novel framework that allows the analyst to interactively refine and test a candidate subgroup by iteratively shrinking it. The sole restriction is that the shrinkage direction only depends on the points outside the current subgroup, but otherwise the analyst may leverage any prior information or machine learning algorithm. Despite this flexibility, our method controls the probability that the discovered subgroup is null (e.g., has a non-positive average treatment effect) under minimal assumptions: for example, in randomized experiments, our method controls the error rate under only bounded moment conditions. Empirically, our method identifies substantially better subgroups than existing methods with inferential guarantees. 

In drug development, the exploration of treatment effect heterogeneity (TEH) within the treated population is important for advancing precision medicine. Drugs may exhibit varying effects across disease subtypes and patient subgroups, often identified through biomarker signatures. Investigating TEH is a continuous scientific process that needs interdisciplinary collaboration throughout the drug development pipeline. Statisticians play a critical leadership role in this process, ensuring robust analysis and interpretation for the study team to make informed decisions. This talk will explore best practices for statisticians in collaborating with interdisciplinary teams to identify TEH in precision drug development. 

We proposed a Workflow for Assessing Treatment effeCt Heterogeneity (WATCH) in clinical drug
development targeted at clinical trial sponsors. WATCH is designed to address the challenges of investigating
treatment effect heterogeneity (TEH) in randomized clinical trials, where sample size and multiplicity limit the
reliability of findings. The proposed workflow includes four steps: Analysis Planning, Initial Data Analysis and
Analysis Dataset Creation, TEH Exploration, and Multidisciplinary Assessment. The workflow offers a general
overview of how treatment effects vary by baseline covariates in the observed data, and guides interpretation of
the observed findings based on external evidence and best scientific understanding.