CS008 - Contributed: Experimental and non-experimental tools to inform healthcare policymaking

Enormous progress has been made in reducing child mortality and disability over the last two decades in low- and middle-income countries. Childhood vaccinations have played an important part in this success story. They represent one of the most cost-effective health technologies, in that they prevent mortality and disability at relatively low cost. Despite the well-documented evidence and consistent investment in national immunisation programmes, WHO estimates that globally 23 million infants were not fully vaccinated in 2020. In Uttar Pradesh, the most populous and poorest state in India and the setting for this study, recent estimates show that 70% of children aged 12 to 23 months are fully vaccinated. While this represents a marked improvement over the past five years, it is clear widespread availability of free immunisation services in public facilities is not sufficient to guarantee high coverage in the population. An experimental evaluation of a door-to-door information intervention to educate mothers on the benefits of child vaccination, reported large positive effects on vaccine uptake in the short-term. The intervention provided the mothers of unvaccinated or incompletely vaccinated children aged 0 to 36 months with health information on the benefits of vaccination. We extend this evaluation, using 23 months follow-up data for mothers of children included in the original experiment, to address two outstanding policy questions.

First, are the effects of the intervention sustained over time? It may be the case that the initial effects of the intervention are attenuated over time, if the intervention merely brings forward vaccinations that would have happened anyway. Under such a scenario, any health benefits of vaccination would be temporary, undermining initial estimates of the cost-effectiveness of the intervention. We also consider whether the intervention led to a sustained change in parental behaviour, be examining effects on the vaccination of particpants' children who were not born at the time of intervention. Such persistence of effects would increase the cost-effectiveness of an intervention.

Second, who benefits from the intervention? Information on heterogenous treatment effects has various uses. It can inform policymakers as to who should be targeted by the intervention to maximise take up of immunisation. It can speak to the question of equity and can potentially shed light on the mechanisms through which the intervention worked. Finally, it can offer policymakers insights on what other interventions may be needed in tandem with demand-side strategies.

The main outcomes were the proportions of children who received (a) diphtheria–pertussis–tetanus (DPT3), or (b) measles vaccinations. We reported whether the overall effects of the initial study were sustained over 23 months, and heterogeneity of the individual-level effects. Attrition was low; 93% of the 722 mother-child dyads who were randomised completed follow-up at 23 months, and baseline characteristics amongst this subsample remained well balanced between the randomised groups. We estimated effects with Causal Forests' (CF), an ensemble Machine Learning methodt hat can predict HTEs according to observable characteristics by searching over high-dimensional functions of covariates rather than requiring the a priori specification of outcome models.

We find that the large short-term effects of the information intervention were sustained at 23 months, with considerable heterogeneity in the individual level-effect estimates. While we detect positive effects for siblings these are not statistically significant, perhaps reflecting reduced statistical power as the analysis only includes 293 siblings. Thus we cannot be confident that the intervention led to lasting behavioural change. This evidence provides important context for the original analysis and can help target educational initiatives to improve vaccine uptake in low-income settings. 

Medical researchers have understood for many years that treatment effect estimates obtained from a randomized clinical trial (RCT) -- termed ``efficacy'' -- can differ from those obtained in a general population -- termed ``effectiveness''. Only in the past decade has extensive work begun in the statistical literature to bridge this gap using formal quantitative methods. As noted by Rod Little in a letter to the editor in the New Yorker ``...randomization in randomized clinical trials concerns the allocation of the treatment, not the selection of individuals for the study. The latter can have an important impact on the average size of a treatment effect,'' with RCT samples often designed, sometimes explicitly, to be more likely to include individuals for whom the treatment may be more effective.

This issue has been various termed ``generalizability'' or ``transportability." Why do we care about transportability? In RCTs we are in the happy situation were treatment assignment is randomized, so confounding due to either observed or unobserved (pre-treatment) covariates is not an issue. But while randomization of treatment eliminates the effect of unobserved confounders, at least net of non-compliance, it does not eliminate the effect of unobserved effect modifiers, which can impact the causal effect of treatment in a population that differs from the RCT sample population. The impact of these interactions on the marginal effect of treatment thus can differ between the RCT population and the final population of interest.

Concurrent with research into transportability has been research into making population inference from non-probability samples. There is a close overlap between these two approaches, particularly with respect to the non-probability inference methods that rely on information from a relevant probability sample of the target population to reduce selection bias effects. When there are relevant censuses or probability samples of the target patient population of interest, these methods can be adapted to transport information from the RCT to the patient population. Because the RCT setting focuses on causal inference, this adaptation involves extensions to estimate counterfactuals. Thus approaches that treat population inference as a missing data problem are a natural fit to connect these two strands of methodological innovation.

In particular, we propose to extend a ``pseudo-weighting'' methodology from other non-probability settings to a ``doubly robust'' estimator that treats sampling probabilities or weights as regression covariates to achieve consistent estimation of population quantities. We explore our proposed approach and compare with some standard existing methods in a simulation study to assess the effectiveness of the approach under differing degrees of selection bias and model misspecification, and compare it with results obtained using the RT data only and with existing methods that use inverse probability weights. We apply it to a study of pulmonary artery catheterization in critically ill patients where we believe differences between the trial sample and the larger population might impact overall estimates of treatment effects. 

Randomized clinical trials (RCTs) are the preferred study design for assessing the causal effects of medical interventions on healthcare policymaking. Real-world RCTs evaluating multifaceted interventions often employ multiple study endpoints to measure treatment success on a small set of underlying constructs. We propose a latent variable model with principal strata of latent compliance types for parsimonious estimation of intervention effects in RCTs with multidimensional longitudinal outcomes and treatment noncompliance. Within each compliance type, a factor regression model is used to relate observed multiple endpoints to latent constructs, which are then modelled by hierarchical mixed-effects regression models. Under this model, high dimensional outcomes are reduced to low dimensional latent factors. This dimension reduction leads to a more parsimonious and efficient test of overall complier average causal effects (CACE) on multiple endpoints, mitigating the potential multiple testing issues associated with multiple endpoints. Furthermore, the inference based on factors can be more interpretable and scientifically relevant. We evaluate the performance of the proposed model using simulation studies, which shows study power can be increased substantially compared with estimating CACE for each endpoint separately. The proposed approach is illustrated by evaluating the treatment efficacy of the Arthritis Health Journal online tool. We evaluated the treatment efficacy of Arthritis Health Journal under one latent variable and two latent variables separately. Significant and beneficial treatment effects on latent variables are detected in both two situations. 

More than 90,000 people are waiting for a kidney transplant in the United States. For those who need both a kidney and another organ, a difficult decision is whether to seek out a living kidney donor or wait to receive both organs from one deceased donor, which can have immunological benefits. The framework of dynamic treatment regimes (DTRs) can inform this choice, but a patient-relevant strategy such as "wait for deceased-donor transplant" is ill-defined because there are multiple versions of treatment (i.e., multiple possible wait times and organ qualities). Existing DTR methods implicitly average over the distribution of versions of treatment in the study data, estimating survival under a so-called "representative intervention." This is undesirable when transporting inferences to a target population such as patients today, who experience shorter wait times thanks to evolutions in allocation policy. We, therefore, propose the concept of a generalized representative intervention (GRI): a random DTR that assigns treatment version by drawing from the distribution of treatments received by strategy compliers in the target population (e.g., patients who joined the waitlist under current allocation policy). We propose a class of inverse-probability-weighted product-limit estimators of survival under a GRI and a robust variance estimator, which perform well in simulations. For ease of estimation with continuous aspects of treatment (e.g., organ quality), weights are reformulated using Bayes' theorem to depend on probabilities only, not densities. We apply our method to a national database of adult kidney-pancreas transplant candidates from 2001-2020 to illustrate that variability in deceased-donor transplant rate and organ quality across years and centers results in qualitative differences in the optimal strategy for patient survival. Specifically, when transplant rate and organ quality are set to their observed distributions under today's allocation policies, survival for the average kidney-pancreas candidate is comparable whether the patient waits for deceased-donor kidney-pancreas transplant or pursues a near-immediate living-donor kidney transplant. This finding implies many potential living donors could be spared the risk of surgery in favor of performing more deceased-donor transplants.

Relevance: Despite a growing literature on statistical decision-making given resource constraints, almost all of the published methods have focused on how to distribute limited resources (as an institution) rather than how to act (as an individual) given the limited resources and distribution policies that are in place. Furthermore, solid-organ transplant research has historically emphasized measures that do not account for waitlist uncertainty and therefore have limited usefulness in patient decision-making, such as post-transplant survival. Our work fills a gap in both fields. We introduce a class of patient-centric estimators that can accommodate evolving organ allocation policies without limiting sample size or follow-up for survival outcomes. The rules for multi-organ allocation in particular have undergone major changes in recent years, and more changes are expected soon with the introduction of heart-kidney and lung-kidney policies, so it is essential to create statistical tools now that can help patients navigate a changing allocation system and make informed decisions about their care.

Background: I am a recently graduated PhD biostatistician now working on the Scientific Registry of Transplant Recipients. In addition to simulating potential allocation policies and evaluating transplant center performance in the U.S., we create patient-facing decision tools to inform treatment planning given policy and resource constraints.

I would be open to presenting a contributed session or a poster. 

Causal inference for environmental health data is often challenging due to the presence of interference: outcomes for observational units depend on some combination of local and nonlocal treatment (Zigler and Papadogeorgou 2021, Zigler et al. 2020, Reich et al. 2021). This is especially relevant when analyzing the effectiveness of air quality interventions at pollution sources (such as coal-fired power plants) on human health, as air pollution exposure is affected by upwind pollution sources, regional differences in demographics, and meteorologic processes. Consequently, the analysis and design of regulatory policies intended to improve public health stands to benefit from causal methods which account for complex sources of treatment interference. In recent years, methods for causal inference with general interference have included the specification of an exposure model, in which treatment assignments are mapped to an exposure value (Aronow and Samii 2017, Karwa and Airoldi 2018, Forastiere et al. 2021); causal estimands of the direct and indirect (i.e., local and spillover) effects of treatment are defined through contrasts of the local treatment assignment and the exposure value. Notably, the exposure model is often defined via a network structure, which is assumed to be fixed and known a priori (Aronow and Samii 2017, Forastiere et al. 2021). However, in environmental settings, treatment interference is often dictated by complex, mechanistic processes that are both stochastic and poorly represented by a network. In this work, we develop methods for causal inference with interference when deterministic exposure models cannot be assumed or are unknown. We offer a Bayesian model for the interference mapping and marginalize estimates of causal effects over uncertainty in the structure of interference. To illustrate the usefulness of our methodology, we analyze the effectiveness of air quality interventions at coal-fired power plants on the prevalence of asthma-related emergency department (ED) visits in Texas. In particular, treatment assignments are mapped to exposure levels via a mechanistic model of air pollution transport (Wikle et al. 2020), and causal estimands are defined to accommodate the corresponding uncertainty in the estimated exposure. We use our results to identify individual upwind power plants that should be targeted for future regulatory intervention, and discuss the relevance of this work to the study of environmental health data at large. This research