Causal decomposition of Black-white disparities in NIH's selection of proposals for panel discussion

Thomas M. Menino Convention & Exhibition Center 

Prior research on NIH R01 grants peer review found that selection of proposals for panel discussion constituted the decision point with the largest contribution to Black-white NIH funding disparities. Although selection into discussion is driven by Preliminary Overall Impact Scores, it is not fully deterministic. In this paper, we use causal analyses on preliminary peer review scores for R01 submissions to evaluate factors contributing to Black-white disparities in selection into discussion. We first use causal decomposition methodology to explore whether hypothetical interventions eliminating Black-white disparities and/or differences in attributes would reduce Black-white disparities in selection of proposals for panel discussion. The attributes we consider include those associated with applicants (degree, career stage, their institution's funding bin) and their submitted proposals (application type, amended status, Preliminary Overall Impact Score, and NIH assigned Administering Organization and Integrated Review Group). Under reasonable assumptions, we find that among the attributes considered, Preliminary Overall Impact Score was the only one that, after equalizing, could dissolve Black-white disparities in selection into discussion. In addition, we use causal decomposition to conduct a thought experiment to study the potential impacts of the NIH's move to binary scoring for the Investigator and Environment criteria in the new Simplified Peer Review Framework on Black-white disparities in selection into discussion. Analyses from our thought experiment suggest that changing to binary scoring of the Investigator-and-Environment factor may not have any effect on disparities in selection into discussion. Overall, under the assumptions of the causal decomposition framework, these results suggest that efforts aimed at reducing disparities in selection into discussion should focus on identifying upstream, policy-relevant, and practically feasible interventions for eliminating racial disparities in Preliminary Overall Impact Scores. In contrast, the transition to the Simplified Peer Review Framework is unlikely to substantially reduce disparities in selection into discussion.

Causal inference

Causal decomposition

Research grant review

Disparity analysis

Policy evaluation

Partial interference