PS6F - Considerations in developing and using open-source statistical software in Pharma

Conference: ASA Biopharmaceutical Section Regulatory-Industry Statistics Workshop 2023
09/29/2023: 1:30 PM - 2:45 PM EDT
Room: White Flint Amphitheather 


In recent years, statisticians and analysts from both industry and regulatory agencies have increased adoption of open-source software such as R. While open-source ecosystems provide unprecedented opportunities for collaboration and innovation, users face huge variability in software quality, in particular reliability, efficiency and maintainability. Developing high quality software with good coding practices, reproducible outputs, and self-sufficient documentation is critical to inform clinical and regulatory decisions. Equally important is the ability to evaluate an open-source tool's quality as a statistician or an analyst.

In this session, we invited and confirmed one industry speaker and one FDA speaker, plus one panelist from industry and one panelist from FDA CDER DBVII. The industry speaker, on behalf of the ASA Biopharmaceutical section Software Engineering working group, will discuss the practice of building high quality R packages for novel statistical methods in an open-source and cross-company team. Following that, the FDA speaker from Division of Analytics and Informatics will share their experience in using R for regulatory review and their considerations in choosing trusted open-source packages. In the panel discussion, the panelists will share their thoughts on how we embrace software and methodology innovation while ensuring quality.


Open source

Statistical software

Software engineering

High-quality R packages

Cross-company collaboration

Regulatory considerations 


Ya Wang, Gilead Sciences


Ning Leng, Genentech

Topic Description

Clinical Trial Conduct and Analysis Tools (e.g., Monitoring, Operations, Visualization)
ASA Biopharmaceutical Section Regulatory-Industry Statistics Workshop 2023


First year of the Software Engineering working group - working together across organizations

The Software Engineering (SWE) Working Group (WG) was formed in August 2022 in the American Statistical Association (ASA) Biopharmaceutical Section (BIOP). The SWE WG facilitates cross-organizational collaboration with regular meetings, and currently includes more than 35 members from over 25 organizations. While most are from the pharmaceutical industry, the working group is open for academic and regulatory members too.
The primary goal of the SWE WG is to engineer R packages that implement important statistical methods to fill in gaps in the open-source statistical software landscape, focusing on the needs of biopharmaceutical applications. The first R package "mmrm" is setting a new standard for fitting mixed models for repeated measures (MMRM) in R.
The secondary goal is to develop and disseminate best practices for engineering high-quality open-source statistical software. The video series "Statistical Software Engineering 101" is introducing specific best practices in an accessible format. Furthermore the workshop "Good Software Engineering Practice for R Packages" has been successfully taught in person at a Basel Biometric Society seminar as well as a China R User Group workshop, and the materials are available publicly to train statisticians on best practices.
Communication is key, and the SWE WG was introduced in a BIOP report and maintains a website including a blog at
The SWE WG plans to develop additional new R packages, covering critical and innovative methodology topics in the health-technology assessment (HTA) space, covariate adjustment and Bayesian inference for MMRMs.
We describe the journey of the SWE WG so far and in particular the ingredients for working together successfully, including mutual interest, getting to know each other, and creating mutual trust. 

Presenting Author

Daniel Sabanes Bove, Roche

Open-source software for regulatory submissions and regulatory environments

Regulatory submissions and regulatory computing environments have traditionally been associated with the use of proprietary software packages. While academic institutions have embraced open-source software, both industry and government have been slower to adopt open-source alternatives. I will discuss some of the challenges and issues with using open-source software in a regulatory environment, followed by some of the lessons learned in the ongoing R Consortium R Submission Pilot, as well as emerging issues. 

Presenting Author

Paul Schuette, FDA/CDER

Panel Discussion

Following the 2 talks, a panel comprised of experts from FDA and pharmaceutical industry will share their thoughts on how we embrace software and methodology innovation while ensuring quality. The panel will also address questions from the audience. 

Presenting Author(s)

Daniel Sabanes Bove, Roche
Eric Nantz, Eli Lilly and Company
Paul Schuette, FDA/CDER
Tae Hyun Jung, FDA