Statistical Analysis Plan (SAP) Automation with Generative AI

Music City Center 

A Statistical Analysis Plan (SAP) is typically written by statisticians and is based primarily on the study protocol, sometimes taking several weeks to finalize. We present our efforts creating tools harnessing generative AI to augment and automate SAP writing.

A key step in SAP writing involves parsing the protocol and mapping its information to the appropriate sections. This information may be copied, summarized, split or fused with additional sources. Protocols are ingested into a knowledge graph. Large Language Models (LLMs) play a role at various stages. A secure Pfizer API is used to query GPT-o1, while vector search and Retrieval-Augmented Generation (RAG) are used as well.

Early versions of our automation tools have been deployed for Non-Interventional, Clinical Pharmacology, and Interventional SAPs. Subject Matter Expert (SME) statisticians have played a pivotal role in development and evaluation of these tools. Our evaluation framework is focused on truthfulness, eloquence and reasoning, and consists of automated and human metrics. User feedback indicates that the automation has successfully increased efficiency.

Generative AI

Statistical Analysis Plan

Clinical Trials

Regulatory documents

Automation

Writing 

Biopharmaceutical Section