Open-Source Risk-Based Quality Management (openRBQM) Framework for Clinical Trial Data Monitoring with AI/ML Extensions

Music City Center 

Background: Risk-Based Quality Management (RBQM) is an adaptive approach to clinical trial monitoring focused on identifying and mitigating risks that could impact patient safety and data quality. Our team developed an open-source RBQM framework that incorporates validated, modularized analytics, featuring Key Risk Indicators (KRIs), quality tolerance limit (QTL), and other statistical monitoring methods to detect clinical trial risks at patient, site, country and study levels. We released over 12 R packages (core + extensions) as the foundation of an internal RBQM analytics system that routinely detect and report clinical trial risk signals for assessment and timely action.

Methods: During a piloting phase with multiple clinical trials, the team created and adopted the new RBQM framework and analytics to identify risk signals from clinical data and track it alongside mitigation actions within a dedicated risk signal management system. The development of this framework included building an automated data pipeline, constructing sophisticated data models, deploying innovative analytic modules including AI/ML, creating dashboards and visualizations, as well as leveraging statistics and data science expertise with technical infrastructures (e.g., GitHub, R/Shiny/JavaScript, AWS Bedrock, Azure DevOps). In one of the extensions, the team implemented a machine learning (ML) module to predict risk signal actions based on historical data, utilizing features such as signal type, severity, study and site characteristics, and previous action patterns. Additionally, a generative AI component was integrated to automate risk signal descriptions and tailored actions suggested based on historical trends.

Results: Early detection of risk signals from clinical trial data with clear mitigation plans using our analytic framework allowed study teams for prompt corrective actions with efficient resources, preventing issues from escalating into major problems in data quality that could compromise the clinical trial's validity and patient's safety. The analytic output contains rich visualizations (interactive plots, data listing, statistical findings) with the ability to drill-down to the underlying data. The innovative process and analytics including AI/ML components have helped clinical trial teams identified and mitigated thousands of key risks at patient, site, country and study levels continuously.

Discussion: Our open-sourced framework with enhanced AI/ML capabilities are being piloted and implemented in ongoing studies and have been well-received by internal clinical trial teams and collaborators in the industry as a PHUSE initiative. The novel AI/ML approaches improved data monitoring efficiency and resource allocation. It provides a more proactive and data-driven framework in clinical trial monitoring and risk management decisions.

Risk-Based Quality Management (RBQM)