Unlocking Insights: Novel Statistical Approaches in Post-Market Safety Studies

Post-marketing surveillance refers to the process of monitoring the safety of drugs once they reach the market, after the successful completion of clinical trials. In this work, we investigate a computational approach using data mining tools to detect safety signals from post-market safety databases, or in other words, to identify adverse events (AEs) with disproportionately high reporting rates compared to other AEs, associated with a particular drug or a drug class. Essentially, we view this as a problem of cluster analysis-based anomaly detection on post-market safety data, where the goal is to 'unsupervisedly' detect the anomalous or the signal AEs. Our findings demonstrate the potential of using a clustering ensemble method to detect drug safety signals. It employs multiple clustering or anomaly detection algorithms, followed by a performance comparison of the candidate algorithms based on a collection of appropriate measures of goodness of clustering results. The method is general enough to include any number of clustering or anomaly detection algorithms and goodness measures, and performs better than any of the candidate algorithms in identifying the signal AEs. Extensive simulation studies illustrate that the ensemble method detects the AE signals from synthetic post-market safety datasets pretty accurately across the different scenarios explored. Based on the cases reported to the FDA Adverse Event Reporting System (FAERS) between 2013 and 2022, we further demonstrate that the ensemble method successfully identifies and confirms most of the adverse events that are known to occur most frequently in reaction to antiepileptic drugs and beta-lactam antibiotics.
 

The identification of a new adverse event (AE) caused by a drug product is one of the key activities in the pharmaceutical industry to ensure the safety profile of a drug product. Machine learning (ML) has the potential to assist with signal detection and supplement traditional pharmacovigilance (PV) surveillance methods. This pilot ML modeling study was designed to detect potential safety signals for two AbbVie products and test the model's capability of detecting safety signals earlier than humans. Drug X, a mature product with post-marketing data, and Drug Y, a recently approved drug in another therapeutic area, were selected. Gradient boosting-based ML approaches (e.g., XGBoost) were applied as the main modeling strategy. For Drug X, eight true signals were present in the test set. Among 12 potential new signals generated, four were true signals with a 50.0% sensitivity rate and a 33.3% positive predictive value (PPV) rate. Among the remaining eight potential new signals, one was confirmed as a signal and detected six months earlier than humans. For Drug Y, nine true signals were present in the test set. Among 13 potential new signals generated, five were true signals with a 55.6% sensitivity rate and a 38.5% PPV rate. Among the remaining eight potential new signals, none were confirmed as true signals upon human review. This model demonstrated acceptable accuracy for safety signal detection and potential for earlier detection when compared to humans. Expert judgment, flexibility, and critical thinking are essential human skills required for the final, accurate assessment of adverse event cases. 

Monitoring the safety of drugs in real world settings following market approval is a crucial aspect of pharmacovigilance. It allows regulatory authorities and healthcare providers to swiftly address any emerging safety concerns. This process assesses whether the therapeutic benefits of a drug continue to outweigh any potential risks when used by the general population, a proactive approach aiming to ensure that patients receive treatments that are both effective and safe.

The analysis of a post-marketing data can be challenging due to the complexity, size, and variability inherent in real-world data. Various methods have been proposed to tackle these challenges, one of which is the tree-based scan statistic, a data mining method looks for excess risk by simultaneously evaluating large set of adverse events, as well as groups of adverse events, adjusting for the multiple testing inherent in the large number of overlapping groups evaluated.

Other interesting methods are sequential probability ratio test or the sequential FDR, which account for temporal nature of adverse events, and allow continuous data analysis as it accumulates while controlling false discovery rate.

In our talk, we will cover the fundamental principles of post-marketing monitoring, Tree Scan method and sequential probability testing in biopharmaceutical research. Practical application and examples will be provided.
 

Post-marketing safety assessment is a crucial aspect of drug development, aiming to ensure the continued safety and efficacy of pharmaceuticals across diverse populations. The International Council for Harmonization (ICH) , such as those in the E17 and M14, have provided essential guidance and frameworks to harmonize the drug development and post-marketing pharmacovigilance.

In this presentation, we examine various design considerations for such post-marketing safety studies. We will also evaluate statistical approaches in assessing drug safety , including aggregate and meta-analytical safety evaluation and consistency assessment across countries/databases. Moreover, the application of artificial intelligence (AI) methods may further enhances the capabilities of post-marketing safety assessments. Techniques such as prospective sequential safety monitoring , Bayesian methodology, or Retrieval-Augmented Generation (RAG), allow for the synthesis of diverse data sources. Together, these innovative designs and analytical techniques can advance the multi-regional post-marketing safety evaluation.

A case study on post-marketing risk assessment will be used to illustrate.