Harnessing Large Language Models in Extracting Longitudinal Smoking History from Unstructured Clinical Notes in Electronic Health Records for Improved Cancer Surveillance

Music City Center 

Accurate smoking documentation in electronic health records is essential for effective risk assessment, screening, prevention, and patient monitoring. However, key smoking information is often absent or inaccurately recorded in structured data, contributing to inconsistencies in longitudinal data arising from recall bias and reporting errors. Large language models (LLMs) offer a promising solution in interpreting clinical text narratives. We developed a framework that utilizes LLMs to extract and harmonize longitudinal smoking histories, incorporating our rule-based smoothing techniques. These techniques aim to improve the quality of post-deployment smoking data by addressing conflicts and inconsistencies in key variables through trend analysis and back calculation. We compared BERT-based models against generative AI models (Gemini 1.5 Flash, PaLM 2 Text-Bison, GPT-4) using a dataset of 1,683 manually-annotated clinical notes from 500 patients across academic and community healthcare systems and deployed them on 80,037 notes from 4988 patients to extract 7 smoking variables, including status, pack-years, duration, and cessation. We assess the clinical utility of the curated longitudinal smoking data in evaluating the effectiveness of different post-treatment cancer surveillance strategies for detecting second malignancies, where smoking is a key critical prognostic factor.