Using Natural Language Processing to Accurately Identify Aortic Stenosis in a Large, Integrated Healthcare Delivery System
2019 140:A12926 in Circulation
Matthew D Solomon, Grace Tabada, Amanda Allen, Sue Hee Sung, Alan S Go
Introduction: Administrative claims data are often used for population management and quality reporting, but diagnosis codes for conditions such as valvular heart disease can be inaccurate and vary across health systems. Echocardiography (echo) data contain detailed clinical information but are generally unstructured and not feasible to extract manually in large scale.
Methods: We developed and validated natural language processing (NLP) algorithms to identify aortic stenosis (AS) from echo reports in Kaiser Permanente Northern California (KPNC) and compared AS identification using NLP vs. administrative codes. Using NLP software (Linguamatics i2e), we initially developed algorithms to identify AS from a development set of >100 echo reports manually confirmed with AS, with iterative refinement using additional development sets (>100 echo reports each) until the NLP algorithm achieved positive and negative predictive values (PPV and NPV) of >95%. We then applied the NLP algorithm to all 2008-2018 echo reports (transthoracic, transesophageal or stress) in KPNC adults and compared results to ICD-9/10 diagnostic code-based definitions from 14 days before to 6 months after the echo date.
Results: The NLP algorithm was developed and refined among >500 echo reports to achieve >95% PPV and NPV. Application of NLP to 957,505 echo reports (N=522,653 patients with mean age 63.3 years, 51% women, 8.5% black, 13.5% Asian/Pacific Islander, 12.9% Hispanic and median [interquartile range] 1 [1 to 2] echoes per person) yielded 104,090 echoes (10.9%) with AS (N=53,791 patients). Among echoes identified by NLP as positive for AS, 36,070 (34.7%) had diagnosis codes for AS between 14 days prior and 6 months after echo. Among echo’s without AS via NLP, 12,626 (1.5%) had diagnosis codes for AS between 14 days before to 6 months after the echo.
Conclusions: An NLP algorithm applied to a large echo database was more accurate than using diagnosis codes for identifying AS and can facilitate more effective individual and population management than relying on administrative data alone. Future NLP development to characterize AS severity will further advance personalized and population-based care strategies for surveillance and treatment.