Large-scale annotation makes it easier to apply deep learning to mammography datasets
A team of researchers has developed a process for large-scale clinical data annotation that makes it easier to apply deep learning to mammography, according to a new study published in the Journal of Digital Imaging.
“Development of large, well-annotated datasets is hindered by several factors including lack of funding, prohibitive requirements in time and medical expertise, and privacy issues that complicate sharing,” wrote Hari M. Trivedi, MD, of the department of radiology and biomedical imaging at the University of California, San Francisco, and colleagues. “For this reason, development of these datasets has traditionally required manual efforts from a large team (such as a through a clinical trial). However, the sheer number of cases required for effective deep learning makes these types of manual methods unfeasible, if not impossible. We postulate that the burden of dataset construction can be significantly reduced by automating the structuring and annotation of existing routine clinical records.”
Most relevant clinical data is stored as free-text, Trivedi et al. explained. Extracting it into structured reports takes a significant amount of time and energy due to “misspellings, abbreviations, acronyms, poor grammatical structure, and variations in reporting styles,” among other challenges.
The authors began with a random sample of more than 3,000 free-text breast pathology reports smaller than 1,024 characters and more than 500 larger than 1,024 characters created from 1997 to 2014. They then built an “annotation pipeline” that included natural language processing (NLP) and IBM Watson’s Natural Language Classifier (NLC). NLP alone was used on reports larger than 1,024 characters due to a character limit associated with IBM Watson. The NLP steps included pre-processing, text mining and classification using machine learning.
Overall, the NLP and IBM Watson’s NLC were effective at annotating the data contained in reports smaller than 1,024 characters with weighted average F-measures above 0.96. For the longer reports, NLP alone had an F-measure of 0.83.
“This technique significantly accelerates the rate of extraction of meaningful data from clinical free-text reports and has important implications for improving the quantity and quality of large-scale datasets available for deep learning,” the authors wrote. “To our knowledge, no such application for processing of free-text pathology records has yet been described.”
Trivedi and colleagues noted that this is just the start of the team’s research.
“Future work will focus on expanding this process to other medical records such as radiology reports and clinical notes as well as testing other automated solutions from Facebook, Google, Amazon, and Microsoft,” the authors concluded. “We hope to design an automated pipeline for large-scale clinical data annotation so that existing clinical records can be efficiently utilized for development of deep learning algorithms.”
Michael has more than 18 years of experience as a professional writer and editor. He has written at length about cardiology, radiology, artificial intelligence and other key healthcare topics.