Artemisia: Validation of a deep learning model for automatic breast density categorization

Abstract

Background: The aim of this study is to validate a deep learning model for the classification of breast density according to American College of Radiology’s breast density patterns. Methods: A convolutional neural network was developed with 10,229 digital screening mammogram images. Once the network was developed and tested, its performance was evaluated before a group of six professionals, the majority report and a commercial software application. We selected randomly 451 new mammographic images from different studies and patients. The categorization process by professionals was repeated in two stages. Results: The agreement between the convolutional neural network and the majority report was k=0.64 (95% CI: 0.58–0.69) in the first stage and k=0.57 (95% CI: 0.52–0.63) in the second stage. The agreement between the CNN and the commercial software application was k=0.54 (95% CI: 0.48–0.60). In both cases, we observed that the concordances of the CNN were within or above the range of professionals’ concordances values. Conclusions: Considering the internal reference standard (majority report) and the external reference standard (commercial software application), we can affirm the CNN achieved professional level performance.