Convolutional neural network misclassification analysis in oral lesions: an error evaluation criterion by image characteristics

Abstract

Background: This retrospective study proposed a analysis of the errors generated by a convolutional neural network (CNN), when performing the automated classification of oral lesions, according to their clinical characteristics, seeking to identify paterns in systemic errors in the intermediate layers of CNN. Methodology: This is a cross-sectional study nested in a previous trial. A CNN performed the automated classification of elementary lesions from clinical images of oral lesions. CNN classification errors formed the dataset for this study. A total, 116 real outputs were identified that diverged from the estimated outputs, representing 7.6% of the total images analyzed by the CNN. Results: The discrepancies between the real output and the estimated output, were associated with problems of sharpness, resolution, focus, human errors and influence of the data augmentation. Conclusion: From the qualitative analysis of failures in the process of automated classification of clinical images, it was possible to confirm the influence of image quality, however, we identified a great influence of the data augmentation process. Knowledge of the evidence that leads models to make their decisions can increase our confidence in their high classification potential.