New unified insights on deep learning in radiological and pathological images: Beyond quantitative performances to qualitative interpretation

Abstract

Abstract Deep learning (DL) has become the main focus of research in the field of artificial intelligence, despite its lack of explainability and interpretability. DL mainly involves automated feature extraction using deep neural networks (DNNs) that can classify radiological and pathological images. Convolutional neural network (CNNs) can be also applied to pathological image analysis, such as the detection of tumors and the quantification of cellular features. However, to our knowledge, no attempts have been made to identify interpretable signatures from CNN features, and few studies have examined the use of CNNs for cytopathology images. Therefore, the aim of the present paper is to provide new unified insights to aid the development of more interpretable CNN-based methods to classify radiological and pathological images and explain the reason for this classification in the form of if-then rules. We first describe the “black box” problem of shallow NNs, the concept of rule extraction, the renewed attack of the “black box” problem in DNN architectures, and the paradigm shift regarding the transparency of DL using rule extraction. Next, we review limitations of DL in pathology in regard to histopathology and cytopathology. We then investigate the discrimination of cytological features and explanations and review recent techniques for interpretable CNN-based methods in histopathology, as well as current approaches being taken to enhance the interpretability of CNN-based methods for radiological images. Finally, we provide new unified insights to extract qualitative interpretable rules for radiological and pathological images.