Pathological Image Classification Research Articles

Richer fusion network for breast cancer classification based on multimodal data

BackgroundDeep learning algorithms significantly improve the accuracy of pathological image classification, but the accuracy of breast cancer classification using only single-mode pathological images still cannot meet the needs of clinical practice. Inspired by the real scenario of pathologists reading pathological images for diagnosis, we integrate pathological images and structured data extracted from clinical electronic medical record (EMR) to further improve the accuracy of breast cancer classification.MethodsIn this paper, we propose a new richer fusion network for the classification of benign and malignant breast cancer based on multimodal data. To make pathological image can be integrated more sufficient with structured EMR data, we proposed a method to extract richer multilevel feature representation of the pathological image from multiple convolutional layers. Meanwhile, to minimize the information loss for each modality before data fusion, we use the denoising autoencoder as a way to increase the low-dimensional structured EMR data to high-dimensional, instead of reducing the high-dimensional image data to low-dimensional before data fusion. In addition, denoising autoencoder naturally generalizes our method to make the accurate prediction with partially missing structured EMR data.ResultsThe experimental results show that the proposed method is superior to the most advanced method in terms of the average classification accuracy (92.9%). In addition, we have released a dataset containing structured data from 185 patients that were extracted from EMR and 3764 paired pathological images of breast cancer, which can be publicly downloaded from http://ear.ict.ac.cn/?page_id=1663.ConclusionsWe utilized a new richer fusion network to integrate highly heterogeneous data to leverage the structured EMR data to improve the accuracy of pathological image classification. Therefore, the application of automatic breast cancer classification algorithms in clinical practice becomes possible. Due to the generality of the proposed fusion method, it can be straightforwardly extended to the fusion of other structured data and unstructured data.

Research on the classification of lymphoma pathological images based on deep residual neural network.

BACKGROUND: Malignant lymphoma is a type of tumor that originated from the lymphohematopoietic system, with complex etiology, diverse pathological morphology, and classification. It takes a lot of time and energy for doctors to accurately determine the type of lymphoma by observing pathological images.OBJECTIVE: At present, an automatic classification technology is urgently needed to assist doctors in analyzing the type of lymphoma.METHODS: In this paper, by comparing the training results of the BP neural network and BP neural network optimized by genetic algorithm (GA-BP), adopts a deep residual neural network model (ResNet-50), with 374 lymphoma pathology images as the experimental data set. After preprocessing the dataset by image flipping, color transformation, and other data enhancement methods, the data set is input into the ResNet-50 network model, and finally classified by the softmax layer.RESULTS: The training results showed that the classification accuracy was 98.63%. By comparing the classification effect of GA-BP and BP neural network, the accuracy of the network model proposed in this paper is improved.CONCLUSIONS: The network model can provide an objective basis for doctors to diagnose lymphoma types.

The Clinical Value of CT Scans for the Early Diagnosis and Treatment of Spinal Fractures and Paraplegia

The early diagnosis and treatment of spinal fractures and paraplegia by CT scan is investigated in depth and its clinical value is discussed in this paper. In this paper, a novel circulatory generation adversarial network, Spine-GAN, is proposed for the diagnosis of various spinal diseases. The algorithmic model can fully automate the segmentation and classification of multiple spinal structures, such as intervertebral discs, vertebrae, and neuroforamina, simultaneously to intelligently generate a complete clinical diagnosis. The innovation of this method is that Spine-GAN not only overcomes the high variability and complexity of spinal structures in MRI images but also preserves the subtle differences between normal and abnormal spinal structures and dynamically learns obscure but important spatial pathological relationships between adjacent structures of the spine, thus overcoming the limitations of small datasets. Spine-GAN enables accurate segmentation, radiological classification, and pathological correlation representation of the three spinal diseases. Specifically, Spine-GAN achieves a pixel accuracy of 96.2% with a specificity and sensitivity distribution of 89.1% and 86%, respectively. The DMML-Net and Spine-GAN algorithm models have important applications and research values in the clinical diagnosis of spinal diseases and MRI image processing, as well as in the intelligent generation of medical image diagnostic reports, which are of great importance for the study of fine-grained image classification of pathological images. It also has a positive impact on the development of the software.

Self-Path: Self-Supervision for Classification of Pathology Images With Limited Annotations.

While high-resolution pathology images lend themselves well to 'data hungry' deep learning algorithms, obtaining exhaustive annotations on these images for learning is a major challenge. In this article, we propose a self-supervised convolutional neural network (CNN) framework to leverage unlabeled data for learning generalizable and domain invariant representations in pathology images. Our proposed framework, termed as Self-Path, employs multi-task learning where the main task is tissue classification and pretext tasks are a variety of self-supervised tasks with labels inherent to the input images. We introduce novel pathology-specific self-supervision tasks that leverage contextual, multi-resolution and semantic features in pathology images for semi-supervised learning and domain adaptation. We investigate the effectiveness of Self-Path on 3 different pathology datasets. Our results show that Self-Path with the pathology-specific pretext tasks achieves state-of-the-art performance for semi-supervised learning when small amounts of labeled data are available. Further, we show that Self-Path improves domain adaptation for histopathology image classification when there is no labeled data available for the target domain. This approach can potentially be employed for other applications in computational pathology, where annotation budget is often limited or large amount of unlabeled image data is available.

Research on Application of Classification Model Based on Stack Generalization in Staging of Cervical Tissue Pathological Images

Cervical cancer is a malignant tumor that threatens women's health and life. Cervical pathology examination, as the gold standard for cervical cancer diagnosis, provides an important basis for the surgical plan and postoperative follow-up strategy for cervical cancer. Cervical biopsy diagnosis includes normal, low-grade squamous intraepithelial lesion (LSIL), high-grade squamous intraepithelial lesion (HSIL) and squamous cell carcinoma (SCC). At present, cervical pathology examination still relies on the doctor's personal clinical experience and subjective judgment, which is time-consuming and may cause misdiagnosis or missed diagnosis. In addition, the current intelligent classification of cervical pathological images still has disadvantages such as imperfect classification system and low classification accuracy. Therefore, this experiment uses the ResNet50 model of the convolutional neural network as the feature extractor, and selects the K-Nearest Neighbour (KNN), Random Forest (RF), Support Vector Machine (SVM) classifiers in Machine Learning to perform cervical tissue pathological images Discrimination, the accuracy of the classification results were 85.83%, 80.33%, and 86.67%. In order to further improve the accuracy of the model and enhance the applicability and stability of the model, this experiment proposes the Stacked Generalization (SK) classification model. The first-layer base learner of the SK classification model selects CNN-KNN, CNN-RF, CNN-SVM, and the second-layer classifier selects Multilayer Perceptron (MLP). Among them, MLP makes the final result by learning the classification performance of the base learner for label discrimination, the accuracy of the classification model after ensemble learning is 90.00%. In addition, this experiment uses the Synthetic Minority Oversampling Technique (SMOTE) algorithm to amplify the training samples, and the amplified data set has a classification accuracy of 89.17% under the training of the SK classification model. The results show that the SK classification model in this experiment has a high classification ability for cervical histopathological images, and has good generalization ability and robustness.

Artificial intelligence in automatic classification of invasive ductal carcinoma breast cancer in digital pathology images.

Background: Breast cancer is one of the most causes of death in women. Early diagnosis and detection of Invasive Ductal Carcinoma (IDC) is an important key for the treatment of IDC. Computer-aided approaches have great potential to improve diagnosis accuracy. In this paper, we proposed a deep learning-based method for the automatic classification of IDC in whole slide images (WSI) of breast cancer. Furthermore, different types of deep neural networks training such as training from scratch and transfer learning to classify IDC were evaluated. Methods: In total, 277524 image patches with 50×50-pixel size form original images were used for model training. In the first method, we train a simple convolutional neural network (named it baseline model) on these images. In the second approach, we used the pre-trained VGG-16 CNN model via feature extraction and fine-tuning for the classification of breast pathology images. Results: Our baseline model achieved a better result for the automatic classification of IDC in terms of F-measure and accuracy (83%, 85%) in comparison with original paper on this data set and achieved a comparable result with a new study that introduced accepted-rejected pooling layer. Also, transfer learning via feature extraction yielded better results (81%, 81%) in comparison with handcrafted features. Furthermore, transfer learning via feature extraction yielded better classification results in comparison with the baseline model. Conclusion: The experimental results demonstrate that using deep learning approaches yielded better results in comparison with handcrafted features. Also, using transfer learning in histopathology image analysis yielded significant results in comparison with training from scratch in much less time.

Reducing annotation effort in digital pathology: A Co-Representation learning framework for classification tasks.

Classification of digital pathology images is imperative in cancer diagnosis and prognosis. Recent advancements in deep learning and computer vision have greatly benefited the pathology workflow by developing automated solutions for classification tasks. However, the cost and time for acquiring high quality task-specific large annotated training data are subject to intra- and inter-observer variability, thus challenging the adoption of such tools. To address these challenges, we propose a classification framework via co-representation learning to maximize the learning capability of deep neural networks while using a reduced amount of training data. The framework captures the class-label information and the local spatial distribution information by jointly optimizing a categorical cross-entropy objective and a deep metric learning objective respectively. A deep metric learning objective is incorporated to enhance the classification, especially in the low training data regime. Further, a neighborhood-aware multiple similarity sampling strategy, and a soft-multi-pair objective that optimizes interactions between multiple informative sample pairs, is proposed to accelerate deep metric learning. We evaluate the proposed framework on five benchmark datasets from three digital pathology tasks, i.e., nuclei classification, mitosis detection, and tissue type classification. For all the datasets, our framework achieves state-of-the-art performance when using approximately only 50% of the training data. On using complete training data, the proposed framework outperforms the state-of-the-art on all the five datasets.

Synthesis of diagnostic quality cancer pathology images by generative adversarial networks.

Deep learning-based computer vision methods have recently made remarkable breakthroughs in the analysis and classification of cancer pathology images. However, there has been relatively little investigation of the utility of deep neural networks to synthesize medical images. In this study, we evaluated the efficacy of generative adversarial networks to synthesize high-resolution pathology images of 10 histological types of cancer, including five cancer types from The Cancer Genome Atlas and the five major histological subtypes of ovarian carcinoma. The quality of these images was assessed using a comprehensive survey of board-certified pathologists (n = 9) and pathology trainees (n = 6). Our results show that the real and synthetic images are classified by histotype with comparable accuracies and the synthetic images are visually indistinguishable from real images. Furthermore, we trained deep convolutional neural networks to diagnose the different cancer types and determined that the synthetic images perform as well as additional real images when used to supplement a small training set. These findings have important applications in proficiency testing of medical practitioners and quality assurance in clinical laboratories. Furthermore, training of computer-aided diagnostic systems can benefit from synthetic images where labeled datasets are limited (e.g. rare cancers). We have created a publicly available website where clinicians and researchers can attempt questions from the image survey (http://gan.aimlab.ca/). © 2020 Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

Breast cancer pathological image classification based on deep learning.

The automatic classification of breast cancer pathological images has important clinical application value. However, to develop the classification algorithm using the artificially extracted image features faces several challenges including the requirement of professional domain knowledge to extract and compute highiquality image features, which are often time-consuming, laborious, and difficult. For overcoming these challenges, this study developed and applied an improved deep convolutional neural network model to perform automatic classification of breast cancer using pathological images. Specifically, in this study, data enhancement and migration learning methods are used to effectively avoid the overfitting problems with deep learning models when they are limited by training image sample size. Experimental results show that a 91% recognition rate or accuracy when applying this improved deep learning model to a publicly available dataset of BreaKHis. Comparing with other previously used models, the new model yields good robustness and generalization.

Classification of digital pathological images of non-Hodgkin's lymphoma subtypes based on the fusion of transfer learning and principal component analysis.

Non-Hodgkin's lymphoma (NHL) is a serious malignant disease. Delayed diagnosis will cause anemia, increased intracranial pressure, organ failure, and even lead to death. The current main trend in this area is to use deep learning (DL) for disease diagnosis. Extracting classification information from the digital pathology images by DL may realize the automated qualitative and quantitative analysis of NHL. Previously, DL has been used to classify NHL digital pathology images with some success. However, shortcomings still exist in the data preprocessing methods and feature extraction. Therefore, this paper presents a method for the classification of NHL subtypes based on the fusion of transfer learning (TL) and principal component analysis (PCA). First, the NHL digital pathology images were preprocessed by image division and segmentation and then input into the transfer models for fine-tuning and feature extraction. Second, PCA was used to map the extracted features. Finally, a neural network was used as a classifier to classify the mapped features. During the fine-tuning of the transfer models, two methods, freezing all feature extraction layers and fine-tuning all layers, were employed to select the optimal model with the best classification result among all the preselected transfer models. On this basis, the use of freezing the layers' location was discussed and analyzed. The results show that the proposed method achieved average fivefold cross-validation accuracies of 100%, 99.73%, and 99.20% for chronic lymphocytic leukemia (CLL), follicular lymphoma (FL), and mantle cell lymphoma (MCL) tumor, and each category has standard deviations 0.00, 0.53, and 0.65, respectively, in the NHL reference dataset. The overall classification accuracy for fivefold cross-validation is 98.93%, which is an increase of 1.26% compared to the latest reported methods, having a lower standard deviation (1.00). The method proposed in this paper achieves a high classification accuracy and strong model generalization for the classification of NHL, which makes it possible to conduct intelligent classification of NHL in clinical practice. Our proposed method has definite clinical value and research significance.

Automated Diagnosis of Pathological Brain Using Fast Curvelet Entropy Features

Automated diagnosis of pathological brain not only reduces the diagnostic error significantly but also improves the patient's quality of life, thereby addressing the sustainability issues. The last few decades have witnessed an intensive research on binary classification of brain magnetic resonance (MR) images. Multiclass classification of pathological brain MR images is a more challenging task and the literature on this problem is still in its infancy. In this paper, we propose a new automated diagnosis system to classify the brain MR images into five different categories. Texture features within MR images play a significant role in accurate and efficient pathological brain detection. This work presents the extraction of such vital texture features by calculating the entropy over the curvelet subbands. Two faster and simpler strategies of fast curvelet transform are separately employed for feature extraction and the derived features are termed as FCEntF-I and FCEntF-II. The features are finally subjected to kernel extreme learning machine (K-ELM) for classification. The effectiveness of the proposed scheme is evaluated on multiclass as well as binary brain MR datasets. Comparisons with state-of-the-art methods indicate the superiority of the proposed scheme. The discriminatory potential of FCEntF-I and FCEntF-II features is found better than its counterparts.

Deep convolutional neural networks with transfer learning for automated brain image classification

MR brain image categorization has been an active research domain from the last decade. Several techniques have been devised in the past for MR image categorization, starting from classical to the deep learning methods like convolutional neural networks (CNNs). Classical machine learning methods need handcrafted features to perform classification. The CNNs, on the other hand, perform classification by extracting image features directly from raw images via tuning the parameters of the convolutional and pooling layer. The features extracted by CNN strongly depend on the size of the training dataset. If the training dataset is small, CNN tends to overfit after several epochs. So, deep CNNs (DCNNs) with transfer learning have evolved. The prime objective of the present work is to explore the capability of different pre-trained DCNN models with transfer learning for pathological brain image classification. Various pre-trained DCNNs, namely Alexnet, Resnet50, GoogLeNet, VGG-16, Resnet101, VGG-19, Inceptionv3, and InceptionResNetV2, were used in the present study. The last few layers of these models were replaced to accommodate new image categories for our application. These models were extensively evaluated on data from Harvard, clinical, and benchmark Figshare repository. The dataset was then partitioned in the ratio 60:40 for training and testing. The validation on the test set reveals that the pre-trained Alexnet with transfer learning exhibited the best performance in less time compared to other proposed models. The proposed method is more generic as it does not need any handcrafted features and can achieve an accuracy value of 100%, 94%, and 95.92% for three datasets. Other performance measures used in the study include sensitivity, specificity, precision, false positive rate, error, F-score, Mathew correlation coefficient, and area under the curve. The results are compared with both the traditional machine learning methods and those using CNN.

Research on Improving the Efficiency of Pathological Diagnosis and Assisting in Diagnosis by Automatic Classification of Kidney Biopsy Pathology Images using Deep Learning Technology

Research on Improving the Efficiency of Pathological Diagnosis and Assisting in Diagnosis by Automatic Classification of Kidney Biopsy Pathology Images using Deep Learning Technology

Pathological Image Classification Based on Hard Example Guided CNN

The diagnosis of biopsy tissue with hematoxylin and eosin (H&E) stained images has been widely used by pathologists to detect the lesions and assess the malignancy. Nevertheless, the diagnostic result relies on the visual observation of pathologists which may vary from person to person under different circumstances. With the advantage of automatically and adaptively learning features at multiple levels of abstraction, Convolutional Neural Networks (CNNs) have rapidly become promising alternatives for pathological image analysis. Therefore, in this paper, we propose an effective method for tumor classification called Hard Example Guided CNN. Our contribution is twofold: firstly, to optimize image representation, we design the CNN architecture as dual-branch, used for extracting global features and local features simultaneously. Secondly, we propose a re-weight training algorithm, which improves learning accuracy and accelerates the convergence by increasing the weight of hard examples. Extensive experiments on multiple datasets demonstrate the superiority of our proposed classification method.

Reverse active learning based atrous DenseNet for pathological image classification

BackgroundDue to the recent advances in deep learning, this model attracted researchers who have applied it to medical image analysis. However, pathological image analysis based on deep learning networks faces a number of challenges, such as the high resolution (gigapixel) of pathological images and the lack of annotation capabilities. To address these challenges, we propose a training strategy called deep-reverse active learning (DRAL) and atrous DenseNet (ADN) for pathological image classification. The proposed DRAL can improve the classification accuracy of widely used deep learning networks such as VGG-16 and ResNet by removing mislabeled patches in the training set. As the size of a cancer area varies widely in pathological images, the proposed ADN integrates the atrous convolutions with the dense block for multiscale feature extraction.ResultsThe proposed DRAL and ADN are evaluated using the following three pathological datasets: BACH, CCG, and UCSB. The experiment results demonstrate the excellent performance of the proposed DRAL + ADN framework, achieving patch-level average classification accuracies (ACA) of 94.10%, 92.05% and 97.63% on the BACH, CCG, and UCSB validation sets, respectively.ConclusionsThe DRAL + ADN framework is a potential candidate for boosting the performance of deep learning models for partially mislabeled training datasets.

A Novel Multispace Image Reconstruction Method for Pathological Image Classification Based on Structural Information.

Pathological image classification is of great importance in various biomedical applications, such as for lesion detection, cancer subtype identification, and pathological grading. To this end, this paper proposed a novel classification framework using the multispace image reconstruction inputs and the transfer learning technology. Specifically, a multispace image reconstruction method was first developed to generate a new image containing three channels composed of gradient, gray level cooccurrence matrix (GLCM) and local binary pattern (LBP) spaces, respectively. Then, the pretrained VGG-16 net was utilized to extract the high-level semantic features of original images (RGB) and reconstructed images. Subsequently, the long short-term memory (LSTM) layer was used for feature selection and refinement while increasing its discrimination capability. Finally, the classification task was performed via the softmax classifier. Our framework was evaluated on a publicly available microscopy image dataset of IICBU malignant lymphoma. Experimental results demonstrated the performance advantages of our proposed classification framework by comparing with the related works.

NHL Pathological Image Classification Based on Hierarchical Local Information and GoogLeNet-Based Representations.

Background Accurate classification for different non-Hodgkin lymphomas (NHL) is one of the main challenges in clinical pathological diagnosis due to its intrinsic complexity. Therefore, this paper proposes an effective classification model for three types of NHL pathological images, including mantle cell lymphoma (MCL), follicular lymphoma (FL), and chronic lymphocytic leukemia (CLL). Methods There are three main parts with respect to our model. First, NHL pathological images stained by hematoxylin and eosin (H&E) are transferred into blue ratio (BR) and Lab spaces, respectively. Then specific patch-level textural and statistical features are extracted from BR images and color features are obtained from Lab images both using a hierarchical way, yielding a set of hand-crafted representations corresponding to different image spaces. A random forest classifier is subsequently trained for patch-level classification. Second, H&E images are cropped and fed into a pretrained google inception net (GoogLeNet) for learning high-level representations and a softmax classifier is used for patch-level classification. Finally, three image-level classification strategies based on patch-level results are discussed including a novel method for calculating the weighted sum of patch results. Different classification results are fused at both feature 1 and image levels to obtain a more satisfactory result. Results The proposed model is evaluated on a public IICBU Malignant Lymphoma Dataset and achieves an improved overall accuracy of 0.991 and area under the receiver operating characteristic curve of 0.998. Conclusion The experimentations demonstrate the significantly increased classification performance of the proposed model, indicating that it is a suitable classification approach for NHL pathological images.

Deep extreme learning machine with leaky rectified linear unit for multiclass classification of pathological brain images

Automatic binary classification of brain magnetic resonance (MR) images has made remarkable progress in the past decade. In comparison, a few pieces of work has been reported on multiclass classification of brain MR images. However, there exist enough scopes for improved automation and accuracy. Most of the existing schemes follow the multi-stage pipeline structure of conventional machine learning framework, where the features are designed manually or hand-crafted. In recent years, deep learning models have attracted great interest from researchers for analyzing medical images that eliminate the traditional steps of machine learning. In this paper, we present an automated method based on deep extreme learning machine (ELM) also termed as multilayer ELM (ML-ELM) for multiclass classification of the pathological brain. ML-ELM is a multilayer architecture stacked with ELM based autoencoders. The effectiveness of leaky rectified linear unit (LReLU) activation function is investigated with ML-ELM. Extensive simulations on a multiclass brain MR image dataset indicate that the ML-ELM with LReLU activation (ML-ELM+LReLU) achieves higher performance with faster training speed compared to its counterparts as well as state-of-the-art schemes. The basic purpose of employing ML-ELM+LReLU algorithm is to eliminate the need for hand-crafted feature extraction and to develop a more stable and generalized system for multiclass brain MR image classification.

Pathological Myopia Image Analysis Using Deep Learning

Abstract Pathological Myopia (PM) is one among the main reason of visual defect in the world. Pathological myopia is associated with decaying changes in the retina. If it remains untreated it may lead to vision loss that can’t be recoverable. The correct diagnosis of pathological myopia will facilitate proper treatment and improve disease management which reduce the growth of the disease. However, it is nearly impossible to scan the whole population. Computer-aided diagnosing tools in eye image study will build the method scalable and economical. This paper focuses on the problems of classification of Pathological myopia images and non-pathological myopia images and optical disc, fovea detection, localization, lesions (atrophy and detachment) segmentation with 400 samples provided by International Symposium on Biomedical Imaging (ISBI). In this paper, a deep learning method with Convolutional Neural Networks (CNN) is used for classification and U-net model is used for Image segmentation which shows that it achieves highly competitive results.

Fractional Wavelet Scattering Network and Applications.

This study introduces a fractional wavelet scattering network (FrScatNet), which is a generalized translation invariant version of the classical wavelet scattering network. In our approach, the FrScatNet is constructed based on the fractional wavelet transform (FRWT). The fractional scattering coefficients are iteratively computed using FRWTs and modulus operators. The feature vectors constructed by fractional scattering coefficients are usually used for signal classification. In this paper, an application example of the FrScatNet is provided in order to assess its performance on pathological images. First, the FrScatNet extracts feature vectors from patches of the original histological images under different orders. Then we classify those patches into target (benign or malignant) and background groups. And the FrScatNet property is analyzed by comparing error rates computed from different fractional orders, respectively. Based on the above pathological image classification, a gland segmentation algorithm is proposed by combining the boundary information and the gland location. The error rates for different fractional orders of FrScatNet are examined and show that the classification accuracy is improved in fractional scattering domain. We also compare the FrScatNet-based gland segmentation method with those proposed in the 2015 MICCAI Gland Segmentation Challenge and our method achieves comparable results. The FrScatNet is shown to achieve accurate and robust results. More stable and discriminative fractional scattering coefficients are obtained by the FrScatNet in this paper. The added fractional order parameter is able to analyze the image in the fractional scattering domain.