Segmentation Of Pathological Images Research Articles

Artemisinin optimization based on malaria therapy: Algorithm and applications to medical image segmentation

This study proposes an efficient metaheuristic algorithm called the Artemisinin Optimization (AO) algorithm. This algorithm draws inspiration from the process of artemisinin medicine therapy for malaria, which involves the comprehensive eradication of malarial parasites within the human body. AO comprises three optimization stages: a comprehensive eliminations phase simulating global exploration, a local clearance phase for local exploitation, and a post-consolidation phase to enhance the algorithm's ability to escape local optima. In the experimental, this paper conducts qualitative analysis experiments on the AO, explaining its characteristics in searching for the optimal solution. Subsequently, AO is then tested on the classical IEEE CEC 2014, and the latest IEEE CEC 2022 benchmark function sets to assess its adaptability to various function types. Comparative analyses are conducted against eight well-established algorithms and eight high-performance improved algorithms. Statistical analyses of convergence curves and qualitative metrics revealed AO's robust competitiveness. Lastly, the AO is incorporated into breast cancer pathology image segmentation applications. Using fifteen authentic medical images at six threshold levels, AO's segmentation performance is compared against eight distinguished algorithms. Experimental results demonstrated AO's superiority in terms of image segmentation accuracy, Feature Similarity Index (FSIM), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM) over the contrast algorithms. These comparative findings emphasize AO's efficacy and its potential in real-world optimization applications. The source codes of this paper will be available in https://aliasgharheidari.com/AO.html and other public websites.

VENet: Variational energy network for gland segmentation of pathological images and early gastric cancer diagnosis of whole slide images

Background and objective: Gland segmentation of pathological images is an essential but challenging step for adenocarcinoma diagnosis. Although deep learning methods have recently made tremendous progress in gland segmentation, they have not given satisfactory boundary and region segmentation results of adjacent glands. These glands usually have a large difference in glandular appearance, and the statistical distribution between the training and test sets in deep learning is inconsistent. These problems make networks not generalize well in the test dataset, bringing difficulties to gland segmentation and early cancer diagnosis.Methods: To address these problems, we propose a Variational Energy Network named VENet with a traditional variational energy Lv loss for gland segmentation of pathological images and early gastric cancer detection in whole slide images (WSIs). It effectively integrates the variational mathematical model and the data-adaptability of deep learning methods to balance boundary and region segmentation. Furthermore, it can effectively segment and classify glands in large-size WSIs with reliable nucleus width and nucleus-to-cytoplasm ratio features.Results: The VENet was evaluated on the 2015 MICCAI Gland Segmentation challenge (GlaS) dataset, the Colorectal Adenocarcinoma Glands (CRAG) dataset, and the self-collected Nanfang Hospital dataset. Compared with state-of-the-art methods, our method achieved excellent performance for GlaS Test A (object dice 0.9562, object F1 0.9271, object Hausdorff distance 73.13), GlaS Test B (object dice 94.95, object F1 95.60, object Hausdorff distance 59.63), and CRAG (object dice 95.08, object F1 92.94, object Hausdorff distance 28.01). For the Nanfang Hospital dataset, our method achieved a kappa of 0.78, an accuracy of 0.9, a sensitivity of 0.98, and a specificity of 0.80 on the classification task of test 69 WSIs.Conclusions: The experimental results show that the proposed model accurately predicts boundaries and outperforms state-of-the-art methods. It can be applied to the early diagnosis of gastric cancer by detecting regions of high-grade gastric intraepithelial neoplasia in WSI, which can assist pathologists in analyzing large WSI and making accurate diagnostic decisions.

Semi-supervised pathological image segmentation via cross distillation of multiple attentions and Seg-CAM consistency

Segmentation of pathological images is a crucial step for accurate cancer diagnosis. However, acquiring dense annotations of such images for training is labor-intensive and time-consuming. To address this issue, Semi-Supervised Learning (SSL) has the potential for reducing the annotation cost, but it is challenged by a large number of unlabeled training images. In this paper, we propose a novel SSL method based on Cross Distillation of Multiple Attentions and Seg-CAM Consistency (CDMA+) to effectively leverage unlabeled images. First, we propose a Multi-attention Tri-decoder Network (MTNet) that consists of a shared encoder and three decoders, with each decoder using a different attention mechanism that calibrates features in different aspects to generate diverse outputs. Second, we introduce Cross Decoder Knowledge Distillation (CDKD) between the three decoders, allowing them to learn from each other’s soft labels to mitigate the negative impact of incorrect pseudo labels during training. Subsequently, motivated by the observation that the Class Activation Maps (CAMs) derived from the classification task could provide a rough segmentation, we employ an auxiliary classification head and introduce a consistency constraint between the CAM and segmentation results, i.e. Seg-CAM consistency. Additionally, uncertainty minimization is applied to the average prediction of the three decoders, which further regularizes predictions on unlabeled images and encourages inter-decoder consistency. Our proposed CDMA+ was compared with eight state-of-the-art SSL methods on two public pathological image datasets, and the experimental results showed that our method outperforms the other approaches under different annotation ratios. The code is available at https://github.com/HiLab-git/CDMA.

Adapting SAM to Histopathology Images for Tumor Bud Segmentation in Colorectal Cancer.

Colorectal cancer (CRC) is the third most common cancer in the United States. Tumor Budding (TB) detection and quantification are crucial yet labor-intensive steps in determining the CRC stage through the analysis of histopathology images. To help with this process, we adapt the Segment Anything Model (SAM) on the CRC histopathology images to segment TBs using SAM-Adapter. In this approach, we automatically take task-specific prompts from CRC images and train the SAM model in a parameter-efficient way. We compare the predictions of our model with the predictions from a trained-from-scratch model using the annotations from a pathologist. As a result, our model achieves an intersection over union (IoU) of 0.65 and an instance-level Dice score of 0.75, which are promising in matching the pathologist's TB annotation. We believe our study offers a novel solution to identify TBs on H&E-stained histopathology images. Our study also demonstrates the value of adapting the foundation model for pathology image segmentation tasks.

SC-Net: Symmetrical conical network for colorectal pathology image segmentation

Background and ObjectiveImage segmentation of histopathology of colorectal cancer is a core task of computer aided medical image diagnosis system. Existing convolutional neural networks generally extract multi-scale information in linear flow structures by inserting multi-branch modules, which is difficult to extract heterogeneous semantic information under multi-level and different receptive field and tough to establish context dependency among different receptive field features. MethodsTo address these issues, we propose a symmetric spiral progressive feature fusion encoder-decoder network called the Symmetric Conical Network (SC-Net). First, we design a Multi-scale Feature Extraction Block (MFEB) matching with the Symmetric Conical Network to obtain multi-branch heterogeneous semantic information under different receptive fields, so as to enrich the diversity of extracted feature information. The encoder is composed of MFEB through spiral and multi-branch arrangement to enhance context dependence between different information flow. Secondly, the information loss of contour, color and others in high-level semantic information through causally stacking MFEB, the Feature Mapping Layer (FML) is designed to map low-level features to high-level semantic features along the down-sampling branch and solve the problem of insufficient global feature extraction in deep levels. ResultsThe SC-Net was evaluated on our self-constructed colorectal cancer dataset, a publicly available breast cancer dataset and a polyp dataset. The results revealed that the mDice of segmentation reached 0.8611, 0.7259 and 0.7144. We compare our model with the state-of-art semantic segmentation UNet++, PSPNet, Attention U-Net, R2U-Net and other advanced segmentation networks. The experimental results demonstrate that we achieve the most advanced performance. ConclusionsThe results indicate that the proposed SC-Net excels in segmenting H&E stained pathology images, effectively preserving morphological features and spatial information even in scenarios with weak texture, poor contrast, and variations in appearance.

Pancreatic cancer pathology image segmentation with channel and spatial long-range dependencies

Based on deep learning, pancreatic cancer pathology image segmentation technology effectively assists pathologists in achieving improved treatment outcomes. However, compared to traditional image segmentation tasks, the large size of tissues in pathology images requires a larger receptive field. While methods based on dilated convolutions or attention mechanisms can enhance the receptive field, they cannot capture long-range feature dependencies. Directly applying self-attention mechanisms to capture long-range dependencies results in intolerable computational complexity. To address these challenges, we introduce a channel and spatial self-attention (CS) Module designed for efficiently capturing both channel and spatial long-range feature dependencies in pancreatic cancer pathological images. Specifically, the channel and spatial self-attention module consists of an adaptive channel self-attention module and a window-shift spatial self-attention module. The adaptive channel self-attention module adaptively pools features to a fixed size to capture long-range feature dependencies. While the window-shift spatial self-attention module captures spatial long-range dependencies in a window-based manner. Additionally, we propose a re-weighted cross-entropy loss to mitigate the impact of long-tail distribution on performance. Our proposed method surpasses state-of-the-art on both our Pancreatic Cancer Pathology Image (PCPI) dataset and the GlaS challenge dataset. The mDice and mIoU have achieved 73.93% and 59.42% in our PCPI dataset.

Semi-supervised breast cancer pathology image segmentation based on fine-grained classification guidance.

Breast cancer pathological image segmentation (BCPIS) holds significant value in assisting physicians with quantifying tumor regions and providing treatment guidance. However, achieving fine-grained semantic segmentation remains a major challenge for this technology. The complex and diverse morphologies of breast cancer tissue structures result in high costs for manual annotation, thereby limiting the sample size and annotation quality of the dataset. These practical issues have a significant impact on the segmentation performance. To overcome these challenges, this study proposes a semi-supervised learning model based on classification-guided segmentation. The model first utilizes a multi-scale convolutional network to extract rich semantic information and then employs a multi-expert cross-layer joint learning strategy, integrating a small number of labeled samples to iteratively provide the model with class-generated multi-cue pseudo-labels and real labels. Given the complexity of the breast cancer samples and the limited sample quantity, an innovative approach of augmenting additional unlabeled data was adopted to overcome this limitation. Experimental results demonstrate that, although the proposed model falls slightly behind supervised segmentation models, it still exhibits significant progress and innovation. The semi-supervised model in this study achieves outstanding performance, with an IoU (Intersection over Union) value of 71.53%. Compared to other semi-supervised methods, the model developed in this study demonstrates a performance advantage of approximately 3%. Furthermore, the research findings indicate a significant correlation between the classification and segmentation tasks in breast cancer pathological images, and the guidance of a multi-expert system can significantly enhance the fine-grained effects of semi-supervised semantic segmentation.

Nuclei segmentation with point annotations from pathology images via self-supervised learning and co-training.

Nuclei segmentation is a crucial task for whole slide image analysis in digital pathology. Generally, the segmentation performance of fully-supervised learning heavily depends on the amount and quality of the annotated data. However, it is time-consuming and expensive for professional pathologists to provide accurate pixel-level ground truth, while it is much easier to get coarse labels such as point annotations. In this paper, we propose a weakly-supervised learning method for nuclei segmentation that only requires point annotations for training. First, coarse pixel-level labels are derived from the point annotations based on the Voronoi diagram and the k-means clustering method to avoid overfitting. Second, a co-training strategy with an exponential moving average method is designed to refine the incomplete supervision of the coarse labels. Third, a self-supervised visual representation learning method is tailored for nuclei segmentation of pathology images that transforms the hematoxylin component images into the H&E stained images to gain better understanding of the relationship between the nuclei and cytoplasm. We comprehensively evaluate the proposed method using two public datasets. Both visual and quantitative results demonstrate the superiority of our method to the state-of-the-art methods, and its competitive performance compared to the fully-supervised methods. Codes are available at https://github.com/hust-linyi/SC-Net.

Omni-Seg: A Scale-Aware Dynamic Network for Renal Pathological Image Segmentation.

Comprehensive semantic segmentation on renal pathological images is challenging due to the heterogeneous scales of the objects. For example, on a whole slide image (WSI), the cross-sectional areas of glomeruli can be 64 times larger than that of the peritubular capillaries, making it impractical to segment both objects on the same patch, at the same scale. To handle this scaling issue, prior studies have typically trained multiple segmentation networks in order to match the optimal pixel resolution of heterogeneous tissue types. This multi-network solution is resource-intensive and fails to model the spatial relationship between tissue types. In this article, we propose the Omni-Seg network, a scale-aware dynamic neural network that achieves multi-object (six tissue types) and multi-scale (5× to 40× scale) pathological image segmentation via a single neural network. The contribution of this article is three-fold: (1) a novel scale-aware controller is proposed to generalize the dynamic neural network from single-scale to multi-scale; (2) semi-supervised consistency regularization of pseudo-labels is introduced to model the inter-scale correlation of unannotated tissue types into a single end-to-end learning paradigm; and (3) superior scale-aware generalization is evidenced by directly applying a model trained on human kidney images to mouse kidney images, without retraining. By learning from 150,000 human pathological image patches from six tissue types at three different resolutions, our approach achieved superior segmentation performance according to human visual assessment and evaluation of image-omics (i.e., spatial transcriptomics).

A Medically Assisted Model for Precise Segmentation of Osteosarcoma Nuclei on Pathological Images.

Osteosarcoma is the most common malignant bone tumor with a high degree of malignancy and misdiagnosis rates. Pathological images are crucial for its diagnosis. However, underdeveloped regions currently lack sufficient high-level pathologists, leading to uncertain diagnostic accuracy and efficiency. Existing research on pathological image segmentation often neglects the differences in staining styles and lack of data, without considering medical backgrounds. To alleviate the difficulty in diagnosing osteosarcoma in underdeveloped areas, an intelligent assisted diagnosis and treatment scheme for osteosarcoma pathological images, ENMViT, is proposed. ENMViT utilizes KIN to achieve normalization of mismatched images with limited GPU resources and uses traditional data enhancement methods, such as cleaning, cropping, mosaic, Laplacian sharpening, and other techniques to alleviate the issue of insufficient data. A multi-path semantic segmentation network combining Transformer and CNN is used to segment images, and the degree of edge offset in the spatial domain is introduced into the loss function. Finally, noise is filtered according to the size of the connecting domain. This paper experimented on more than 2000 osteosarcoma pathological images from Central South University. The experimental results demonstrate that this scheme performs well in each stage of the osteosarcoma pathological image processing, and the segmentation results' IoU index is 9.4% higher than the comparative models, demonstrating its significant value in the medical industry.

An Accelerated Pipeline for Multi-label Renal Pathology Image Segmentation at the Whole Slide Image Level.

Deep-learning techniques have been used widely to alleviate the labour-intensive and time-consuming manual annotation required for pixel-level tissue characterization. Our previous study introduced an efficient single dynamic network - Omni-Seg - that achieved multi-class multi-scale pathological segmentation with less computational complexity. However, the patch-wise segmentation paradigm still applies to Omni-Seg, and the pipeline is time-consuming when providing segmentation for Whole Slide Images (WSIs). In this paper, we propose an enhanced version of the Omni-Seg pipeline in order to reduce the repetitive computing processes and utilize a GPU to accelerate the model's prediction for both better model performance and faster speed. Our proposed method's innovative contribution is two-fold: (1) a Docker is released for an end-to-end slide-wise multi-tissue segmentation for WSIs; and (2) the pipeline is deployed on a GPU to accelerate the prediction, achieving better segmentation quality in less time. The proposed accelerated implementation reduced the average processing time (at the testing stage) on a standard needle biopsy WSI from 2.3 hours to 22 minutes, using 35 WSIs from the Kidney Tissue Atlas (KPMP) Datasets. The source code and the Docker have been made publicly available at https://github.com/ddrrnn123/Omni-Seg.

Shortcomings and areas for improvement in digital pathology image segmentation challenges.

Digital pathology image analysis challenges have been organised regularly since 2010, often with events hosted at major conferences and results published in high-impact journals. These challenges mobilise a lot of energy from organisers, participants, and expert annotators (especially for image segmentation challenges). This study reviews image segmentation challenges in digital pathology and the top-ranked methods, with a particular focus on how reference annotations are generated and how the methods' predictions are evaluated. We found important shortcomings in the handling of inter-expert disagreement and the relevance of the evaluation process chosen. We also noted key problems with the quality control of various challenge elements that can lead to uncertainties in the published results. Our findings show the importance of greatly increasing transparency in the reporting of challenge results, and the need to make publicly available the evaluation codes, test set annotations and participants' predictions. The aim is to properly ensure the reproducibility and interpretation of the results and to increase the potential for exploitation of the substantial work done in these challenges.

Democratizing Pathological Image Segmentation with Lay Annotators via Molecular-empowered Learning.

Multi-class cell segmentation in high-resolution Giga-pixel whole slide images (WSI) is critical for various clinical applications. Training such an AI model typically requires labor-intensive pixel-wise manual annotation from experienced domain experts (e.g., pathologists). Moreover, such annotation is error-prone when differentiating fine-grained cell types (e.g., podocyte and mesangial cells) via the naked human eye. In this study, we assess the feasibility of democratizing pathological AI deployment by only using lay annotators (annotators without medical domain knowledge). The contribution of this paper is threefold: (1) We proposed a molecular-empowered learning scheme for multi-class cell segmentation using partial labels from lay annotators; (2) The proposed method integrated Giga-pixel level molecular-morphology cross-modality registration, molecular-informed annotation, and molecular-oriented segmentation model, so as to achieve significantly superior performance via 3 lay annotators as compared with 2 experienced pathologists; (3) A deep corrective learning (learning with imperfect label) method is proposed to further improve the segmentation performance using partially annotated noisy data. From the experimental results, our learning method achieved F1 = 0.8496 using molecular-informed annotations from lay annotators, which is better than conventional morphology-based annotations (F1 = 0.7015) from experienced pathologists. Our method democratizes the development of a pathological segmentation deep model to the lay annotator level, which consequently scales up the learning process similar to a non-medical computer vision task. The official implementation and cell annotations are publicly available at https://github.com/hrlblab/MolecularEL.

MSSA‐Net: A novel multi‐scale feature fusion and global self‐attention network for lesion segmentation

AbstractIn medical image segmentation tasks, it is typical to adopt convolutional neural networks with a serial encoder‐decoder structure. However, mainstream networks cannot simultaneously achieve sufficient extraction of global features and the fusion of multi‐scale information, which may lead to unpromising results for the segmentation of pathological images. Therefore, this article proposed a novel multi‐scale feature fusion and global self‐attention network (MSSA‐Net) for medical image segmentation. Specifically, we designed a parallel double‐encoder network with a multi‐scale feature fusion encoder (MS‐Encoder) and a self‐attention encoder (SA‐Encoder). The SA‐Encoder introduces the transformer's global self‐attention mechanism to extract global features, and the MS‐Encoder adopts atrous spatial pyramid pooling (ASPP) to realize multi‐scale fusion. We have evaluated the proposed MSSA‐Net using three medical segmentation datasets, covering various imaging modalities such as colonoscopy and magnetic resonance imaging. Experiments on the CVC‐ClinicDC, the 2015 MICCAI subchallenge on automatic polyp detection dataset, and anatomical tracings of lesions after stroke (ATLAS) show that our MSSA‐Net outperforms mainstream methods such as DoubleU‐Net and TransUNet. Moreover, MSSA‐Net can predict more accurate segmentation masks, especially in the case of ATLAS, which has challenging images such as multiple shadow areas and discrete lesions.

ANT-UNet: Accurate and Noise-Tolerant Segmentation for Pathology Image Processing

Pathology image segmentation is an essential step in early detection and diagnosis for various diseases. Due to its complex nature, precise segmentation is not a trivial task. Recently, deep learning has been proved as an effective option for pathology image processing. However, its efficiency is highly restricted by inconsistent annotation quality. In this article, we propose an accurate and noise-tolerant segmentation approach to overcome the aforementioned issues. This approach consists of two main parts: a preprocessing module for data augmentation and a new neural network architecture, ANT-UNet. Experimental results demonstrate that, even on a noisy dataset, the proposed approach can achieve more accurate segmentation with 6% to 35% accuracy improvement versus other commonly used segmentation methods. In addition, the proposed architecture is hardware friendly, which can reduce the amount of parameters to one-tenth of the original and achieve 1.7× speed-up.

Joint fully convolutional and graph convolutional networks for weakly-supervised segmentation of pathology images

Tissue/region segmentation of pathology images is essential for quantitative analysis in digital pathology. Previous studies usually require full supervision (e.g., pixel-level annotation) which is challenging to acquire. In this paper, we propose a weakly-supervised model using joint Fully convolutional and Graph convolutional Networks (FGNet) for automated segmentation of pathology images. Instead of using pixel-wise annotations as supervision, we employ an image-level label (i.e., foreground proportion) as weakly-supervised information for training a unified convolutional model. Our FGNet consists of a feature extraction module (with a fully convolutional network) and a classification module (with a graph convolutional network). These two modules are connected via a dynamic superpixel operation, making the joint training possible. To achieve robust segmentation performance, we propose to use mutable numbers of superpixels for both training and inference. Besides, to achieve strict supervision, we employ an uncertainty range constraint in FGNet to reduce the negative effect of inaccurate image-level annotations. Compared with fully-supervised methods, the proposed FGNet achieves competitive segmentation results on three pathology image datasets (i.e., HER2, KI67, and H&E) for cancer region segmentation, suggesting the effectiveness of our method. The code is made publicly available at https://github.com/zhangjun001/FGNet.

Structure convolutional extreme learning machine and case-based shape template for HCC nucleus segmentation

Accurate segmentation of hepatocellular carcinoma (HCC) nuclei is of great importance in automatic pathologic diagnosis. This paper proposes structure convolutional extreme learning machine (SC-ELM) and case-based shape template (CBST) methods for HCC nucleus segmentation, which could tackle complex nucleus scenarios including adhesion or overlap. First, SC-ELM is developed for global segmentation of pathology images, which is used for coarse segmentation. Then, each connected region is considered as a nucleus clump and a probability model with three energy functions is proposed for contour refinement of nucleus clumps. Finally, for complex nucleus clumps, the CBST method combined with pixel-based classification is utilized for unclear or lost boundary inference. Experimentations with 127 liver pathology images demonstrate the performance advantages of our proposed method as compared with related work.

Acceleration of Two Point Correlation Function Calculation for Pathology Image Segmentation.

The segmentation of tissue regions in high-resolution microscopy is a challenging problem due to both the size and appearance of digitized pathology sections. The two point correlation function (TPCF) has proved to be an effective feature to address the textural appearance of tissues. However the calculation of the TPCF functions is computationally burdensome and often intractable in the gigapixel images produced by slide scanning devices for pathology application. In this paper we present several approaches for accelerating deterministic calculation of point correlation functions using theory to reduce computation, parallelization on distributed systems, and parallelization on graphics processors. Previously we show that the correlation updating method of calculation offers an 8-35× speedup over frequency domain methods and decouples efficient computation from the select scales of Fourier methods. In this paper, using distributed computation on 64 compute nodes provides a further 42× speedup. Finally, parallelization on graphics processors (GPU) results in an additional 11-16× speedup using an implementation capable of running on a single desktop machine.

Partitioning Histopathological Images: An Integrated Framework for Supervised Color-Texture Segmentation and Cell Splitting

For quantitative analysis of histopathological images, such as the lymphoma grading systems, quantification of features is usually carried out on single cells before categorizing them by classification algorithms. To this end, we propose an integrated framework consisting of a novel supervised cell-image segmentation algorithm and a new touching-cell splitting method. For the segmentation part, we segment the cell regions from the other areas by classifying the image pixels into either cell or extra-cellular category. Instead of using pixel color intensities, the color-texture extracted at the local neighborhood of each pixel is utilized as the input to our classification algorithm. The color-texture at each pixel is extracted by local Fourier transform (LFT) from a new color space, the most discriminant color space (MDC). The MDC color space is optimized to be a linear combination of the original RGB color space so that the extracted LFT texture features in the MDC color space can achieve most discrimination in terms of classification (segmentation) performance. To speed up the texture feature extraction process, we develop an efficient LFT extraction algorithm based on image shifting and image integral. For the splitting part, given a connected component of the segmentation map, we initially differentiate whether it is a touching-cell clump or a single nontouching cell. The differentiation is mainly based on the distance between the most likely radial-symmetry center and the geometrical center of the connected component. The boundaries of touching-cell clumps are smoothed out by Fourier shape descriptor before carrying out an iterative, concave-point and radial-symmetry based splitting algorithm. To test the validity, effectiveness and efficiency of the framework, it is applied to follicular lymphoma pathological images, which exhibit complex background and extracellular texture with nonuniform illumination condition. For comparison purposes, the results of the proposed segmentation algorithm are evaluated against the outputs of superpixel, graph-cut, mean-shift, and two state-of-the-art pathological image segmentation methods using ground-truth that was established by manual segmentation of cells in the original images. Our segmentation algorithm achieves better results than the other compared methods. The results of splitting are evaluated in terms of under-splitting, over-splitting, and encroachment errors. By summing up the three types of errors, we achieve a total error rate of 5.25% per image.