Loss Of Contrast Research Articles

Machine cross-domain remaining useful life prediction via contrastive adversarial variational recurrent method

The performance degradation process of the machine is non-stationary which varies with the operating environment and is reflected as a temporal shift phenomenon. Models or methods that assume the test and training sets have the same distribution will no longer be suitable for solving problems where domain shifts exist. This brings new challenges for accurately evaluating the machine’s remaining useful life (RUL). This paper studies a contrastive adversarial variational recurrent method for machine RUL prediction under different service conditions. In this new approach, the variational recurrent networks are developed to extract distribution features and latent spaces of raw data, meanwhile, the adversarial strategies are applied to enable the learning of domain-invariant features to make the model achieve cross-domain task processing. To supervise the learning process of the model to learn more mutual information between the extracted features and the raw input data, a contrastive loss is also introduced in the proposed method. Sufficient experiments were conducted to verify the feasibility and superiority of the suggested approach, including 12 sets of cross-scenario tests on the turbofan engine dataset C-MAPSS from NASA. Experimental findings confirm that the proposed method performs satisfactorily and competitively even compared to current state-of-the-art works.

Adaptive Multi-Feature Attention Network for Image Dehazing

Currently, deep-learning-based image dehazing methods occupy a dominant position in image dehazing applications. Although many complicated dehazing models have achieved competitive dehazing performance, effective methods for extracting useful features are still under-researched. Thus, an adaptive multi-feature attention network (AMFAN) consisting of the point-weighted attention (PWA) mechanism and the multi-layer feature fusion (AMLFF) is presented in this paper. We start by enhancing pixel-level attention for each feature map. Specifically, we design a PWA block, which aggregates global and local information of the feature map. We also employ PWA to make the model adaptively focus on significant channels/regions. Then, we design a feature fusion block (FFB), which can accomplish feature-level fusion by exploiting a PWA block. The FFB and PWA constitute our AMLFF. We design an AMLFF, which can integrate three different levels of feature maps to effectively balance the weights of the inputs to the encoder and decoder. We also utilize the contrastive loss function to train the dehazing network so that the recovered image is far from the negative sample and close to the positive sample. Experimental results on both synthetic and real-world images demonstrate that this dehazing approach surpasses numerous other advanced techniques, both visually and quantitatively, showcasing its superiority in image dehazing.

Zero-shot prompt-based video encoder for surgical gesture recognition.

In order to produce a surgical gesture recognition system that can support a wide variety of procedures, either a very large annotated dataset must be acquired, or fitted models must generalize to new labels (so-called zero-shot capability). In this paper we investigate the feasibility of latter option. Leveraging the bridge-prompt framework, we prompt-tune a pre-trained vision-text model (CLIP) for gesture recognition in surgical videos. This can utilize extensive outside video data such as text, but also make use of label meta-data and weakly supervised contrastive losses. Our experiments show that prompt-based video encoder outperforms standard encoders in surgical gesture recognition tasks. Notably, it displays strong performance in zero-shot scenarios, where gestures/tasks that were not provided during the encoder training phase are included in the prediction phase. Additionally, we measure the benefit of inclusion text descriptions in the feature extractor training schema. Bridge-prompt and similar pre-trained+prompt-tuned video encoder models present significant visual representation for surgical robotics, especially in gesture recognition tasks. Given the diverse range of surgical tasks (gestures), the ability of these models to zero-shot transfer without the need for any task (gesture) specific retraining makes them invaluable.

Mask-guided BERT for few-shot text classification

Transformer-based language models have achieved significant success in various domains. However, the data-intensive nature of the transformer architecture requires much labeled data, which is challenging in low-resource scenarios (i.e., few-shot learning (FSL)). The main challenge of FSL is the difficulty of training robust models on small amounts of samples, which frequently leads to overfitting. Here we present Mask-BERT, a simple and modular framework to help BERT-based architectures tackle FSL. The proposed approach fundamentally differs from existing FSL strategies such as prompt tuning and meta-learning. The core idea is to selectively apply masks on text inputs and filter out irrelevant information, which guides the model to focus on discriminative tokens that influence prediction results. In addition, to make the text representations from different categories more separable and the text representations from the same category more compact, we introduce a contrastive learning loss function. Experimental results on open-domain and medical-domain datasets demonstrate the effectiveness of Mask-BERT. Code and data are available at: github.com/WenxiongLiao/mask-bert

Boosting cross-modal retrieval in remote sensing via a novel unified attention network

With the rapid advent and abundance of remote sensing data in different modalities, cross-modal retrieval tasks have gained importance in the research community. Cross-modal retrieval belongs to the research paradigm in which the query is of one modality and the retrieved output is of the other modality. In this paper, the remote sensing (RS) data modalities considered are the earth observation optical data (aerial photos) and the corresponding hand-drawn sketches. The main challenge of the cross-modal retrieval research objective for optical remote sensing images and the corresponding sketches is the distribution gap between the shared embedding space of the modalities. Prior attempts to resolve this issue have not yielded satisfactory outcomes regarding accurately retrieving cross-modal sketch-image RS data. The state-of-the-art architectures used conventional convolutional architectures, which focused on local pixel-wise information about the modalities to be retrieved. This limits the interaction between the sketch texture and the corresponding image, making these models susceptible to overfitting datasets with particular scenarios. To circumvent this limitation, we suggest establishing multi-modal correspondence using a novel architecture of the combined self and cross-attention algorithms, SPCA-Net to minimize the modality gap by employing attention mechanisms for the query and other modalities. Efficient cross-modal retrieval is achieved through the suggested attention architecture, which empirically emphasizes the global information of the relevant query modality and bridges the domain gap through a unique pairwise cross-attention network. In addition to the novel architecture, this paper introduces a unique loss function, label-specific supervised contrastive loss, tailored to the intricacies of the task and to enhance the discriminative power of the learned embeddings. Extensive evaluations are conducted on two sketch-image remote sensing datasets, Earth-on-Canvas and RSketch. Under the same experimental conditions, the performance metrics of our proposed model beat the state-of-the-art architectures by significant margins of 16.7%, 18.9%, 33.7%, and 40.9% correspondingly.

Triplet Contrastive Representation Learning for Unsupervised Vehicle Re-identification

Part feature learning plays a crucial role in achieving fine-grained semantic understanding in unsupervised vehicle re-identification. However, existing approaches directly model part and global features, which can easily lead to severe gradient vanishing issues due to their unequal feature information and unreliable pseudo-labels. To address this problem, in this paper, we propose a triplet contrastive representation learning (TCRL) framework, which leverages cluster features to bridge the part features and global features for unsupervised vehicle re-identification. Specifically, TCRL devises three memory banks to store the instance/cluster features and proposes a proxy contrastive loss (PCL) to make contrastive learning between adjacent memory banks, thus presenting the associations between the part and global features as a transition of the part-cluster and cluster-global associations. Since the cluster memory bank copes with all the vehicle features, it can summarize them into a discriminative feature representation. To deeply exploit the instance/cluster information, TCRL proposes two additional loss functions. For the instance-level feature, a hybrid contrastive loss (HCL) re-defines the sample correlations by approaching the positive instance features and pushing all negative instance features away. For the cluster-level feature, a weighted regularization cluster contrastive loss (WRCCL) refines the pseudo labels by penalizing the mislabeled images according to the instance similarity. Extensive experiments show that TCRL outperforms many state-of-the-art unsupervised vehicle re-identification approaches.

Early diagnosis of Chikungunya virus utilizing square core photonic crystal fiber (SC-PCF) with extremely high relative sensitivity

Chikungunya virus (CHIKV) poses a significant public health threat due to its capacity to cause widespread and debilitating outbreaks. The virus is responsible for CHIKV fever, a disease characterized by severe joint pain, sudden onset of fever, headache, muscle pain, and rash. The virus has been reported in various regions globally, with outbreaks occurring in parts of Africa, Asia, the Americas, and the Indian subcontinent. Consequently, the scientific community expends substantial efforts in developing dependable, rapid, highly sensitive, and cost-effective techniques in order to identify the CHIKV virus. In this study, an innovative biomedical sensor using photonic crystal fiber technology enables precise detection of the CHIKV virus through uric acid, normal and infected plasma, red blood cells, and platelets in the blood. The introduced sensor identifies those kinds with extremely increased relative sensitivity and minimal losses in contrast to alternative photonic crystal fiber-based biosensors. The introduced sensor showcases a minimal confinement loss of 2.25 × 10− 13 cm− 1, a relative sensitivity of 99.37%, an effective area of 1.36 × 105 µm2, with a minimal effective material loss of 0.001966 cm–1, a numerical aperture of 0.1874, and low dispersion of 0.06. Also, the demonstrated sensor is able to function within the terahertz spectrum, covering a substantial span from 0.8 to 2.6 THz. Furthermore, an extensive comparison analysis is performed between the showcased sensor and related literature on photonic crystal fibers to verify the reliability and effectiveness of the introduced structure.

Integrating Contrastive Learning and Cycle Generative Adversarial Networks for Non-invasive Fetal ECG Extraction.

Fetal electrocardiogram (FECG) contains crucial information about the fetus during pregnancy, making the extraction of FECG signal essential for monitoring fetal health. However, extracting FECG signal from abdominal electrocardiogram (AECG) poses several challenges: (1) FECG signal is often contaminated by noise, and (2) FECG signal is frequently overshadowed by high-amplitude maternal electrocardiogram (MECG). To address these issues and enhance the accuracy of signal extraction, this paper proposes an improved Cycle Generative Adversarial Networks (CycleGAN) with integrated contrastive learning for FECG signal extraction. The model introduces a dual-attention mechanism in the generator of the generative adversarial network, incorporating amulti-head self-attention(MSA) module and achannel-wise self-attention (CSA) module to enhance the quality of generated signals. Additionally, a contrastive triplet loss is integrated into the CycleGAN loss function, optimizing training to increase the similarity between the extracted FECG signal and the scalp fetal electrocardiogram. The proposed method is evaluated using the ADFECG dataset and the PCDB dataset both from the Physionet. In terms of signal extraction quality, Mean Squared Error is reduced to 0.036, Mean Absolute Error (MAE) to 0.009, and Pearson Correlation Coefficient reaches 0.924. When validating the model performance, Structural Similarity Index achieves 95.54%, Peak Signal-to-Noise Ratio (PSNR) reaches 38.87dB, and R-squared (R2) attains 95.12%. Furthermore, the positive predictive value (PPV), sensitivity (SEN) and F1-score for QRS wave cluster detection on the ADFECG dataset also reached 99.56%, 99.43% and 99.50%, respectively. On the PCDB dataset, the positive predictive value (PPV), sensitivity (SEN) and F1-score for QRS wave cluster detection also reached 98.24%, 98.60% and 98.42%, respectively. All of them are higher than other methods. Therefore, the proposed model has important applications in effective monitoring of fetal health during pregnancy.

MolMVC: Enhancing molecular representations for drug-related tasks through multi-view contrastive learning.

Effective molecular representation is critical in drug development. The complex nature of molecules demands comprehensive multi-view representations, considering 1D, 2D, and 3D aspects, to capture diverse perspectives. Obtaining representations that encompass these varied structures is crucial for a holistic understanding of molecules in drug-related contexts. In this study, we introduce an innovative multi-view contrastive learning framework for molecular representation, denoted as MolMVC. Initially, we use a Transformer encoder to capture 1D sequence information and a Graph Transformer to encode the intricate 2D and 3D structural details of molecules. Our approach incorporates a novel attention-guided augmentation scheme, leveraging prior knowledge to create positive samples tailored to different molecular data views. To align multi-view molecular positive samples effectively in latent space, we introduce an adaptive multi-view contrastive loss (AMCLoss). In particular, we calculate AMCLoss at various levels within the model to effectively capture the hierarchical nature of the molecular information. Eventually, we pre-train the encoders via minimizing AMCLoss to obtain the molecular representation, which can be used for various down-stream tasks. In our experiments, we evaluate the performance of our MolMVC on multiple tasks, including molecular property prediction (MPP), drug-target binding affinity (DTA) prediction and cancer drug response (CDR) prediction. The results demonstrate that the molecular representation learned by our MolMVC can enhance the predictive accuracy on these tasks and also reduce the computational costs. Furthermore, we showcase MolMVC's efficacy in drug repositioning across a spectrum of drug-related applications. The code and pre-trained model are publicly available at https://github.com/Hhhzj-7/MolMVC.

MILG: Realistic lip-sync video generation with audio-modulated image inpainting

Existing lip synchronization (lip-sync) methods generate accurately synchronized mouths and faces in a generated video. However, they still confront the problem of artifacts in regions of non-interest (RONI), e.g., background and other parts of a face, which decreases the overall visual quality. To solve these problems, we innovatively introduce diverse image inpainting to lip-sync generation. We propose Modulated Inpainting Lip-sync GAN (MILG), an audio-constraint inpainting network to predict synchronous mouths. MILG utilizes prior knowledge of RONI and audio sequences to predict lip shape instead of image generation, which can keep the RONI consistent. Specifically, we integrate modulated spatially probabilistic diversity normalization (MSPD Norm) in our inpainting network, which helps the network generate fine-grained diverse mouth movements guided by the continuous audio features. Furthermore, to lower the training overhead, we modify the contrastive loss in lip-sync to support small-batch-size and few-sample training. Extensive experiments demonstrate that our approach outperforms the existing state-of-the-art of image quality and authenticity while keeping lip-sync.

Probabilistic Contrastive Learning for Long-Tailed Visual Recognition.

Long-tailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples. Such imbalance issue considerably impairs the performance of standard supervised learning algorithms, which are mainly designed for balanced training sets. Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance. However, the performance of supervised contrastive learning is plagued by an inherent challenge: it necessitates sufficiently large batches of training data to construct contrastive pairs that cover all categories, yet this requirement is difficult to meet in the context of class-imbalanced data. To overcome this obstacle, we propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space, and samples contrastive pairs accordingly. In fact, estimating the distributions of all classes using features in a small batch, particularly for imbalanced data, is not feasible. Our key idea is to introduce a reasonable and simple assumption that the normalized features in contrastive learning follow a mixture of von Mises-Fisher (vMF) distributions on unit space, which brings two-fold benefits. First, the distribution parameters can be estimated using only the first sample moment, which can be efficiently computed in an online manner across different batches. Second, based on the estimated distribution, the vMF distribution allows us to sample an infinite number of contrastive pairs and derive a closed form of the expected contrastive loss for efficient optimization. Other than long-tailed problems, ProCo can be directly applied to semi-supervised learning by generating pseudo-labels for unlabeled data, which can subsequently be utilized to estimate the distribution of the samples inversely. Theoretically, we analyze the error bound of ProCo. Empirically, extensive experimental results on supervised/semi-supervised visual recognition and object detection tasks demonstrate that ProCo consistently outperforms existing methods across various datasets.

Cross-Modal Consistency for Single-Modal MR Image Segmentation.

Multi-modal magnetic resonance (MR) image segmentation is an important task in disease diagnosis and treatment, but it is usually difficult to obtain multiple modalities for a single patient in clinical applications. To address these issues, a cross-modal consistency framework is proposed for a single-modal MR image segmentation. To enable single-modal MR image segmentation in the inference stage, a weighted cross-entropy loss and a pixel-level feature consistency loss are proposed to train the target network with the guidance of the teacher network and the auxiliary network. To fuse dual-modal MR images in the training stage, the cross-modal consistency is measured according to Dice similarity entropy loss and Dice similarity contrastive loss, so as to maximize the prediction similarity of the teacher network and the auxiliary network. To reduce the difference in image contrast between different MR images for the same organs, a contrast alignment network is proposed to align input images with different contrasts to reference images with a good contrast. Comprehensive experiments have been performed on a publicly available prostate dataset and an in-house pancreas dataset to verify the effectiveness of the proposed method. Compared to state-of-the-art methods, the proposed method can achieve better segmentation. The proposed image segmentation method can fuse dual-modal MR images in the training stage and only need one-modal MR images in the inference stage. The proposed method can be used in routine clinical occasions when only single-modal MR image with variable contrast is available for a patient.

Uncovering the Over-Smoothing Challenge in Image Super-Resolution: Entropy-Based Quantification and Contrastive Optimization.

PSNR-oriented models are a critical class of super-resolution models with applications across various fields. However, these models tend to generate over-smoothed images, a problem that has been analyzed previously from the perspectives of models or loss functions, but without taking into account the impact of data properties. In this paper, we present a novel phenomenon that we term the center-oriented optimization (COO) problem, where a model's output converges towards the center point of similar high-resolution images, rather than towards the ground truth. We demonstrate that the strength of this problem is related to the uncertainty of data, which we quantify using entropy. We prove that as the entropy of high-resolution images increases, their center point will move further away from the clean image distribution, and the model will generate over-smoothed images. Implicitly optimizing the COO problem, perceptual-driven approaches such as perceptual loss, model structure optimization, or GAN-based methods can be viewed. We propose an explicit solution to the COO problem, called Detail Enhanced Contrastive Loss (DECLoss). DECLoss utilizes the clustering property of contrastive learning to directly reduce the variance of the potential high-resolution distribution and thereby decrease the entropy. We evaluate DECLoss on multiple super-resolution benchmarks and demonstrate that it improves the perceptual quality of PSNR-oriented models. Moreover, when applied to GAN-based methods, such as RaGAN, DECLoss helps to achieve state-of-the-art performance, such as 0.093 LPIPS with 24.51 PSNR on 4× downsampled Urban100, validating the effectiveness and generalization of our approach.

DCANet: Dense Convolutional Attention Network for infrared small target detection

Infrared small target detection and tracking technology plays a essential role in systems like infrared warning, guidance, and missile defense. Efficiently and reliably detecting fast-moving, small targets is a key technological challenge. This study introduces a dense convolutional attention network as a potential solution to address the identified challenge. This network incorporates a dense convolutional interaction module that effectively minimizes information loss across different levels. Additionally, a simple, parameter-free attention module is introduced to adaptively extract features and enhance the emphasis on important information. Furthermore, a dense pixel contrastive learning module is introduced to capture more features of targets with similar sizes by constructing positive and negative sample pairs and designing a pixel contrastive loss function. Experimental validation was conducted on four representative datasets, demonstrating the success of our approach in terms of performance superiority over other methods.

Deep dual incomplete multi-view multi-label classification via label semantic-guided contrastive learning

Multi-view multi-label learning (MVML) aims to train a model that can explore the multi-view information of the input sample to obtain its accurate predictions of multiple labels. Unfortunately, a majority of existing MVML methods are based on the assumption of data completeness, making them useless in practical applications with partially missing views or some uncertain labels. Recently, many approaches have been proposed for incomplete data, but few of them can handle the case of both missing views and labels. Moreover, these few existing works commonly ignore potentially valuable information about unknown labels or do not sufficiently explore latent label information. Therefore, in this paper, we propose a label semantic-guided contrastive learning method named LSGC for the dual incomplete multi-view multi-label classification problem. Concretely, LSGC employs deep neural networks to extract high-level features of samples. Inspired by the observation of exploiting label correlations to improve the feature discriminability, we introduce a graph convolutional network to effectively capture label semantics. Furthermore, we introduce a new sample-label contrastive loss to explore the label semantic information and enhance the feature representation learning. For missing labels, we adopt a pseudo-label filling strategy and develop a weighting mechanism to explore the confidently recovered label information. We validate the framework on five standard datasets and the experimental results show that our method achieves superior performance in comparison with the state-of-the-art methods.

FIDWATCH: Federated incremental distillation for continuous monitoring of IoT security threats

The fast evolutions of Internet of Things (IoT) technologies have been accelerating their applicability in different sectors of life and becoming a pillar for sustainable development. However, this revolutionary expansion led to a substantial increase in attack surface, raising many concerns about security threats and their possible consequences. Machine learning has significantly contributed to designing intrusion detection systems (IDS) but suffers from critical limitations such as data privacy and sovereignty, data imbalance, concept drift, and catastrophic forgetting. This collectively makes existing IDSs an improper choice for securing IoT environments. This paper presents a federated learning approach called FIDWATCH to continuously monitor and detect a broad range of IoT security threats. The local side of FIDWATCH introduces contrastive focal loss to enhance the ability of the local model (teacher) to discriminate between diverse types of IoT security threats while putting an increased emphasis on hard-to-classify samples. A fine-grained Knowledge Distillation (KD) is introduced to allow the client to distill the required teacher's knowledge into a lighter, more compact model termed the pupil model. This greatly assists the competence and flexibility of the model in resource-constrained scenarios. Furthermore, an adaptive incremental updating method is introduced in FIDWATCH to allow the global model to exploit the distilled knowledge and refine the shared dataset. This helps generate global anchors for improving the robustness of the mode against the distributional shift, thereby improving model alignment and compliance with the dynamics of IoT security threats. Proof-of-concept simulations are performed on data from two public datasets (BoT-IoT and ToN-IoT), demonstrating the superiority of FIDWATCH over cutting-edge performance with an average f1-score of 97.07% and 95.63%, respectively.

WDICD: A novel simulated dataset and structure-aware framework for semantic segmentation of spacecraft component

The identification and localization of non-cooperative target, especially for the components such as solar panel and docking ring, is crucial for missions involving the on-orbit space services, navigation, and positioning. However, on-orbit spacecraft images are difficult to obtain, and it requires substantial effort to label them with pixel-wise accuracy for semantic segmentation. To address the above problems, we construct a simulated Wide-Depth Illumination-Composite Dataset (WDICD) and introduces an automatic pixel-level semantic annotation method. Furthermore, this paper proposes a structure-aware framework for semantic segmentation of spacecraft component, which introduces a boundary-aware auxiliary task and constructs a boundary-distance-aware contrast loss based on batch-wise grouping. The related boundary-aware tasks alleviate the prediction error of semantic segmentation near the boundary. Experimental results demonstrate that the proposed method achieves SOTA mIoU results in spacecraft component segmentation against complex backgrounds on two datasets. Particularly, the test results on spaceborne device show that the proposed framework balances precision and efficiency without generating additional computational complexity.

Fatigue feature extraction and classification algorithm of forehead single-channel electroencephalography signals

Aiming at the problem that the feature extraction ability of forehead single-channel electroencephalography (EEG) signals is insufficient, which leads to decreased fatigue detection accuracy, a fatigue feature extraction and classification algorithm based on supervised contrastive learning is proposed. Firstly, the raw signals are filtered by empirical modal decomposition to improve the signal-to-noise ratio. Secondly, considering the limitation of the one-dimensional signal in information expression, overlapping sampling is used to transform the signal into a two-dimensional structure, and simultaneously express the short-term and long-term changes of the signal. The feature extraction network is constructed by depthwise separable convolution to accelerate model operation. Finally, the model is globally optimized by combining the supervised contrastive loss and the mean square error loss. Experiments show that the average accuracy of the algorithm for classifying three fatigue states can reach 75.80%, which is greatly improved compared with other advanced algorithms, and the accuracy and feasibility of fatigue detection by single-channel EEG signals are significantly improved. The results provide strong support for the application of single-channel EEG signals, and also provide a new idea for fatigue detection research.

Skeleton-guided multi-scale dual-coordinate attention aggregation network for retinal blood vessel segmentation

Deep learning plays a pivotal role in retinal blood vessel segmentation for medical diagnosis. Despite their significant efficacy, these techniques face two major challenges. Firstly, they often neglect the severe class imbalance in fundus images, where thin vessels in the foreground are proportionally minimal. Secondly, they are susceptible to poor image quality and blurred vessel edges, resulting in discontinuities or breaks in vascular structures. In response, this paper proposes the Skeleton-guided Multi-scale Dual-coordinate Attention Aggregation (SMDAA) network for retinal vessel segmentation. SMDAA comprises three innovative modules: Dual-coordinate Attention (DCA), Unbalanced Pixel Amplifier (UPA), and Vessel Skeleton Guidance (VSG). DCA, integrating Multi-scale Coordinate Feature Aggregation (MCFA) and Scale Coordinate Attention Decoding (SCAD), meticulously analyzes vessel structures across various scales, adept at capturing intricate details, thereby significantly enhancing segmentation accuracy. To address class imbalance, we introduce UPA, dynamically allocating more attention to misclassified pixels, ensuring precise extraction of thin and small blood vessels. Moreover, to preserve vessel structure continuity, we integrate vessel anatomy and develop the VSG module to connect fragmented vessel segments. Additionally, a Feature-level Contrast (FCL) loss is introduced to capture subtle nuances within the same category, enhancing the fidelity of retinal blood vessel segmentation. Extensive experiments on three public datasets (DRIVE, STARE, and CHASE_DB1) demonstrate superior performance in comparison to current methods. The code is available at https://github.com/wangwxr/SMDAA_NET.

Boundary-aware semantic clustering network for segmentation of prostate zones from T2-weighted MRI

Objective. Automatic segmentation of prostatic zones from MRI can improve clinical diagnosis of prostate cancer as lesions in the peripheral zone (PZ) and central gland (CG) exhibit different characteristics. Existing approaches are limited in their accuracy in localizing the edges of PZ and CG. The proposed boundary-aware semantic clustering network (BASC-Net) improves segmentation performance by learning features in the vicinity of the prostate zonal boundaries, instead of only focusing on manually segmented boundaries. Approach. BASC-Net consists of two major components: the semantic clustering attention (SCA) module and the boundary-aware contrastive (BAC) loss. The SCA module implements a self-attention mechanism that extracts feature bases representing essential features of the inner body and boundary subregions and constructs attention maps highlighting each subregion. SCA is the first self-attention algorithm that utilizes ground truth masks to supervise the feature basis construction process. The features extracted from the inner body and boundary subregions of the same zone were integrated by BAC loss, which promotes the similarity of features extracted in the two subregions of the same zone. The BAC loss further promotes the difference between features extracted from different zones. Main results. BASC-Net was evaluated on the NCI-ISBI 2013 Challenge and Prostate158 datasets. An inter-dataset evaluation was conducted to evaluate the generalizability of the proposed method. BASC-Net outperformed nine state-of-the-art methods in all three experimental settings, attaining Dice similarity coefficients of 79.9% and 88.6% for PZ and CG, respectively, in the NCI-ISBI dataset, 80.5% and 89.2% for PZ and CG, respectively, in Prostate158 dataset, and 73.2% and 87.4% for PZ and CG, respectively, in the inter-dataset evaluation. Significance. As prostate lesions in PZ and CG have different characteristics, the zonal boundaries segmented by BASC-Net will facilitate prostate lesion detection.