Segmentation Method Research Articles

Development of advanced AI-based segmentation and prediction method for air entrainment in plunging water jets

Understanding air entrainment phenomena induced by plunging water jets is critical in the fields of nuclear and hydraulic engineering. Air entrainment is one of the key safety design parameters for nuclear systems. However, most existing studies rely on empirical correlations or curve-fitting models to estimate bubble penetration depth, and no agreed-upon calculation principle exists for varying jet conditions. To address these limitations, this research developed two advanced AI approaches: an improved YOLOv5 for segmenting air entrainment and the NSGA-III-BPNN method for predicting penetration depth. The improved YOLOv5 enables real-time segmentation and extraction of air entrainment motion and dynamics under diverse conditions, demonstrating high scalability and robustness. The penetration depth estimated using the improved YOLOv5 shows greater accuracy compared to conventional empirical correlationsand is more efficient than traditional image post-processing techniques for classifying shape regimes based on dynamic air entrainment patterns. To overcome the limitations of object segmentation, which typically relies on video or image data, the NSGA-III-BPNN method predicts maximum penetration depths with greater accuracy than YOLOv5, offering a more effective prediction model for air entrainment penetration depth. By leveraging advanced AI techniques, the research not only provides valuable segmentation data for refining computational fluid dynamics (CFD) modeling but also paves the way for significant advancements in both nuclear and hydraulic engineering.

A 3D Vision-Based Weld Seam Extraction Method for Arbitrarily Placed Spatial Planar Weldments

Vision-based weld seam extraction poses a significant challenge for weldments with complex spatial structures in automated welding. Existing research primarily focuses on identifying weld seams from weldments with given positions and postures, while practical weld path planning requires multiple weld seams identified within arbitrarily placed weldments. This paper proposes a methodology that identifies weld seams from arbitrarily placed spatial planar weldments in a single run. First, by introducing a turntable calibrated with respect to a 3D camera, we perform 3D reconstruction on an arbitrarily placed spatial planar weldment. Second, an improved RANSAC algorithm based on Euclidean clustering is proposed to carry out plane segmentation, focusing on segmentation accuracy. Finally, we present a novel weld seam extraction algorithm leveraging the half-edge data structure to efficiently compute weld seams from the segmented planes. The experiments conducted in this study demonstrate that the average segmentation errors (as an indirect indicator of weld seam extraction error) are reduced by 90.3% to 99.8% over conventional segmentation methods, and the standard deviations are reduced by 64.8% to 97.0%.

Automated seminal root angle measurement with corrective annotation.

Measuring seminal root angle is an important aspect of root phenotyping, yet automated methods are lacking. We introduce SeminalRootAngle, a novel open-source automated method that measures seminal root angles from images. To ensure our method is flexible and user-friendly we build on an established corrective annotation training method for image segmentation. We tested SeminalRootAngle on a heterogeneous dataset of 662 spring barley rhizobox images, which presented challenges in terms of image clarity and root obstruction. Validation of our new automated pipeline against manual measurements yielded a Pearson correlation coefficient of 0.71. We also measure inter-annotator agreement, obtaining a Pearson correlation coefficient of 0.68, indicating that our new pipeline provides similar root angle measurement accuracy to manual approaches. We use our new SeminalRootAngle tool to identify single nucleotide polymorphisms (SNPs) significantly associated with angle and length, shedding light on the genetic basis of root architecture.

A crop disease severity index derived from transfer learning and feature fusion using enhanced OPTICS algorithm

The adoption of automated methods for the identification and assessment of tomato-related disorders is highly sought-after in the agriculture sector. Using this technology is crucial for reducing wasteful spending, increasing the efficiency of treatments, and ultimately growing more resilient crops by reducing losses in agricultural output and maximising the effectiveness of these processes. An automated method has been suggested for accurately identifying and classifying diseases using a single photograph. The described method for disease detection in tomato plants makes use of a computer vision-based technique. Image processing, ML, and deep learning are just a few of the methods that this strategy uses. The goal of this approach is to prevent tomato crops from being damaged by various illnesses by reducing the need of conventional procedures. Bacterial spot, early blight, late blight, leaf mould, spider mites, target spot, spotted spider mite, mosaic virus, and yellow leaf curl are all examples of these illnesses. The following ten diseases frequently strike tomato crops in India. By utilising picture segmentation in combination with the Enhanced OPTICS algorithm (EOPTICSA), the affected area of the tomato plant may be precisely detected and defined after image pre-processing procedures have been used. It may be necessary to look for certain visual signs in order to diagnose the previously mentioned illnesses. The primary goal of this study was to evaluate the efficacy of the EOPTICSA method for detecting diseases in plant leaves. To eliminate the geometric features associated with colour, texture, and leaf arrangement in the provided plant pictures, image segmentation and edge detection methods are employed. Using these methods allows us to achieve our goal. Various efficacy measures are used to assess and provide a technique recommendation. This research shows that when performance metrics are used to implement these strategies, the suggested strategy outperforms the current methods in terms of accuracy, precision, and F1-score. The process of detecting sickness involves several consecutive steps. Capturing images, segmenting them, detecting edges, and determining the infection’s severity are all steps in this process. To accomplish the goal of recognising and categorising different types of diseases that might impact tomato plants, the method of transfer learning is employed. As soon as the problem is identified, it is recommended to take proactive measures to help individuals and organisations involved in agriculture address the effects of these disorders using appropriate measures.

Efficient and accurate semi-supervised semantic segmentation for industrial surface defects

Deep learning-based defect detection methods have gained widespread application in industrial quality inspection. However, limitations such as insufficient sample sizes, low data utilization, and issues with accuracy and speed persist. This paper proposes a semi-supervised semantic segmentation framework that addresses these challenges through perturbation invariance at both the image and feature space. The framework employs diverse perturbation cross-pseudo-supervision to reduce dependency on extensive labeled datasets. Our lightweight method incorporates edge pixel-level semantic information and shallow feature fusion to enhance real-time performance and improve the accuracy of defect edge detection and small target segmentation in industrial inspection. Experimental results demonstrate that the proposed method outperforms the current state-of-the-art (SOTA) semi-supervised semantic segmentation methods across various industrial scenarios. Specifically, our method achieves a mean Intersection over Union (mIoU) 3.11% higher than the SOTA method on our dataset and 4.39% higher on the public KolektorSDD dataset. Additionally, our semantic segmentation network matches the speed of the fastest network, U-net, while achieving a mIoU 2.99% higher than DeepLabv3Plus.

CelloType: A Unified Model for Segmentation and Classification of Tissue Images.

Cell segmentation and classification are critical tasks in spatial omics data analysis. We introduce CelloType, an end-to-end model designed for cell segmentation and classification of biomedical microscopy images. Unlike the traditional two-stage approach of segmentation followed by classification, CelloType adopts a multi-task learning approach that connects the segmentation and classification tasks and simultaneously boost the performance of both tasks. CelloType leverages Transformer-based deep learning techniques for enhanced accuracy of object detection, segmentation, and classification. It outperforms existing segmentation methods using ground-truths from public databases. In terms of classification, CelloType outperforms a baseline model comprised of state-of-the-art methods for individual tasks. Using multiplexed tissue images, we further demonstrate the utility of CelloType for multi-scale segmentation and classification of both cellular and non-cellular elements in a tissue. The enhanced accuracy and multi-task-learning ability of CelloType facilitate automated annotation of rapidly growing spatial omics data.

Artificial intelligence-based analysis of behavior and brain images in cocaine-self-administered marmosets

BackgroundThe sophisticated behavioral and cognitive repertoires of non-human primates (NHPs) make them suitable subjects for studies involving cocaine self-administration (SA) schedules. However, ethical considerations, adherence to the 3Rs principle (replacement, reduction and refinement), and other factors make it challenging to obtain NHPs individuals for research. Consequently, there is a need for methods that can comprehensively analyze small datasets using artificial intelligence (AI). New methodsWe employed AI to identify cocaine dependence patterns from collected data. First, we collected behavioral data from cocaine SA marmosets (Callithrix jacchus) to develop a dependence prediction model. SHapley Additive exPlanations (SHAP) values were used to demonstrate the importance of various variables. Additionally, we collected positron emission tomographic (PET) images showing dopamine transporter (DAT) binding potential and developed an algorithm for PET image segmentation. ResultsThe prediction model indicated that the Random Forest (RF) algorithm performed best, with an area under the curve (AUC) of 0.92. The top five variables influencing the model were identified using SHAP values. The PET image segmentation model achieved an accuracy of 0.97, a mean squared error of 0.02, an intersection over union (IoU) of 0.845, and a Dice coefficient of 0.913. Comparison with existing methods and conclusionUtilizing data from the marmoset SA experiment, we developed an ML-based dependence prediction model and analyzed variable importance rankings using SHAP. AI-based imaging segmentation methods offer a valuable tool for evaluating DAT availability in NHPs following chronic cocaine administration.

Multi-center external validation of an automated method segmenting and differentiating atypical lipomatous tumors from lipomas using radiomics and deep-learning on MRI

As differentiating between lipomas and atypical lipomatous tumors (ALTs) based on imaging is challenging and requires biopsies, radiomics has been proposed to aid the diagnosis. This study aimed to externally and prospectively validate a radiomics model differentiating between lipomas and ALTs on MRI in three large, multi-center cohorts, and extend it with automatic and minimally interactive segmentation methods to increase clinical feasibility. Three study cohorts were formed, two for external validation containing data from medical centers in the United States (US) collected from 2008 until 2018 and the United Kingdom (UK) collected from 2011 until 2017, and one for prospective validation consisting of data collected from 2020 until 2021 in the Netherlands. Patient characteristics, MDM2 amplification status, and MRI scans were collected. An automatic segmentation method was developed to segment all tumors on T1-weighted MRI scans of the validation cohorts. Segmentations were subsequently quality scored. In case of insufficient quality, an interactive segmentation method was used. Radiomics performance was evaluated for all cohorts and compared to two radiologists. The validation cohorts included 150 (54% ALT), 208 (37% ALT), and 86 patients (28% ALT) from the US, UK and NL. Of the 444 cases, 78% were automatically segmented. For 22%, interactive segmentation was necessary due to insufficient quality, with only 3% of all patients requiring manual adjustment. External validation resulted in an AUC of 0.74 (95% CI: 0.66, 0.82) in US data and 0.86 (0.80, 0.92) in UK data. Prospective validation resulted in an AUC of 0.89 (0.83, 0.96). The radiomics model performed similar to the two radiologists (US: 0.79 and 0.76, UK: 0.86 and 0.86, NL: 0.82 and 0.85). The radiomics model extended with automatic and minimally interactive segmentation methods accurately differentiated between lipomas and ALTs in two large, multi-center external cohorts, and in prospective validation, performing similar to expert radiologists, possibly limiting the need for invasive diagnostics. Hanarth fonds.

Energy curve based enhanced smell agent optimizer for optimal multilevel threshold selection of thermographic breast image segmentation

Multilevel thresholding image segmentation will subdivide an image into several meaningful regions or objects, which makes the image more informative and easier to analyze. Optimal multilevel thresholding approaches are extensively used for segmentation because they are easy to implement and offer low computational cost. Multilevel thresholding image segmentation is frequently performed using popular methods such as Otsu’s between-class variance and Kapur’s entropy. Numerous researchers have used evolutionary algorithms to identify the best multilevel thresholds based on the above approaches using histogram. This paper uses the Energy Curve (EC) based thresholding method instead of the histogram. Chaotic Bidirectional Smell Agent Optimization with Adaptive Control Strategy (ChBSAOACS), a powerful evolutionary algorithm, is developed and employed in this paper to create and execute an effective method for multilevel thresholding segmentation of breast thermogram images based on energy curves. The proposed algorithm was tested for viability on standard breast thermogram images. All experimental data are examined quantitatively and qualitatively to verify the suggested method’s efficacy.

RetVes segmentation: A pseudo-labeling and feature knowledge distillation optimization technique for retinal vessel channel enhancement

Recent advancements in retinal vessel segmentation, which employ transformer-based and domain-adaptive approaches, show promise in addressing the complexity of ocular diseases such as diabetic retinopathy. However, current algorithms face challenges in effectively accommodating domain-specific variations and limitations of training datasets, which fail to represent real-world conditions comprehensively. Manual inspection by specialists remains time-consuming despite technological progress in medical imaging, underscoring the pressing need for automated and robust segmentation techniques. Additionally, these methods have deficiencies in handling unlabeled target sets, requiring extra preprocessing steps and manual intervention, which hinders their scalability and practical application in clinical settings. This research introduces a novel framework that employs semi-supervised domain adaptation and contrastive pre-training to address these limitations. The proposed model effectively learns from target data by implementing a novel pseudo-labeling approach and feature-based knowledge distillation within a temporal convolutional network (TCN) and extracts robust, domain-independent features. This approach enhances cross-domain adaptation, significantly enhancing the model's versatility and performance in clinical settings. The semi-supervised domain adaptation component overcomes the challenges posed by domain shifts, while pseudo-labeling utilizes the data's inherent structure for enhanced learning, which is particularly beneficial when labeled data is scarce. Evaluated on the DRIVE and CHASE_DB1 datasets, which contain clinical fundus images, the proposed model achieves outstanding performance, with accuracy, sensitivity, specificity, and AUC values of 0.9792, 0.8640, 0.9901, and 0.9868 on DRIVE, and 0.9830, 0.9058, 0.9888, and 0.9950 on CHASE_DB1, respectively, outperforming current state-of-the-art vessel segmentation methods. The partitioning of datasets into training and testing sets ensures thorough validation, while extensive ablation studies with thorough sensitivity analysis of the model's parameters and different percentages of labeled data further validate its robustness.

Framework for electrochemical-mechanical behavior of all-solid-state batteries: From the reconstruction method to multi-physics and multi-scale modeling

All-solid-state batteries (ASSBs) are high-energy, high-power batteries. To enhance the understanding of the electrochemical-mechanical behavior in ASSBs across different scales, we developed a multi-physics and multi-scale modeling framework. This framework incorporates elastoplastic finite deformation and electrode microstructures of ASSBs, and the role of gradient plasticity in the governing equation for multiple physical fields was discussed. Utilizing X-ray computed tomography, we reconstructed the microstructure through a machine learning (ML)-informed image segmentation process. Our study clarifies the impact of electrode microstructures on concentration, stress, voltage, delamination and buckling from AM to electrode scale. Comparative analysis of the Feret diameter distribution of active materials (AMs) shows that ML-informed image segmentation outperforms two traditional segmentation methods. We observed that the asynchronous diffusion saturation of AMs, varying in shape and size, significantly influences the electrochemical-mechanical behavior of ASSBs, resulting in complicated debonding indices and J-integral distribution at the interface. The proposed upscaling homogenization procedure is demonstrated to be efficient for buckling analysis, with the shape mode closely matching existing experimental observations. These results shed light on the critical multi-physics and multi-scale coupling mechanisms in ASSBs.

Automated segmentation of midbrain nuclei using deep learning and multisequence MRI: a longitudinal study on iron accumulation with age

Abstract Elevated levels of brain iron, particularly within the basal ganglia, have been associated with cognitive and motor impairment in normal aging and neurodegenerative conditions. The subthalamic nucleus (STN), substantia nigra (SN), and red nucleus (RN), despite their high iron content and contribution to motor and cognitive processes, are less frequently studied. This oversight can largely be attributed to the challenges posed by in-vivo assessments of these small, deep-seated midbrain structures. We developed and validated an automated tool for the segmentation of the STN, SN, and RN. Multi-sequence magnetic resonance imaging (MRI) data including T1-weighted, FLAIR, Quantitative Susceptibility Mapping (QSM) and R2* alongside manual delineation on QSM images of 40 individuals, was used to train segmentation models based on nnU-Net deep-learning framework. A combination of QSM and FLAIR sequences was found to be optimal for structure segmentation (mean Dice scores of 0.84, 0.91 and 0.94 for STN, SN and RN, respectively). We next applied the automated segmentation method to an independent 3-year longitudinal dataset including 175 healthy adults (age range at baseline: 20-79 years old). Structural equation modelling was used to assess iron accumulation over time using age, sex, baseline iron and regional volume as factors of interest. Cross-sectionally, older age was linearly associated with higher iron load in SN and STN; the association was non-linear in RN. Longitudinally, results indicated significant iron accumulation in the STN (Mean increase = 0.02, p = 0.005) and SN (Mean increase = 0.035, p = 0.001), but not in the RN (Mean increase = 0.015, p = 0.2). Our findings demonstrated high performance of nnU-Net in automated segmentation, and advanced our understanding of iron accumulation in midbrain nuclei in aging.

Dataset-level color augmentation and multi-scale exploration methods for polyp segmentation

Automatic segmentation of polyps from colonoscopy images plays a critical role in early screening and treatment of colorectal cancer. Although deep learning methods have made significant progress, precise polyp segmentation faces two challenges: (1) the imbalance of color appearances in the limited training dataset hinders the generalization of the model, and (2) polyps have the diverse scales, locations and shapes with blurred boundary. To address the issues, Dataset-Level Color Augmentation (DLCA) and Convolutional Multi-scale Attention Module (CMAM) are proposed. DLCA employs the dataset-level color knowledge and generate new color appearances, to avoid the model learning false associations with colors. Meanwhile, CMAM simultaneously explores the region and boundary clues and model multi-scale context, which improves the accuracy of polyp localization and fine-grained segmentation. We conduct comprehensive experiments and compare our network with state-of-the-art methods. The proposed model is superior on multiple polyp datasets, especially on ETIS, where mDice and mIoU reach 0.839 and 0.766. Furthermore, it is validated that DLCA can be widely applied to most polyp segmentation methods, and CMAM is practical and plug-and-play. Finally, the model demonstrates promising generalization to breast ultrasound and skin lesion segmentation.

HPNet: Text Detection Network with Hybrid Attention and Pixel Aggregation for Irregularly-Shaped Nearby Texts

Scene text detection is a challenging topic in computer vision, characterized by complex illumination, irregular shape, and arbitrary size. While recent advancements have been made in scene text detection, it remains difficult to simultaneously distinguish nearby text and accommodate irregularly shaped text. Therefore, this paper introduces HPNet, an enhanced text detector, based on the segmentation method that predicts two-scale results. To improve the shape robustness, the Hybrid Attentional Feature Fusion (HAFF) module is integrated into Feature Pyramid Networks (FPN) to dynamically perform feature fusion. Additionally, to distinguish nearby text, the model predicts the text region covering text instances and the text kernel covering the central region of the text. The improved Pixel Aggregation (PA) algorithm is then utilized to guide the expansion from the text kernel to the text region. Experiments on IC15, Total-Text, and CTW1500 validate the effectiveness of these improvements and the superiority of HPNet. Compared with the previous method PSENet for nearby texts, the proposed HPNet has improved inference speed by 63.6% and F-measure metric by 2.6%, 3.7%, and 2.5% on three datasets, respectively.

Implicit Is Not Enough: Explicitly Enforcing Anatomical Priors inside Landmark Localization Models.

The task of localizing distinct anatomical structures in medical image data is an essential prerequisite for several medical applications, such as treatment planning in orthodontics, bone-age estimation, or initialization of segmentation methods in automated image analysis tools. Currently, Anatomical Landmark Localization (ALL) is mainly solved by deep-learning methods, which cannot guarantee robust ALL predictions; there may always be outlier predictions that are far from their ground truth locations due to out-of-distribution inputs. However, these localization outliers are detrimental to the performance of subsequent medical applications that rely on ALL results. The current ALL literature relies heavily on implicit anatomical constraints built into the loss function and network architecture to reduce the risk of anatomically infeasible predictions. However, we argue that in medical imaging, where images are generally acquired in a controlled environment, we should use stronger explicit anatomical constraints to reduce the number of outliers as much as possible. Therefore, we propose the end-to-end trainable Global Anatomical Feasibility Filter and Analysis (GAFFA) method, which uses prior anatomical knowledge estimated from data to explicitly enforce anatomical constraints. GAFFA refines the initial localization results of a U-Net by approximately solving a Markov Random Field (MRF) with a single iteration of the sum-product algorithm in a differentiable manner. Our experiments demonstrate that GAFFA outperforms all other landmark refinement methods investigated in our framework. Moreover, we show that GAFFA is more robust to large outliers than state-of-the-art methods on the studied X-ray hand dataset. We further motivate this claim by visualizing the anatomical constraints used in GAFFA as spatial energy heatmaps, which allowed us to find an annotation error in the hand dataset not previously discussed in the literature.

Pricing and Hedging Contingent Claims by Entropy Segmentation and Fenchel Duality

We present a new approach to the problem of characterizing and choosing equivalent martingale pricing measures for a contingent claim, in a finite-state incomplete market. This is the entropy segmentation method achieved by means of convex programming, thanks to which we divide the claim no-arbitrage prices interval into two halves, the buyer’s and the seller’s prices at successive entropy levels. Classical buyer’s and seller’s prices arise when the entropy level approaches 0. Next, we apply Fenchel duality to these primal programs to characterize the hedging positions, unifying in the same expression the cases of super (resp. sub) replication (arising when the entropy approaches 0) and partial replication (when entropy tends to its maximal value). We finally apply linear programming to our hedging problem to find in a price slice of the dual feasible set an optimal partial replicating portfolio with minimal CVaR. We apply our methodology to a cliquet style guarantee, using Heston’s dynamic with parameters calibrated on EUROSTOXX50 index quoted prices of European calls. This way prices and hedging positions take into account the volatility risk.

Structural tensor and frequency guided semi-supervised segmentation for medical images.

The method of semi-supervised semantic segmentation entails training with a limited number of labeled samples alongside many unlabeled samples, aiming to reduce dependence on pixel-level annotations. Most semi-supervised semantic segmentation methods primarily focus on sample augmentation in spatial dimensions to reduce the shortage of labeled samples. These methods tend to ignore the structural information of objects. In addition, frequency-domain information also supplies another perspective to evaluate information from images, which includes different properties compared to the spatialdomain. In this study, we attempt to answer these two questions: (1) is it helpful to provide structural information of objects in semi-supervised semantic segmentation tasks for medical images? (2) is it more effective to evaluate the segmentation performance in the frequency domain compared to the spatial domain for semi-supervised medical image segmentation? Therefore, we seek to introduce structural and frequency information to improve the performance of semi-supervised semantic segmentation for medicalimages. We present a novel structural tensor loss (STL) to guide feature learning on the spatial domain for semi-supervised semantic segmentation. Specifically, STL utilizes the structural information encoded in the tensors to enforce the consistency of objects across spatial regions, thereby promoting more robust and accurate feature extraction. Additionally, we proposed a frequency-domain alignment loss (FAL) to enable the neural networks to learn frequency-domain information across different augmented samples. It leverages the inherent patterns present in frequency-domain representations to guide the network in capturing and aligning features across diverse augmentation variations, thereby enhancing the model's robustness for the inputting variations. We conduct our experiments on three benchmark datasets, which include MRI (ACDC) for cardiac, CT (Synapse) for abdomen organs, and ultrasound image (BUSI) for breast lesion segmentation. The experimental results demonstrate that our method outperforms state-of-the-art semi-supervised approaches regarding the Dice similaritycoefficient. We find the proposed approach could improve the final performance of the semi-supervised medical image segmentation task. It will help reduce the need for medical image labels. Our code will are available at https://github.com/apple1986/STLFAL.

Sea–Land Segmentation of Remote-Sensing Images with Prompt Mask-Attention

Remote-sensing technology has gradually become one of the most important ways to extract sea–land boundaries due to its large scale, high efficiency, and low cost. However, sea–land segmentation (SLS) is still a challenging problem because of data diversity and inconsistency, “different objects with the same spectrum” or “the same object with different spectra”, and noise and interference problems, etc. In this paper, a new sea–land segmentation method (PMFormer) for remote-sensing images is proposed. The contributions are mainly two points. First, based on Mask2Former architecture, we introduce the prompt mask by normalized difference water index (NDWI) of the target image and prompt encoder architecture. The prompt mask provides more reasonable constraints for attention so that the segmentation errors are alleviated in small region boundaries and small branches, which are caused by insufficiency of prior information by large data diversity or inconsistency. Second, for the large intra-class difference problem in the foreground–background segmentation in sea–land scenes, we use deep clustering to simplify the query vectors and make them more suitable for binary segmentation. Then, traditional NDWI and eight other deep-learning methods are thoroughly compared with the proposed PMFormer on three open sea–land datasets. The efficiency of the proposed method is confirmed, after the quantitative analysis, qualitative analysis, time consumption, error distribution, etc. are presented by detailed contrast experiments.

Mask-free Iterative Refinement Network for weakly-supervised Few-shot Semantic Segmentation

The dependence on densely labeled samples in Few-shot Semantic Segmentation (FSS) poses challenges in terms of sample annotation. This paper introduces a more challenging few-shot semantic segmentation task that only utilizes intra-class images as weak supervision. To address this task, we propose a Mask-free Iterative Refinement Network consisting of a mask generation module (MGM) and an iterative refinement module (IRM), respectively addressing the inherent two challenges of locating segmented objects and deriving class-specific features in the absence of support mask and semantic labels. MGM generates pseudo-masks for the support image without requiring any training, which provides a rough estimation of the object locations, serving as an initial guidance for segmentation. IRM is designed to capture the co-occurrence class-specific information between the support and query images for segmentation and progressively refine the predicted mask in a bootstrap manner with the guidance of the class-specific information. Experimental results on three FSS benchmarks, i.e., FSS-1000, PASCAL-5i, and COCO-20i demonstrate that our proposed method achieves comparable or decent performance compared to existing zero-shot and supervised segmentation methods.

DBCAN: DFormer-Based Cross-Attention Network for RGB Depth Semantic Segmentation

Existing RGB-depth semantic segmentation methods primarily rely on symmetric two-stream Convolutional Neural Networks (CNNs) to extract RGB and spatial features separately. However, these architectures have limitations in incorporating spatial features and efficiently fusing RGB and depth information. In this study, we propose a novel architecture called the DFormer-Based Cross-Attention Network (DBCAN), which utilizes DFormer as an encoder for feature extraction and integrates several modifications to address these challenges. While DFormer is leveraged for its strong feature extraction capabilities, our modifications in the decoder focus on improving cross-modal fusion and spatial feature incorporation. We introduce three modules in the decoding process: the Object-Region Generated Module (ORGM), the Feature-Region Relation Module (FRRM), and the Spatial-Semantic Fusion Module (SSFM), which enhance feature interaction and segmentation accuracy. Experimental results on the NYUDepthv2 and SUN-RGBD datasets demonstrate that DBCAN achieves state-of-the-art performance, highlighting the effectiveness of our architectural enhancements in overcoming the limitations of existing models.