U-shaped Network Research Articles

Robotic Grasping Detection Algorithm Based on 3D Vision Dual-Stream Encoding Strategy

The automatic generation of stable robotic grasping postures is crucial for the application of computer vision algorithms in real-world settings. This task becomes especially challenging in complex environments, where accurately identifying the geometric shapes and spatial relationships between objects is essential. To enhance the capture of object pose information in 3D visual scenes, we propose a planar robotic grasping detection algorithm named SU-Grasp, which simultaneously focuses on local regions and long-distance relationships. Built upon a U-shaped network, SU-Grasp introduces a novel dual-stream encoding strategy using the Swin Transformer combined with spatial semantic enhancement. Compared to existing baseline methods, our algorithm achieves superior performance across public datasets, simulation tests, and real-world scenarios, highlighting its robust understanding of complex spatial environments.

MFHARFNet: multi-branch feature hybrid and adaptive receptive field network for image segmentation

Abstract Accurate segmentation of medical images is crucial for disease diagnosis and understanding disease changes. Deep learning methods, utilizing encoder-decoder structures, have demonstrated cutting-edge performance in various medical image segmentation tasks. However, the pooling operation in the encoding stage results in feature loss, which makes the network lack the ability to fuse multi-scale information at different levels, hinders its effective perception of multi-scale information, and leads to poor segmentation performance. Drawing inspiration from the U-shaped network, this study introduces a multi-branch feature hybrid attention and adaptive receptive field network (MFHARFNet) for medical image segmentation. Building upon the encoder-decoder framework, we initially devise a multi-branch feature hybrid attention module (MFHAM) to seamlessly integrate feature maps of varying scales, capturing both fine-grained features and coarse-grained semantics across the entire scale. Furthermore, we redesign the skip connection to amalgamate feature information from different branches in the encoder stage and efficiently transmit it to the decoder, providing the decoder with global context feature maps at different levels. Finally, the adaptive receptive field (ARF) module is introduced in the decoder feature reconstruction stage to adapt and focus on related fields, ensuring the model’s adaptation to different segmentation target features, and achieving different weights for the output of different convolution kernels to improve segmentation performance. We comprehensively evaluate our method on medical image segmentation tasks, by using four public datasets across CT and MRI. Remarkably, MFHARFNet method consistently outperforms other state-of-the-art methods, exceeding UNet by 2.1%, 0.9%, 6.6% and 1.0% on Dice on ATLAS, LiTs, BraTs2019 and Spine and intervertebral disc datasets, respectively. In addition, MFHARFNet minimizes network parameters and computational complexity as much as possible. The source codes are in https://github.com/OneHundred99/MFHARFNet.

Classification of Hyperspectral Images of Explosive Fragments Based on Spatial–Spectral Combination

The identification and recovery of explosive fragments can provide a reference for the evaluation of explosive power and the design of explosion-proof measures. At present, fragment detection usually uses a few bands in the visible light or infrared bands for imaging, without fully utilizing multi-band spectral information. Hyperspectral imaging has high spectral resolution and can provide multidimensional reference information for the fragments to be classified. Therefore, this article proposed a spatial–spectral joint method for explosive fragment classification by combining hyperspectral imaging technology. In a laboratory environment, this article collected hyperspectral images of explosion fragments scattered in simulated scenes. In order to extract effective features from redundant spectral information and improve classification accuracy, this paper adopted a classification framework based on deep learning. This framework used a convolutional neural network–bidirectional long short-term memory network (CNN-BiLSTM) as the spectral information classification model and a U-shaped network (U-Net) as the spatial segmentation model. The experimental results showed that the overall accuracy exceeds 95.2%. The analysis results indicated that the method of spatial–spectral combination can accurately identify explosive fragment targets. It validated the feasibility of using hyperspectral imaging for explosive fragment classification in laboratory environments. Due to the complex environment of the actual explosion site, this study still needs to be validated in outdoor environments. Our next step is to use airborne hyperspectral imaging to identify explosive fragments in outdoor environments.

Quadratic Unet for seismic random noise attenuation

In the field of seismic data processing, seismic denoising is essential to improve the quality of seismic data. Various deep learning methods have been proposed, showing promising performance for seismic denoising. However, most deep learning methods are based on lin ear neurons, resulting in limited expressive ability for complex seismic signals. To enhance denoising capability using non-linear neurons, we propose a supervised quadratic U-shaped network (QUnet) for seismic random noise attenuation. The quadratic neurons in QUnet are represented by a second-order polynomial of input data and weighted parameters. Nu merical synthetic seismic data experiments show that the proposed QUnet method achieves higher signal-to-noise ratio results and preserves the continuity of reflectors more effectively. We have also tested the effectiveness of QUnet on both prestack field data and post-stack seismic data. The detailed structures and weak signals of seismic data are better preserved by our proposed QUnet method.

Semantic segmentation of optical satellite images for the illegal construction detection using transfer learning

As urban populations grow, the demand for new infrastructure intensifies. However, some builders often neglect to obtain necessary permits, resulting in unstable structures that endanger public safety and disrupt city planning. Illegal construction poses significant challenges to economic development and social harmony. To tackle this issue, there is an urgent need for an automated illegal building monitoring system that can accurately identify and notify authorities about unlawful constructions. Current methods are often inefficient due to limited data access, inadequate analysis, and lack of automation.Recent technological advancements suggest that using satellite imagery alongside semantic segmentation is a highly effective approach for detecting illegal buildings. This process not only ensures the safety and compliance of structures but also enhances overall regulatory enforcement. This research distinguishes itself from previous studies by primarily focusing on illegal construction. It employs a diverse range of machine learning models, including the U-Shaped Network and Visual Geometry Group, and incorporates customized evaluation metrics that are not typically found in earlier research. Additionally, it utilizes real-world data, enhancing its practical relevance, and features IoT integration for real-time monitoring capabilities.We chose U-Net as our architecture due to its symmetrical design, which facilitates effective feature extraction. Its skip connections play a crucial role in preserving important features during the segmentation process. U-Net is known for its high accuracy and adaptability across various domains, making it versatile for our needs. Additionally, it performs well even with limited data, demonstrating robustness in diverse conditions. Furthermore, its compatibility with transfer learning enhances its applicability. For all these reasons, U-Net was selected as our model.Our experiments revealed that the Unet_mini model outperformed all others, demonstrating its effectiveness in addressing illegal construction challenges in urban areas. Qualitative results further illustrate the model's robustness, showing a significant reduction in false positives and improved accuracy in identifying illegal structures. This comprehensive approach not only improves detection rates but also contributes to more efficient regulatory enforcement, thereby promoting safer urban environments.This research distinguishes itself from previous studies by primarily focusing on illegal construction detection rather than general segmentation tasks. It employs a diverse range of machine learning models, including the U-Shaped Network and Visual Geometry Group, and integrates customized evaluation metrics that are not typically found in earlier research. Furthermore, it utilizes real-world data, enhancing its practical relevance, and incorporates IoT integration for real-time monitoring capabilities, setting it apart from existing studies.

Automated Grasp Labeling and Detection Framework with Pixel-Level Precision

Due to the differences in shape, material, and color of objects, the detection of planar grasps by robots remains challenging. Traditional methods rely on discrete grasp configurations for annotation, ignoring many possible grasp configurations. This leads to poor network generalization, making it challenging to handle a diverse range of grasped objects. Manual reannotation to continuous labels can effectively address this issue but comes with a significant cost. Therefore, this paper proposes a Pixel-level Grasp framework. Firstly, APGLG can automatically generate pixel-level grasp labels and discrete labels into continuous labels, effectively increasing the information content of individual data and improving the network generalization performance. Then, we propose the Max-Grasp-Net, built on the Multi-axis Vision Transformer and Dynamic Convolution Decomposition, to construct a U-shaped Network structure. A grasp decoder is incorporated, and deep supervision is applied to enhance network generalization. On the Cornell dataset, we achieve the best results, a grasping accuracy of 99.55%, an average success rate of 97.92% in single-object grasping, and 95.83% in multi-object grasping. We verified the effectiveness of our label generation algorithms and network innovation through actual grasp experiments.

MIMO-Uformer: A Transformer-Based Image Deblurring Network for Vehicle Surveillance Scenarios

Motion blur is a common problem in the field of surveillance scenarios, and it obstructs the acquisition of valuable information. Thanks to the success of deep learning, a sequence of CNN-based architecture has been designed for image deblurring and has made great progress. As another type of neural network, transformers have exhibited powerful deep representation learning and impressive performance based on high-level vision tasks. Transformer-based networks leverage self-attention to capture the long-range dependencies in the data, yet the computational complexity is quadratic to the spatial resolution, which makes transformers infeasible for the restoration of high-resolution images. In this article, we propose an efficient transformer-based deblurring network, named MIMO-Uformer, for vehicle-surveillance scenarios. The distinct feature of the MIMO-Uformer is that the basic-window-based multi-head self-attention (W-MSA) of the Swin transformer is employed to reduce the computational complexity and then incorporated into a multi-input and multi-output U-shaped network (MIMO-UNet). The performance can benefit from the operation of multi-scale images by MIMO-UNet. However, most deblurring networks are designed for global blur, while local blur is more common under vehicle-surveillance scenarios since the motion blur is primarily caused by local moving vehicles. Based on this observation, we further propose an Intersection over Patch (IoP) factor and a supervised morphological loss to improve the performance based on local blur. Extensive experiments on a public and a self-established dataset are carried out to verify the effectiveness. As a result, the deblurring behavior based on PSNR is improved at least 0.21 dB based on GOPRO and 0.74 dB based on the self-established datasets compared to the existing benchmarks.

Multi-task OCTA image segmentation with innovative dimension compression

Optical Coherence Tomography Angiography (OCTA) plays a crucial role in the early detection and continuous monitoring of ocular diseases, which relies on accurate multi-tissue segmentation of retinal images. Existing OCTA segmentation methods typically focus on single-task designs that do not fully utilize the information of volume data in these images. To bridge this gap, our study introduces H2C-Net, a novel network architecture engineered for simultaneous and precise segmentation of various retinal structures, including capillaries, arteries, veins, and the fovea avascular zone (FAZ). At its core, H2C-Net consists of a plug-and-play Height-Channel Module (H2C) and an Enhanced U-shaped Network (GPC-Net). The H2C module cleverly converts the height information of the OCTA volume data into channel information through the Squeeze operation, realizes the lossless dimensionality reduction from 3D to 2D, and provides the "Soft layering" information by unidirectional pooling. Meanwhile, in order to guide the network to focus on channels for training, U-Net is enhanced with group normalization, channel attention mechanism, and Parametric Rectified Linear Unit (PReLU), which reduces the dependence on batch size and enhances the network's ability to extract salient features. Extensive experiments on two subsets of the publicly available OCTA-500 dataset have shown that H2C-Net outperforms existing state-of-the-art methods. It achieves average Intersection over Union (IoU) scores of 82.84 % and 88.48 %, marking improvements of 0.81 % and 1.59 %, respectively. Similarly, the average Dice scores are elevated to 90.40 % and 93.76 %, exceeding previous benchmarks by 0.42 % and 0.94 %. The proposed H2C-Net exhibits excellent performance in OCTA image segmentation, providing an efficient and accurate multi-task segmentation solution in ophthalmic diagnostics. The code is publicly available at: https://github.com/IAAI-SIT/H2C-Net.

Visual detection of drilling robot position for rockburst prevention in mining processing by a new image dehazing method

The positioning of drill pipe based on visual detection is a crucial aspect in achieving unmanned operation of drilling robot for rockburst prevention. However, the images directly collected from mechanized mining face are always polluted. In order to eliminate the interference of dust and haze in the image, an image dehazing method based on improved CycleGAN model is proposed in this paper. Firstly, a pipe image dehazing dataset for the rockburst prevention drilling robot is collected and established. Moreover, a generator architecture with a multi-scale U-shaped network structure is designed to improve the quality and accuracy of image recovery. A new reconstruction block is designed and an SK fusion layer is introduced to improve the feature extraction capability of the model, and a MU-CycleGAN network structure is constructed. Finally, an experimental platform for drill pipe image dehazing of the drilling robot for rockburst prevention is set up in the intelligent mining equipment laboratory. Experimental results showed that the PSNR and SSIM of the image dehazing model achieved 27.04 and 0.946, and the success rate of drill pipe grabbing has increased by 13.75%. Experimental results reveal that the proposed framework achieves superior image enhancement performance than the comparison algorithms.

MixUNETR: A U-shaped network based on W-MSA and depth-wise convolution with channel and spatial interactions for zonal prostate segmentation in MRI

Magnetic resonance imaging (MRI) plays a pivotal role in diagnosing and staging prostate cancer. Precise delineation of the peripheral zone (PZ) and transition zone (TZ) within prostate MRI is essential for accurate diagnosis and subsequent artificial intelligence-driven analysis. However, existing segmentation methods are limited by ambiguous boundaries, shape variations and texture complexities between PZ and TZ. Moreover, they suffer from inadequate modeling capabilities and limited receptive fields. To address these challenges, we propose a Enhanced MixFormer, which integrates window-based multi-head self-attention (W-MSA) and depth-wise convolution with parallel design and cross-branch bidirectional interaction. We further introduce MixUNETR, which use multiple Enhanced MixFormers as encoder to extract features from both PZ and TZ in prostate MRI. This augmentation effectively enlarges the receptive field and enhances the modeling capability of W-MSA, ultimately improving the extraction of both global and local feature information from PZ and TZ, thereby addressing mis-segmentation and challenges in delineating boundaries between them. Extensive experiments were conducted, comparing MixUNETR with several state-of-the-art methods on the Prostate158, ProstateX public datasets and private dataset. The results consistently demonstrate the accuracy and robustness of MixUNETR in MRI prostate segmentation. Our code of methods is available at https://github.com/skyous779/MixUNETR.git.

MWIR image deep denoising reconstruction based on single-pixel imaging

Single-pixel imaging enables cost-effective and highly sensitive mid-wave infrared imaging, with the added convenience facilitated by deep learning algorithms. However, most networks train separate models depending on compression rates, frequently neglecting real-world infrared noise, leading to inconveniences and suboptimal reconstruction performance. To address these issues, we propose to integrate intrinsic and external deep denoising priors for the reconstruction of compressive mid-wave infrared images from single-pixel measurements. The generalized alternating projection is unfolded into a designed network to harness the benefits of robustness and internal priors. Furthermore, to mitigate real-world noise in measurements, a Swin Transformer U-shaped network that embeds noise variance is introduced to learn external denoise prior. A comparative analysis of simulated and real infrared single-pixel imaging in experiments highlights the promising performance of the proposed method.

Automatic Segmentation of Ultrasound-Guided Quadratus Lumborum Blocks Based on Artificial Intelligence.

Ultrasound-guided quadratus lumborum block (QLB) technology has become a widely used perioperative analgesia method during abdominal and pelvic surgeries. Due to the anatomical complexity and individual variability of the quadratus lumborum muscle (QLM) on ultrasound images, nerve blocks heavily rely on anesthesiologist experience. Therefore, using artificial intelligence (AI) to identify different tissue regions in ultrasound images is crucial. In our study, we retrospectively collected 112 patients (3162 images) and developed a deep learning model named Q-VUM, which is a U-shaped network based on the Visual Geometry Group 16 (VGG16) network. Q-VUM precisely segments various tissues, including the QLM, the external oblique muscle, the internal oblique muscle, the transversus abdominis muscle (collectively referred to as the EIT), and the bones. Furthermore, we evaluated Q-VUM. Our model demonstrated robust performance, achieving mean intersection over union (mIoU), mean pixel accuracy, dice coefficient, and accuracy values of 0.734, 0.829, 0.841, and 0.944, respectively. The IoU, recall, precision, and dice coefficient achieved for the QLM were 0.711, 0.813, 0.850, and 0.831, respectively. Additionally, the Q-VUM predictions showed that 85% of the pixels in the blocked area fell within the actual blocked area. Finally, our model exhibited stronger segmentation performance than did the common deep learning segmentation networks (0.734 vs. 0.720 and 0.720, respectively). In summary, we proposed a model named Q-VUM that can accurately identify the anatomical structure of the quadratus lumborum in real time. This model aids anesthesiologists in precisely locating the nerve block site, thereby reducing potential complications and enhancing the effectiveness of nerve block procedures.

An illumination-guided dual attention vision transformer for low-light image enhancement

Existing Retinex-based low-light image enhancement methods often overlook corruptions hidden in darkness or pattern collapse caused by the lit-up process. Recent deep learning approaches suggest the use of U-shaped networks with Vision in Transformer (VIT) to address these issues. However, most VIT-based methods focus on channel modeling to reduce expensive computational costs, but in which the restored images suffer from spatial illumination inconsistencies, artifacts, and blurriness. To end for this, we propose a novel one-stage Retinex-based Illumination-Guided Dual transformer model (IGDFormer) to lit up low-light images. The model consists of an estimator and a restorer. The estimator generates a light-up feature map and a lit-up image through pure Convolutional Neural Networks (CNNs). The restorer denoises the lit-up image with a U-shaped network equipped with an Illumination-Guided Dual Attention Block (IGDAB). Specifically, IGDAB consists of cascaded channel attention and window attention that achieves cross-channel/spatial modeling. Channel attention alleviates inductive bias through the CNN-Transformer collaborative layer, and window attention introduces spatial domains knowledge by partitioning and shifting. In addition, the light-up features act as values guide the interaction modeling of non-local illumination intensities in both the channel and spatial domains. Extensive experiments were conducted on 5 low-light image enhancement benchmarks and 1 dark object detection benchmark, which demonstrate that the efficacy of our IGDFormer and its superiority in restoring spatial details compared to other state-of-the-art methods. The code is available at https://github.com/YanJieWen/IGDFormer-light-up-dark.

Embedded U-shaped network with cross-hierarchical feature adaptation fusion for remote sensing image haze removal

ABSTRACT Recently, U-Net architecture has been extensively explored for remote sensing (RS) image haze removal, demonstrating remarkable performance. However, most existing RS image haze removal methods based on the U-Net fail to fully utilize feature information extracted in the encoding layers during decoding, resulting in unsatisfactory haze removal results. Moreover, most haze removal algorithms neglect the interaction of cross-level feature information, which is particularly important for scene recovery and context information constraints. To address these issues, this paper presents an Embedded U-shaped Network with Cross-Hierarchical Feature Adaptation Fusion Network (EUCHA). Specifically, an Embedded U-shaped Network Framework (EUNF) is proposed to fuse the features extracted from different scale encoding layers as supplementary information into the corresponding decoding layers by embedding multiple parallel U-shaped subnetworks, which alleviates feature information dilution during the decoding process and enhances the network receptive field. Furthermore, we propose a Cross-Hierarchical Feature Adaptive Fusion (CHAF) module, which effectively and adaptively fuses multiple adjacent hierarchical features extracted by channel attention and pixel attention through learnable factors to enhance the expressive ability of features. Experiments were conducted on both real and synthetic datasets and compared with the nine most advanced algorithms. The results showed that EUCHA algorithm performs better in removing artefacts in both RS images and natural images. The source code is available at https://github.com/GongHongY/EUCHA.

Asym-UNet: An asymmetric U-shape Network for breast lesions ultrasound images segmentation

Precise segmentation of breast ultrasound images is essential for early breast cancer screening. However, the segmentation process is challenging due to the diverse morphology of tumors, blurred boundaries, and the similar grey intensity distribution between lesions and normal tissues. To overcome these obstacles, we propose the Asym-UNet, an asymmetric U-shaped network tailored for segmenting breast lesions with high accuracy and reliability. This network features a Multi-branch Residual Encoder (MRE), an External Attention Module (EAM), and a Boundary Detection Module (BDM). The MRE is designed to capture distinctive representations of breast lesions, facilitating accurate subsequent segmentation. The EAM directs the network to focus on the lesion areas while filtering out irrelevant information, thereby enhancing diagnostic precision. Furthermore, the BDM, incorporated to refine the boundaries of predicted masks, provides additional supervision and guidance through a multi-level prediction layer. We conducted evaluations of the Asym-UNet against thirteen other segmentation methods using two publicly available datasets. The results indicate that our network achieves state-of-the-art performance in breast tumor segmentation.

A Novel Skip-Connection Strategy by Fusing Spatial and Channel Wise Features for Multi-Region Medical Image Segmentation.

Recent methods often introduce attention mechanisms into the skip connections of U-shaped networks to capture features. However, these methods usually overlook spatial information extraction in skip connections and exhibit inefficiency in capturing spatial and channel information. This issue prompts us to reevaluate the design of the skip-connection mechanism and propose a new deep-learning network called the Fusing Spatial and Channel Attention Network, abbreviated as FSCA-Net. FSCA-Net is a novel U-shaped network architecture that utilizes the Parallel Attention Transformer (PAT) to enhance the extraction of spatial and channel features in the skip-connection mechanism, further compensating for downsampling losses. We design the Cross-Attention Bridge Layer (CAB) to mitigate excessive feature and resolution loss when downsampling to the lowest level, ensuring meaningful information fusion during upsampling at the lowest level. Finally, we construct the Dual-Path Channel Attention (DPCA) module to guide channel and spatial information filtering for Transformer features, eliminating ambiguities with decoder features and better concatenating features with semantic inconsistencies between the Transformer and the U-Net decoder. FSCA-Net is designed explicitly for fine-grained segmentation tasks of multiple organs and regions. Our approach achieves over 48% reduction in FLOPs and over 32% reduction in parameters compared to the state-of-the-art method. Moreover, FSCA-Net outperforms existing segmentation methods on seven public datasets, demonstrating exceptional performance.

SSGU-CD: A combined semantic and structural information graph U-shaped network for document-level Chemical-Disease interaction extraction

Document-level interaction extraction for Chemical-Disease is aimed at inferring the interaction relations between chemical entities and disease entities across multiple sentences. Compared with sentence-level relation extraction, document-level relation extraction can capture the associations between different entities throughout the entire document, which is found to be more practical for biomedical text information. However, current biomedical extraction methods mainly concentrate on sentence-level relation extraction, making it difficult to access the rich structural information contained in documents in practical application scenarios. We put forward SSGU-CD, a combined Semantic and Structural information Graph U-shaped network for document-level Chemical-Disease interaction extraction. This framework effectively stores document semantic and structure information as graphs and can fuse the original context information of documents. Using the framework, we propose a balanced combination of cross-entropy loss function to facilitate collaborative optimization among models with the aim of enhancing the ability to extract Chemical-Disease interaction relations. We evaluated SSGU-CD on the document-level relation extraction dataset CDR and BioRED, and the results demonstrate that the framework can significantly improve the extraction performance.

US-Net: U-shaped network with Convolutional Attention Mechanism for ultrasound medical images

Ultrasound imaging, characterized by low contrast, high noise, and interference from surrounding tissues, poses significant challenges in lesion segmentation. To tackle these issues, we introduce an enhanced U-shaped network that incorporates several novel features for precise, automated segmentation. Firstly, our model utilizes a convolution-based self-attention mechanism to establish long-range dependencies in feature maps, crucial for small dataset applications, accompanied by a soft thresholding method for noise reduction. Secondly, we employ multi-sized convolutional kernels to enrich feature processing, coupled with curvature calculations to accentuate edge details via a soft-attention approach. Thirdly, an advanced skip connection strategy is implemented in the UNet architecture, integrating information entropy to assess and utilize texture-rich channels, thereby improving semantic detail in the encoder before merging with decoder outputs. We validated our approach using a newly curated dataset, VPUSI (Vascular Plaques Ultrasound Images), alongside the established datasets, BUSI, TN3K and DDTI. Comparative experiments on these datasets show that our model outperforms existing state-of-the-art techniques in segmentation accuracy.

CTDUNet: A Multimodal CNN-Transformer Dual U-Shaped Network with Coordinate Space Attention for Camellia oleifera Pests and Diseases Segmentation in Complex Environments.

Camellia oleifera is a crop of high economic value, yet it is particularly susceptible to various diseases and pests that significantly reduce its yield and quality. Consequently, the precise segmentation and classification of diseased Camellia leaves are vital for managing pests and diseases effectively. Deep learning exhibits significant advantages in the segmentation of plant diseases and pests, particularly in complex image processing and automated feature extraction. However, when employing single-modal models to segment Camellia oleifera diseases, three critical challenges arise: (A) lesions may closely resemble the colors of the complex background; (B) small sections of diseased leaves overlap; (C) the presence of multiple diseases on a single leaf. These factors considerably hinder segmentation accuracy. A novel multimodal model, CNN-Transformer Dual U-shaped Network (CTDUNet), based on a CNN-Transformer architecture, has been proposed to integrate image and text information. This model first utilizes text data to address the shortcomings of single-modal image features, enhancing its ability to distinguish lesions from environmental characteristics, even under conditions where they closely resemble one another. Additionally, we introduce Coordinate Space Attention (CSA), which focuses on the positional relationships between targets, thereby improving the segmentation of overlapping leaf edges. Furthermore, cross-attention (CA) is employed to align image and text features effectively, preserving local information and enhancing the perception and differentiation of various diseases. The CTDUNet model was evaluated on a self-made multimodal dataset compared against several models, including DeeplabV3+, UNet, PSPNet, Segformer, HrNet, and Language meets Vision Transformer (LViT). The experimental results demonstrate that CTDUNet achieved an mean Intersection over Union (mIoU) of 86.14%, surpassing both multimodal models and the best single-modal model by 3.91% and 5.84%, respectively. Additionally, CTDUNet exhibits high balance in the multi-class segmentation of Camellia oleifera diseases and pests. These results indicate the successful application of fused image and text multimodal information in the segmentation of Camellia disease, achieving outstanding performance.

DERE-Net: A dual-encoder residual enhanced U-Net for muscle fiber segmentation of H&E images

Accurate segmentation of hematoxylin-eosin (H&E) muscle fiber images is crucial for the diagnosis of weightless muscle atrophy. However, uneven contrast, blurred fiber boundaries and adhesions in H&E images are still challenges for accurate segment complete muscle fiber. Although the existing single-encoder based U-shaped networks can extract local information about the target in H&E images, ignores remote dependencies between muscle fibers and low-level detail features, resulting in poor segmentation accuracy when facing H&E images with complex contrast. Therefore, we propose a dual-encoder residual enhanced U-Net segmentation model (DERE-Net) for effective segment muscle fibers in H&E images. DERE-Net uses Transformer and residual CNN to extract global context information and local features of muscle fibers in parallel, and fuses these two features to enable the model to accurately identify the muscle fibers. The multi-scale semantic information fusion (MSIF) module also utilizes the differences between multi-scale features to recover undetected muscle fiber, thus improving the network’s ability to recognize the object regions. In addition, the attention module is added to the skip connection, making the muscle fiber regions information more prominent and improving the robustness of the model. To demonstrate the DERE-Net effect, the comparative experiments are conducted on our own muscle fiber H&E image dataset, GLAS and MoNuSeg. DERE-Net achieved excellent performance on the muscle fiber H&E dataset (DSC 0.919), GLAS dataset (DSC 0.912), and MoNuSeg dataset (DSC 0.821). These results indicate that DERE-Net can accurately predict muscle fiber regions, providing a new method for accurate diagnosis of muscle atrophy in the future.