Detail Information Research Articles

Feature fusion method based on spiking neural convolutional network for edge detection

NSNP-type neuron is a new type of neuron model inspired by nonlinear spiking mechanisms in nonlinear spiking neural P systems. In order to address the loss problem of edge detail information in edge detection methods based on deep learning, we propose a feature fusion method based on NSNP-type neurons. The architecture of this feature fusion method consists of two modules: feature extraction module and feature fusion module. In particular, the feature fusion module is composed of convolutional blocks constructed by NSNP-type neurons for multi-level feature fusions, and CoT blocks with Transformer style is introduced to extract rich contextual information from low-level features and high-level features. To fuse multi-level features and preserve contextual information, we design a new loss function that not only preserves feature prediction loss and fusion loss, but also considers contour-related and texture-related information. The proposed method is evaluated on BSDS500 and NYUDv2 data sets and compare it with 9 baseline methods and 12 CNN-based methods, and we achieve ODS of 0.808 and OIS of 0.827 on BSDS500. The comparison results demonstrate the advantages of the proposed method for edge detection. The source code is available at https://github.com/xhuph66/FF-CNSNP-master.

ИССЛЕДОВАНИЕ ОБЪЕКТОВ, ПРЕДСТАВЛЯЮЩИХ ЭКОЛОГИЧЕСКУЮ УГРОЗУ, В ЗАЛИВЕ ЦИВОЛЬКИ (НОВАЯ ЗЕМЛЯ, КАРСКОЕ МОРЕ)

The article presents results of field studies of the environmental assessment of the bays of the eastern coast of Novaya Zemlya in connection with potentially dangerous objects flooded in this region, which are the nuclear fleet operation waste products. In the course of these studies, technology was used based on remote-controlled and towed uninhabited underwater vehicles (ROV and UTSI), as well as side-view sonar (SSS) developed at IO RAS. To detail information about the condition of emergency facilities and operational waste flooded in 60 years at Tsivolki bay, a deep-water UTSI “Video Module” was used. It is equipped with video and photo cameras, a high-frequency SSS, a depth gauge, an altimeter, a laser scale pointer, as well as auxiliary equipment that ensures the functioning of research systems. Observations in the Gulf of Tsivolki using instrumental technologies made it possible to identify underwater objects whose existence was known only from archival data. Monitoring of the condition of objects was provided using RAM-family gamma-ray spectrometers developed at the Kurchatov Institute Research Center. The nature of the recorded gamma-ray spectra generally corresponds to the measurement results obtained during the assembly survey in 2010 and 2015, and indicates the integrity of the internal protective barriers.

LEES-Net: Fast, lightweight unsupervised curve estimation network for low-light image enhancement and exposure suppression

Images in dark scenes are affected not only by low-light conditions but also by mixed exposures, posing great challenges to human and machine vision. Most of the current methods predominantly focus on ameliorating low-exposure images. However, for mixed-exposure low-light images, the high-exposure areas of the images will be over-enhanced, resulting in losing some detail information and not getting a well visual effect. To address this problem, we design a new unsupervised low-light enhancement method, called LEES-Net, with good robustness for mixed exposure cases. LEES-Net can transform the problem of low-light enhancement and exposure suppression into a curve estimation problem, thereby effectively reducing the complexity associated with image enhancement. By adding the attention mechanisms, the low-exposure and high-exposure areas in the images can be targeted and dynamically adjusted, which makes the enhanced images have well visual effects. Through extensive experiments, our method outperforms other state-of-the-art unsupervised methods with good generalization ability, robustness and visual effects. Furthermore, we propose a more lightweight network with merely 4.464 K parameters and 0.002 s inference speed, called LEES-Net+, which can keep the enhanced performance of LEES-Net with smaller computational cost and parameters, and is more suitable for deployment on devices with limited resources and real-time requirements.

CSFFNet: Lightweight cross‐scale feature fusion network for salient object detection in remote sensing images

AbstractSalient object detection (SOD), one of the most important applications in the field of computer vision, aims to extract the most visually appealing regions of scenes. However, the improvement of the accuracy of existing salient object detection in optical remote sensing images (ORSI‐SOD) is usually accompanied by an increase of network complexity, which affects the application of these models. Motivated by this, a novel lightweight edge‐supervised neural network for ORSI‐SOD is proposed, named CSFFNet. Specifically, the backbone (ResNet34) is first lightened by feature encoding module (FEM), building a lightweight subnet for feature extraction. Then, in the transformer‐based feature pyramid enhancement module (FPEM), the convolutional features obtained in the FEM are enhanced by long‐distance dependence to obtain multi‐scale features containing rich saliency cues. Based on this, the feature fusion module (FFM) is designed to capture cross‐scale long‐range dependencies and effectively fuse high‐level semantic information with low‐level detail information. Thus, the increase in network complexity due to multi‐level decoding is avoided. Finally, the segmentation results are optimized by using salient edges as auxiliary information, which effectively improves the contrast and completeness of the results. Experimental results on two public datasets demonstrate that the lightweight CSFFNet achieves competitive or even better performance compared with state‐of‐the‐art methods.

Dual Cross-Attention Multi-Stage Embedding Network for Low-Light Image Enhancement

The low-light image enhancement task aims to improve the visibility of information in the dark to obtain more data and utilize it, while also improving the visual quality of the image. In this paper, we propose a dual cross-attention multi-stage embedding network (DCMENet) for fast and accurate enhancement of low-light images into high-quality images with high visibility. The problem that enhanced images tend to have more noise in them, which affects the image quality, is improved by introducing an attention mechanism in the encoder–decoder structure. In addition, the encoder–decoder can focus most of its attention on the dark areas of the image and better attend to the detailed features in the image that are obscured by the dark areas. In particular, the poor performance of the Transformer when the dataset size is small is solved by fusing the CNN-Attention and Transformer in the encoder. Considering the purpose of the low-light image enhancement task, we raise the importance of recovering image detail information to the same level as reconstructing the lighting. For features such as texture details in images, cascade extraction using spatial attention and pixel attention can reduce the model complexity while the performance is also improved. Finally, the global features obtained by the encoder–decoder are fused into the shallow feature extraction structure to reconstruct the illumination while guiding the network for the focused extraction of information in the dark. The proposed DCMENet achieves the best results in both objective quality assessment and subjective evaluation, while for the computer vision tasks working in low-light environments as well, the enhanced images using the DCMENet proposed in this paper show the best performance.

Modern Technology and Scope of Precision Agriculture (PA)

Traditional agriculture practices are time consuming, less productive and uneconomic. Due to uncontrolled use of resources and inputs frequently occurred by farmers caused environmental degradation, land loss and economic loss of farmers. Smart farming known as Precision agriculture which uses the modern technology and tools such as Global position system, Geographic information system, remote sensing and information and communication in integrations provides detail information about crop growth, soil conditions, nutrient level, irrigation systems as data sets which helps to take decisions in right time so that it increases the crop productivity. The use of tools like GPS, GIS and Remote sensing optimize the use of natural resources and inputs for a given crop production and quality. Agriculture becomes more productive and consistent by digital technology and effective use of resources and time. The main gist of this study is to present the scope and implications of the precision agriculture (PA) in future.

Improved Edge Folding Algorithm for 3D Building Models Taking into Account the Visual Features

Simplifying 3D building models, effectively reducing model complexity and improving mapping efficiency, is an important part of 3D GIS. In order to address the problem that the simplification basis considered by most 3D building model data compression algorithms is mainly based on geometric features and fails to retain the visual features of the model, this paper proposed a half-folding simplification algorithm for 3D building models that took the visual features into account. The algorithm incorporated regional boundary constraints in the pre-processing stage to focus on preserving the boundary features of the model, and incorporated visual constraints, such as the vertex importance, edge length, and texture, in the calculation of the edge folding cost to construct a new error metric model. The simplification order was changed by redefining the edge folding cost, and a trade-off between compression rate and retention rate was made by using half-edge folding to maximise the retention of the visual features of the building. The experimental results showed that the algorithm could effectively reduce the number of elements drawn by the system and achieve a high level of detail information retention in the visual feature region of the model with a good visual representation.

An image zooming method based on the coupling threshold in the wavelet packet transform domain

Aiming at the problem of blurring edge and detail information in the process of image zooming, this paper proposes a new method of image zooming based on wavelet packet transform by combining the characteristics of anisotropic diffusion. First, the initial zoomed image with higher resolution is obtained by wavelet transform, and the wavelet packet decomposition is performed to obtain more high-frequency wavelet packet coefficients reflecting image details. Second, due to the existence of noise, the relationship between wavelet packet transform and anisotropic diffusion is obtained by studying the process of wavelet packet threshold denoising, and the expression of coupling threshold based on diffusion function is given and applied to the high-frequency wavelet packet coefficients. Finally, the original image is used as the low-frequency part after passing the soft threshold, and reconstructed with the denoised high-frequency part to obtain the final zoomed image. The traditional zooming algorithms as well as the learning-based zooming algorithms are selected for comparison. The results show that the algorithm in this paper effectively avoids the blurring of edges and details under the premise of ensuring the similarity between the zoomed image and the original image, so that the zoomed image can obtain more high-frequency information, and achieves the purpose of removing the noise as well as enhancing the detailed information of the image. The effectiveness of this paper’s algorithm in edge protection can be seen from the results of the comparison with the deep learning-based zooming algorithms.

A systematic approach for enhancement of homogeneous background images using structural information

Image enhancement is an indispensable pre-processing step for several image processing applications. Mainly, histogram equalization is one of the widespread techniques used by various researchers to improve the image quality by expanding the pixel values to fill the entire dynamic grayscale. It results in the visual artifact, structural information loss near edges due to the information loss (due to many-to-one mapping), and alteration in average luminance to a higher value. This paper proposes an enhancement algorithm based on structural information for homogeneous background images. The intensities are divided into two segments using the median value to preserve the average luminance. Unlike traditional techniques, this algorithm incorporates the spatial locations in the equalization process instead of the number of intensity values occurrences. The occurrences of each intensity concerning their spatial locations are combined using Rènyi entropy to enumerate a discrete function. An adaptive clipping limit is applied to the discrete function to control the enhancement rate. Then histogram equalization is performed on each segment separately, and the equalized segments are integrated to produce an enhanced image. The algorithm’s effectiveness is validated by evaluating the proposed method on CEED, CSIQ, LOL, and TID2013 databases. Experimental results reveal that the proposed method improves the contrast while preserving structural information, detail information, and average luminance. They are quantified by the high value of contrast improvement index, structural similarity index, and discrete entropy, and low value of average mean brightness error values of the proposed method when compared with the methods available in the literature, including deep learning architectures.

Quality guided reversible data hiding with contrast enhancement

Reversible data hiding with contrast enhancement (RDH-CE) enables contrast enhancement of the image and lossless recovery of host image after embedded data extraction. Although the current RDH-CE methods are effective in enhancing carrier images and ensuring reversibility, image quality control is still lacking, which may lead to poor image visualization. In more serious situations, the image detail information may be lost. The peak pair selection mechanism plays an extremely crucial role in image quality enhancement; however, existing work often neglects this. In this paper, we propose an RDH-CE algorithm that pursues high image quality maintenance, aiming to maintain image quality by guiding and controlling the selection of peaks and the direction of histogram shift through quality metrics. Specifically, instead of simply selecting only one pair of peaks at each peak selection stage, our algorithm selects several pairs of peaks that meet the requirements as candidate peak pairs. It then first performs simulated histogram equalization on these candidate peaks, followed by analyzing the situation of the quality metrics generated by each pair of peaks accordingly, and then selects the optimal peaks for the current iteration based on the performance of each pair of candidate peaks to be used in the equalization process. Our selection mechanism also dynamically changes the peak selection strategy based on before-and-after changes in the actual metrics, which can better ensure image quality stability over the iterations. Under the evaluations of a series of image quality metrics, the proposed method can still maintain high quality and visual effect under high iteration and has strong stability for image quality control, thus demonstrating the effectiveness and superiority of the proposed method.

Multi-scale Perception and Feature Refinement Network for multi-class segmentation of intracerebral hemorrhage in CT images

Intracerebral hemorrhage (ICH) poses a severe threat to human health and well-being. Automatic segmentation of hematomas in CT images can provide essential diagnostic assistance to physicians, and ensure improved treatment and recovery outcomes for patients. Existing methods for intracerebral hemorrhage segmentation mainly focus on identifying hemorrhage areas, without the ability to accurately distinguish and outline different types of hematomas. However, different types of hemorrhage exhibit a high degree of similarity in terms of gray matter level and shape, and the scale of hematomas can vary significantly. To address this issue, we propose a Multi-scale Perception and Feature Refinement Network (MPFR-Net) for automatic segmentation of both intraparenchymal and intraventricular hemorrhages. Specifically, we propose a Multi-scale Perception Module (MPM), which consists of the integration of features at different levels and the local and global multi-scale branches. MPM allows for the efficient extraction of multi-scale features and the establishment of long-range relationships between the target and background. Additionally, we propose a Feature Refinement Module (FRM) to refine fuzzy detail information that has been lost after down-sampling to the deep layer, while simultaneously supplementing the small target information from shallow features. To improve the clinical adaptability of our method, we further collect 608 patient cases from multiple hospitals to construct a multi-center dataset, termed as ICH-Seg in which each case contains both intraparenchymal and intraventricular hemorrhages. From the quantitative and visual results, MPFR-Net outperforms previous methods on both private and public datasets, showing promising segmentation for intracerebral hemorrhage and potential clinical applications.

Blind Super-Resolution Network with Dual-Channel Attention for Images Captured by Sub-Millimeter-Diameter Fiberscope

A blind super-resolution network with dual-channel attention is proposed for images captured by the 0.37 mm diameter sub-millimeter fiberscope. The fiberscope can used in scenarios where other image acquisition devices cannot be applied based on its flexible, soft, and minimally invasive characteristics. However, the images have black reticulated noise and only 3000 pixels. To improve image quality, the Butterworth band-stop filter is used to reduce the frequency of the reticulated noise. By optimizing the blind super-resolution model, high-quality images can be reconstructed that do not require a lot of synthetic paired fiberscope image data. Perceptual loss is utilized as a loss function, and channel and spatial attention mechanisms are introduced to the model to enhance the high-frequency detail information of the reconstructed image. In the comparative experiment with other methods, our method showed improvements of 2.25 in peak signal-to-noise ratio (PSNR) and 0.09 in structural similarity (SSIM) based on objective evaluation metrics. The learned perceptual image patch similarity (LPIPS) based on learning was reduced by 0.6. Furthermore, four different methods were used to enhance the resolution of the fiberscope images by a factor of four. The results of this paper improve the information entropy and Laplace clarity by 0.44 and 2.54, respectively, compared to the average of other methods. Validation results show that the approach in this paper is more applicable to sub-millimeter-diameter fiberscopes.

Abnormal neuroanatomical patterns as potential diagnostic biomarkers for cocaine use disorder.

Cocaine use disorder (CUD) is a global health problem with serious consequences for both individuals and society. Previous studies on abnormal anatomical patterns in CUD have mainly used voxel-based morphometry to investigate grey matter volume changes, while surface-based morphometry (SBM) has been found to provide detail information on cortical thickness (CT), surface area and cortical meancurve, which can contribute to a better understanding of structural brain changes associated with CUD. In this study, SBM was conducted to investigate abnormal neuroanatomical patterns in CUD and whether these abnormal patterns could be used as potential diagnostic biomarkers for CUD. Sixty-eight CUD individuals and 52 matched healthy controls were enrolled, and all participants performed once MRI scanning and clinical assessments. We found that CUD individuals exhibited altered morphological indicators across widespread brain regions and these abnormal anatomical alterations were significantly predictive of CUD status. Furthermore, the CT reduction of right insula was significantly associated with years of cocaine use in CUD. These findings revealed the association of abnormal anatomical patterns in specific brain regions in CUD, which further improve the understanding of CUD pathophysiology and provide the alternative diagnostic biomarkers for CUD.

Imaging quality characterization and optimization of segmented planar imaging system based on honeycomb dense azimuth sampling lens array.

Segmented planar photoelectric imaging is an advanced computational imaging technology that utilizes photonic integrated circuits (PICs) to achieve the miniaturization of imaging systems. The original radial-spoke lens array has dense radial sampling and coarse azimuthal sampling. The sparsity and inhomogeneity of spatial frequency sampling lead to the loss of spatial frequency information and blurred reconstructed images. In this paper, a honeycomb dense azimuth sampling lens array is proposed, and three baseline pairing methods are designed, which can realize dense azimuth sampling, effectively increase spatial frequency sampling and improve the imaging quality. The signal transmission model of the segmented planar imaging system is established and the imaging process is simulated and analyzed. The simulation results show that the honeycomb lens array improves the azimuth sampling density and spatial frequency coverage, and its imaging quality is significantly improved compared with the hexagonal lens array and the radial-spoke lens array. Furthermore, the optimal choice of the baseline pairing method and the error range of the fill factor and are also given in this paper. The results indicate that the mixed pairing method first ensures low and medium-frequency dense sampling, and then increases high-frequency sampling, which makes the imaging results better than those of the other two baseline pairing methods in terms of image contour, contrast and image detail information. The sampling density of the spatial frequency and the imaging quality can be improved by increasing the fill factor. In the actual manufacturing process, the allowable error range of the fill factor of the lens array is within 5%. The research results will provide theoretical support for the design and development of segmented planar imaging system.

Attention-based dual-path feature fusion network for automatic skin lesion segmentation

Automatic segmentation of skin lesions is a critical step in Computer Aided Diagnosis (CAD) of melanoma. However, due to the blurring of the lesion boundary, uneven color distribution, and low image contrast, resulting in poor segmentation result. Aiming at the problem of difficult segmentation of skin lesions, this paper proposes an Attention-based Dual-path Feature Fusion Network (ADFFNet) for automatic skin lesion segmentation. Firstly, in the spatial path, a Boundary Refinement (BR) module is designed for the output of low-level features to filter out irrelevant background information and retain more boundary details of the lesion area. Secondly, in the context path, a Multi-scale Feature Selection (MFS) module is constructed for high-level feature output to capture multi-scale context information and use the attention mechanism to filter out redundant semantic information. Finally, we design a Dual-path Feature Fusion (DFF) module, which uses high-level global attention information to guide the step-by-step fusion of high-level semantic features and low-level detail features, which is beneficial to restore image detail information and further improve the pixel-level segmentation accuracy of skin lesion. In the experiment, the ISIC 2018 and PH2 datasets are employed to evaluate the effectiveness of the proposed method. It achieves a performance of 0.890/ 0.925 and 0.933 /0.954 on the F1-score and SE index, respectively. Comparative analysis with state-of-the-art segmentation methods reveals that the ADFFNet algorithm exhibits superior segmentation performance.

Weakly Supervised Cross-Domain Person Re-Identification Algorithm Based on Small Sample Learning

This paper proposes a weakly supervised cross-domain person re-identification (Re-ID) method based on small sample data. In order to reduce the cost of data collection and annotation, the model design focuses on extracting and abstracting the information contained in the data under limited conditions. In this paper, we focus on the problems of strong data dependence, weak cross-domain capability and low accuracy in Re-ID in weakly supervised scenarios. Our contributions are as follows: first, we implement a joint training framework with the help of small sample learning and cross-domain migration for Re-ID. Second, with the help of residual compensation and fusion attention module, the RCFA module is designed, and the model framework is built on this basis to improve the cross-domain ability of the model. Third, to solve the problem of low accuracy caused by insufficient data coverage of small samples, a fusion of shallow features and deep features is designed to enable the model to weighted fusion of shallow detail information and deep semantic information. Finally, by selecting different camera images in Market1501 dataset and DukeMTMC-reID dataset as small samples, respectively, and introducing another dataset data for joint training, we demonstrate the feasibility of this joint training framework, which can perform weakly supervised cross-domain Re-ID based on small sample data.

Overcomplete graph convolutional denoising autoencoder for noisy skeleton action recognition

AbstractCurrent skeleton‐based action recognition methods usually assume the input skeleton is complete and noise‐free. However, it is inevitable that the captured skeletons are incomplete due to occlusions or noisy due to changes in the environment. When dealing with these data, even State Of The Art (SOTA) recognition backbones experience significant degradation in recognition accuracy. Though a few methods have been proposed to address this issue, they still lack flexibility, efficiency and interpretability. In this work, an overcomplete Graph Convolutional Denoising Autoencoder (GCDAE) is proposed which can act as a flexible preprocessing module for pretrained recognition backbones and improve their robustness. Taking advantages of the overcomplete and fully graph convolutional structure, GCDAE is able to rectify noisy joints while keeping information of unspoiled details efficiently. On two large scale skeleton datasets NTU RGB+D 60 and 120, the introducing of GCDAE brings significant robustness improvements to SOTA backbones towards different types of noises.

3D multi-modality Transformer-GAN for high-quality PET reconstruction

Positron emission tomography (PET) scans can reveal abnormal metabolic activities of cells and provide favorable information for clinical patient diagnosis. Generally, standard-dose PET (SPET) images contain more diagnostic information than low-dose PET (LPET) images but higher-dose scans can also bring higher potential radiation risks. To reduce the radiation risk while acquiring high-quality PET images, in this paper, we propose a 3D multi-modality edge-aware Transformer-GAN for high-quality SPET reconstruction using the corresponding LPET images and T1 acquisitions from magnetic resonance imaging (T1-MRI). Specifically, to fully excavate the metabolic distributions in LPET and anatomical structural information in T1-MRI, we first use two separate CNN-based encoders to extract local spatial features from the two modalities, respectively, and design a multimodal feature integration module to effectively integrate the two kinds of features given the diverse contributions of features at different locations. Then, as CNNs can describe local spatial information well but have difficulty in modeling long-range dependencies in images, we further apply a Transformer-based encoder to extract global semantic information in the input images and use a CNN decoder to transform the encoded features into SPET images. Finally, a patch-based discriminator is applied to ensure the similarity of patch-wise data distribution between the reconstructed and real images. Considering the importance of edge information in anatomical structures for clinical disease diagnosis, besides voxel-level estimation error and adversarial loss, we also introduce an edge-aware loss to retain more edge detail information in the reconstructed SPET images. Experiments on the phantom dataset and clinical dataset validate that our proposed method can effectively reconstruct high-quality SPET images and outperform current state-of-the-art methods in terms of qualitative and quantitative metrics.

Semantic Segmentation of Images Based on Multi-Feature Fusion and Convolutional Neural Networks

Image semantic segmentation technology is one of the core research contents in the field of computer vision. With the improvement of computer performance and the continuous development of deep learning technology, researchers have more and more enthusiasm to study the actual effect and performance of image semantic segmentation. The results of deep semantic segmentation allow computers to have a more detailed and accurate understanding of images, and have a wide range of application needs in the fields of autonomous driving, intelligent security, medical imaging, remote sensing images, etc. However, the existing image semantic segmentation algorithms have the disadvantages of easy discontinuous results and insufficient prediction accuracy. In this paper, we take deep learning-based image semantic segmentation technology as the research object to explore the improvement of the image semantic segmentation algorithm and its application in road scenarios. First, this paper proposes MCU-Net method based on residual fusion and multi-scale contextual information. MCU-Net uses residual fusion module to deepen the network structure and improve the ability of U-Net to acquire deeper features. Then a top-down and bottom-up path is constructed for feature information between different levels, and the spatial and semantic information contained in shallow and deep features in the network is fully utilized by fusing features from different levels. In addition, an enhanced void space pyramid pooling module is added for feature information between the same levels, which enables the output features to have a larger range of semantic information. Second, this paper proposes the DAMCU-Net method based on attention mechanism and edge detection based on MCU-Net. DAMCU-Net extracts global contextual information by the attention mechanism optimization module, while fusing features using dense jump connections to facilitate the network to recover more spatial detail information during upsampling, and uses the FReLU activation function to improve the segmentation capability of the network for complex targets. For the edge information lost in the feature extraction process, the edge detection branch is added to supplement the feature information of the main path by feature fusion to achieve the optimization of the edge information.

RTAU-Net: A novel 3D rectal tumor segmentation model based on dual path fusion and attentional guidance

Background and objectiveAccording to the Global Cancer Statistics 2020, colorectal cancer has the third-highest diagnosis rate (10.0 %) and the second-highest mortality rate (9.4 %) among the 36 types. Rectal cancer accounts for a large proportion of colorectal cancer. The size and shape of the rectal tumor can directly affect the diagnosis and treatment by doctors. The existing rectal tumor segmentation methods are based on two-dimensional slices, which cannot analyze a patient's tumor as a whole and lose the correlation between slices of MRI image, so the practical application value is not high. MethodsIn this paper, a three-dimensional rectal tumor segmentation model is proposed. Firstly, image preprocessing is performed to reduce the effect caused by the unbalanced proportion of background region and target region, and improve the quality of the image. Secondly, a dual-path fusion network is designed to extract both global features and local detail features of rectal tumors. The network includes two encoders, a residual encoder for enhancing the spatial detail information and feature representation of the tumor and a transformer encoder for extracting global contour information of the tumor. In the decoding stage, we merge the information extracted from the dual paths and decode them. In addition, for the problem of the complex morphology and different sizes of rectal tumors, a multi-scale fusion channel attention mechanism is designed, which can capture important contextual information of different scales. Finally, visualize the 3D rectal tumor segmentation results. ResultsThe RTAU-Net is evaluated on the data set provided by Shanxi Provincial Cancer Hospital and Xinhua Hospital. The experimental results showed that the Dice of tumor segmentation reached 0.7978 and 0.6792, respectively, which improved by 2.78 % and 7.02 % compared with suboptimal model. ConclusionsAlthough the morphology of rectal tumors varies, RTAU-Net can precisely localize rectal tumors and learn the contour and details of tumors, which can relieve physicians' workload and improve diagnostic accuracy.