Downsampling Process Research Articles

A small‐target traffic sign detection algorithm based on partial conv and atrous spatial pyramid

AbstractTraffic sign detection is essential to an intelligent driving assistance system. The deep learning‐based algorithm proposed in this paper aims to address the issue of low detection accuracy caused by the small size and high density of traffic signs in real‐world traffic scenarios. First, to improve the feature extraction module of the backbone network and to increase the model's ability to capture contextual information, partial convolution (PConv) is introduced. Second, to prevent information loss during the downsampling process, a cross‐stage atrous spatial pyramid (ASPPFCSPC) is constructed using atrous convolution. This method combines feature map information from various scales and expands the receptive field. Lastly, the small‐target detection precision is improved by incorporating an additional small‐target detection head, which uses high‐resolution feature maps for shallow features. The detection head is decoupled to extract the location and class information of the predicted target separately, thereby enhancing the generalization ability of the proposed model. Experiments have demonstrated the superiority of the proposed algorithm, as testing on the TT100K dataset resulted in a mAP@0.5 of 91.2% and a mAP@0.5:0.95 of 71.8% using the proposed algorithm.

Asymmetric Network Combining CNN and Transformer for Building Extraction from Remote Sensing Images.

The accurate extraction of buildings from remote sensing images is crucial in fields such as 3D urban planning, disaster detection, and military reconnaissance. In recent years, models based on Transformer have performed well in global information processing and contextual relationship modeling, but suffer from high computational costs and insufficient ability to capture local information. In contrast, convolutional neural networks (CNNs) are very effective in extracting local features, but have a limited ability to process global information. In this paper, an asymmetric network (CTANet), which combines the advantages of CNN and Transformer, is proposed to achieve efficient extraction of buildings. Specifically, CTANet employs ConvNeXt as an encoder to extract features and combines it with an efficient bilateral hybrid attention transformer (BHAFormer) which is designed as a decoder. The BHAFormer establishes global dependencies from both texture edge features and background information perspectives to extract buildings more accurately while maintaining a low computational cost. Additionally, the multiscale mixed attention mechanism module (MSM-AMM) is introduced to learn the multiscale semantic information and channel representations of the encoder features to reduce noise interference and compensate for the loss of information in the downsampling process. Experimental results show that the proposed model achieves the best F1-score (86.7%, 95.74%, and 90.52%) and IoU (76.52%, 91.84%, and 82.68%) compared to other state-of-the-art methods on the Massachusetts building dataset, the WHU building dataset, and the Inria aerial image labeling dataset.

Exploiting K-Space in Magnetic Resonance Imaging Diagnosis: Dual-Path Attention Fusion for K-Space Global and Image Local Features.

Magnetic resonance imaging (MRI) diagnosis, enhanced by deep learning methods, plays a crucial role in medical image processing, facilitating precise clinical diagnosis and optimal treatment planning. Current methodologies predominantly focus on feature extraction from the image domain, which often results in the loss of global features during down-sampling processes. However, the unique global representational capacity of MRI K-space is often overlooked. In this paper, we present a novel MRI K-space-based global feature extraction and dual-path attention fusion network. Our proposed method extracts global features from MRI K-space data and fuses them with local features from the image domain using a dual-path attention mechanism, thereby achieving accurate MRI segmentation for diagnosis. Specifically, our method consists of four main components: an image-domain feature extraction module, a K-space domain feature extraction module, a dual-path attention feature fusion module, and a decoder. We conducted ablation studies and comprehensive comparisons on the Brain Tumor Segmentation (BraTS) MRI dataset to validate the effectiveness of each module. The results demonstrate that our method exhibits superior performance in segmentation diagnostics, outperforming state-of-the-art methods with improvements of up to 63.82% in the HD95 distance evaluation metric. Furthermore, we performed generalization testing and complexity analysis on the Automated Cardiac Diagnosis Challenge (ACDC) MRI cardiac segmentation dataset. The findings indicate robust performance across different datasets, highlighting strong generalizability and favorable algorithmic complexity. Collectively, these results suggest that our proposed method holds significant potential for practical clinical applications.

Road Crack Detection by Combining Dynamic Snake Convolution and Attention Mechanism

As one of the early manifestations of road pavement structure degradation, road cracks will accelerate the deterioration of the road if not detected and repaired in time. Aiming at the problems of low recall and incomplete crack detection in current road detection, based on the U-Net network, this paper proposed an Attention-Dynamic Snake Convolution U-Net (ADSC-U-Net) network. Firstly, the dynamic snake-shaped convolution was added to the normal downsampling process to make the network adaptively focus on the slender and curved local features, which can solve the problem of low accuracy of small crack detection. Secondly, the attention mechanism was used to pay better attention to the significant features of positive samples under the condition of a large proportion gap between positive and negative samples, which solved the problem of the poor crack integrity detection effect. Finally, the dataset was expanded by random vertical and horizontal flip operations, which solved the problem of network training overfitting caused by the small-scale datasets. The experimental results showed that, when the input image had a resolution of 480 × 320, evaluation indices P, R, and F1 of ADSC-U-Net on the self-built dataset were 74.44%, 68.77%, and 69.42%, respectively. Compared to SegNet, DeepLab, and DeepCrack, the P was improved by 1.90%, 2.49%, and 11.64%, respectively; the R was improved by 8.01%, 4.70%, and 59.58%, respectively; and the comprehensive evaluation index F1 was improved by 5.73%, 4.02%, and 55.87%, respectively, which proves the effectiveness of the proposed method.

Enhanced recognition of insulator defects on power transmission lines via proposal-based detection model with integrated improvement methods

Deep learning-driven transmission line inspection is a critical area for smart power grid development. Despite advances in deep learning for insulator defect detection, challenges remain in model robustness and adaptability for the varying real-world adaptability, especially for insignificant defects in complex backgrounds. This study presents a comprehensive improvement strategy for detecting insulators and cross-scale broken defects on transmission lines, employing a proposal-based detection model. The model introduces a holistic pipeline of improved methods, including backbone modification, anchor box scale recalibration, and improvements in Region of Interest (RoI) downsampling alignment and Intersection over Union (IoU) loss function. Various backbone networks, including convolutional network (ConvNet) and Vision Transformer (ViT) structures, are constructed and integrated with attention modules, specifically designed to amplify the perception of insulators and defective regions. The geometric scale of anchor boxes is reconstructed using a developed clustering algorithm, considering the elongated characteristics of insulator strings to improve the adaptability of anchor boxes. Bilinear interpolation is utilized to mitigate spatial misalignment issues during the downsampling process of Region Proposal Network (RPN)-based proposals. The experimental results indicate that the improved models with the Swin Transformer (Swin-T) backbone framework achieve the mean Average Precision (mAP)@0.5 of 88.42% and mAP@0.7 of 60.52%, with a defect recall rate of 81.94%. Additionally, the improved IoU loss function contributes to the performance of the model at higher IoU thresholds. The results of this study contribute to the further development of defect detection frameworks for power vision applications.

Improved YOLOv8 Algorithm for Water Surface Object Detection.

To address the issues of decreased detection accuracy, false detections, and missed detections caused by scale differences between near and distant targets and environmental factors (such as lighting and water waves) in surface target detection tasks for uncrewed vessels, the YOLOv8-MSS algorithm is proposed to be used to optimize the detection of water surface targets. By adding a small target detection head, the model becomes more sensitive and accurate in recognizing small targets. To reduce noise interference caused by complex water surface environments during the downsampling process in the backbone network, C2f_MLCA is used to enhance the robustness and stability of the model. The lightweight model SENetV2 is employed in the neck component to improve the model's performance in detecting small targets and its anti-interference capability. The SIoU loss function enhances detection accuracy and bounding box regression precision through shape awareness and geometric information integration. Experiments on the publicly available dataset FloW-Img show that the improved algorithm achieves an mAP@0.5 of 87.9% and an mAP@0.5:0.95 of 47.6%, which are improvements of 5% and 2.6%, respectively, compared to the original model.

Soldered chip reversed polarity detection based on semi-combinatorial attention and size-sensitive IoU

Abstract Chips that have been automatically soldered may have defects such as short circuits, reversed polarity, offsets, and missing components. This paper proposes a reversed polarity detection algorithm (RPDNet) for soldered chips based on semi-combinatorial attention and size-sensitive IoU to solve the polarity reversal problem. To distinguish characters or patterns similar to polarity markings in the background, a semi-combination attention module (SCAM) is designed, which effectively distinguishes the subtle feature differences between polarity markings and background elements by integrating the long-range dependence of the self-attention mechanism and the local focusing ability of the general attention mechanism, thereby reducing misdetection of polarity markings. Aiming at the problem of blurred chip polarity markings, a differential pooling module (DPM) is proposed, which optimizes the downsampling process of fuzzy polarity markings by combining the differential features obtained from subtracting the results of max pooling from average pooling with features derived solely from average pooling, thereby reducing the information loss of fuzzy markings during the feature extraction process. In order to solve the lack of sensitivity of CIoU to the side length of the bounding box, a size-sensitive IoU(SIoU) is proposed. Through the size-sensitivity factor and area difference loss term, the model strengthens the focus on boundary regression and achieves faster and more accurate target positioning. Using the Chip-Set provided by the company for experiments and tests, the detection accuracy reached 97.79% and the processing speed reached 107 FPS.

ADS-VQA: Adaptive sampling model for video quality assessment

No-reference video quality assessment (NR-VQA) for user-generated content (UGC) plays a crucial role in ensuring the quality of video services. Although some works have achieved impressive results, their performance-complexity trade-off is still sub-optimal. On the one hand, overly complex network structures and additional inputs require more computing resources. On the other hand, the simple sampling methods have tended to overlook the temporal characteristics of the videos, resulting in the degradation of local textures and potential distortion of the thematic content, consequently leading to the performance decline of the VQA technologies. Therefore, in this paper, we propose an enhanced NR-VQA model, known as the Adaptive Sampling Strategy for Video Quality Assessment (ADS-VQA). Temporally, we conduct non-uniform sampling on videos utilizing features from the lateral geniculate nucleus (LGN) to capture the temporal characteristics of videos. Spatially, a dual-branch structure is designed to supplement spatial features across different levels. The one branch samples patches at their raw resolution, effectively preserving the local texture detail. The other branch performs a downsampling process guided by saliency cues, attaining global semantic features with a diminished computational expense. Experimental results demonstrate that the proposed approach achieves high performance at a lower computational cost than most state-of-the-art VQA models on four popular VQA databases.

FCNet: Flexible Convolution Network for Infrared Small Ship Detection

The automatic monitoring and detection of maritime targets hold paramount significance in safeguarding national sovereignty, ensuring maritime rights, and advancing national development. Among the principal means of maritime surveillance, infrared (IR) small ship detection technology stands out. However, due to their minimal pixel occupancy and lack of discernible color and texture information, IR small ships have persistently posed a formidable challenge in the realm of target detection. Additionally, the intricate maritime backgrounds often exacerbate the issue by inducing high false alarm rates. In an effort to surmount these challenges, this paper proposes a flexible convolutional network (FCNet), integrating dilated convolutions and deformable convolutions to achieve flexible variations in convolutional receptive fields. Firstly, a feature enhancement module (FEM) is devised to enhance input features by fusing standard convolutions with dilated convolutions, thereby obtaining precise feature representations. Subsequently, a context fusion module (CFM) is designed to integrate contextual information during the downsampling process, mitigating information loss. Furthermore, a semantic fusion module (SFM) is crafted to fuse shallow features with deep semantic information during the upsampling process. Additionally, squeeze-and-excitation (SE) blocks are incorporated during upsampling to bolster channel information. Experimental evaluations conducted on two datasets demonstrate that FCNet outperforms other algorithms in the detection of IR small ships on maritime surfaces. Moreover, to propel research in deep learning-based IR small ship detection on maritime surfaces, we introduce the IR small ship dataset (Maritime-SIRST).

Medical image classification: Knowledge transfer via residual U-Net and vision transformer-based teacher-student model with knowledge distillation

With the widespread integration of deep learning techniques in the domain of medical image analysis, there is a prevailing consensus regarding their efficacy in handling high-dimensional and intricate medical image data. However, it is imperative to acknowledge that while complex deep models exhibit a remarkable capacity for processing high-dimensional and intricate data, they often necessitate a substantial allocation of computational resources and time. Furthermore, lightweight models, despite their computational efficiency, tend to underperform when compared to their more intricate counterparts in terms of performance. Hence, the prevailing aspiration is to transfer the cognitive prowess of complex models to their lightweight counterparts. Addressing the aforementioned concern, this study proposes a knowledge distillation approach that encompasses joint feature and soft label transfer. It entails the transference of knowledge from the teacher model’s intermediate features and predictive outcomes to the student model. The student model leverages this knowledge to emulate the behavior of the teacher model, thereby enhancing the precision of its own predictions. Building upon this foundation, we introduce a Res-Transformer teacher model based on the U-Net architecture and a student model known as ResU-Net, which is grounded in residual modules. The Res-Transformer teacher model employs multi-layer residual attention during the downsampling process to capture deep-level features of the image. Subsequently, we have incorporated a Multi-layer Perceptual Attention module (MPA) for each skip connection layer, facilitating the integration of hierarchical upsampled information to restore fine-grained details within the feature maps. The ResU-Net student model enhances network stability through the utilization of residual modules and optimizes skip connections to recover any lost image information during convolutional operations. Lastly, we conducted experimental assessments on multiple disease datasets. The results reveal that the ACC of the Res-Transformer model achieves an impressive 96.9%. Furthermore, through the knowledge distillation method, rich knowledge is effectively transferred to the ResU-Net model, resulting in a remarkable ACC improvement of 7.2%.

APEIOU Integration for Enhanced YOLOV7: Achieving Efficient Plant Disease Detection

Plant diseases can severely hamper plant growth and yield. Currently, these diseases often manifest diverse symptoms, characterized by small targets and high quantities. However, existing algorithms inadequately address these challenges. Therefore, this paper proposes improving plant disease detection by enhancing a YOLOV7-based model. Initially, we strengthen multi-scale feature fusion using the fourth prediction layer. Subsequently, we reduce model parameters and the computational load with the DW-ELAN structure, followed by optimizing the downsampling process using the improved SPD-MP module. Additionally, we enhance the Soft-SimAM attention mechanism to prioritize crucial feature components and suppress irrelevant information. To distinguish overlapping predicted and actual bounding box centroids, we propose the APEIOU loss function and refine the offset formula and grid matching strategy, significantly increasing positive samples. We train the improved model using transfer learning. The experimental results show significant enhancements: the mAP, F1 score, Recall, and Precision are 96.75%, 0.94, 89.69%, and 97.64%, respectively. Compared to the original YOLOV7, the improvements are 5.79%, 7.00%, 9.43%, and 3.30%. The enhanced model outperforms the original, enabling the more precise detection of plant diseases.

TD Swin-UNet: Texture-Driven Swin-UNet with Enhanced Boundary-Wise Perception for Retinal Vessel Segmentation.

Retinal vessel segmentation plays a crucial role in medical image analysis, aiding ophthalmologists in disease diagnosis, monitoring, and treatment guidance. However, due to the complex boundary structure and rich texture features in retinal blood vessel images, existing methods have challenges in the accurate segmentation of blood vessel boundaries. In this study, we propose the texture-driven Swin-UNet with enhanced boundary-wise perception. Firstly, we designed a Cross-level Texture Complementary Module (CTCM) to fuse feature maps at different scales during the encoding stage, thereby recovering detailed features lost in the downsampling process. Additionally, we introduced a Pixel-wise Texture Swin Block (PT Swin Block) to improve the model's ability to localize vessel boundary and contour information. Finally, we introduced an improved Hausdorff distance loss function to further enhance the accuracy of vessel boundary segmentation. The proposed method was evaluated on the DRIVE and CHASEDB1 datasets, and the experimental results demonstrate that our model obtained superior performance in terms of Accuracy (ACC), Sensitivity (SE), Specificity (SP), and F1 score (F1), and the accuracy of vessel boundary segmentation was significantly improved.

GANSharp: High-definition image reconstruction using generative adversarial networks

In the realm of digital imaging, enhancing low-resolution images to high-definition quality is a pivotal challenge, particularly crucial for applications in medical imaging, security, and remote sensing. Traditional methods, primarily relying on basic interpolation techniques, often result in images that lack detail and fidelity. GANSharp introduces an innovative GAN-based framework that substantially improves the generator network, incorporating adversarial and perceptual loss functions for enhanced image reconstruction. The core issue addressed is the loss of critical information during down-sampling processes. To counteract this, we proposed a GAN-based method leveraging deep learning algorithms, trained using sets of both low- and high-resolution images. Our approach, which focuses on expanding the generator network’s size and depth and integrating adversarial and perceptual loss, was thoroughly evaluated on various benchmark datasets. The experimental results showed remarkable outcomes. On the Set5 dataset, our method achieved a PSNR of 34.18 dB and a SSIM of 0.956. Comparatively, on the Set14 dataset, it yielded a PSNR of 31.16 dB and an SSIM of 0.920, and on the B100 dataset, it achieved a PSNR of 30.51 dB and an SSIM of 0.912. These results were superior or comparable to those of existing advanced algorithms, demonstrating the proposed method’s potential in generating high-quality, high-resolution images. Our research underscores the potency of GANs in image super-resolution, making it a promising tool for applications spanning medical diagnostics, security systems, and remote sensing. Future exploration could extend to the utilization of alternative loss functions and novel training techniques, aiming to further refine the efficacy of GAN-based image restoration algorithms.

TD-Net:tiny defect detection network for industrial products

AbstractThe detection of tiny defects in industrial products is important for improving the quality of industrial products and maintaining production safety. Currently, image-based defect detection methods are ineffective in detecting tiny and variously shaped defects. Therefore, this paper proposes a tiny defect detection network (TD-Net) for industrial products to improve the effectiveness of tiny defect detection. TD-Net improves the overall defect detection effect, especially the detection effect of tiny defects, by solving the problems of downsampling of tiny defects, pre-filtering of conflicting deep and shallow semantic information, and cascading fusion of multi-scale information. Specifically, this paper proposes the Defect Downsampling (DD) module to realize the defect information supplementation during the backbone downsampling process and improve the problem that the stepwise convolution easily misses the detection of tiny defects. Meanwhile, the Semantic Information Interaction Module (SIIM) is proposed, which fuses deep and shallow semantic features, and is designed to interact the fused features with shallow features to optimize the detection of tiny defects. Finally, the Scale Information Fusion Module (SIFM) is proposed to improve the Path Aggregation Network (PANet) for cascading fusion and information focus on different scale information, which enables further improvement of defect detection performance of TD-Net. Extensive experimental results on the NEU–DET data set (76.8$$\%$$ % mAP), the Peking University PCB defect data set (96.2$$\%$$ % mAP) and the GC10-DET data set (71.5$$\%$$ % mAP) show that the proposed TD-Net achieves competitive results compared with SOTA methods with the equivalent parameter quantity.

A Novel Deep Learning Network Model for Extracting Lake Water Bodies from Remote Sensing Images

Extraction of lake water bodies from remote sensing images provides reliable data support for water resource management, environmental protection, natural disaster early warning, and scientific research, and helps to promote sustainable development, protect the ecological environment and human health. With reference to the classical encoding-decoding semantic segmentation network, we propose the network model R50A3-LWBENet for lake water body extraction from remote sensing images based on ResNet50 and three attention mechanisms. R50A3-LWBENet model uses ResNet50 for feature extraction, also known as encoding, and squeeze and excitation (SE) block is added to the residual module, which highlights the deeper features of the water body part of the feature map during the down-sampling process, and also takes into account the importance of the feature map channels, which can better capture the multiscale relationship between pixels. After the feature extraction is completed, the convolutional block attention module (CBAM) is added to give the model a global adaptive perception capability and pay more attention to the water body part of the image. The feature map is up-sampled using bilinear interpolation, and the features at different levels are fused, a process also known as decoding, to finalize the extraction of the lake water body. Compared with U-Net, AU-Net, RU-Net, ARU-Net, SER34AUNet, and MU-Net, the R50A3-LWBENet model has the fastest convergence speed and the highest MIoU accuracy with a value of 97.6%, which is able to better combine global and local information, refine the edge contours of the lake’s water body, and have stronger feature extraction capability and segmentation performance.

Event-Based Optical Flow via Transforming Into Motion-Dependent View.

Event cameras respond to temporal dynamics, helping to resolve ambiguities in spatio-temporal changes for optical flow estimation. However, the unique spatio-temporal event distribution challenges the feature extraction, and the direct construction of motion representation through the orthogonal view is less than ideal due to the entanglement of appearance and motion. This paper proposes to transform the orthogonal view into a motion-dependent one for enhancing event-based motion representation and presents a Motion View-based Network (MV-Net) for practical optical flow estimation. Specifically, this motion-dependent view transformation is achieved through the Event View Transformation Module, which captures the relationship between the steepest temporal changes and motion direction, incorporating these temporal cues into the view transformation process for feature gathering. This module includes two phases: extracting the temporal evolution clues by central difference operation in the extraction phase and capturing the motion pattern by evolution-guided deformable convolution in the perception phase. Besides, the MV-Net constructs an eccentric downsampling process to avoid response weakening from the sparsity of events in the downsampling stage. The whole network is trained end-to-end in a self-supervised manner, and the evaluations conducted on four challenging datasets reveal the superior performance of the proposed model compared to state-of-the-art (SOTA) methods.

RAU-Net++: River Channel Extraction Methods for Remote Sensing Images of Cold and Arid Regions

Extracting river channels from remote sensing images is crucial for locating river water bodies and efficiently managing water resources, especially in cold and arid regions. The dynamic nature of river channels in these regions during the flood season necessitates a method that can finely delineate the edges of perennially changing river channels and accurately capture information about variable fine river branches. To address this need, we propose a river channel extraction method designed specifically for detecting fine river branches in remote sensing images within cold and arid regions. The method introduces a novel river attention U-shaped network structure (RAU-Net++), leveraging the rich convolutional features of VGG16 for effective feature extraction. For optimal feature extraction along channel edges and fine river branches, we incorporate a CBAM attention module into the upper sampling area at the end of the encoder. Additionally, a residual attention feature fusion module (RAFF) is embedded at each short jump connection in the dense jump connection. Dense skip connections play a crucial role in extracting detailed texture features from river channel features with varying receptive fields obtained during the downsampling process. The integration of the RAFF module mitigates the loss of river information, optimizing the extraction of lost river detail feature information in the original dense jump connection. This tightens the combination between the detailed texture features of the river and the high-level semantic features. To enhance network performance and reduce pixel-level segmentation errors in medium-resolution remote sensing imagery, we employ a weighted loss function comprising cross-entropy (CE) loss, dice loss, focal loss, and Jaccard loss. The RAU-Net++ demonstrates impressive performance metrics, with precision, IOU, recall, and F1 scores reaching 99.78%, 99.39%, 99.71%, and 99.75%, respectively. Meanwhile, both ED and ED′ of the RAU-Net++ are optimal, with values of 1.411 and 0.003, respectively. Moreover, its effectiveness has been validated on NWPU-RESISC45 datasets. Experimental results conclusively demonstrate the superiority of the proposed network over existing mainstream methods.

Local-enhanced multi-scale aggregation swin transformer for semantic segmentation of high-resolution remote sensing images

ABSTRACT Semantic segmentation of remote sensing images is crucial for various practical applications. In the field of deep learning, convolutional neural network (CNN) has been the primary approach for semantic segmentation over the past decade. Recently, Transformer-based models have achieved superior segmentation performance due to their exceptional global modelling capabilities. However, the Transformer-based models tend to focus more on extracting global contextual information, leading to suboptimal performance in segmenting local edges and difficulties in preserving fine-grained details during the patch token downsampling process. Inspired by the local receptive field of CNN, this article proposes a Local-Enhanced Multi-Scale Aggregation Swin Transformer (LMA-Swin) for semantic segmentation of high-resolution remote sensing images. Specifically, we adopt Swin Transformer as main encoder, introduce convolutional blocks as auxiliary encoder, and design a feature modulation module (FMM) to integrate the local contextual modelling ability of CNN into the Transformer backbone. Additionally, we propose a novel cross-aggregation decoder (CAD) to effectively aggregate shallow edge information and deep semantic information, thereby enhancing the discriminative ability for multi-scale objects. On the ISPRS Vaihingen and Potsdam datasets, experimental results illustrate noteworthy improvement in segmentation performance accomplished through the proposed approach. Code: https://github.com/patricklee16/LMA-Swin.

Csswin-unet: a Swin-unet network for semantic segmentation of remote sensing images by aggregating contextual information and extracting spatial information

ABSTRACTImage interpretation algorithms based on deep learning are becoming increasingly important in land cover information acquisition. We propose CSSwin-unet, a network designed for the semantic segmentation of remote sensing images. CSSwin-unet is based on Swin-unet, follows the U-shaped codec structure of U-Net, but utilizes Swin transformer blocks with superior global modelling capabilities to constitute the codec. In addition, we design a parallel branch in the encoder with a context aggregation module (CAM) to enhance contextual information extraction and alleviate the semantic ambiguity problem resulting from occlusion. To address the problem of semantic information mismatch between codecs and improve the model’s ability to extract spatial information, we constructed a space extraction module (SEM) in the skip connections, which replaces the direct copying of encoder features in Swin-unet. To reduce information loss during the downsampling process and strengthen the segmentation capacity of the network, we designed a feature shrinkage module (FSM) in the downsampling session. We conducted comprehensive ablation experiments on a dataset we produced ourselves and compared the results with other advanced methods. The test results showed significant improvement, with mIoU, mF1, and OA values improving by 2.83%, 2.47%, and 2.05%, respectively, compared to the second best performing model, Swin-unet. The above results prove the excellent performance of CSSwin-unet.

Vertical Layering of Quantized Neural Networks for Heterogeneous Inference.

Although considerable progress has been obtained in neural network quantization for efficient inference, existing methods are not scalable to heterogeneous devices as one dedicated model needs to be trained, transmitted, and stored for one specific hardware setting, incurring considerable costs in model training and maintenance. In this paper, we study a new vertical-layered representation of neural network weights for encapsulating all quantized models into a single one. It represents weights as a group of bits (vertical layers) organized from the most significant bit (also called the basic layer) to less significant bits (enhance layers). Hence, a neural network with an arbitrary quantization precision can be obtained by adding corresponding enhance layers to the basic layer. However, we empirically find that models obtained with existing quantization methods suffer severe performance degradation if adapted to vertical-layered weight representation. To this end, we propose a simple once quantization-aware training (QAT) scheme for obtaining high-performance vertical-layered models. Our design incorporates a cascade downsampling mechanism with the multi-objective optimization employed to train the shared source model weights such that they can be updated simultaneously, considering the performance of all networks. After the model is trained, to construct a vertical-layered network, the lowest bit-width quantized weights become the basic layer, and every bit dropped along the downsampling process act as an enhance layer. Our design is extensively evaluated on CIFAR-100 and ImageNet datasets. Experiments show that the proposed vertical-layered representation and developed once QAT scheme are effective in embodying multiple quantized networks into a single one and allow one-time training, and it delivers comparable performance as that of quantized models tailored to any specific bit-width.