Large Kernel Research Articles

Efficient Optimized YOLOv8 Model with Extended Vision.

In the field of object detection, enhancing algorithm performance in complex scenarios represents a fundamental technological challenge. To address this issue, this paper presents an efficient optimized YOLOv8 model with extended vision (YOLO-EV), which optimizes the performance of the YOLOv8 model through a series of innovative improvement measures and strategies. First, we propose a multi-branch group-enhanced fusion attention (MGEFA) module and integrate it into YOLO-EV, which significantly boosts the model's feature extraction capabilities. Second, we enhance the existing spatial pyramid pooling fast (SPPF) layer by integrating large scale kernel attention (LSKA), improving the model's efficiency in processing spatial information. Additionally, we replace the traditional IOU loss function with the Wise-IOU loss function, thereby enhancing localization accuracy across various target sizes. We also introduce a P6 layer to augment the model's detection capabilities for multi-scale targets. Through network structure optimization, we achieve higher computational efficiency, ensuring that YOLO-EV consumes fewer computational resources than YOLOv8s. In the validation section, preliminary tests on the VOC12 dataset demonstrate YOLO-EV's effectiveness in standard object detection tasks. Moreover, YOLO-EV has been applied to the CottonWeedDet12 and CropWeed datasets, which are characterized by complex scenes, diverse weed morphologies, significant occlusions, and numerous small targets. Experimental results indicate that YOLO-EV exhibits superior detection accuracy in these complex agricultural environments compared to the original YOLOv8s and other state-of-the-art models, effectively identifying and locating various types of weeds, thus demonstrating its significant practical application potential.

DLCH-YOLO: An Object Detection Algorithm for Monitoring the Operation Status of Circuit Breakers in Power Scenarios

In the scenario of power system monitoring, detecting the operating status of circuit breakers is often inaccurate due to variable object scales and background interference. This paper introduces DLCH-YOLO, an object detection algorithm aimed at identifying the operating status of circuit breakers. Firstly, we propose a novel C2f_DLKA module based on Deformable Large Kernel Attention. This module adapts to objects of varying scales within a large receptive field, thereby more effectively extracting multi-scale features. Secondly, we propose a Semantic Screening Feature Pyramid Network designed to fuse multi-scale features. By filtering low-level semantic information, it effectively suppresses background interference to enhance localization accuracy. Finally, the feature extraction network incorporates Generalized-Sparse Convolution, which combines depth-wise separable convolution and channel mixing operations, reducing computational load. The DLCH-YOLO algorithm achieved a 91.8% mAP on our self-built power equipment dataset, representing a 4.7% improvement over the baseline network Yolov8. With its superior detection accuracy and real-time performance, DLCH-YOLO outperforms mainstream detection algorithms. This algorithm provides an efficient and viable solution for circuit breaker status detection.

A Multi-Scale Feature Fusion Based Lightweight Vehicle Target Detection Network on Aerial Optical Images

Vehicle detection with optical remote sensing images has become widely applied in recent years. However, the following challenges have remained unsolved during remote sensing vehicle target detection. These challenges include the dense and arbitrary angles at which vehicles are distributed and which make it difficult to detect them; the extensive model parameter (Param) that blocks real-time detection; the large differences between larger vehicles in terms of their features, which lead to a reduced detection precision; and the way in which the distribution in vehicle datasets is unbalanced and thus not conducive to training. First, this paper constructs a small dataset of vehicles, MiVehicle. This dataset includes 3000 corresponding infrared and visible image pairs, offering a more balanced distribution. In the infrared part of the dataset, the proportions of different vehicle types are as follows: cars, 48%; buses, 19%; trucks, 15%; freight, cars 10%; and vans, 8%. Second, we choose the rotated box mechanism for detection with the model and we build a new vehicle detector, ML-Det, with a novel multi-scale feature fusion triple cross-criss FPN (TCFPN), which can effectively capture the vehicle features in three different positions with an mAP improvement of 1.97%. Moreover, we propose LKC–INVO, which allows involution to couple the structure of multiple large kernel convolutions, resulting in an mAP increase of 2.86%. We also introduce a novel C2F_ContextGuided module with global context perception, which enhances the perception ability of the model in the global scope and minimizes model Params. Eventually, we propose an assemble–disperse attention module to aggregate local features so as to improve the performance. Overall, ML-Det achieved a 3.22% improvement in accuracy while keeping Params almost unchanged. In the self-built small MiVehicle dataset, we achieved 70.44% on visible images and 79.12% on infrared images with 20.1 GFLOPS, 78.8 FPS, and 7.91 M. Additionally, we trained and tested our model on the following public datasets: UAS-AOD and DOTA. ML-Det was found to be ahead of many other advanced target detection algorithms.

DELR-Net: a network for 3D multimodal medical image registration in more lightweight application scenarios.

3D multimodal medical image deformable registration plays a significant role in medical image analysis and diagnosis. However, due to the substantial differences between images of different modalities, registration is challenging and requires high computational costs. Deep learning-based registration methods face these challenges. The primary aim of this paper is to design a 3D multimodal registration network that ensures high-quality registration results while reducing the number of parameters. This study designed a Dual-Encoder More Lightweight Registration Network (DELR-Net). DELR-Net is a low-complexity network that integrates Mamba and ConvNet. The State Space Sequence Module and the Dynamic Large Kernel block are used as the main components of the dual encoders, while the Dynamic Feature Fusion block is used as the main component of the decoder. This study conducted experiments on 3D brain MR images and abdominal MR and CT images. Compared to existing registration methods, DELR-Net achieved better registration results while maintaining a lower number of parameters. Additionally, generalization experiments on other modalities showed that DELR-Net has superior generalization capabilities. DELR-Net significantly improves the limitations of 3D multimodal medical image deformable registration, achieving better registration performance with fewer parameters.

Dual-Path Large Kernel Learning and Its Applications in Single-Image Super-Resolution.

To enhance the performance of super-resolution models, neural networks frequently employ module stacking. However, this approach inevitably results in an excessive proliferation of parameter counts and information redundancy, ultimately constraining the deployment of these models on mobile devices. To surmount this limitation, this study introduces the application of Dual-path Large Kernel Learning (DLKL) to the task of image super-resolution. Within the DLKL framework, we harness a multiscale large kernel decomposition technique to efficiently establish long-range dependencies among pixels. This network not only maintains excellent performance but also significantly mitigates the parameter burden, achieving an optimal balance between network performance and efficiency. When compared with other prevalent algorithms, DLKL exhibits remarkable proficiency in generating images with sharper textures and structures that are more akin to natural ones. It is particularly noteworthy that on the challenging texture dataset Urban100, the network proposed in this study achieved a significant improvement in Peak Signal-to-Noise Ratio (PSNR) for the ×4 upscaling task, with an increase of 0.32 dB and 0.19 dB compared with the state-of-the-art HAFRN and MICU networks, respectively. This remarkable result not only validates the effectiveness of the present model in complex image super-resolution tasks but also highlights its superior performance and unique advantages in the field.

DLKN: enhanced lightweight image super-resolution with dynamic large kernel network

DLKN: enhanced lightweight image super-resolution with dynamic large kernel network

Real-Time ConvNext-Based U-Net with Feature Infusion for Egg Microcrack Detection

Real-time automatic detection of microcracks in eggs is crucial for ensuring egg quality and safety, yet rapid detection of micron-scale cracks remains challenging. This study introduces a real-time ConvNext-Based U-Net model with Feature Infusion (CBU-FI Net) for egg microcrack detection. Leveraging edge features and spatial continuity of cracks, we incorporate an edge feature infusion module in the encoder and design a multi-scale feature aggregation strategy in the decoder to enhance the extraction of both local details and global semantic information. By introducing large convolution kernels and depth-wise separable convolution from ConvNext, the model significantly reduces network parameters compared to the original U-Net. Additionally, a composite loss function is devised to address class imbalance issues. Experimental results on a dataset comprising over 3400 graded egg microcrack image patches demonstrate that CBU-FI Net achieves a reduction in parameters to one-third the amount in the original U-Net, with an inference speed of 21 ms per image (1 million pixels). The model achieves a Crack-IoU of 65.51% for microcracks smaller than 20 μm and a Crack-IoU and MIoU of 60.76% and 80.22%, respectively, for even smaller cracks (less than 5 μm), achieving high-precision, real-time detection of egg microcracks. Furthermore, on the publicly benchmarked CrackSeg9k dataset, CBU-FI Net achieves an inference speed of 4 ms for 400 × 400 resolution images, with an MIoU of 81.38%, proving the proposed method’s robustness and generalization capability across various cracks and complex backgrounds.

Study on the Damage Characteristics of Wheat Kernels under Continuous Compression Conditions.

Peeling wheat yields higher-quality flour. During processing in a flaking machine, wheat kernels undergo continuous compression within the machine's chamber. As this compression persists, damage to the kernels intensifies and accumulates, eventually leading to kernel breakage. To study the damage characteristics of wheat kernels during peeling, this study established a continuous damage model based on Hertzian contact theory and continuous damage theory. The model's accuracy was validated through experiments, culminating in the calculation of critical parameters for wheat peeling. This study focused on different wheat varieties (Ningmai 22 and Jichun 1) and kernel sizes (the thicknesses of the small, medium, and large kernels were standardized as follows: Ningmai 22-2.67 ± 0.07 mm, 2.81 ± 0.07 mm, and 2.95 ± 0.07 mm; Jichun 1-2.98 ± 0.11 mm, 3.20 ± 0.11 mm, and 3.42 ± 0.11 mm). Continuous compression tests were conducted using a mass spectrometer, and critical damage parameters were analyzed and calculated by integrating the theoretical model with experimental data. The test results showed that the average maximum crushing force (Fc) for small, medium, and large-sized kernels of Ningmai 22 was 96.71 ± 2.27 N, 110.17 ± 2.68 N, and 128.41 ± 2.85 N, respectively. The average maximum crushing deformation (αc) was 0.65 ± 0.08 mm, 0.68 ± 0.13 mm, and 0.77 ± 0.17 mm, respectively. The average elastic-plastic critical pressure (Fs) was 50.21 N, 60.13 N, and 59.08 N, respectively, and the average critical values of elastic-plastic deformation (αs) were 0.37 mm, 0.38 mm, and 0.39 mm, respectively. For Jichun 1, the average maximum crushing force (Fc) for small-, medium-, and large-sized kernels was 113.34 ± 3.15 N, 125.28 ± 3.64 N, and 136.15 ± 3.29 N, respectively. The average maximum crushing deformation (αc) was 0.75 ± 0.11 mm, 0.83 ± 0.15 mm, and 0.88 ± 0.18 mm, respectively. The average elastic-plastic critical pressure (Fs) was 58.11 N, 64.17 N, and 85.05 N, respectively, and the average critical values of elastic-plastic deformation (αs) were 0.45 mm, 0.47 mm, and 0.52 mm, respectively. The test results indicated that during mechanical compression, if the deformation is less than αs, the continued application of the compression load will not result in kernel crushing. However, if the deformation exceeds αs, continued compression will lead to kernel crushing, with the required number of compressions decreasing as the deformation increases. If the deformation surpasses αc, a single compression load is sufficient to cause kernel crushing. Since smaller wheat kernels are more susceptible to breakage during processing, the peeling pressure (F) within the chamber should be controlled to remain below the average elastic-plastic critical pressure (Fs) of small-sized wheat kernels. Additionally, the kernel deformation (α) induced by the flow rate and loading in the chamber should be kept below the average elastic-plastic critical deformation (αs) of small-sized wheat kernels. This paper provides a theoretical foundation for the structural design and optimization of processing parameters for wheat peeling machines.

YOLOv8-E: An Improved YOLOv8 Algorithm for Eggplant Disease Detection

During the developmental stages, eggplants are susceptible to diseases, which can impact crop yields and farmers’ economic returns. Therefore, timely and effective detection of eggplant diseases is crucial. Deep learning-based object detection algorithms can automatically extract features from images of eggplants affected by diseases. However, eggplant disease images captured in complex farmland environments present challenges such as varying disease sizes, occlusion, overlap, and small target detection, making it difficult for existing deep-learning models to achieve satisfactory detection performance. To address this challenge, this study proposed an optimized eggplant disease detection algorithm, YOLOv8-E, based on You Only Look Once version 8 nano (YOLOv8n). Firstly, we integrate switchable atrous convolution (SAConv) into the C2f module to design the C2f_SAConv module, replacing some of the C2f modules in the backbone network of YOLOv8n, enabling our proposed algorithm to better extract eggplant disease features. Secondly, to facilitate the deployment of the detection model on mobile devices, we reconstruct the Neck network of YOLOv8n using the SlimNeck module, making the model lighter. Additionally, to tackle the issue of missing small targets, we embed the large separable kernel attention (LSKA) module within SlimNeck, enhancing the model’s attention to fine-grained information. Lastly, we combined intersection over union with auxiliary bounding box (Inner-IoU) and minimum point distance intersection over union (MPDIoU), introducing the Inner-MPDIoU loss to speed up convergence of the model and raise detection precision of overlapped and occluded targets. Ablation studies demonstrated that, compared to YOLOv8n, the mean average precision (mAP) and F1 score of YOLOv8-E reached 79.4% and 75.7%, respectively, which obtained a 5.5% increment and a 4.5% increase, while also reducing the model size and computational complexity. Furthermore, YOLOv8-E achieved higher detection performance than other mainstream algorithms. YOLOv8-E exhibits significant potential for practical application in eggplant disease detection.

Caracterización morfoagronómica de poblaciones de maíz criollo (Zea mays L.), provincia de Manabí, Ecuador

Ecuador is a country with a wide genetic diversity of maize and there are populations of native maize conserved by farmers that have not yet been characterized. These genetic resources could be conserved and used in plant breeding programs. The objective of this research was to characterize the morpho-agronomic diversity of 38 populations of native maize from the province of Manabí, Ecuador, using 19 quantitative and 11 qualitative morpho-agronomic descriptors. During the dry season of 2022 (July - December) at the Portoviejo Experimental Station of the National Institute of Agricultural Research (INIAP), plots of 8 m2 were established for each population of native maize, with 0.3 m between plants and 0.8 m between furrows and each furrow was 5 m long. Cluster analysis showed the formation of four groups, where the populations of hard kernels with large ears and soft kernels with short ears were separated into different groups. The quantitative variables ear height, panicle length, percentage of lodging, number of kernels per row and biomass of the inflorescence rachis recorded "D" indices of 0.75, showing themselves as discriminant variables in the formation of the groups, while the most discriminating qualitative variables were kernel type (χ2 = 49.09***, P= 0.742, V= 0.64), kernel color (χ2= 51.955***, P= 0.75, V=0.64), row arrangement (χ2= 18.11*, P=0.56, V=0.39), and kernel surface shape (χ2 = 20.52*, P=0.58, V=0.41). The native maize races identified were Candela, Cubano, Tuxpeño, Tusilla, and Uchima, observing significant genetic diversity in the populations studied. It was concluded that the characterized native maize populations were a valuable genetic resource for the conservation and use of this cereal. Keywords: genotypes, landraces, genetic variability, conservation of genetic resources.

Boundary-aware convolutional attention network for liver segmentation in ultrasound images

Liver ultrasound is widely used in clinical practice due to its advantages of non-invasiveness, non-radiation, and real-time imaging. Accurate segmentation of the liver region in ultrasound images is essential for accelerating the auxiliary diagnosis of liver-related diseases. This paper proposes BACANet, a deep learning algorithm designed for real-time liver ultrasound segmentation. Our approach utilizes a lightweight network backbone for liver feature extraction and incorporates a convolutional attention mechanism to enhance the network's ability to capture global contextual information. To improve early localization of liver boundaries, we developed a selective large kernel convolution module for boundary feature extraction and introduced explicit liver boundary supervision. Additionally, we designed an enhanced attention gate to efficiently convey liver body and boundary features to the decoder to enhance the feature representation capability. Experimental results across multiple datasets demonstrate that BACANet effectively completes the task of liver ultrasound segmentation, achieving a balance between inference speed and segmentation accuracy. On a public dataset, BACANet achieved a DSC of 0.921 and an IOU of 0.854. On a private test dataset, BACANet achieved a DSC of 0.950 and an IOU of 0.907, with an inference time of approximately 0.32 s per image on a CPU processor.

DFEDC: Dual fusion with enhanced deformable convolution for medical image segmentation

Considering the complexity of lesion regions in medical images, current researches relying on CNNs typically employ large-kernel convolutions to expand the receptive field and enhance segmentation quality. However, these convolution methods are hindered by substantial computational requirements and limited capacity to extract contextual and multi-scale information, making it challenging to efficiently segment complex regions. To address this issue, we propose a dual fusion with enhanced deformable convolution network, namely DFEDC, which dynamically adjusts the receptive field and simultaneously integrates multi-scale feature information to effectively segment complex lesion areas and process boundaries. Firstly, we combine global channel and spatial fusion in a serial way, which integrates and reuses global channel attention and fully connected layers to achieve lightweight extraction of channel and spatial information. Additionally, we design a structured deformable convolution (SDC) that structures deformable convolution with inceptions and large kernel attention, and enhances the learning of offsets through parallel fusion to efficiently extract multi-scale feature information. To compensate for the loss of spatial information of SDC, we introduce a hybrid 2D and 3D feature extraction module to transform feature extraction from a single dimension to a fusion of 2D and 3D. Extensive experimental results on the Synapse, ACDC, and ISIC-2018 datasets demonstrate that our proposed DFEDC achieves superior results.

Dense pedestrian face detection in complex environments

To address the problem of dense crowd face detection in complex environments, this paper proposes a face detection model named Deep and Compact Face Detection (DCFD), which adopts an improved lightweight EfficientNetV2 network to replace the backbone network of RetinaFace. A large kernel attention mechanism is introduced to address the face detection task more accurately. The backbone network, an improved efficient channel attention (ECA) mechanism, is added to further improve the algorithm performance. The feature fusion module is an improved neural architecture search feature pyramid network (NAS-FPN) that significantly improves the face detection accuracy in different scenes. To balance the training process of positive and negative samples, we use the focus loss function to replace the traditional cross-entropy loss function. In different environments, the DCFD algorithm has shown efficient face detection performance. This algorithm provides not only a feasible and effective solution for solving the problem of face detection in dense groups but also an important basis for improving the accuracy of face detection models in practical applications.

Lightweight Substation Equipment Defect Detection Algorithm for Small Targets.

Substation equipment defect detection has always played an important role in equipment operation and maintenance. However, the task scenarios of substation equipment defect detection are complex and different. Recent studies have revealed issues such as a significant missed detection rate for small-sized targets and diminished detection precision. At the same time, the current mainstream detection algorithms are highly complex, which is not conducive to deployment on resource-constrained devices. In view of the above problems, a small target and lightweight substation main scene equipment defect detection algorithm is proposed: Efficient Attentional Lightweight-YOLO (EAL-YOLO), which detection accuracy exceeds the current mainstream model, and the number of parameters and floating point operations (FLOPs) are also advantageous. Firstly, the EfficientFormerV2 is used to optimize the model backbone, and the Large Separable Kernel Attention (LSKA) mechanism has been incorporated into the Spatial Pyramid Pooling Fast (SPPF) to enhance the model's feature extraction capabilities; secondly, a small target neck network Attentional scale Sequence Fusion P2-Neck (ASF2-Neck) is proposed to enhance the model's ability to detect small target defects; finally, in order to facilitate deployment on resource-constrained devices, a lightweight shared convolution detection head module Lightweight Shared Convolutional Head (LSCHead) is proposed. Experiments show that compared with YOLOv8n, EAL-YOLO has improved its accuracy by 2.93 percentage points, and the mAP50 of 12 types of typical equipment defects has reached 92.26%. Concurrently, the quantity of FLOPs and parameters has diminished by 46.5% and 61.17% respectively, in comparison with YOLOv8s, meeting the needs of substation defect detection.

Multi-scale synchronous contextual network for fine-grained urban flow inference

In modern urban planning and traffic management, accurately inferring fine-grained urban flows is crucial. This reduces the need for extensive deployment of sensing devices and yields significant financial savings. However, most existing methods overlook the complexity of flow dependencies in urban areas and the specificity of each external influencing factor when addressing this urban inference challenge. To enhance the accuracy of fine-grained urban flow inference, this paper proposes a novel multi-scale synchronous contextual framework named MS-SCL. This model introduces a multi-scale global modeling module designed to capture comprehensive information about urban traffic dynamics. It integrates linear bottleneck attention and two distinct large kernel convolutions, enabling the synchronous learning of three global representations across different scales. Moreover, MS-SCL employs a multi-view approach to generate individual feature maps for each external influencing factor, rather than merging them together. This strategy provides a more precise differentiation of the impact from each external factor on the traffic flow inference. Extensive experiments conducted on three urban datasets validate that the MS-SCL model outperforms current baseline methods. The source code for the algorithm is available at https://github.com/panlinnn/MS-SCL.

Hybrid Transformer and Convolution for Image Compressed Sensing

In recent years, deep unfolding networks (DUNs) have received widespread attention in the field of compressed sensing (CS) reconstruction due to their good interpretability and strong mapping capabilities. However, existing DUNs often improve the reconstruction effect at the expense of a large number of parameters, and there is the problem of information loss in long-distance feature transmission. Based on the above problems, we propose an unfolded network architecture that mixes Transformer and large kernel convolution to achieve sparse sampling and reconstruction of natural images, namely, a reconstruction network based on Transformer and convolution (TCR-Net). The Transformer framework has the inherent ability to capture global context through a self-attention mechanism, which can effectively solve the challenge of long-range dependence on features. TCR-Net is an end-to-end two-stage architecture. First, a data-driven pre-trained encoder is used to complete the sparse representation and basic feature extraction of image information. Second, a new attention mechanism is introduced to replace the self-attention mechanism in Transformer, and a hybrid Transformer and convolution module based on optimization-inspired is designed. Its iterative process leads to the unfolding framework, which approximates the original image stage by stage. Experimental results show that TCR-Net outperforms existing state-of-the-art CS methods while maintaining fast computational speed. Specifically, when the CS ratio is 0.10, the average PSNR on the test set used in this paper is improved by at least 0.8%, the average SSIM is improved by at least 1.5%, and the processing speed is higher than 70FPS. These quantitative results show that our method has high computational efficiency while ensuring high-quality image restoration.

Field Obstacle Detection and Location Method Based on Binocular Vision

When uncrewed agricultural machinery performs autonomous operations in the field, it inevitably encounters obstacles such as persons, livestock, poles, and stones. Therefore, accurate recognition of obstacles in the field environment is an essential function. To ensure the safety and enhance the operational efficiency of autonomous farming equipment, this study proposes an improved YOLOv8-based field obstacle detection model, leveraging depth information obtained from binocular cameras for precise obstacle localization. The improved model incorporates the Large Separable Kernel Attention (LSKA) module to enhance the extraction of field obstacle features. Additionally, the use of a Poly Kernel Inception (PKI) Block reduces model size while improving obstacle detection across various scales. An auxiliary detection head is also added to improve accuracy. Combining the improved model with binocular cameras allows for the detection of obstacles and their three-dimensional coordinates. Experimental results demonstrate that the improved model achieves a mean average precision (mAP) of 91.8%, representing a 3.4% improvement over the original model, while reducing floating-point operations to 7.9 G (Giga). The improved model exhibits significant advantages compared to other algorithms. In localization accuracy tests, the maximum average error and relative error in the 2–10 m range for the distance between the camera and five types of obstacles were 0.16 m and 2.26%. These findings confirm that the designed model meets the requirements for obstacle detection and localization in field environments.

NPFormer: Interpretable rotating machinery fault diagnosis architecture design under heavy noise operating scenarios

While existing models have achieved significant progress on fault diagnosis tasks, what interpretable temporal dependencies they capture to resist noise remains unclear. Previous studies have ignored the inherent non-stationary property of fault data, which is the basis for capturing reliable temporal dependencies under heavy noise operating scenarios. To tackle the aforementioned problems, we propose a non-stationary perception network (NPFormer). Firstly, we revisit the large kernel design in the early stage of the network, and we demonstrate that this can make the attention map more stable for modeling non-stationary series, which can be a more powerful paradigm with heavy noise interference. Secondly, an adaptive non-stationarity embedding block is developed, which breaks the raw data into multiple components and adaptively attenuates the non-stationarity of mechanical signals for better predictability. Finally, we design multi-scale fusion attention to capture fine-grained features with the global scope, while achieving localized spatial–temporal correlations. Extensive experiments have been conducted on three public datasets and the practical aero-engine engineering application, showing that NPFormer significantly outperforms existing state-of-the-art methods. The weight information is visualized to reveal how NPFormer works, which can demonstrate that the stable attention map is a clear clue of anti-noise. We quantitatively reveal the effect of noise on the model, and the model’s anti-noise clues, during model inference. Code and models will be available for further study.

LKFlowNet: A deep neural network based on large kernel convolution for fast and accurate nonlinear fluid-changing prediction

LKFlowNet: A deep neural network based on large kernel convolution for fast and accurate nonlinear fluid-changing prediction

GlobalSR: Global context network for single image super-resolution via deformable convolution attention and fast Fourier convolution

Vision Transformer have achieved impressive performance in image super-resolution. However, they suffer from low inference speed mainly because of the quadratic complexity of multi-head self-attention (MHSA), which is the key to learning long-range dependencies. On the contrary, most CNN-based methods neglect the important effect of global contextual information, resulting in inaccurate and blurring details. If one can make the best of both Transformers and CNNs, it will achieve a better trade-off between image quality and inference speed. Based on this observation, firstly assume that the main factor affecting the performance in the Transformer-based SR models is the general architecture design, not the specific MHSA component. To verify this, some ablation studies are made by replacing MHSA with large kernel convolutions, alongside other essential module replacements. Surprisingly, the derived models achieve competitive performance. Therefore, a general architecture design GlobalSR is extracted by not specifying the core modules including blocks and domain embeddings of Transformer-based SR models. It also contains three practical guidelines for designing a lightweight SR network utilizing image-level global contextual information to reconstruct SR images. Following the guidelines, the blocks and domain embeddings of GlobalSR are instantiated via Deformable Convolution Attention Block (DCAB) and Fast Fourier Convolution Domain Embedding (FCDE), respectively. The instantiation of general architecture, termed GlobalSR-DF, proposes a DCA to extract the global contextual feature by utilizing Deformable Convolution and a Hadamard product as the attention map at the block level. Meanwhile, the FCDE utilizes the Fast Fourier to transform the input spatial feature into frequency space and then extract image-level global information from it by convolutions. Extensive experiments demonstrate that GlobalSR is the key part in achieving a superior trade-off between SR quality and efficiency. Specifically, our proposed GlobalSR-DF outperforms state-of-the-art CNN-based and ViT-based SISR models regarding accuracy-speed trade-offs with sharp and natural details.