Receptive Fields Research Articles

TRRHA: A two-stream re-parameterized refocusing hybrid attention network for synthesized view quality enhancement

In multi-view video systems, the decoded texture video and its corresponding depth video are utilized to synthesize virtual views from different perspectives using the depth-image-based rendering (DIBR) technology in 3D-high efficiency video coding (3D-HEVC). However, the distortion of the compressed multi-view video and the disocclusion problem in DIBR can easily cause obvious holes and cracks in the synthesized views, degrading the visual quality of the synthesized views. To address this problem, a novel two-stream re-parameterized refocusing hybrid attention (TRRHA) network is proposed to significantly improve the quality of synthesized views. Firstly, a global multi-scale residual information stream is applied to extract the global context information by using refocusing attention module (RAM), and the RAM can detect the contextual feature and adaptively learn channel and spatial attention feature to selectively focus on different areas. Secondly, a local feature pyramid attention information stream is used to fully capture complex local texture details by using re-parameterized refocusing attention module (RRAM). The RRAM can effectively capture multi-scale texture details with different receptive fields, and adaptively adjust channel and spatial weights to adapt to information transformation at different sizes and levels. Finally, an efficient feature fusion module is proposed to effectively fuse the extracted global and local information streams. Extensive experimental results show that the proposed TRRHA achieves significantly better performance than the state-of-the-art methods. The source code will be available at https://github.com/647-bei/TRRHA.

High-precision and lightweight small-target detection algorithm for low-cost edge intelligence

The proliferation of edge devices driven by advancements in Internet of Things (IoT) technology has intensified the challenge of achieving high-precision small target detection, as it demands extensive computational resources. This amplifies the conflict between the need for precise detection and the requirement for cost-efficiency across numerous edge devices. To solve this problem, this paper introduces an enhanced target detection algorithm, MSGD-YOLO, built upon YOLOv8. The Faster Implementation of CSP Bottleneck with 2 convolutions (C2f) module is enhanced through the integration of the Ghost module and dynamic convolution, resulting in a more lightweight architecture while enhancing feature generation. Additionally, Spatial Pyramid Pooling with Enhanced Local Attention Network (SPPELAN) replaces Spatial Pyramid Pooling Fast (SPPF) to expand the receptive field, optimizing multi-level feature aggregation for improved performance. Furthermore, a novel Multi-Scale Ghost Convolution (MSGConv) and Multi-Scale Generalized Feature Pyramid Network (MSGPFN) are introduced to enhance feature fusion and integrate multi-scale information. Finally, four optimized dynamic convolutional detection heads are employed to capture target features more accurately and improve small target detection precision. Evaluation on the VisDrone2019 dataset shows that compared with YOLOv8-n, MSGD-YOLO improves mAP@50 and mAP@50–95 by 14.1% and 11.2%, respectively. In addition, the model not only achieves a 16.1% reduction in parameters but also attains a processing speed of 24.6 Frames Per Second (FPS) on embedded devices, thereby fulfilling real-time detection requirements.

Enhancing a You Only Look Once-Plated Detector via Auxiliary Textual Coding for Multi-Scale Rotating Remote Sensing Objects in Transportation Monitoring Applications

With the rapid development of intelligent information technologies, remote sensing object detection has played an important role in different field applications. Particularly in recent years, it has attracted widespread attention in assisting with food safety supervision, which still faces troubling issues between oversized parameters and low performance that are challenging to solve. Hence, this article proposes a novel remote sensing detection framework for multi-scale objects with a rotating status and mutual occlusion, defined as EYMR-Net. This proposed approach is established on the YOLO-v7 architecture with a Swin Transformer backbone, which offers multi-scale receptive fields to mine massive features. Then, an enhanced attention module is added to exploit the spatial and dimensional interrelationships among different local characteristics. Subsequently, the effective rotating frame regression mechanism via circular smoothing labels is introduced to the EYMR-Net structure, addressing the problem of horizontal YOLO (You Only Look Once) frames ignoring direction changes. Extensive experiments on DOTA datasets demonstrated the outstanding performance of EYMR-Net, which achieved an impressive mAP0.5 of up to 74.3%. Further ablation experiments verified that our proposed approach obtains a balance between performance and efficiency, which is beneficial for practical remote sensing applications in transportation monitoring and supply chain management.

Revolutionizing swarm dynamics: the role of receptive fields in enhancing convergence and stability

The classic Vicsek model, while influential in understanding swarm behavior, has limitations in achieving motion consensus and convergence speed, especially under varying conditions of density and noise. This study aims to introduce a novel receptive field mechanism to the Vicsek model to enhance its performance in terms of motion consensus and convergence speed within swarms. The modified model divides a particle’s surrounding area into excitation and inhibition zones based on distinct functions. This structural modification is designed to enrich evolutionary behavior and improve consensus convergence capabilities. Experimental outcomes indicate that the proposed model achieves faster convergence rates towards motion consensus under various density and noise conditions compared to traditional models. Specifically, while classic Vicsek models fail to converge to an overall polarization state under high noise levels and exhibit quasi-periodic oscillations, the enhanced model demonstrates stable convergence without oscillatory behavior across both low- and high-noise environments. The findings highlight the superior evolutionary consistency characteristics of the improved model, offering new theoretical and practical insights into the stability and controllability of swarms. This advancement presents significant implications for the development of more robust swarm systems.

Improving Differential-Neural Cryptanalysis

Our first objective is to enhance the capabilities of differential-neural distinguishers by applying more deep-learning techniques, focusing on handling more rounds and improving accuracy. Inspired by the Inception Block in GoogLeNet, we adopted a design that uses multiple parallel convolutional layers with varying kernel sizes before the residual block to capture multi-dimensional information. Additionally, we expanded the convolutional kernels in the residual blocks, enlarging the network's receptive field. In the case of Speck32/64, our efforts yield accuracy improvements in rounds 6, 7, and 8, enabling the successful training of a 9-round differential-neural distinguisher. As for Simon32/64, we developed a differential-neural distinguisher capable of effectively handling 12 rounds while achieving noteworthy accuracy enhancements in rounds 9, 10, and 11. Additionally, we utilized neutral bits to ensure the required data distribution for launching a successful key recovery attack when using multiple-ciphertext pairs as input for the neural network. Meanwhile, we redefined the formula for time complexity based on the differences in prediction speeds of the distinguisher between a single-core CPU and a GPU. Combining these various advancements allows us to considerably reduce the time and data complexity of key recovery attacks on 13-round Speck32/64. Furthermore, we used knowledge distillation techniques to reduce the model size, accelerating the distinguisher's prediction speed and reducing the time complexity. In particular, we achieved a successful 14-round key recovery attack by exhaustively guessing a 1-round subkey. For Simon32/64, we accomplished a 17-round key recovery attack for the first time and reduced the time complexity of the 16-round key recovery attack.

DLCH-YOLO: An Object Detection Algorithm for Monitoring the Operation Status of Circuit Breakers in Power Scenarios

In the scenario of power system monitoring, detecting the operating status of circuit breakers is often inaccurate due to variable object scales and background interference. This paper introduces DLCH-YOLO, an object detection algorithm aimed at identifying the operating status of circuit breakers. Firstly, we propose a novel C2f_DLKA module based on Deformable Large Kernel Attention. This module adapts to objects of varying scales within a large receptive field, thereby more effectively extracting multi-scale features. Secondly, we propose a Semantic Screening Feature Pyramid Network designed to fuse multi-scale features. By filtering low-level semantic information, it effectively suppresses background interference to enhance localization accuracy. Finally, the feature extraction network incorporates Generalized-Sparse Convolution, which combines depth-wise separable convolution and channel mixing operations, reducing computational load. The DLCH-YOLO algorithm achieved a 91.8% mAP on our self-built power equipment dataset, representing a 4.7% improvement over the baseline network Yolov8. With its superior detection accuracy and real-time performance, DLCH-YOLO outperforms mainstream detection algorithms. This algorithm provides an efficient and viable solution for circuit breaker status detection.

Advanced power quality monitoring with dilated convolutional neural networks for smart nanogrids

This study presents the development of an advanced power quality monitoring (PQM) system using dilated convolutional deep neural networks (DCDNN) for smart nanogrids. Traditional convolutional neural networks (CNNs) face challenges in efficiently handling high-dimensional data and identifying nuanced features due to their limited receptive field. To overcome these limitations, our DCDNN model employs a novel use of dilated convolutions to expand the receptive field without increasing network complexity, enabling the capture of long-range dependencies essential for accurate classification of power quality disturbances (PQDs). In addition, the integration of multi-channel time-series data processing, including voltage, current, and frequency measurements, is unique to our approach. The model processes multi-channel time-series data, including voltage, current, and frequency measurements, achieving an average accuracy of 98.9% across 29 PQD classes and 100% accuracy in 6 main classes, with a reduced detection time of 47 µs per segment. This approach not only improves real-time disturbance detection but also enhances the reliability and efficiency of electrical systems. The results demonstrate significant improvements over existing methods, underscoring the potential of DCDNNs in advancing PQM practices. Future work will focus on validating the model in real-world conditions and optimizing its computational efficiency.

MixUNETR: A U-shaped network based on W-MSA and depth-wise convolution with channel and spatial interactions for zonal prostate segmentation in MRI

Magnetic resonance imaging (MRI) plays a pivotal role in diagnosing and staging prostate cancer. Precise delineation of the peripheral zone (PZ) and transition zone (TZ) within prostate MRI is essential for accurate diagnosis and subsequent artificial intelligence-driven analysis. However, existing segmentation methods are limited by ambiguous boundaries, shape variations and texture complexities between PZ and TZ. Moreover, they suffer from inadequate modeling capabilities and limited receptive fields. To address these challenges, we propose a Enhanced MixFormer, which integrates window-based multi-head self-attention (W-MSA) and depth-wise convolution with parallel design and cross-branch bidirectional interaction. We further introduce MixUNETR, which use multiple Enhanced MixFormers as encoder to extract features from both PZ and TZ in prostate MRI. This augmentation effectively enlarges the receptive field and enhances the modeling capability of W-MSA, ultimately improving the extraction of both global and local feature information from PZ and TZ, thereby addressing mis-segmentation and challenges in delineating boundaries between them. Extensive experiments were conducted, comparing MixUNETR with several state-of-the-art methods on the Prostate158, ProstateX public datasets and private dataset. The results consistently demonstrate the accuracy and robustness of MixUNETR in MRI prostate segmentation. Our code of methods is available at https://github.com/skyous779/MixUNETR.git.

Multi-scale sensing and multi-dimensional feature enhancement for surface defect detection of hot-rolled steel strip

ABSTRACT Surface defect detection of hot-rolled steel strip is a crucial process in its production. Aiming at the problem of poor detection accuracy caused by complex shape of defect features and imbalance between object scale and feature scale, this paper proposes a multi-scale sensing and multi-dimensional feature enhancement algorithm. Firstly, a multi-branch residual aggregation module (MRAM) is proposed, which uses the interactive features of different receptive fields between layers to capture global and local texture features, improving the feature extraction capability for defects of different scales. Then, the multi-dimensional attention enhancement head (MAEH) is proposed, which integrates feature information from different prediction branches and realises feature information enhancement in scale, space and channel dimensions for the aggregated information to improve the accuracy and robustness of steel surface defect detection. The results show that the proposed algorithm achieves 82.7% mAP and 89 FPS on the NEU-DET dataset, and 73.3% mAP and 94 FPS on the GC10-DET dataset. Compared with state-of-the-art algorithms, the proposed algorithm has the highest mAP, strong generalisation ability, and good real-time performance, which proves its effectiveness in detecting hot-rolled steel strip surface defects.

Computational ghost imaging using the dilated ghost network

Computational ghost imaging has poor imaging quality at low sampling rates. To solve this issue, a dilated ghost network is proposed to improve the quality of noisy images reconstructed by computational ghost imaging in this paper. The network builds upon the traditional GhostNet framework, which minimizes computational costs while maintaining stable performance. By incorporating dilated convolutions, the network expands the receptive field without adding additional parameters. However, the use of dilated convolutions can cause a gridding problem, which degrades the imaging quality. To solve this problem, a sawtooth wave-like structure is introduced into the network. Simulation experiments, combined with metrics such as SSIM, PSNR, and RMSE, validate both the existence of the gridding problem and the effectiveness of the sawtooth wave-like structure in overcoming it. The experimental results indicate that the proposed method significantly outperforms traditional and compressive-sensing-based CGI methods.

SNN using color-opponent and attention mechanisms for object recognition

The current spiking neural network (SNN) relies on spike-timing-dependent plasticity (STDP) primarily for shape learning in object recognition tasks, overlooking the equally critical aspect of color information. To address this gap, our study introduces an unsupervised variant of STDP that incorporates principles from color-opponency mechanisms (COM) and classical receptive fields (CRF) found in the biological visual system, facilitating the integration of color information during parameter updates within the SNN architecture. Our approach initially preprocesses images into two distinct feature maps: one for shape and another for color. Then, signals derived from COM and intensity concurrently drive the STDP process, thereby updating parameters associated with both color and shape feature maps. Furthermore, we propose a channel-wise attention mechanism to enhance differentiation among objects sharing similar shapes or colors. Specifically, this mechanism utilizes convolution to generate an output spike-wave, identifying a winner based on earliest spike timing and maximal potential. The winning kernel computes attention, which is then applied via convolution to each input image feature map, generating post-feature maps. A STDP-like normalization rule compares firing times between pre- and post-feature maps, dynamically adjusting channel weights to optimize object recognition during the training phase.We assessed the proposed algorithm using SNN with both single-layer and multi-layer architectures across three datasets. Experimental findings highlight its efficacy and superiority in complex object recognition tasks compared to state-of-the-art (SOTA) algorithms. Notably, our approach achieved a significant 20% performance improvement over the SOTA on the Caltech-101 dataset. Moreover, the algorithm is well-suited for hardware implementation and energy efficiency, leveraging a winner-selection mechanism based on the earliest spike time.

Positive and Negative Retinotopic Codes in the Human Hippocampus.

The hippocampus is thought to coordinate sensory-mnemonic information streams in the brain, representing both the apex of the visual processing hierarchy and the central hub of mnemonic processing. Yet, the mechanisms underlying sensory-mnemonic interactions in the hippocampus are poorly understood. Recent work in cortex suggests that a retinotopic code - typically thought to be exclusive to visual areas - may help organize internal and external information at the cortical apex via opponent interactions. Here, we leverage high-resolution 7T functional MRI to test whether a bivalent retinotopic code structures activity within the human hippocampus and mediates hippocampal-cortical interactions. In seven densely-sampled individuals, we defined the retinotopic preferences of individual voxels within the hippocampus and cortex during a visual mapping task, as well as their functional connectivity during independent runs of resting-state fixation. Our findings reveal a robust retinotopic code in the hippocampus, characterized by stable population receptive fields (pRFs) with consistent preferred visual field locations across experimental runs. Notably, this retinotopic code is comprised of roughly equal proportions of positive and negative pRFs, aligning with the hypothesized role of negative pRFs in mnemonic processing. Finally, the signed amplitude of hippocampal pRFs predicts functional connectivity between retinotopic hippocampal and cortical voxels. Taken together, these results suggest that retinotopic coding may scaffold internal mnemonic and external sensory information processing within the hippocampus, and across hippocampal-cortical interactions.

Improved Small Object Detection Algorithm CRL-YOLOv5.

Detecting small objects in images poses significant challenges due to their limited pixel representation and the difficulty in extracting sufficient features, often leading to missed or false detections. To address these challenges and enhance detection accuracy, this paper presents an improved small object detection algorithm, CRL-YOLOv5. The proposed approach integrates the Convolutional Block Attention Module (CBAM) attention mechanism into the C3 module of the backbone network, which enhances the localization accuracy of small objects. Additionally, the Receptive Field Block (RFB) module is introduced to expand the model's receptive field, thereby fully leveraging contextual information. Furthermore, the network architecture is restructured to include an additional detection layer specifically for small objects, allowing for deeper feature extraction from shallow layers. When tested on the VisDrone2019 small object dataset, CRL-YOLOv5 achieved an mAP50 of 39.2%, representing a 5.4% improvement over the original YOLOv5, effectively boosting the detection precision for small objects in images.

Multiresolution cascaded attention U-Net for localization and segmentation of optic disc and fovea in fundus images

Identification of retinal diseases in automated screening methods, such as those used in clinical settings or computer-aided diagnosis, usually depends on the localization and segmentation of the Optic Disc (OD) and fovea. However, this task is difficult since these anatomical features have irregular spatial, texture, and shape characteristics, limited sample sizes, and domain shifts due to different data distributions across datasets. This study proposes a novel Multiresolution Cascaded Attention U-Net (MCAU-Net) model that addresses these problems by optimally balancing receptive field size and computational efficiency. The MCAU-Net utilizes two skip connections to accurately localize and segment the OD and fovea in fundus images. We incorporated a Multiresolution Wavelet Pooling Module (MWPM) into the CNN at each stage of U-Net input to compensate for spatial information loss. Additionally, we integrated a cascaded connection of the spatial and channel attentions as a skip connection in MCAU-Net to concentrate precisely on the target object and improve model convergence for segmenting and localizing OD and fovea centers. The proposed model has a low parameter count of 0.8 million, improving computational efficiency and reducing the risk of overfitting. For OD segmentation, the MCAU-Net achieves high IoU values of 0.9771, 0.945, and 0.946 for the DRISHTI-GS, DRIONS-DB, and IDRiD datasets, respectively, outperforming previous results for all three datasets. For the IDRiD dataset, the MCAU-Net locates the OD center with an Euclidean Distance (ED) of 16.90 pixels and the fovea center with an ED of 33.45 pixels, demonstrating its effectiveness in overcoming the common limitations of state-of-the-art methods.

Few-shot bioacoustic event detection utilizing spectro-temporal receptive field

Bioacoustic event detection is a task of identifying specific animal sounds within a biological audio recordings. Specialists only can label temporal information of animal sounds since knowledge for biological sound is demanded for labeling and they can give only few annotations due to high labor intensity caused by large duration of biological recordings. Therefore, the task is set to few-shot learning scheme and method called prototypical network effectively deal with the few-shot learning scheme by learning representation space which represent each class for given few examples. In this work, we utilized spectro-temporal receptive field, which is inspired auditory cortex which responses actively to certain spectro-temporal modulation, as a convolutional layer kernel of prototypical network. Bioacoustic events retain plentiful spectro-temporal modulation that STRF is expected to capture animal sounds effectively. Also, STRF kernels are constructed to fixed shape rather than trained that a model utilizing STRF kernels would learn representation space with less parameter. We built a model called Two Branch STRFNet (TB-STRFNet), in which STRF branch captures spectro-temporal modulation by STRF kernels and the other branch captures detailed time-frequency information which can be smashed in STRF branch. TB-STRFNet outperformed other models showing that effectiveness of auditory system inspired method.

Beyond Granularity: Enhancing Continuous Sign Language Recognition with Granularity-Aware Feature Fusion and Attention Optimization

The advancement of deep learning techniques has significantly propelled the development of the continuous sign language recognition (cSLR) task. However, the spatial feature extraction of sign language videos in the RGB space tends to focus on the overall image information while neglecting the perception of traits at different granularities, such as eye gaze and lip shape, which are more detailed, or posture and gestures, which are more macroscopic. Exploring the efficient fusion of visual information of different granularities is crucial for accurate sign language recognition. In addition, applying a vanilla Transformer to sequence modeling in cSLR exhibits weak performance because specific video frames could interfere with the attention mechanism. These limitations constrain the capability to understand potential semantic characteristics. We introduce a feature fusion method for integrating visual features of disparate granularities and refine the metric of attention to enhance the Transformer’s comprehension of video content. Specifically, we extract CNN feature maps with varying receptive fields and employ a self-attention mechanism to fuse feature maps of different granularities, thereby obtaining multi-scale spatial features of the sign language framework. As for video modeling, we first analyze why the vanilla Transformer failed in cSLR and observe that the magnitude of the feature vectors of video frames could interfere with the distribution of attention weights. Therefore, we utilize the Euclidean distance among vectors to measure the attention weights instead of scaled-dot to enhance dynamic temporal modeling capabilities. Finally, we integrate the two components to construct the model MSF-ET (Multi-Scaled feature Fusion–Euclidean Transformer) for cSLR and train the model end-to-end. We perform experiments on large-scale cSLR benchmarks—PHOENIX-2014 and Chinese Sign Language (CSL)—to validate the effectiveness.

Functional specialization and distributed processing across marmoset lateral prefrontal subregions.

A prominent aspect of primate lateral prefrontal cortex organization is its division into several cytoarchitecturally distinct subregions. Neurophysiological investigations in macaques have provided evidence for the functional specialization of these subregions, but an understanding of the relative representational topography of sensory, social, and cognitive processes within them remains elusive. One explanatory factor is that evidence for functional specialization has been compiled largely from a patchwork of findings across studies, in many animals, and with considerable variation in stimulus sets and tasks. Here, we addressed this by leveraging the common marmoset (Callithrix jacchus) to carry out large-scale neurophysiological mapping of the lateral prefrontal cortex using high-density microelectrode arrays, and a diverse suite of test stimuli including faces, marmoset calls, and spatial working memory task. Task-modulated units and units responsive to visual and auditory stimuli were distributed throughout the lateral prefrontal cortex, while those with saccade-related activity or face-selective responses were restricted to 8aV, 8aD, 10, 46V, and 47. Neurons with contralateral visual receptive fields were limited to areas 8aV and 8aD. These data reveal a mixed pattern of functional specialization in the lateral prefrontal cortex, in which responses to some stimuli and tasks are distributed broadly across lateral prefrontal cortex subregions, while others are more limited in their representation.

Predicting visual function by interpreting a neuronal wiring diagram

As connectomics advances, it will become commonplace to know far more about the structure of a nervous system than about its function. The starting point for many investigations will become neuronal wiring diagrams, which will be interpreted to make theoretical predictions about function. Here I demonstrate this emerging approach with the Drosophila optic lobe, analysing its structure to predict that three Dm3 (refs. 1–4) and three TmY (refs. 2,4) cell types are part of a circuit that serves the function of form vision. Receptive fields are predicted from connectivity, and suggest that the cell types encode the local orientation of a visual stimulus. Extraclassical5,6 receptive fields are also predicted, with implications for robust orientation tuning7, position invariance8,9 and completion of noisy or illusory contours10,11. The TmY types synapse onto neurons that project from the optic lobe to the central brain12,13, which are conjectured to compute conjunctions and disjunctions of oriented features. My predictions can be tested through neurophysiology, which would constrain the parameters and biophysical mechanisms in neural network models of fly vision14.

Nested transformer decoder using dense skip-connections for change detection in high-resolution remote sensing image

ABSTRACT Change detection is a critical task in remote sensing image analysis, with applications in land management and urban development. Convolutional neural networks (CNNs) have shown success in this area, but existing CNN-based methods often struggle with irrelevant changes caused by interfering factors, leading to unsatisfactory results. To address this, some studies have replaced CNN layers with transformers to enhance the receptive field and utilize global information better. However, this can make models overly reliant on global information, potentially overlooking shallow, fine-grained features, especially at the edges of changed targets. To overcome these limitations, we propose a Nested Transformer Decoder (NTD) that combines the strengths of transformers and CNNs by introducing dense skip connections. While having a large receptive field, it can also better retain shallow and fine-grained features by adding skip-connection. These innovations enable our model to more accurately identify changes and preserve the edges of the changed areas, including small changes that are difficult to detect using existing methods. Additionally, we introduce a Residual Fusion Module (RFM) to reduce false detections caused by irrelevant factors like light and shadow. The RFM employs residual information and spatiotemporal attention to suppress differences in multi-scale features, leading to better detection outcomes. The experimental results demonstrate that our proposed model outperforms the current state-of-the-art change detection methods in various metrics, and represents a novel and effective solution to the challenge of change detection in remote sensing images. Experimental results show that our model surpasses state-of-the-art change detection methods across various metrics. Tested on three public change detection datasets, our method improves the Intersection over Union (IoU) for the change class by 2.83% to 16.39% and enhances the F1-score by 1.60% to 9.63%. These results demonstrate that the proposed approach provides an effective solution to the challenges of change detection in remote sensing images.

A transformer-based convolutional method to model inverse cascade in forced two-dimensional turbulence

The present work proposes a novel transformer-based convolutional neural network (TransCNN) method to effectively model the inverse energy cascade in two dimensional (2D) turbulence. The TransCNN structure combines large-scale features extracted by transformer with small-scale features from convolutional layers, thus is considered suitable for multi-scale modeling. The novel TransCNN method has been applied to model sub-grid scale (SGS) stress for large-eddy simulation (LES) of 2D turbulence, under the extremely challenging situation that the LES grid is too coarse to resolve the external forcing scale. The data-driven model trained by the novel TransCNN structure is compared to two deep CNN models with varying complexities. All models exhibit proficiency during a priori tests. Notably, TransCNN surpasses its counterparts in predictive accuracy and generalizability in a posteriori tests. An investigation into the receptive fields reveals that the TransCNN model can efficiently leverage global information with the transformer structure, which is key to its superior performance in representing the inverse energy cascade in the 2D turbulent simulations.