Multi-level Feature Fusion Research Articles

Lightweight railroad semantic segmentation network and distance estimation for railroad Unmanned aerial vehicle images

Derailments significantly harm railroads in terms of severity and fatality rates. Manually monitoring railway tracks is a tedious and often insufficient task to prevent derailment accidents. This necessitates the development of an automated monitoring system to oversee the condition of railway tracks. In the foreseeable future, Unmanned Aerial Vehicles (UAVs), artificial intelligence, and computer vision will play a crucial role in periodically assessing the railway environment to ensure passenger safety. This paper proposes a lightweight and efficient railroad semantic segmentation network (Lite-RSNet), which segments real-time aerial railroad images into rails, gauges, and backgrounds. The segmented regions then compute the distance between rails and obstacles, identifying potential derailment hazards in railroad applications. Lite-RSNet incorporates a multi-level feature fusion framework that includes a residual layer, a multi-head attention layer, a Concurrent Spatial and Channel Squeeze Excitation (CSSE) block, and a Convolutional Block Attention Module (CBAM). The designed approach enhances onboard processing efficiency by streamlining the model to use fewer parameters and reduce storage demands. Performance assessment of the model utilized the Rail Segmentation Dataset (RSD), which consists of high-resolution, low-altitude aerial photographs of Indian railroads. The Lite-RSNet model outperforms current leading methods, maintaining a lean structure with only 6 million parameters and achieving notable metrics of 0.971 for the Dice Coefficient (DC) and 0.947 for the Jaccard Index (JI) on the test dataset. Furthermore, distance estimation between an obstacle and rail has been implemented using image processing techniques as an additional work of this proposal to know about the caution of an obstacle.

Densely integrated multi-branch attentive net for image dehazing

Single image dehazing is a fundamental but challenging task in image processing. Various deep learning-based methods have achieved great dehazing performance. However, there are still hazy residues, even color distortion and texture loss when removing haze from complex outdoor images in dense hazy scenes. A densely integrated multi-branch attentive net for image dehazing is proposed in the paper to address the above problems. The network includes a multi-scale feature extraction module and a dense-feature fusion module. The multi-scale feature extraction module adopts a multi-branch structure composed of residual channel attention blocks, which can expand the receptive field and filter the extracted features of diverse scales by weighting for fusion. It raises network learning accuracy. The dense-feature fusion module contains a multi-level feature fusion module for front and back layers, a color information renovation module, and a feature enhancement module. It achieves dynamically adjusting the channel weights of features at diverse scales, learning rich context information and suppressing redundant information, and compensating for the lack of color and texture information. The dense-feature fusion module bolsters the generalization capability of the network. The proposed method achieves superior objective and subjective evaluation results via quantitative and qualitative experiments on synthetic hazy images and natural hazy images, with better generalization ability and dehazing effect than the current SOTA dehazing algorithms, and effectively ameliorates the color distortion and incomplete dehazing.

Scene classification for remote sensing image of land use and land cover using dual-model architecture with multilevel feature fusion

ABSTRACT Scene classification for remote sensing image (RSI) of land use and land cover (LULC) involves identifying discriminative features of interest in different classes. Spurred by the powerful feature extraction capability of Convolutional Neural Networks (CNNs), LULC classification for RSI has rapidly developed in recent years. Although multi-models have better classification performance than single-models, combining multi-models is still the key to maximizing classification accuracy. Thus, this paper proposes a dual-model architecture with multilevel feature fusion called XE-Net. Specifically, high, middle, and low-level features are extracted by the Xception and EfficientNet-V2, respectively, and through transfer learning, the model’s weighting parameters from the ImageNet data are shared. Moreover, the designed sibling feature fusion algorithm fuses the triple-level features extracted from the dual models sequentially according to the same level. Besides, the proposed multi-scale feature fusion method systematically enhances the fused three-scale features to improve the discriminative feature. Finally, the discriminative feature is input into the classifier to obtain the classification results. The maximum average overall accuracy obtained from sufficient experiments using XE-Net on the RSSCN-7 dataset is 96.84%, while WHU-19, UCM-21, OPTIMAL-31, NWPU-RESISC45, and AID attain 99.58%, 99.37%, 97.07%, 95.03%, and 95.78%, respectively, demonstrating our model’s superiority.

A feature enhanced EEG compression model using asymmetric encoding–decoding network * *This work was supported by the National Natural Science Foundation of China (No. NSFC62227801) and the Lingang Laboratory, Grant No. LG-TKN-202205-01.

Objective. Recently, the demand for wearable devices using electroencephalography (EEG) has increased rapidly in many fields. Due to its volume and computation constraints, wearable devices usually compress and transmit EEG to external devices for analysis. However, current EEG compression algorithms are not tailor-made for wearable devices with limited computing and storage. Firstly, the huge amount of parameters makes it difficult to apply in wearable devices; secondly, it is tricky to learn EEG signals’ distribution law due to the low signal-to-noise ratio, which leads to excessive reconstruction error and suboptimal compression performance. Approach. Here, a feature enhanced asymmetric encoding–decoding network is proposed. EEG is encoded with a lightweight model, and subsequently decoded with a multi-level feature fusion network by extracting the encoded features deeply and reconstructing the signal through a two-branch structure. Main results. On public EEG datasets, motor imagery and event-related potentials, experimental results show that the proposed method has achieved the state of the art compression performance. In addition, the neural representation analysis and the classification performance of the reconstructed EEG signals also show that our method tends to retain more task-related information as the compression ratio increases and retains reliable discriminative information after EEG compression. Significance. This paper tailors an asymmetric EEG compression method for wearable devices that achieves state-of-the-art compression performance in a lightweight manner, paving the way for the application of EEG-based wearable devices.

SLMFNet: Enhancing land cover classification of remote sensing images through selective attentions and multi-level feature fusion.

Land cover classification (LCC) is of paramount importance for assessing environmental changes in remote sensing images (RSIs) as it involves assigning categorical labels to ground objects. The growing availability of multi-source RSIs presents an opportunity for intelligent LCC through semantic segmentation, offering a comprehensive understanding of ground objects. Nonetheless, the heterogeneous appearances of terrains and objects contribute to significant intra-class variance and inter-class similarity at various scales, adding complexity to this task. In response, we introduce SLMFNet, an innovative encoder-decoder segmentation network that adeptly addresses this challenge. To mitigate the sparse and imbalanced distribution of RSIs, we incorporate selective attention modules (SAMs) aimed at enhancing the distinguishability of learned representations by integrating contextual affinities within spatial and channel domains through a compact number of matrix operations. Precisely, the selective position attention module (SPAM) employs spatial pyramid pooling (SPP) to resample feature anchors and compute contextual affinities. In tandem, the selective channel attention module (SCAM) concentrates on capturing channel-wise affinity. Initially, feature maps are aggregated into fewer channels, followed by the generation of pairwise channel attention maps between the aggregated channels and all channels. To harness fine-grained details across multiple scales, we introduce a multi-level feature fusion decoder with data-dependent upsampling (MLFD) to meticulously recover and merge feature maps at diverse scales using a trainable projection matrix. Empirical results on the ISPRS Potsdam and DeepGlobe datasets underscore the superior performance of SLMFNet compared to various state-of-the-art methods. Ablation studies affirm the efficacy and precision of SAMs in the proposed model.

SAR Target Recognition Method based on Adaptive Weighted Decision Fusion of Deep Features

Background: This paper proposes a synthetic aperture radar (SAR) target recognition method based on adaptive weighted decision fusion of multi-level deep features. Methods: The trained ResNet-18 is employed to extract multi-level deep features from SAR images. Afterwards, based on the joint sparse representation (JSR) model, the multi-level deep features are represented to obtain the corresponding reconstruction error vectors. Considering the differences in the abilities of different levels of features to distinguish the target, the reconstruction error vectors are analyzed based on entropy theory, and their corresponding weights are adaptively obtained. Finally, the fused reconstruction error result is obtained through adaptively weighted fusion, and the target label is determined accordingly. Results: Experiments are conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset under different conditions, and the proposed method is compared with published methods, including multi-feature decision fusion, JSR-based decision fusion and other types of ResNets. Conclusion: The experimental results under standard operating condition (SOC) and extended operating conditions (EOCs) including depression angle variance and noise corruption validate the advantages of the proposed method.

Multi-level feature interaction image super-resolution network based on convolutional nonlinear spiking neural model

Image super-resolution (ISR) is designed to recover lost detail information from low-resolution images, resulting in high-quality and high-definition high-resolution images. In the existing single ISR (SISR) methods based on convolutional neural networks (CNN), however, most of the models cannot effectively combine global and local information and are also easy to ignore the correlation between different hierarchical feature information. To address these problems, this study proposes a multi-level feature interactive image super-resolution network, which is constructed by the convolutional units inspired by nonlinear spiking mechanism in nonlinear spiking neural P systems, including shallow feature processing, deep feature extraction and fusion, and reconstruction modules. The different omni domain self-attention blocks are introduced to extract global information in the deep feature extraction and fusion stage and formed a feature enhancement module having a Transformer structure using a novel convolutional unit for extracting local information. Furthermore, to adaptively fuse features between different hierarchies, we design a multi-level feature fusion module, which not only can adaptively fuse features between different hierarchies, but also can better interact with contextual information. The proposed model is compared with 16 state-of-the-art or baseline models on five benchmark datasets. The experimental results show that the proposed model not only achieves good reconstruction performance, but also strikes a good balance between model parameters and performance.

G2ViT: Graph Neural Network-Guided Vision Transformer Enhanced Network for retinal vessel and coronary angiograph segmentation

Blood vessel segmentation is a crucial stage in extracting morphological characteristics of vessels for the clinical diagnosis of fundus and coronary artery disease. However, traditional convolutional neural networks (CNNs) are confined to learning local vessel features, making it challenging to capture the graph structural information and fail to perceive the global context of vessels. Therefore, we propose a novel graph neural network-guided vision transformer enhanced network (G2ViT) for vessel segmentation. G2ViT skillfully orchestrates the Convolutional Neural Network, Graph Neural Network, and Vision Transformer to enhance comprehension of the entire graphical structure of blood vessels. To achieve deeper insights into the global graph structure and higher-level global context cognizance, we investigate a graph neural network-guided vision transformer module. This module constructs graph-structured representation in an unprecedented manner using the high-level features extracted by CNNs for graph reasoning. To increase the receptive field while ensuring minimal loss of edge information, G2ViT introduces a multi-scale edge feature attention module (MEFA), leveraging dilated convolutions with different dilation rates and the Sobel edge detection algorithm to obtain multi-scale edge information of vessels. To avoid critical information loss during upsampling and downsampling, we design a multi-level feature fusion module (MLF2) to fuse complementary information between coarse and fine features. Experiments on retinal vessel datasets (DRIVE, STARE, CHASE_DB1, and HRF) and coronary angiography datasets (DCA1 and CHUAC) indicate that the G2ViT excels in robustness, generality, and applicability. Furthermore, it has acceptable inference time and computational complexity and presents a new solution for blood vessel segmentation.

Residual Dense Swin Transformer for Continuous-Scale Super-Resolution Algorithm

The single-image super-resolution task benefits has a wide range of application scenarios, so has long been a hotspot in the field of computer vision. However, designing a continuous-scale super-resolution algorithm with excellent performance is still a difficult problem to solve. In order to solve this problem, we propose a continuous-scale SR algorithm based on a Transformer, which is called residual dense Swin Transformer (RDST). Firstly, we design a residual dense Transformer block (RDTB) to enhance the information flow before and after the network and extract local fusion features. Then, we use multilevel feature fusion to obtain richer feature information. Finally, we use the upsampling module based on the local implicit image function (LIIF) to obtain continuous-scale super-resolution results. We test RDST on multiple benchmarks. The experimental results show that RDST achieves SOTA performance in the fixed scale of super-resolution tasks in the distribution, and significantly improves (0.1∼0.6 dB) the arbitrary scale of super-resolution tasks out of distribution. Sufficient experiments show that our RDST can use fewer parameters, and its performance is better than the SOTA SR method.

Unsupervised multi-level spatio-spectral fusion transformer for hyperspectral image super-resolution

Fusing a low spatial resolution hyperspectral image (LR-HSI) with a high spatial resolution multispectral image (HR-MSI) is widely used for HSI super-resolution. Recent works still face problems in exploring global spatio-spectral correlation and lack effective utilization of multi-level features of the inputs (i.e., HR-MSI and LR-HSI), which results in a lack of similarity between the reconstruction and the inputs, ultimately causing significant spatio-spectral distortion. To solve these problems, we design an Unsupervised Multi-level Spatio-spectral Fusion Transformer (UMSFT). In UMSFT, a novel multi-level features fusion strategy is constructed, which fuses the hierarchy features of the inputs via proposed Spatio-spectral Feature Fusion Attention Blocks (S2F-FAB) in a level-by-level manner, thereby fully exploring the interaction between the hierarchical features. The S2F-FAB is specially designed for HSI-MSI fusion consisting of two components: (1) a spatial fusion module (Spa-FM) is designed for spatial domain fusion, and its output is set as Values (V) of a transformer; (2) a novel spectral feature cross attention (Spe- FCA) formulates the features of LR-HSI and HR-MSI as Queries (Q) and Keys (K), respectively, and achieves spectral domain fusion by applying attention mechanism along the spectral dimension. Incorporating spatial detail reconstruction and spectral feature integration into the attention mechanism, the S2F-FAB efficiently exploits the spatio-spectral correlation between target HR-HSI and inputs. Experimental results on three public datasets and our real-world images show the superiority of our method as compared with eleven state-of-the-art methods. Codes will be available at https://github.com/Caoxuheng/HIFtool.

Lightweight robotic grasping detection network based on dual attention and inverted residual

Grasping detection is one of the crucial capabilities for robot systems. Deep learning has achieved remarkable outcomes in robot grasping tasks; however, many deep neural networks were at the expense of high computation cost with memory requirements, which hindered their deployment on computing-constrained devices. To solve this problem, this paper proposes an end-to-end lightweight network with dual attention and inverted residual strategies (LiDAIR), which adopts a generative pixel-level prediction to achieve grasp detection. The LiDAIR is composed of the convolution modules (Conv), the inverted residual convolution module (IRCM), the convolutional block attention connection module (CBACM), and the transposed convolution modules (TConv). The Convs are utilized in downsampling processes to extract the input image features. Then, the IRCM is proposed as a bridge between the downsampling and upsampling phases. In the upsampling phase, the CBACM is designed to focus on the valuable regions from spatial and channel dimensions, where the skip connection is employed to attain multi-level feature fusion. Afterwards, the TConvs are used to restore image resolution. The LiDAIR is lightweight with 704K parameters and enjoys a good tradeoff among lightweight structure, accuracy, and speed. It was evaluated on both the Cornell data set and the Jacquard data set within 10 ms inference time, and the detection accuracy on both the data sets were 97.7% and 92.7%, respectively.

UCDCN: a nested architecture based on central difference convolution for face anti-spoofing

The significance of facial anti-spoofing algorithms in enhancing the security of facial recognition systems cannot be overstated. Current approaches aim to compensate for the model’s shortcomings in capturing spatial information by leveraging spatio-temporal information from multiple frames. However, the additional branches to extract inter-frame details increases the model’s parameter count and computational workload, leading to a decrease in inference efficiency. To address this, we have developed a robust and easily deployable facial anti-spoofing algorithm. In this paper, we propose Central Difference Convolution UNet++ (UCDCN), which takes advantage of central difference convolution and improves the characterization ability of invariant details in diverse environments. Particularly, we leverage domain knowledge from image segmentation and propose a multi-level feature fusion network structure to enhance the model’s ability to capture semantic information which is beneficial for face anti-spoofing tasks. In this manner, UCDCN greatly reduces the number of model parameters as well as achieves satisfactory metrics on three popular benchmarks, i.e., Replay-Attack, Oulu-NPU and SiW.

Patchlpr: a multi-level feature fusion transformer network for LiDAR-based place recognition

Patchlpr: a multi-level feature fusion transformer network for LiDAR-based place recognition

Research on Visual Inspection Method of Tree Whitening Quality based on Multi-level Feature Fusion

Background: At present, the traditional manual whitewashing method is used in tree whitewashing in China. The quality of tree whitening can only be judged by the naked eye. Objective: Up to now, the quality of tree whitewashing is still judged manually. In order to improve work efficiency, an automatic evaluation method of tree whitewashing quality based on multi-level feature fusion is proposed. Methods: The images extract texture features from white-washed trees (gray-level co-occurrence matrix) and shape features (gradient direction histogram) from pixel-level fusion to obtain a global characteristic matrix. Using a support vector machine (SVM), random forests, and clustering algorithm (KNN), three classifiers were selected to identify the integration characteristics of the training test. In order to reduce the correlation of feature information and improve the execution efficiency of the classifier, the global feature matrix was optimized by combining principal component analysis (PCA) with pixel-level fusion and feature-level fusion. Results: The experiment results showed that the classification accuracy of the support vector machine is 94.00%, which is higher than that of the random forest classifier (92.67) and KNN classifier (92.67%). Meanwhile, the support vector machine is superior to the other two classification algorithms in recall rate, accuracy rate, and algorithm execution efficiency. The results showed that the execution efficiency of each classification algorithm was improved after the optimization of the feature data. The support vector machine classification algorithm is more stable than the other two algorithms. Conclusion: The feature fusion method combined with PCA can improve the execution efficiency and recognition precision of classifiers to a certain extent. For the feature matrix obtained by different data processing methods, the SVM classifier performs more stably and reliably than the random forest classifier and KNN in the inspection of tree whitening quality.

Multi-level feature fusion and joint refinement for simultaneous object pose estimation and camera localization

Object pose estimation and camera localization are critical in various applications. However, achieving algorithm universality, which refers to category-level pose estimation and scene-independent camera localization, presents challenges for both techniques. Although the two tasks keep close relationships due to spatial geometry constraints, different tasks require distinct feature extractions. This paper pays attention to a unified RGB-D based framework that simultaneously performs category-level object pose estimation and scene-independent camera localization. The framework consists of a pose estimation branch called SLO-ObjNet, a localization branch called SLO-LocNet, a pose confidence calculation process and object-level optimization. At the start, we obtain the initial camera and object results from SLO-LocNet and SLO-ObjNet. In these two networks, we design there-level feature fusion modules as well as the loss function to achieve feature sharing between two tasks. Then the proposed approach involves a confidence calculation process to determine the accuracy of object poses obtained. Additionally, an object-level Bundle Adjustment (BA) optimization algorithm is further used to improve the precision of these techniques. The BA algorithm establishes relationships among feature points, objects, and cameras with the usage of camera-point, camera-object, and object-point metrics. To evaluate the performance of this approach, experiments are conducted on localization and pose estimation datasets including REAL275, CAMERA25, LineMOD, YCB-Video, 7 Scenes, ScanNet and TUM RGB-D. The results show that this approach outperforms existing methods in terms of both estimation and localization accuracy. Additionally, SLO-LocNet and SLO-ObjNet are trained on ScanNet data and tested on 7 Scenes and TUM RGB-D datasets to demonstrate its universality performance. Finally, we also highlight the positive effects of fusion modules, loss function, confidence process and BA for improving overall performance.

PGKD-Net: Prior-guided and Knowledge Diffusive Network for Choroid Segmentation

The thickness of the choroid is considered to be an important indicator of clinical diagnosis. Therefore, accurate choroid segmentation in retinal OCT images is crucial for monitoring various ophthalmic diseases. However, this is still challenging due to the blurry boundaries and interference from other lesions. To address these issues, we propose a novel prior-guided and knowledge diffusive network (PGKD-Net) to fully utilize retinal structural information to highlight choroidal region features and boost segmentation performance. Specifically, it is composed of two parts: a Prior-mask Guided Network (PG-Net) for coarse segmentation and a Knowledge Diffusive Network (KD-Net) for fine segmentation. In addition, we design two novel feature enhancement modules, Multi-Scale Context Aggregation (MSCA) and Multi-Level Feature Fusion (MLFF). The MSCA module captures the long-distance dependencies between features from different receptive fields and improves the model’s ability to learn global context. The MLFF module integrates the cascaded context knowledge learned from PG-Net to benefit fine-level segmentation. Comprehensive experiments are conducted to evaluate the performance of the proposed PGKD-Net. Experimental results show that our proposed method achieves superior segmentation accuracy over other state-of-the-art methods. Our code is made up publicly available at: https://github.com/yzh-hdu/choroid-segmentation.

SiamRAAN: Siamese Residual Attentional Aggregation Network for Visual Object Tracking

The Siamese network-based tracker calculates object templates and search images independently, and the template features are not updated online when performing object tracking. Adapting to interference scenarios with performance-guaranteed tracking accuracy when background clutter, illumination variation or partial occlusion occurs in the search area is a challenging task. To effectively address the issue with the abovementioned interference and to improve location accuracy, this paper devises a Siamese residual attentional aggregation network framework for self-adaptive feature implicit updating. First, SiamRAAN introduces Self-RAAN into the backbone network by applying residual self-attention to extract effective objective features. Then, we introduce Cross-RAAN to update the template features online by focusing on the high-relevance parts in the feature extraction process of both the object template and search image. Finally, a multilevel feature fusion module is introduced to fuse the RAAN-enhanced feature information and improve the network’s ability to perceive key features. Extensive experiments conducted on benchmark datasets (GOT-10K, LaSOT, OTB-50, OTB-100 and UAV123) demonstrated that our SiamRAAN delivers excellent performance and runs at 51 FPS in various challenging object tracking tasks. Code is available at https://github.com/MallowYi/SiamRAAN.

Lightweight camouflaged object detection model based on multilevel feature fusion

The intrinsic similarity between camouflaged objects and background environment impedes the automatic detection/segmentation of camouflaged objects, and novel network architectures for deep learning are promising to overcome this challenge and improve detection accuracy. However, these existing network architectures for distinguishing between camouflaged objects and their backgrounds do not account for the constraint of detection speed, which results in high computational complexity and the inability to meet the requirements of rapid detection. Therefore, based on the human visual system, this study proposes a single-stage lightweight camouflage object detection network using multilevel feature fusion, integrating features of various feature layers and receptive field sizes. Using three benchmark datasets for normal camouflaged objects, the lightweight network (LINet) model demonstrated an accuracy superior to those of six existing mainstream camouflaged object detection methods. Its detection speed, 126.3 frames per second, is significantly higher than those of the existing mainstream methods, enabling rapid detection with a maximum increase of 187.62%. The accuracy of LINet is the minimum and maximum for Resnet101 and Resnet152, respectively. These findings pave the way for diverse applications of camouflaged target detection algorithms.

A robust multi-branch multi-attention-mechanism EEGNet for motor imagery BCI decoding

BackgroundMotor-Imagery-based Brain-Computer Interface (MI-BCI) is a promising technology to assist communication, movement, and neurological rehabilitation for motor-impaired individuals. Electroencephalography (EEG) decoding techniques using deep learning (DL) possess noteworthy advantages due to automatic feature extraction and end-to-end learning. However, the DL-based EEG decoding models tend to show large variations due to intersubject variability of EEG, which results from inconsistencies of different subjects’ optimal hyperparameters. New methodsThis study proposes a multi-branch multi-attention mechanism EEGNet model (MBMANet) for robust decoding. It applies the multi-branch EEGNet structure to achieve various feature extractions. Further, the different attention mechanisms introduced in each branch attain diverse adaptive weight adjustments. This combination of multi-branch and multi-attention mechanisms allows for multi-level feature fusion to provide robust decoding for different subjects. ResultsThe MBMANet model has a four-classification accuracy of 83.18% and kappa of 0.776 on the BCI Competition IV-2a dataset, which outperforms other eight CNN-based decoding models. This consistently satisfactory performance across all nine subjects indicates that the proposed model is robust. ConclusionsThe combine of multi-branch and multi-attention mechanisms empowers the DL-based models to adaptively learn different EEG features, which provides a feasible solution for dealing with data variability. It also gives the MBMANet model more accurate decoding of motion intentions and lower training costs, thus improving the MI-BCI’s utility and robustness.

Two-Level Spatio-Temporal Feature Fused Two-Stream Network for Micro-Expression Recognition.

Micro-expressions, which are spontaneous and difficult to suppress, reveal a person's true emotions. They are characterized by short duration and low intensity, making the task of micro-expression recognition challenging in the field of emotion computing. In recent years, deep learning-based feature extraction and fusion techniques have been widely used for micro-expression recognition, particularly methods based on Vision Transformer that have gained popularity. However, the Vision Transformer-based architecture used in micro-expression recognition involves a significant amount of invalid computation. Additionally, in the traditional two-stream architecture, although separate streams are combined through late fusion, only the output features from the deepest level of the network are utilized for classification, thus limiting the network's ability to capture subtle details due to the lack of fine-grained information. To address these issues, we propose a new two-level spatio-temporal feature fused with a two-stream architecture. This architecture includes a spatial encoder (modified ResNet) for learning texture features of the face, a temporal encoder (Swin Transformer) for learning facial muscle motor features, a feature fusion algorithm for integrating multi-level spatio-temporal features, a classification head, and a weighted average operator for temporal aggregation. The two-stream architecture has the advantage of extracting richer features compared to the single-stream architecture, leading to improved performance. The shifted window scheme of Swin Transformer restricts self-attention computation to non-overlapping local windows and allows cross-window connections, significantly improving the performance and reducing the computation compared to Vision Transformer. Moreover, the modified ResNet is computationally less intensive. Our proposed feature fusion algorithm leverages the similarity in output feature shapes at each stage of the two streams, enabling the effective fusion of multi-level spatio-temporal features. This algorithm results in an improvement of approximately 4% in both the F1 score and the UAR. Comprehensive evaluations conducted on three widely used spontaneous micro-expression datasets (SMIC-HS, CASME II, and SAMM) consistently demonstrate the superiority of our approach over comparative methods. Notably, our approach achieves a UAR exceeding 0.905 on CASME II, making it one of the few frameworks in the published micro-expression recognition literature to achieve such high performance.