Feature Enhancement Module Research Articles

Image forgery localization integrating multi-scale and boundary features

Abstract Image forgery localization identifies tampered regions within an image by extracting distinctive forgery features. Current methods mainly use convolutional neural networks (CNNs) to extract features. However, CNNs’ limited receptive field emphasizes local features, impeding the global modeling of crucial lower-level features like edges and textures, leading to decreased precision. Moreover, prior methods use pyramid networks for multi-scale feature extraction but show deficiencies in multi-scale and interlayer modeling, leading to inadequate multi-scale information representation and limiting flexibility to tampered regions of varying sizes. To address these issues, this paper proposes a Transformer-based model integrating multi-scale and boundary features. The model employs a Pyramid Vision Transformer as the encoder, using self-attention over convolution to enhance global context modeling. Building on this, the model incorporates a multi-scale feature enhancement module that enriches forgery features by paralleling various convolutional layers. Features at various encoder stages are integrated through a cross-stage interaction module, enabling multi-level feature correlation for a strong feature representation. Furthermore, the model includes a forgery boundary information-guided branch, which focuses precisely on tampered region structures without introducing irrelevant noise. Experiments demonstrate that our model surpasses previous methods in localization accuracy, with F1 and AUC improving by 8.5% and 2.2% in pre-training, respectively.

A Lightweight Hand Attitude Estimation Method Based on GCN Feature Enhancement

In this study, a hand pose estimation method based on GCN feature enhancement is proposed to address the problem of the time-consuming nature and neglection of the internal relationships between hand joint points, which results in the low accuracy of hand pose estimation. Firstly, a lightweight feature extraction network RexNet is used, and deep separable convolutions are used instead of ordinary convolutions to reduce the model parameters and computational complexity. Secondly, deconvolution is added to the backend of the network to obtain preliminary estimation results of joint points. Finally, the GCN feature enhancement module is used to modify the preliminary estimation results to improve the accuracy of hand pose estimation. The proposed method is tested for accuracy on the CMU-Hand and RHD datasets. The results show that the proposed method achieves an AUC metric of 80.1% on the CMU-Hand dataset and 97.0% on the RHD dataset, and the accuracy of hand pose estimation is high.

Efficient indoor 3D object detection in point clouds using the Kinect sensor

Abstract Only using the point clouds collected by a Kinect sensor to perform 3D object detection is a very challenging task for indoor sense understanding. In this paper, we propose a novel point clouds based 3D object detection framework named Weighted VoteNet (W-VoteNet), which systematically enhances seed points, voting points, and proposal objects to varying degree, thereby enhancing the detection accuracy of indoor scene point clouds. Firstly, a new neighborhood feature enhancement module was designed to enhance the features of the seed point, which can better utilize the neighborhood feature of the seed point. Secondly, a weighted voting module is implemented to increase the accuracy of voting, allowing more prospect seed points to participate in voting. Finally, a new semantic relation reasoning module is proposed to get the semantic relationship features of the proposal object, which can further decrease false alarm rate. The three modules we proposed all contribute to more accurate voting and more effective proposals. The experimental results on two benchmark indoor 3D object detection datasets, SUN RGB-D and ScanNet V2, demonstrate the effectiveness of our approach. The source code is available at: https://github.com/Guo-JQ/W-VoteNet.

Refined feature enhancement network for object detection

Convolutional neural networks-based object detection techniques have achieved positive performances. However, due to the limitations of local receptive field, some existing object detection methods cannot effectively capture global information in feature extraction phases, and thus lead to unsatisfactory detection performance. Moreover, the feature information extracted by the backbone network may be redundant. To alleviate these problems, in this paper we propose a refined feature enhancement network (RFENet) for object detection. Specifically, we first propose a feature enhancement module (FEM) to capture more global and local information from feature maps with certain long-range dependencies. We further propose a multi-branch dilated attention mechanism (MDAM) to refine the extracted features in a weighted form, which can select more important spatial and channel information and broaden the receptive field of the network. Finally, we validate RFENet on MS-COCO2017, PASCAL VOC2012, and PASCAL VOC07+12 datasets, respectively. Compared to the baseline network, our RFENet improves by 2.4 AP on MS-COCO2017 dataset, 3.4 mAP on PASCAL VOC2012 dataset, and 2.7 mAP on PASCAL VOC07+12 dataset. Extensive experiments show that our RFENet can perform competitively on different datasets. The code is available at https://github.com/object9detection/RFENet.

Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection

Due to the increasing sophistication of face forgery techniques, the images generated are becoming more and more realistic and difficult for human eyes to distinguish. These face forgery techniques can cause problems such as fraud and social engineering attacks in facial recognition and identity verification areas. Therefore, researchers have worked on face forgery detection studies and have made significant progress. Current face forgery detection algorithms achieve high detection accuracy within-dataset. However, it is difficult to achieve satisfactory generalization performance in cross-dataset scenarios. In order to improve the cross-dataset detection performance of the model, this paper proposes a multi-feature fusion network based on two-stream extraction and multi-scale enhancement. First, we design a two-stream feature extraction module to obtain richer feature information. Secondly, the multi-scale feature enhancement module is proposed to focus the model more on information related to the current sub-region from different scales. Finally, the forgery detection module calculates the overlap between the features of the input image and real images during the training phase to determine the forgery regions. The method encourages the model to mine forgery features and learns generic and robust features not limited to a particular feature. Thus, the model achieves high detection accuracy and performance. We achieve the AUC of 99.70% and 90.71% on FaceForensics++ and WildDeepfake datasets. The generalization experiments on Celeb-DF-v2 and WildDeepfake datasets achieve the AUC of 80.16% and 65.15%. Comparison experiments with multiple methods on other benchmark datasets confirm the superior generalization performance of our proposed method while ensuring model detection accuracy. Our code can be found at: https://github.com/1241128239/MFNet.

SFFNet: Shallow Feature Fusion Network Based on Detection Framework for Infrared Small Target Detection

Infrared small target detection (IRSTD) is the process of recognizing and distinguishing small targets from infrared images that are obstructed by crowded backgrounds. This technique is used in various areas, including ground monitoring, flight navigation, and so on. However, due to complex backgrounds and the loss of information in deep networks, infrared small target detection remains a difficult undertaking. To solve the above problems, we present a shallow feature fusion network (SFFNet) based on detection framework. Specifically, we design the shallow-layer-guided feature enhancement (SLGFE) module, which guides multi-scale feature fusion with shallow layer information, effectively mitigating the loss of information in deep networks. Then, we design the visual-Mamba-based global information extension (VMamba-GIE) module, which leverages a multi-branch structure combining the capability of convolutional layers to extract features in local space with the advantages of state space models in the exploration of long-distance information. The design significantly extends the network’s capacity to acquire global contextual information, enhancing its capability to handle complex backgrounds. And through the effective fusion of the SLGFE and VMamba-GIE modules, the exorbitant computation brought by the SLGFE module is substantially reduced. The experimental results on two publicly available infrared small target datasets demonstrate that the SFFNet surpasses other state-of-the-art algorithms.

LEM-Detector: An Efficient Detector for Photovoltaic Panel Defect Detection

Photovoltaic panel defect detection presents significant challenges due to the wide range of defect scales, diverse defect types, and severe background interference, often leading to a high rate of false positives and missed detections. To address these challenges, this paper proposes the LEM-Detector, an efficient end-to-end photovoltaic panel defect detector based on the transformer architecture. To address the low detection accuracy for Crack and Star crack defects and the imbalanced dataset, a novel data augmentation method, the Linear Feature Augmentation (LFA) module, specifically designed for linear features, is introduced. LFA effectively improves model training performance and robustness. Furthermore, the Efficient Feature Enhancement Module (EFEM) is presented to enhance the receptive field, suppress redundant information, and emphasize meaningful features. To handle defects of varying scales, complementary semantic information from different feature layers is leveraged for enhanced feature fusion. A Multi-Scale Multi-Feature Pyramid Network (MMFPN) is employed to selectively aggregate boundary and category information, thereby improving the accuracy of multi-scale target recognition. Experimental results on a large-scale photovoltaic panel dataset demonstrate that the LEM-Detector achieves a detection accuracy of 94.7% for multi-scale defects, outperforming several state-of-the-art methods. This approach effectively addresses the challenges of photovoltaic panel defect detection, paving the way for more reliable and accurate defect identification systems. This research will contribute to the automatic detection of surface defects in industrial production, ultimately enhancing production efficiency.

Defect intelligent recognition of membrane product based on deep learning

Defect detection plays a crucial role in the manufacturing industry, ensuring the quality of industrial products. Despite advancements in this field, current defect detection methods face two primary challenges: (1) extracting visually similar features from the background poses difficult and (2) these methods struggle to identify tiny defects in the target objects. We present a feature enhancement module called the SE-CAR, which aims to handle the identified problems effectively. This module is designed to efficiently capture tiny defects in images, prioritize defect information features, and ultimately enhance the model’s predictive performance and accuracy in defect recognition tasks. In addition, Distance-IoU Non-Maximum Suppression is employed as a substitution for the original Non-Maximum Suppression. This enhances the recognition accuracy of bounding boxes, ensuring that the model maintains high detection accuracy even after complex scenarios. Moreover, the proposed methodology exhibits broad applicability across a wide spectrum of prevalent defect detection paradigms. In empirical experiments, we employ the YOLOv7 architecture as the foundation framework, integrating the proposed methodologies for the purpose of detecting defects in the membrane. The empirical evidence demonstrates a notable improvement in the performance of detecting defects in membrane products using the SE-CAR feature enhancement module and Distance-IoU Non-Maximum Suppression algorithm. In comparison to baseline networks, an increase in mAp by 6.29%, precision by 1.63%, and recall by 8.76%.

Underwater Acoustic Target Recognition Based on Sub-Regional Feature Enhancement and Multi-Activated Channel Aggregation

Feature selection and fusion in ship radiated noise-based underwater target recognition have remained challenging tasks. This paper proposes a novel feature extraction method based on multi-dimensional feature selection and fusion. Redundant features are filtered through feature visualization techniques. The Sub-regional Feature Enhancement modules (SFE) and Multi-activated Channel Aggregation modules (MCA) within the neural network are utilized to achieve underwater target recognition. Experimental results indicate that our network, named Sub-Regional Channel Aggregation Net (SRCA-Net), utilizing 3-s sound segments for ship radiated noise recognition, surpasses existing models, achieving an accuracy of 78.52% on the public DeepShip dataset.

PBNet: Combining Transformer and CNN in Passport Background Texture Printing Image Classification

Passport background texture classification has always been an important task in border checks. Current manual methods struggle to achieve satisfactory results in terms of consistency and stability for weakly textured background images. For this reason, this study designs and develops a CNN and Transformer complementary network (PBNet) for passport background texture image classification. We first design two encoders by Transformer and CNN to produce complementary features in the Transformer and CNN domains, respectively. Then, we cross-wisely concatenate these complementary features to propose a feature enhancement module (FEM) for effectively blending them. In addition, we introduce focal loss to relieve the overfitting problem caused by data imbalance. Experimental results show that our PBNet significantly surpasses the state-of-the-art image segmentation models based on CNNs, Transformers, and even Transformer and CNN combined models designed for passport background texture image classification.

Multi-level feature fusion network for neuronal morphology classification.

Neuronal morphology can be represented using various feature representations, such as hand-crafted morphometrics and deep features. These features are complementary to each other, contributing to improving performance. However, existing classification methods only utilize a single feature representation or simply concatenate different features without fully considering their complementarity. Therefore, their performance is limited and can be further improved. In this paper, we propose a multi-level feature fusion network that fully utilizes diverse feature representations and their complementarity to effectively describe neuronal morphology and improve performance. Specifically, we devise a Multi-Level Fusion Module (MLFM) and incorporate it into each feature extraction block. It can facilitate the interaction between different features and achieve effective feature fusion at multiple levels. The MLFM comprises a channel attention-based Feature Enhancement Module (FEM) and a cross-attention-based Feature Interaction Module (FIM). The FEM is used to enhance robust morphological feature presentations, while the FIM mines and propagates complementary information across different feature presentations. In this way, our feature fusion network ultimately yields a more distinctive neuronal morphology descriptor that can effectively characterize neurons than any singular morphological representation. Experimental results show that our method effectively depicts neuronal morphology and correctly classifies 10-type neurons on the NeuronMorpho-10 dataset with an accuracy of 95.18%, outperforming other approaches. Moreover, our method performs well on the NeuronMorpho-12 and NeuronMorpho-17 datasets and possesses good generalization.

Visual tracking algorithm based on template updating and dual feature enhancement

Aiming at the problem of tracking failure due to target deformation, flipping and occlusion in visual tracking, a template updating algorithm based on image structural similarity is proposed by dynamically updating the template to adapt to the changes of the target during tracking, specifically, a queue is used to save the recent N -frame tracking results, and a decision is made on whether to update the template or not based on the structural similarity score between the current tracking results and the template image, and if updated, the template is matched from the historical N -frame tracking results as the new template. If updated, the optimal target image is matched from the historical N -frame tracking results as a new template for subsequent tracking. The tracking feature enhancement module and segmentation feature enhancement module are also designed based on SiamMask network. The tracking feature enhancement module consists of non-local operations and convolutional downsampling, which is used to establish contextual correlation, enhance the target features, suppress the background interference, improve the tracking robustness, and solve the feature attenuation problem due to the occlusion of the target. The segmentation feature enhancement module introduces the convolutional block attention module and deformable convolution to improve the network's ability to capture channel and spatial features, adaptively learn the shape and contour information of the target, and enhance the network's segmentation accuracy of the tracked target. , which in turn improves the tracking accuracy. Experiments show that the proposed algorithm performs well and stably in solving the above problems, improving the expected average overlap rate by 5.2%, 5.3%, and 2.5%, and the robustness by 6%, 7.9%, and 15.6%on the VOT2016, VOT2018, and VOT2019 datasets , respectively, and achieving a real-time speed of 91 frames per second, when compared to the baseline SiamMask.

Multi-feature fusion and multi-attention deep network for enhancing road extraction in remote sensing images

ABSTRACT Road detection in remote sensing (RS) images plays a critical role in applications ranging from urban planning to autonomous navigation systems. However, accurate road extraction remains a challenging task due to the presence of textual-similar objects that can be visually confused with roads, and shadows that can obscure road features. For this regard, we present a novel end-to-end network that leverages multi-feature fusion and multi-attention. Multi-level dual residual blocks are introduced to capture multi-scale and multi-level road features. A channel attention feature fusion module fuses road features from the decoder while suppressing coarse-grain noises. Moreover, a difference channel feature fusion module filters out the interfering texture-similar features and irrelevant background features. Subsequently, a spatial channel feature enhancement module is designed to identify the mis-segmented regions and reclassify these pixels. We conducted extensive experiments on diverse datasets, demonstrating the effectiveness of our approach in improving road detection accuracy, especially in areas with shadows and textual-similar objects. The results indicate that our approach outperforms state-of-the-art methods in handling textual-similar objects and shadow-occluded roads, making it a valuable contribution to the field of road extraction in remote sensing images.

SODD-YOLOv8: an insulator defect detection algorithm based on feature enhancement and variable row convolution

Abstract To address the detection challenges in defective images of insulators in transmission lines, including tiny object size, significant scale variations, a wide variety of defects, and complex background interference. In this study, an improved insulator defect detection algorithm is proposed, based on the YOLOv8s framework and combining feature enhancement and deformable convolution techniques. Firstly, to address the image feature distortion problem caused by aerial photography, a deformable convolutional feature extraction module (DCFEM) is introduced, which is designed to enhance the model’s ability to adapt to the local geometric deformation, so as to effectively recover the distorted feature information in the image. Moreover, to enhance the detection ability of the model for small objects, a small object feature enhancement module is designed, which adopts an efficient multi-scale attention mechanism, and aims to enhance the feature extraction ability of small objects, improve the sensitivity to small-size defects, and improve the detection accuracy. Eventually, to optimize the computational efficiency of the model, the average pooling-sparse convolution-batch normalization (BN) module is proposed. This module combines average pooling, sparse convolution and BN techniques to achieve a lightweight model while maintaining a high level of feature extraction capability. Experimental results on the China power line insulator dataset show that the improved model achieves a 4.3 percentage point improvement in the mAP metric compared to YOLOv8s, and the number of parameters in the model is reduced by 10%. The proposed scheme not only improves the accuracy and efficiency of defect detection, but also reduces the demand for computational resources, thus providing a more reliable and efficient solution for insulator defect detection in practical applications.

MATNet: Multilevel attention-based transformers for change detection in remote sensing images

Remote sensing image change detection is crucial for natural disaster monitoring and land use change. As the resolution increases, the scenes covered by remote sensing images become more complex, and traditional methods have difficulties in extracting detailed information. With the development of deep learning, the field of change detection has new opportunities. However, existing algorithms mainly focus on the difference analysis between bi-temporal images, while ignoring the semantic information between images, resulting in the global and local information not being able to interact effectively. In this paper, we introduce a new transformer-based multilevel attention network (MATNet), which is capable of extracting multilevel features of global and local information, enabling information interaction and fusion, and thus modeling the global context more effectively. Specifically, we extract multilevel semantic features through the Transformer encoder, and utilize the Feature Enhancement Module (FEM) to perform feature summing and differencing on the multilevel features in order to better extract the local detail information, and thus better detect the changes in small regions. In addition, we employ a multilevel attention decoder (MAD) to obtain information in spatial and spectral dimensions, which can effectively fuse global and local information. In experiments, our method performs excellently on CDD, DSIFN-CD, LEVIR-CD, and SYSU-CD datasets, with F1 scores and OA reaching 95.67%∕87.75%∕90.94%∕86.82% and 98.95%∕95.93%∕99.11%∕90.53% respectively.

MAEE-Net: SAR ship target detection network based on multi-input attention and edge feature enhancement

Synthetic Aperture Radar (SAR) imagery has a wide range of applications in search and rescue ships lost contact and military reconnaissance. When detecting multi-scale targets, better determination of the target edge is conducive to improving the detection accuracy of the model, but most of the existing methods lack research on this aspect. To fix the problems mentioned earlier, this paper suggests using a SAR Ship target detection network called MAEE-Net. In this paper, a multi-input attention-based feature fusion module (MAFM) and an edge feature enhancement module (EFEM) are proposed. MAFM uses attention mechanism with multi-input and multiple-output to improve attention to shallow feature map target and suppress invalid information, so as to improve the information utilization rate of each layer. To make the network better at detecting the edges of ships, EFEM uses double-branched structure to carry out fine-grained information retention and edge feature extraction. PIoU v2 is introduced to enhance multi-target processing capability. Experiments were carried out on SSDD dataset and SAR-Ship-Dataset, the overall detection accuracy was as high as 98.6% and 94.7%. The detection accuracy was 93.5% and 99.3% on inshore and offshore sub-datasets of SSDD dataset. Experimental results on two datasets show that our model is impactful.

Faster-slow network fused with enhanced fine-grained features for action recognition

Two-stream methods, which separate human actions and backgrounds into temporal and spatial streams visually, have shown promising results in action recognition datasets. However, prior researches emphasize motion modeling but overlook the robust correlation between motion features and spatial information, causing restriction of the model’s ability to recognize behaviors entailing occlusions or rapid changes. Therefore, we introduce Faster-slow, an improved framework for frame-level motion features. It introduces a Behavioural Feature Enhancement (BFE) module based on a novel two-stream network with different temporal resolutions. BFE consists of two components: MM, which incorporates motion-aware attention to capture dependencies between adjacent frames; STC, which enhances spatio-temporal and channel information to generate optimized features. Overall, BFE facilitates the extraction of finer-grained motion information, while ensuring a stable fusion of information across both streams. We evaluate the Faster-slow on the Atomic Visual Actions dataset, and the Faster-AVA dataset constructed in this paper, yielding promising experimental results.

Active learning based on multi-enhanced views for classification of multiple patterns in lung ultrasound images

There are several main patterns in lung ultrasound (LUS) images, including A-lines, B-lines, consolidation and pleural effusion. LUS images of healthy lungs typically only exhibit A-lines, while other patterns may emerge or coexist in LUS images associated with different lung diseases. The accurate categorization of these primary patterns is pivotal for effective lung disease screening. However, two challenges complicate the classification task: the first is the inherent blurring of feature differences between main patterns due to ultrasound imaging properties; and the second is the potential coexistence of multiple patterns in a single case, with only the most dominant pattern being clinically annotated. To address these challenges, we propose the active learning based on multi-enhanced views (MEVAL) method to achieve more precise pattern classification in LUS. To accentuate feature differences between multiple patterns, we introduce a feature enhancement module by applying vertical linear fitting and k-means clustering. The multi-enhanced views are then employed in parallel with the original images, thus enhancing MEVAL’s awareness of feature differences between multiple patterns. To tackle the patterns coexistence issue, we propose an active learning strategy based on confidence sets and misclassified sets. This strategy enables the network to simultaneously recognize multiple patterns by selectively labeling of a small number of images. Our dataset comprises 5075 LUS images, with approximately 4% exhibiting multiple patterns. Experimental results showcase the effectiveness of the proposed method in the classification task, with accuracy of 98.72%, AUC of 0.9989, sensitivity of 98.76%, and specificity of 98.16%, which outperforms than the state-of-the-art deep learning-based methods. A series of comprehensive ablation studies suggest the effectiveness of each proposed component and show great potential in clinical application.

A three-stage model for camouflaged object detection

Camouflaged objects are typically assimilated into their backgrounds and exhibit fuzzy boundaries. The complex environmental conditions and the high intrinsic similarity between camouflaged targets and their surroundings pose significant challenges in accurately locating and segmenting these objects in their entirety. While existing methods have demonstrated remarkable performance in various real-world scenarios, they still face limitations when confronted with difficult cases, such as small targets, thin structures, and indistinct boundaries. Drawing inspiration from human visual perception when observing images containing camouflaged objects, we propose a three-stage model that enables coarse-to-fine segmentation in a single iteration. Specifically, our model employs three decoders to sequentially process subsampled features, cropped features, and high-resolution original features. This proposed approach not only reduces computational overhead but also mitigates interference caused by background noise. Furthermore, considering the significance of multi-scale information, we have designed a multi-scale feature enhancement module that enlarges the receptive field while preserving detailed structural cues. Additionally, a boundary enhancement module has been developed to enhance performance by leveraging boundary information. Subsequently, a mask-guided fusion module is proposed to generate fine-grained results by integrating coarse prediction maps with high-resolution feature maps. Our network shows superior performance without introducing unnecessary complexities. Upon acceptance of the paper, the source code will be made publicly available at https://github.com/clelouch/TSNet.

RFNet: Multivariate long sequence time-series forecasting based on recurrent representation and feature enhancement

Multivariate time series exhibit complex patterns and structures involving interactions among multiple variables and long-term temporal dependencies, making multivariate long sequence time series forecasting (MLSTF) exceptionally challenging. Despite significant progress in Transformer-based methods in the MLSTF domain, many models still rely on stacked encoder–decoder architectures to capture complex time series patterns. This leads to increased computational complexity and overlooks spatial pattern information in multivariate time series, thereby limiting the model’s performance. To address these challenges, we propose RFNet, a lightweight model based on recurrent representation and feature enhancement. We partition the time series into fixed-size subsequences to retain local contextual temporal pattern information and cross-variable spatial pattern information. The recurrent representation module employs gate attention mechanisms and memory units to capture local information of the subsequences and obtain long-term correlation information of the input sequence by integrating information from different memory units. Meanwhile, we utilize a shared multi-layer perceptron (MLP) to capture global pattern information of the input sequence. The feature enhancement module explicitly extracts complex spatial patterns in the time series by transforming the input sequence. We validate the performance of RFNet on ten real-world datasets. The results demonstrate an improvement of approximately 55.3% over state-of-the-art MLSTF models, highlighting its significant advantage in addressing multivariate long sequence time series forecasting problems.