Feature Enhancement Module Research Articles

Few-Shot Relation Extraction via the Entity Feature Enhancement and Attention-Based Prototypical Network

In order to overcome the problems that the feature representation and classification effect of the existing methods need to be improved in complex contexts, this paper presents a novel few-shot relation extraction approach via the entity feature enhancement and attention-based prototypical network. The proposed model uses the pretrained RoBERTa model as the encoder while using the BiLSTM module for directional feature extraction. We further incorporate the entity feature enhancement module to improve the feature representation ability of the model. At last, the attention-based prototypical network is used to predict relations. The experimental results show that the proposed method not only outperforms the baseline models on the datasets from the bridge inspection and health domains but also achieves competitive results on the FewRel dataset in the general domain.

Domain migration representation learning for blind magnetic resonance image super-resolution

Recently, magnetic resonance (MR) imaging has been widely used in clinical treatment to assist doctors in diagnosis because of its high resolution of soft tissue and no radiation. However, limited by imaging equipment and imaging time, the resolution of MR imaging is low. The previous super-resolution algorithms used for MR images use paired datasets to generate down-sampled images by artificially assuming the image degradation process, and forming a sample pair to monitor and train the super-resolution network. However, the super-resolution network obtained by this way will cause serious degradation of the network performance in practical application. In this paper, we proposed DMN, a blind super-resolution framework based on domain migration, which decouples the blind super-resolution network with the idea of domain migration, and avoids the false detail generation problem of the blind super-resolution network in the MR images super-resolution task through independent training. At the same time, we propose a multi-level parallel transformer feature enhancement module (MPTB), which takes noise as the original input and embeds noise features step by step to enhance the fitting ability of the network while reducing the complexity of the network structure design. Besides, we also use the k-space loss function to form a constraint function, so that the domain migration network can fully consider the k-space characteristics of MR images. The experimental results on the MR images of the knee joint and brain in the FastMRI dataset show that the super-resolution performance of our proposed DMN network is better than the previous super-resolution networks.

Feature enhancement guided network for yield estimation of high-density jujube

BackgroundAutomatic and precise jujube yield prediction is important for the management of orchards and the allocation of resources. Traditional yield prediction techniques are based on object detection, which predicts a box to achieve target statistics, but are often used in sparse target settings. Those techniques, however, are challenging to use in real-world situations with particularly dense jujubes. The box labeling is labor- and time-intensive, and the robustness of the system is adversely impacted by severe occlusions. Therefore, there is an urgent need to develop a robust method for predicting jujube yield based on images. But in addition to the extreme occlusions, it is also challenging due to varying scales, complex backgrounds, and illumination variations.Results In this work, we developed a simple and effective feature enhancement guided network for yield estimation of high-density jujube. It has two key designs: Firstly, we proposed a novel label representation method based on uniform distribution, which provides a better characterization of object appearance compared to the Gaussian-kernel-based method. This new method is simpler to implement and has shown greater success. Secondly, we introduced a feature enhancement guided network for jujube counting, comprising three main components: backbone, density regression module, and feature enhancement module. The feature enhancement module plays a crucial role in perceiving the target of interest effectively and guiding the density regression module to make accurate predictions. Notably, our method takes advantage of this module to improve the overall performance of our network. To validate the effectiveness of our method, we conducted experiments on a collected dataset consisting of 692 images containing a total of 40,344 jujubes. The results demonstrate the high accuracy of our method in estimating the number of jujubes, with a mean absolute error (MAE) of 9.62 and a mean squared error (MSE) of 22.47. Importantly, our method outperforms other state-of-the-art methods by a significant margin, highlighting its superiority in jujube yield estimation.Conclusions The proposed method provides an efficient image-based technique for predicting the yield of jujubes. The study will advance the application of artificial intelligence for high-density target recognition in agriculture and forestry. By leveraging this technique, we aim to enhance the level of planting automation and optimize resource allocation.

HPG-GAN: High-Quality Prior-Guided Blind Face Restoration Generative Adversarial Network

To address the problems of low resolution, compression artifacts, complex noise, and color loss in image restoration, we propose a High-Quality Prior-Guided Blind Face Restoration Generative Adversarial Network (HPG-GAN). This mainly consists of Coarse Restoration Sub-Network (CR-Net) and Fine Restoration Sub-Network (FR-Net). HPG-GAN extracts high-quality structural and textural priors and facial feature priors from coarse restoration images to reconstruct clear and high-quality facial images. FR-Net includes the Facial Feature Enhancement Module (FFEM) and the Asymmetric Feature Fusion Module (AFFM). FFEM enhances facial feature information using high-definition facial feature priors obtained from ArcFace. AFFM fuses and selects asymmetric high-quality structural and textural information from ResNet34 to recover overall structural and textural information. The comparative evaluations on synthetic and real-world datasets demonstrate superior performance and visual restoration effects compared to state-of-the-art methods. The ablation experiments validate the importance of each module. HPG-GAN is an effective and robust blind face deblurring and restoration network. The experimental results demonstrate the effectiveness of the proposed network, which achieves better visual quality against state-of-the-art methods.

Color-Coated Steel Sheet Roof Building Extraction from External Environment of High-Speed Rail Based on High-Resolution Remote Sensing Images

The identification of color-coated steel sheet (CCSS) roof buildings in the external environment is of great significance for the operational security of high-speed rail systems. While high-resolution remote sensing images offer an efficient approach to identify CCSS roof buildings, achieving accurate extraction is challenging due to the complex background in remote sensing images and the extensive scale range of CCSS roof buildings. This research introduces the deformation-aware feature enhancement and alignment network (DFEANet) to address these challenges. DFEANet adaptively adjusts the receptive field to effectively separate the foreground and background facilitated by the deformation-aware feature enhancement module (DFEM). Additionally, feature alignment and gated fusion module (FAGM) is proposed to refine boundaries and preserve structural details, which can ameliorate the misalignment between adjacent features and suppress redundant information during the fusion process. Experimental results on remote sensing images along the Beijing–Zhangjiakou high-speed railway demonstrate the effectiveness of DFEANet. Ablation studies further underscore the enhancement in extraction accuracy due to the proposed modules. Overall, the DFEANet was verified as capable of assisting in the external environment security of high-speed rails.

Multi-Scale Feature Fusion and Structure-Preserving Network for Face Super-Resolution

Deep convolutional neural networks have demonstrated significant performance improvements in face super-resolution tasks. However, many deep learning-based approaches tend to overlook the inherent structural information and feature correlation across different scales in face images, making the accurate recovery of face structure in low-resolution cases challenging. To address this, this paper proposes a method that fuses multi-scale features while preserving the facial structure. It introduces a novel multi-scale residual block (MSRB) to reconstruct key facial parts and structures from spatial and channel dimensions, and utilizes pyramid attention (PA) to exploit non-local self-similarity, improving the details of the reconstructed face. Feature Enhancement Modules (FEM) are employed in the upscale stage to refine and enhance current features using multi-scale features from previous stages. The experimental results on CelebA, Helen and LFW datasets provide evidence that our method achieves superior quantitative metrics compared to the baseline, the Peak Signal-to-Noise Ratio (PSNR) outperforms the baseline by 0.282 dB, 0.343 dB, and 0.336 dB. Furthermore, our method demonstrates improved visual performance on two additional no-reference datasets, Widerface and Webface.

CloudDeepLabV3+: a lightweight ground-based cloud segmentation method based on multi-scale feature aggregation and multi-level attention feature enhancement

ABSTRACT The segmentation of ground-based cloud images is the basis for obtaining numerous cloud parameters. To achieve high-precision adaptive cloud image segmentation requirements, this study designs a lightweight ground-based cloud image adaptive segmentation method named CloudDeepLabV3+ that integrates multi-scale features aggregation and multi-level attention feature enhancement. Firstly, a novel lightweight EfficientNetV2-S is designed as a feature extraction backbone to reduce network parameters. Secondly, a heterogeneous receptive field splicing atrous spatial pyramid pooling module is designed. It enhances the correlation of information between layers, and realizes multiscale information fusion. The feature enhancement module based on the self-attention mechanism intensifies the representation of local and global features. Thirdly, the feature alignment module based on the attention mechanism is constructed to pull deep and shallow features for alignment. Finally, we implement ablation study on the key components of method and comparison experiment with other advanced methods using five evaluation metrics. Results show that the key components play an important role in multiscale information fusion. It promotes the accuracy of cloud image feature extraction while reducing the loss of detailed features. Generalization performance verification indicates the excellent performance of the proposed model in advanced cloud feature extraction and cloud-mask generation.

FPIseg: Iterative segmentation network based on feature pyramid for few‐shot segmentation

AbstractFew‐shot segmentation (FSS) enables rapid adaptation to the segmentation task of unseen‐classes object based on a few labelled support samples. Currently, the focal point of research in the FSS field is to align features between support and query images, aiming to improve the segmentation performance. However, most existing FSS methods implement such support/query alignment by solely leveraging middle‐level feature for generalization, ignoring the category semantic information contained in high‐level feature, while pooling operation inevitably lose spatial information of the feature. To alleviate these issues, the authors propose the Iterative Segmentation Network Based on Feature Pyramid (FPIseg), which mainly consists of three modules: Feature Pyramid Fusion Module (FPFM), Region Feature Enhancement Module (RFEM), and Iterative Optimization Segmentation Module (IOSM). Firstly, FPFM fully utilizes the foreground information from the support image to implement support/query alignment under multi‐scale, multi‐level semantic backgrounds. Secondly, RFEM enhances the foreground detail information of aligned feature to improve generalization ability. Finally, ISOM iteratively segments the query image to optimize the prediction result and improve segmentation performance. Extensive experiments on the PASCAL‐5i and COCO‐20i datasets show that FPIseg achieves considerable segmentation performance under both 1‐shot and 5‐shot settings.

Cross-Modality Double Bidirectional Interaction and Fusion Network for RGB-T Salient Object Detection

RGB-T salient object detection (SOD) aims to detect and segment saliency regions on RGB images and the corresponding thermal maps. The ability of alleviating the modality difference between RGB and thermal modality plays a vital role in the development of RGB-T SOD. However, most of the existing methods try to integrate multi-modal information through various fusion strategies, or reduce the modality difference via unidirectional or undifferentiated bidirectional interaction, but failing in some challenging scenes. To deal with the above question, a novel Cross-Modality Double Bidirectional Interaction and Fusion Network (CMDBIF-Net) for RGB-T SOD is proposed. Specifically, we construct an interactive branch to indirectly bridge the RGB and thermal modalities. In addition, we propose a double bidirectional interaction (DBI) module composed of a forward interaction block (FIB) and a backward interaction block (BIB) to reduce the cross-modality differences. Moreover, a multi-scale feature enhancement and fusion (MSFEF) module is introduced to integrate the multi-modal features with considering the internal gap of different modality. Finally, we use a cascaded decoder and a cross-level feature enhancement (CLFE) module to generate high-quality saliency map. Extensive experiments are conducted on three publicly available RGB-T SOD datasets shows that the proposed CMDBIF-Net achieves outstanding performance against the state-of-the-art (SOTA) RGB-T SOD methods.

ODL Net: Object detection and location network for small pears around the thinning period

In the process of efficient management of intelligent orchards, due to the short cycle and high intensity of fruit thinning, it is urgent to realize the automatic operation of fruit thinning in orchards. However, affected by the complex orchard environment, the color of fruit and the background are similar, and the more important problem is that the fruit is small-scale. These factors bring great challenges to fruit detection before and after the thinning period. For this reason, a detection algorithm for fruits of small green objects is proposed, namely, ODL Net. By integrating the semantic enhancement module and label assignment Center-Box, the small size problem of the target fruit is alleviated. The feature enhancement module and position enhancement module are constructed to enhance the fusion effect of features and improve the detection accuracy. To better verify the performance of the algorithm, this study takes a pear orchard as an example to produce two datasets before and after pear thinning. The experimental results show that the detection accuracy of ODL Net can reach 56.2% and 65.1% before and after the fruit thinning period, respectively, and the recall rate can reach 61.3% and 70.8%, respectively, which are significantly higher than those of other mainstream algorithms at present. The new algorithm can effectively assist the orchard automatic fruit thinning operation and provide the basis for orchard yield measurement after the fruit thinning period. This study can provide a theoretical basis for the scientific management of intelligent orchards.

Disentangled Representation and Enhancement Network for Vein Recognition

Recently, vein recognition has been paid more considerable attention in biometric recognition fields. In the process of vein image acquisition, due to the influence of external factors such as illumination change, the texture information of vein images with the same identity information may change, which enhances the difference of intra-class and extremely degrades the performance of vein recognition systems. To address this problem, we proposed a Disentangled Representation and Enhancement Network for vein recognition called DRE-Net. First, robust vein shape masks are obtained by the designed vein segmentation algorithms, which are utilized as label information in DRE-Net to acquire the shape features of vein images. Second, a disentangled representation network which contains two encoders and two decoders is designed to disentangle the texture features and shape features of vein images. Besides, a Multi-Scale Attention Residual Block (MSARB) is presented to better mine the vein network information and enhance the representation ability of DRE-Net for vein images. Finally, a Weight-Guided Feature Enhancement Module (WGFEM) is presented to obtain more discriminative representations for vein recognition by reducing the importance of texture features and increasing the importance of shape features, which decreases the difference of intra-class caused by illumination change. Extensive experiments have been carried out on three benchmark vein databases, and the experimental results demonstrate that our proposed model outperforms the state-of-the-art methods.

Slope Failure and Landslide Detection in Huangdao District of Qingdao City Based on an Improved Faster R-CNN Model

To reduce the significant losses caused by slope failures and landslides, it is of great significance to detect and predict these disasters scientifically. This study focused on Huangdao District of Qingdao City in Shandong Province, using the improved Faster R-CNN network to detect slope failures and landslides. This study introduced a multi-scale feature enhancement module into the Faster R-CNN model. The module enhances the network’s perception of different scales of slope failures and landslides by deeply fusing high-resolution weak semantic features with low-resolution strong semantic features. Our experiments show that the improved Faster R-CNN model outperformed the traditional version, and that ResNet50 performed better than VGG16 with an AP value of 90.68%, F1 value of 0.94, recall value of 90.68%, and precision value of 98.17%. While the targets predicted by VGG16 were more dispersed and the false detection rate was higher than that of ResNet50, VGG16 was shown to have an advantage in predicting small-scale slope failures and landslides. The trained Faster R-CNN network model detected geological hazards of slope failure and landslide in Huangdao District, missing only two landslides, thereby demonstrating high detection accuracy. This method can provide an effective technical means for slope failures and landslides target detection and has practical implications.

CCA-FPN: Channel and content adaptive object detection

Feature pyramid network (FPN) is a typical detector commonly for solving the issue of object detection at different scales. However, the lateral connections in FPN lead to the loss of feature information due to the reduction of feature channels. Moreover, the top-down feature fusion will weaken the feature representation in the process of feature delivery because of features with different semantic information. In this paper, we propose a feature pyramid network with channel and content adaptive feature enhancement module (CCA-FPN), which uses a channel adaptive guided mechanism module (CAGM) and multi-scale content adaptive feature enhancement module (MCAFEM) to alleviate these problems. We conduct comprehensive experiments on the MS COCO dataset. By replacing FPN with CCA-FPN in ATSS, our models achieve 1.3 percentage points higher Average Precision (AP) when using ResNet50 as backbone. Furthermore, our CCA-FPN achieves 0.3 percentage points higher than the AugFPN which is the state-of-the-art FPN-based detector.

Progressive spatial-angular feature enhancement network for light field image super-resolution

The intensity and direction of the light field (LF) can be recorded simultaneously by using LF cameras. However, since LF cameras sacrifice spatial resolution for higher angular resolution, the images acquired by LF cameras tend to have low spatial resolution. Therefore, LF image super-resolution (SR) has become an integral part of LF studies. Many existing LF image SR methods fail to fully utilize angular and spatial information due to only using partial sub-aperture images (SAIs). In this paper, we propose a progressive spatial-angular feature enhancement network (PSAFENet) to deal with the problem of missing information in LF image SR. Specifically, we first extract the spatial features of SAIs, the spatial and angular features contained in the macro-pixel images (MacPIs) by three different feature extraction modules. Then, these features are fed into a spatial-angular feature enhancement (SAFE) module to perform enhancement of spatial-angular information on the SAIs. To improve the reconstruction accuracy, we also use the information multi-distillation block (IMDB) to remove the redundant information before upsampling. Our network can well merge the angular and spatial information into each SAI, which facilitates the reconstruction of the LF images. Experimental results on five public datasets show that the proposed PSAFENet method outperforms existing methods in both qualitative and quantitative comparisons.

LSD-YOLOv5: A Steel Strip Surface Defect Detection Algorithm Based on Lightweight Network and Enhanced Feature Fusion Mode.

In the field of metallurgy, the timely and accurate detection of surface defects on metallic materials is a crucial quality control task. However, current defect detection approaches face challenges with large model parameters and low detection rates. To address these issues, this paper proposes a lightweight recognition model for surface damage on steel strips, named LSD-YOLOv5. First, we design a shallow feature enhancement module to replace the first Conv structure in the backbone network. Second, the Coordinate Attention mechanism is introduced into the MobileNetV2 bottleneck structure to maintain the lightweight nature of the model. Then, we propose a smaller bidirectional feature pyramid network (BiFPN-S) and combine it with Concat operation for efficient bidirectional cross-scale connectivity and weighted feature fusion. Finally, the Soft-DIoU-NMS algorithm is employed to enhance the recognition efficiency in scenarios where targets overlap. Compared with the original YOLOv5s, the LSD-YOLOv5 model achieves a reduction of 61.5% in model parameters and a 28.7% improvement in detection speed, while improving recognition accuracy by 2.4%. This demonstrates that the model achieves an optimal balance between detection accuracy and speed, while maintaining a lightweight structure.

Road Traffic Sign Detection Method Based on RTS R-CNN Instance Segmentation Network.

With the rapid development of the autonomous driving industry, there is increasing research on related perception tasks. However, research on road surface traffic sign detection tasks is still limited. There are two main challenges to this task. First, when the target object's pixel ratio is small, the detection accuracy often decreases. Second, the existing publicly available road surface traffic sign datasets have limited image data. To address these issues, this paper proposes a new instance segmentation network, RTS R-CNN, for road surface traffic sign detection tasks based on Mask R-CNN. The network can accurately perceive road surface traffic signs and provide important information for the autonomous driving decision-making system. Specifically, CSPDarkNet53_ECA is proposed in the feature extraction stage to enhance the performance of deep convolutional networks by increasing inter-channel interactions. Second, to improve the network's detection accuracy for small target objects, GR-PAFPN is proposed in the feature fusion part, which uses a residual feature enhancement module (RFA) and atrous spatial pyramid pooling (ASPP) to optimize PAFPN and introduces a balanced feature pyramid module (BFP) to handle the imbalanced feature information at different resolutions. Finally, data augmentation is used to generate more data and prevent overfitting in specific scenarios. The proposed method has been tested on the open-source dataset Ceymo, achieving a Macro F1-score of 87.56%, which is 2.3% higher than the baseline method, while the inference speed reaches 23.5 FPS.

Learning from small data for hyperspectral image classification

With the development of deep learning technique, many hyperspectral image classification (HIC) methods achieve great progress based on the application of convolution neural network (CNN). However, most of them still face small data cases. In this paper, we study the challenges of small data in HIC and propose a stronger HIC small data learning framework (SDLF). More specifically, a feature enhancement module named high-low dimension hybrid representation (HLDR) is designed to improve the feature differentiation power. To generate higher quality classification labels for unlabeled data, a smooth label generation module (SLG) is introduced based on Spatio-Temporal constraint. Then, we introduce a dynamically training sample amplification (TSA) under a unified curriculum learning strategy to add generated reliable labels into training samples. Finally, integrating HLDR in feature representation, SLG in label generation, and TSA in training amplification, the proposed SDLF framework is constructed, which can ensure correctness of HIC network. Experimental results demonstrate that our SDLF can not only learn robust feature representation to enhance the differentiation power but also select reliably generated labels to enrich the training data, which is useful for learning from small data. Also, our SDLF method achieves a superior performance with small labeled training data on three typical datasets.

A coarse-to-fine multi-scale feature hybrid low-dose CT denoising network

With the growing development and wide clinical application of CT technology, the potential radiation damage to patients has sparked public concern. However, reducing the radiation dose may cause large amounts of noise and artifacts in the reconstructed images, which may affect the accuracy of the clinical diagnosis. Therefore, improving the quality of low-dose CT scans has become a popular research topic. Generative adversarial networks (GAN) have provided new research ideas for low-dose CT (LDCT) denoising. However, utilizing only image decomposition or adding new functional subnetworks cannot effectively fuse the same type of features with different scales (or different types of features). Thus, most current GAN-based denoising networks often suffer from low feature utilization and increased network complexity. To address these problems, we propose a coarse-to-fine multiscale feature hybrid low-dose CT denoising network (CMFHGAN). The generator consists of a global denoising module, local texture feature enhancement module, and self-calibration feature fusion module. The three modules complement each other and guarantee overall denoising performance. In addition, to further improve the denoising performance, we propose a multi-resolution inception discriminator with multiscale feature extraction ability. Experiments were performed on the Mayo and Piglet datasets, and the results showed that the proposed method outperformed the state-of-the-art denoising algorithms.

A multimodal feature fusion image dehazing method with scene depth prior

AbstractCurrent dehazing networks usually only learn haze features in a single‐image colour space and often suffer from uneven dehazing, colour, and edge degradation when confronted with different scales of ground objects in the depth space of the scene. The authors propose a multimodal feature fusion image dehazing method with scene depth prior based on a decoder–encoder backbone network. The multimodal feature fusion module was first designed. In this module, affine transformation and polarized self‐attention mechanism are used to realize the fusion of image colour and depth prior feature, to improve the representation ability of the model for different scale ground haze feature in‐depth space. Then, the feature enhancement module (FEM) is added, and deformable convolution and difference convolution methods are used to enhance the representation ability of the model for the geometric and texture feature of the ground objects. The publicly available dehazing datasets are used for comparison and ablation experiments. The results show that compared with the existing classical dehazing networks, the peak signal‐to‐noise ratio (PSNR) and SSIM of the authors’ proposed method have been significantly improved, have a more uniform dehazing effect in different depth spaces, and maintain the colour and edge details of the ground objects very well.

Real-time detection of a camouflaged object in unstructured scenarios based on hierarchical aggregated attention lightweight network

Accurate and real-time grasping of household items is a challenge for home service robots in complex indoor environments. In this paper, using transparent cups as the recognition object, an accurate, reliable, and real-time visual algorithm is proposed for the elderly who needs nursing care to solve the recognition problems of overlapping, blurred, and small cups in complex unstructured indoor scenes. In order to meet the lightweight deployment of home service robots while considering both high-scale and low-scale semantic features, this paper designs a multi-scale fusion lightweight backbone network based on the Split Shuffle Block (SSB) and Group Shuffle Block (GSB) feature encoding units. Among them, depthwise separable convolution (DwConv) uses to reduce the amount of coding unit parameters, and feature shuffle uses to promote information exchange and the ability of information expression. In order to accurately identify the transparent cup camouflaged in complex backgrounds, this paper proposed a lightweight feature enhancement module that combines multi-scale hierarchical aggregation attention and multi-branch parallel convolution structure. The proposed module used an adaptive weighting strategy and a channel normalization weighting strategy to highlight the active regions of boundary features in each branch feature map, enhance the exchange of boundary information, and reduce the loss of detailed information. The experimental results on the IndoorCup and MobileCup datasets show that the detection accuracy of the proposed method is 93.6% and 92.6%, respectively, and the model calculation amount is only 0.91 M, which can lightweight deploy on indoor mobile robots for real-time detection. From the results of qualitative comparisons, the proposed method has strong robustness. It can effectively suppress false and missed detection caused by background interference. Likewise, it also effectively identifies camouflage objects and small objects in complex backgrounds.