Adaptive Feature Fusion Research Articles

A Robustness Study on Early Fire Image Recognitions

With the advancement of society and the rapid urbanization process, there is an escalating need for effective fire detection systems. This study endeavors to bolster the efficacy and dependability of fire detection systems in intricate settings by refining the existing You Only Look Once version 5 (YOLOv5) algorithm and introducing algorithms grounded on fire characteristics. Primarily, the Convolutional Block Attention Module (CBAM) attention mechanism is introduced to steer the model towards substantial features, thereby amplifying detection precision. Subsequently, a multi-scale feature fusion network, employing the Adaptive Spatial Feature Fusion Module (ASFF), is embraced to proficiently amalgamate feature information from various scales, thereby enhancing the model’s comprehension of image content and subsequently fortifying detection resilience. Moreover, refining the loss function and integrating a larger detection head further fortify the model’s capability to discern diminutive targets. Experimental findings illustrate that the refined YOLOv5 algorithm attains accuracy advancements of 8% and 8.2% on standard and small target datasets, respectively. To ascertain the practical viability of the refined YOLOv5 algorithm, this study introduces a temperature-based flame detection algorithm. By amalgamating and deploying both algorithms, the ultimate experimental outcomes reveal that the integrated algorithm not only elevates accuracy but also achieves a frame rate of 57 frames, aligning with the prerequisites for practical deployment.

A Detection Algorithm for Citrus Huanglongbing Disease Based on an Improved YOLOv8n.

Given the severe impact of Citrus Huanglongbing on orchard production, accurate detection of the disease is crucial in orchard management. In the natural environments, due to factors such as varying light intensities, mutual occlusion of citrus leaves, the extremely small size of Huanglongbing leaves, and the high similarity between Huanglongbing and other citrus diseases, there remains an issue of low detection accuracy when using existing mainstream object detection models for the detection of citrus Huanglongbing. To address this issue, we propose YOLO-EAF (You Only Look Once-Efficient Asymptotic Fusion), an improved model based on YOLOv8n. Firstly, the Efficient Multi-Scale Attention Module with cross-spatial learning (EMA) is integrated into the backbone feature extraction network to enhance the feature extraction and integration capabilities of the model. Secondly, the adaptive spatial feature fusion (ASFF) module is used to enhance the feature fusion ability of different levels of the model so as to improve the generalization ability of the model. Finally, the focal and efficient intersection over union (Focal-EIOU) is utilized as the loss function, which accelerates the convergence process of the model and improves the regression precision and robustness of the model. In order to verify the performance of the YOLO-EAF method, we tested it on the self-built citrus Huanglongbing image dataset. The experimental results showed that YOLO-EAF achieved an 8.4% higher precision than YOLOv8n on the self-built dataset, reaching 82.7%. The F1-score increased by 3.33% to 77.83%, and the mAP (0.5) increased by 3.3% to 84.7%. Through experimental comparisons, the YOLO-EAF model proposed in this paper offers a new technical route for the monitoring and management of Huanglongbing in smart orange orchards.

Dehazing & Reasoning YOLO: Prior knowledge-guided network for object detection in foggy weather

Fast and accurate object detection in foggy weather is crucial for visual tasks such as autonomous driving and video surveillance. Existing methods typically preprocess images with enhancement techniques before the object detector, so that the real-time performance of object detection decreases to some extent. Meanwhile, many popular object detection models rely solely on visual features for localization and classification. When fog is present, visual features would be so adversely impacted that the detection accuracy sharply decreases. Therefore, we propose an end-to-end prior knowledge-guided network called DR-YOLO for object detection in foggy weather. DR-YOLO integrates the atmospheric scattering model and the co-occurrence relation graph as prior knowledge into the entire training process of the detector. Firstly, Restoration Subnet Module (RSM) is designed to employ the atmospheric scattering model to guide the learning direction of the detector for dehazing features. Specifically, it is only adopted during the training process and does not increase the time cost of detection process. Secondly, for guiding the detector to pay more attention to potential co-occurring objects in the same scene, we introduce Relation Reasoning Attention Module (RRAM) that utilizes the co-occurrence relation graph to supplement deficient visual features in foggy weather. In addition, DR-YOLO employs Adaptive Feature Fusion Module (AFFM) to effectively merge the key features from the backbone and neck for the needs of RRAM and RSM. Finally, we conduct experiments on clear, synthetic and real-world foggy datasets to demonstrate the effectiveness of DR-YOLO. The source code is available at https://github.com/wenxinss/DR-YOLO.

Contraband detection method in X-ray image based on improved CenterNet network

Security inspection has been played a crucial role in ensuring the safety of railway passenger transportation. The detection of contraband in X-ray images is an important part of safety inspection work. Since the X-ray image of security inspection is a pseudo color image with a unique imaging style, and the restricted objects are mostly small targets, the accuracy of the object detection algorithm using visible light images directly is not high. This research will propose an improved CenterNet method, and design a 16 times down sampling rate backbone network, which is more suitable for detecting contraband in X-ray images. This article will introduce attention mechanisms to increase attention to effective and key areas in X-ray images, and reveal the mechanisms of spatial and channel attention in X-ray images. Furthermore, by designing an adaptive feature fusion mechanism, the drawback of the original CenterNet by using only the last layer of output features is compensated. Experiments have shown that on the SIXray dataset, mAP is improved by 3.3% compared to the original CenterNet method, and 5.5%, 2.4%, 3.6%, 10.1% compared to SSD, Yolo V3, FPN, DERT algorithms. The detection accuracy of restricted items in X-ray images is effectively improved.

Detection Based on Semantics and a Detail Infusion Feature Pyramid Network and a Coordinate Adaptive Spatial Feature Fusion Mechanism Remote Sensing Small Object Detector

In response to the challenges of remote sensing imagery, such as unmanned aerial vehicle (UAV) aerial imagery, including differences in target dimensions, the dominance of small targets, and dense clutter and occlusion in complex environments, this paper optimizes the YOLOv8n model and proposes an innovative small-object-detection model called DDSC-YOLO. First, a DualC2f structure is introduced to improve the feature-extraction capabilities of the model. This structure uses dual-convolutions and group convolution techniques to effectively address the issues of cross-channel communication and preserving information in the original input feature mappings. Next, a new attention mechanism, DCNv3LKA, was developed. This mechanism uses adaptive and fine-grained information-extraction methods to simulate receptive fields similar to self-attention, allowing adaptation to a wide range of target size variations. To address the problem of false and missed detection of small targets in aerial photography, we designed a Semantics and Detail Infusion Feature Pyramid Network (SDI-FPN) and added a dedicated detection scale specifically for small targets, effectively mitigating the loss of contextual information in the model. In addition, the coordinate adaptive spatial feature fusion (CASFF) mechanism is used to optimize the original detection head, effectively overcoming multi-scale information conflicts while significantly improving small target localization accuracy and long-range dependency perception. Testing on the VisDrone2019 dataset shows that the DDSC-YOLO model improves the mAP0.5 by 9.3% over YOLOv8n, and its performance on the SSDD and RSOD datasets also confirms its superior generalization capabilities. These results confirm the effectiveness and significant progress of our novel approach to small target detection.

A robust feature matching algorithm based on adaptive feature fusion combined with image superresolution reconstruction

A robust feature matching algorithm based on adaptive feature fusion combined with image superresolution reconstruction

Adaptive Feature Fusion and Improved Attention Mechanism-Based Small Object Detection for UAV Target Tracking

Adaptive Feature Fusion and Improved Attention Mechanism-Based Small Object Detection for UAV Target Tracking

An Improved Pedestrian Detection Model Based on YOLOv8 for Dense Scenes

In dense scenes, pedestrians often exhibit a variety of symmetrical features, such as symmetry in body contour, posture, clothing, and appearance. However, pedestrian detection poses challenges due to the mutual occlusion of pedestrians and the small scale of distant pedestrians in the image. To address these challenges, we propose a pedestrian detection algorithm tailored for dense scenarios called YOLO-RAD. In this algorithm, we integrate the concept of receiving field attention (RFA) into the Conv and C2f modules to enhance the feature extraction capability of the network. A self-designed four-layer adaptive spatial feature fusion (ASFF) module is introduced, and shallow pedestrian feature information is added to enhance the multi-scale feature fusion capability. Finally, we introduce a small-target dynamic head structure (DyHead-S) to enhance the capability of detecting small-scale pedestrians. Experimental results on WiderPerson and CrowdHuman, two challenging dense pedestrian datasets, show that compared with YOLOv8n, our YOLO-RAD algorithm has achieved significant improvement in detection performance, and the detection performance of mAP@0.5 has increased by 2.5% and 6%, respectively. The detection performance of mAP@0.5:0.95 was improved by 2.7% and 6.8%, respectively. Therefore, the algorithm can effectively improve the performance of pedestrian detection in dense scenes.

Multi-scale Attention GAN with Adaptive Feature Fusion for Unsupervised Animation Face Generation

Multi-scale Attention GAN with Adaptive Feature Fusion for Unsupervised Animation Face Generation

Lightweight helmet target detection algorithm combined with Effici-Bi-Level Routing Attention.

Wearing helmets is essential in two-wheeler traffic to reduce the incidence of injuries caused by accidents. We present FB-YOLOv7, an improved detection network based on the YOLOv7-tiny model. The objective of this network is to tackle the problems of both missed detection and false detection that result from the difficulties in identifying small targets and the constraints in equipment performance during helmet detection. By applying an enhanced Bi-Level Routing Attention, the network can improve its capacity to extract global characteristics and reduce information distortion. Furthermore, we deploy the AFPN framework and effectively resolve information conflict using asymptotic adaptive feature fusion technology. Incorporating the EfficiCIoU loss significantly improves the prediction box's accuracy. Experimental trials done on specific datasets reveal that FB-YOLOv7 attains an accuracy of 87.2% and 94.6% on the mean average precision (mAP@.5). Additionally, it maintains a high level of efficiency with frame rates of 129 and 126 frames per second (FPS). FB-YOLOv7 surpasses the other six widely-used detection networks in terms of detection accuracy, network implementation requirements, sensitivity in detecting small targets, and potential for practical applications.

4AC-YOLOv5: an improved algorithm for small target face detection

In real scenes, small target faces often encounter various conditions, such as intricate background, occlusion and scale change, which leads to the problem of omission or misdetection of face detection results. To solve this puzzle, an improved algorithm of small target face detection 4AC-YOLOv5 is proposed. First, the algorithm by introducing a new layer to detect faces at a much smaller size, through the fusion of more shallow information, enhance the network perception of small objects, the accuracy of small target detection is improved; second, to improve the neck structure, to add the adaptive feature fusion network AFPN to replace FPN + PAN, to prevent the large information gap between non-adjacent Level to some extent, and to fully retain and integrate different scale characteristic information; and finally, improve the C3 module and propose a new multiscale residual module C3_MultiRes. Improving the expressive power of the network by introducing a multibranched structure and gradually increasing resolution somewhat reduces the complexity of the model calculation. The experimental results show that the precision of the improved model reached 94.54%, 93.08% and 84.98% in easy, medium and hard levels of WiderFace data set, respectively, and the results of detection are better than the original network. 4AC-YOLOv5 can meet the requirements of small target face detection in complex environment.

GF-UNet:A Cropland Extraction Method Based on Attention Gate and Adaptive Feature Fusion

GF-UNet:A Cropland Extraction Method Based on Attention Gate and Adaptive Feature Fusion

UAV Complex-Scene Single-Target Tracking Based on Improved Re-Detection Staple Algorithm

With the advancement of remote sensing technology, the demand for the accurate monitoring and tracking of various targets utilizing unmanned aerial vehicles (UAVs) is increasing. However, challenges such as object deformation, motion blur, and object occlusion during the tracking process could significantly affect tracking performance and ultimately lead to tracking drift. To address this issue, this paper introduces a high-precision target-tracking method with anomaly tracking status detection and recovery. An adaptive feature fusion strategy is proposed to improve the adaptability of the traditional sum of template and pixel-wise learners (Staple) algorithm to changes in target appearance and environmental conditions. Additionally, the Moth Flame Optimization (MFO) algorithm, known for its strong global search capability, is introduced as a re-detection algorithm in case of tracking failure. Furthermore, a trajectory-guided Gaussian initialization technique and an iteration speed update strategy are proposed based on sexual pheromone density to enhance the tracking performance of the introduced re-detection algorithm. Comparative experiments conducted on UAV123 and UAVDT datasets demonstrate the excellent stability and robustness of the proposed algorithm.

Detection Model of Tea Disease Severity under Low Light Intensity Based on YOLOv8 and EnlightenGAN

In response to the challenge of low recognition rates for similar phenotypic symptoms of tea diseases in low-light environments and the difficulty in detecting small lesions, a novel adaptive method for tea disease severity detection is proposed. This method integrates an image enhancement algorithm based on an improved EnlightenGAN network and an enhanced version of YOLO v8. The approach involves first enhancing the EnlightenGAN network through non-paired training on low-light-intensity images of various tea diseases, guiding the generation of high-quality disease images. This step aims to expand the dataset and improve lesion characteristics and texture details in low-light conditions. Subsequently, the YOLO v8 network incorporates ResNet50 as its backbone, integrating channel and spatial attention modules to extract key features from disease feature maps effectively. The introduction of adaptive spatial feature fusion in the Neck part of the YOLOv8 module further enhances detection accuracy, particularly for small disease targets in complex backgrounds. Additionally, the model architecture is optimized by replacing traditional Conv blocks with ODConv blocks and introducing a new ODC2f block to reduce parameters, improve performance, and switch the loss function from CIOU to EIOU for a faster and more accurate recognition of small targets. Experimental results demonstrate that YOLOv8-ASFF achieves a tea disease detection accuracy of 87.47% and a mean average precision (mAP) of 95.26%. These results show a 2.47 percentage point improvement over YOLOv8, and a significant lead of 9.11, 9.55, and 7.08 percentage points over CornerNet, SSD, YOLOv5, and other models, respectively. The ability to swiftly and accurately detect tea diseases can offer robust theoretical support for assessing tea disease severity and managing tea growth. Moreover, its compatibility with edge computing devices and practical application in agriculture further enhance its value.

A transformer-guided cross-modality adaptive feature fusion framework for esophageal gross tumor volume segmentation

Background and ObjectiveAccurate segmentation of esophageal gross tumor volume (GTV) indirectly enhances the efficacy of radiotherapy for patients with esophagus cancer. In this domain, learning-based methods have been employed to fuse cross-modality positron emission tomography (PET) and computed tomography (CT) images, aiming to improve segmentation accuracy. This fusion is essential as it combines functional metabolic information from PET with anatomical information from CT, providing complementary information. While the existing three-dimensional (3D) segmentation method has achieved state-of-the-art (SOTA) performance, it typically relies on pure-convolution architectures, limiting its ability to capture long-range spatial dependencies due to convolution's confinement to a local receptive field. To address this limitation and further enhance esophageal GTV segmentation performance, this work proposes a transformer-guided cross-modality adaptive feature fusion network, referred to as TransAttPSNN, which is based on cross-modality PET/CT scans. MethodsSpecifically, we establish an attention progressive semantically-nested network (AttPSNN) by incorporating the convolutional attention mechanism into the progressive semantically-nested network (PSNN). Subsequently, we devise a plug-and-play transformer-guided cross-modality adaptive feature fusion model, which is inserted between the multi-scale feature counterparts of a two-stream AttPSNN backbone (one for the PET modality flow and another for the CT modality flow), resulting in the proposed TransAttPSNN architecture. ResultsThrough extensive four-fold cross-validation experiments on the clinical PET/CT cohort. The proposed approach acquires a Dice similarity coefficient (DSC) of 0.76 ± 0.13, a Hausdorff distance (HD) of 9.38 ± 8.76 mm, and a Mean surface distance (MSD) of 1.13 ± 0.94 mm, outperforming the SOTA competing methods. The qualitative results show a satisfying consistency with the lesion areas. ConclusionsThe devised transformer-guided cross-modality adaptive feature fusion module integrates the strengths of PET and CT, effectively enhancing the segmentation performance of esophageal GTV. The proposed TransAttPSNN has further advanced the research of esophageal GTV segmentation.

An adaptive multi-scale feature fusion and adaptive mixture-of-experts multi-task model for industrial equipment health status assessment and remaining useful life prediction

Health status (HS) assessment and remaining useful life (RUL) prediction are two essential tasks in prognostics and health management. Though HS assessment and RUL prediction based on multi-task learning have achieved considerable progress, the following three challenges still exist. Firstly, it is assumed that these two tasks are related, and the inter-task correlation is ignored. Secondly, it is difficult to extract complex degradation features using the traditional deep network as the shared-bottom structure. Thirdly, inter-task heterogeneity is not considered, which limits the model's generalization ability to different tasks. To overcome these challenges, this work proposes an adaptive multi-scale feature fusion and adaptive mixture-of-experts multi-task model for HS assessment and RUL prediction. By exploiting inter-task affinity to measure the correlation between tasks, it provides guidance for multi-task modeling of industrial equipment. Then, an adaptive multi-scale feature fusion mechanism is designed as the shared-bottom structure to improve the feature extraction ability of the multi-task model. Besides, considering inter-task heterogeneity, an adaptive mixture-of-experts mechanism is developed to adaptively fuse expert features for specific tasks, which enhances the model's adaptability and generalization to different tasks. Finally, experiments on engine degradation data and tool wear data demonstrate that the proposed model outperforms other state-of-the-art baselines.

MFEFNet: A Multi-Scale Feature Information Extraction and Fusion Network for Multi-Scale Object Detection in UAV Aerial Images

Unmanned aerial vehicles (UAVs) are now widely used in many fields. Due to the randomness of UAV flight height and shooting angle, UAV images usually have the following characteristics: many small objects, large changes in object scale, and complex background. Therefore, object detection in UAV aerial images is a very challenging task. To address the challenges posed by these characteristics, this paper proposes a novel UAV image object detection method based on global feature aggregation and context feature extraction named the multi-scale feature information extraction and fusion network (MFEFNet). Specifically, first of all, to extract the feature information of objects more effectively from complex backgrounds, we propose an efficient spatial information extraction (SIEM) module, which combines residual connection to build long-distance feature dependencies and effectively extracts the most useful feature information by building contextual feature relations around objects. Secondly, to improve the feature fusion efficiency and reduce the burden brought by redundant feature fusion networks, we propose a global aggregation progressive feature fusion network (GAFN). This network adopts a three-level adaptive feature fusion method, which can adaptively fuse multi-scale features according to the importance of different feature layers and reduce unnecessary intermediate redundant features by utilizing the adaptive feature fusion module (AFFM). Furthermore, we use the MPDIoU loss function as the bounding-box regression loss function, which not only enhances model robustness to noise but also simplifies the calculation process and improves the final detection efficiency. Finally, the proposed MFEFNet was tested on VisDrone and UAVDT datasets, and the mAP0.5 value increased by 2.7% and 2.2%, respectively.

Deep learning for high-resolution seismic imaging

Seismic imaging techniques play a crucial role in interpreting subsurface geological structures by analyzing the propagation and reflection of seismic waves. However, traditional methods face challenges in achieving high resolution due to theoretical constraints and computational costs. Leveraging recent advancements in deep learning, this study introduces a neural network framework that integrates Transformer and Convolutional Neural Network (CNN) architectures, enhanced through Adaptive Spatial Feature Fusion (ASFF), to achieve high-resolution seismic imaging. Our approach directly maps seismic data to reflection models, eliminating the need for post-processing low-resolution results. Through extensive numerical experiments, we demonstrate the outstanding ability of this method to accurately infer subsurface structures. Evaluation metrics including Root Mean Square Error (RMSE), Correlation Coefficient (CC), and Structural Similarity Index (SSIM) emphasize the model's capacity to faithfully reconstruct subsurface features. Furthermore, noise injection experiments showcase the reliability of this efficient seismic imaging method, further underscoring the potential of deep learning in seismic imaging.

A method for grasp detection of flexible four-finger gripper

The flexible four-finger gripper, as a specialized robotic end-effector, is highly valued for its ability to passively adapt to the shape of objects and perform non-destructive grasping. However, the development of grasping detection algorithms for flexible four-finger grippers remains relatively unexplored. This paper addresses the unique characteristics of the flexible four-finger gripper by proposing a grasping detection method based on deep learning. Firstly, the Acute Angle Representation model (AAR-model), which is based on the structure of the flexible four-finger gripper and consists of grasp points and angles, is designed as the grasping representation model that reduces unnecessary rotations of the gripper and improves its versatility in grasping objects. Then, the Flexible Gripper Adaptive Attribute model (FGAA-model) is proposed to represent the grasping attributes of objects, calculate the grasp angles that meet the criteria of the AAR-model, and aggregate the AAR-models on the image data into a unified set, thereby circumventing the time-consuming process of pixel-level annotation. Finally, the Adaptive Grasping Neural Net (AGNN), which is based on Adaptive Feature Fusion and the Grasp Aware Network (AFFGA), is introduced by eliminating redundant network detection headers, fusing color and depth images as inputs, and incorporating a Series Atrous Spatial Pyramid (SASP) structure to produce more accurate grasp poses. Our method not only attains a remarkable accuracy of 97.62% on the Cornell dataset but also swiftly completes grasping detection within 25 ms. In practical robotic arm grasping tests, where a robot is outfitted with a flexible four-finger gripper, it successfully grasps unknown objects with a 96% success rate. These results underscore the reliability and real-time performance of our method, significantly enhancing the gripper's adaptability and precision when handling objects of varying sizes and shapes. This advancement provides a powerful technical solution for robots utilizing flexible four-finger grippers, enabling autonomous, real-time, and highly accurate grasping maneuvers. Moreover, it addresses the persistent challenge of the scarcity of efficient grasping detection techniques tailored for flexible four-finger grippers.

A general-purpose pointer meter reading detection method based on adaptive feature fusion and data augmentation

Pointer meters continue to serve as the primary monitoring instruments within substations. To solve the issue of poor model accuracy and low reading accuracy when reading multiple types of pointer meters, this paper proposes a universal cascaded convolutional neural network for accurate reading of various types of pointer instruments. The proposed detection method consists of two stages: instrument digital positioning and instrument pointer fitting. In the digital(figure of a number) positioning stage, an instrument data augmentation method for “large background and small target” is proposed to improve the model's sensitivity to digital position and reduce the interference of redundant background information to the model. In addition, an adaptive feature fusion pyramid structure is proposed for more effective feature fusion and enhanced fuzzy small target recognition. Compared with the baseline model, the method introduced in this study realizes a 0.47 % improvement in target detection accuracy. In our proposed dataset, the average recognition accuracy for the digital position location phase is 98.65 %,the average reading errors in square instruments, circular instruments, and instruments with uneven scales are 0.76 %, 0.73 %, and 0.67 % respectively. The proposed universal detection method in this paper achieves ultra-high precision and can be applied to reading pointer instruments of different types and scales. Compared with existing pointer meter reading methods, the proposed detection method showcases superior simplicity and effectiveness. Furthermore, it showcases a reduced measurement error, thereby bolstering its practical utility in industrial contexts.