Fusion Algorithm Research Articles

Research on Damage Assessment Method for Reinforced Concrete T-Girder Bridges Under Munitions Strikes

As a transportation hub, the girder bridge is often an important target for both sides in combat, and it is very important for wartime decision-making and subsequent fire planning to carry out assessment on the effect of the damage to the bridge after the fight. The use of images for target damage assessment is more secure and efficient compared with the traditional method of manual inspection. However, the current image-based damage assessment of girder bridges has the problems of neglecting the damage to bridge piers and the poor stability of the damage features. To address these issues, this paper proposes a functional damage assessment method for beam bridges after being hit by ammunition based on visible light images. This method decomposes the girder bridge into two main sub-targets (the abutment and deck), analyzes its functional characteristics, designs the residual bearing capacity of the bridge and the number of passable lanes in the damage features, proposes an algorithm for conversion from image features to damage features and a fusion algorithm for the damage features, and establishes the rules for the evaluation of the damage level accordingly. The MATLAB experiment verifies that the method is able to extract the residual bearing capacity and the number of passable lanes from the image of the post-fighting girder bridge and to obtain the damage assessment results, which provides a new method for the assessment of the effect of damage to girder bridges.

A Novel Temporal Fusion Channel Network with Multi-Channel Hybrid Attention for the Remaining Useful Life Prediction of Rolling Bearings

The remaining useful life (RUL) prediction of rolling bearings is crucial for optimizing maintenance schedules, reducing downtime, and extending machinery lifespan. However, existing multi-channel feature fusion methods do not fully capture the correlations between channels and time points in multi-dimensional sensor data. To address the above problems, this paper proposes a multi-channel feature fusion algorithm based on a hybrid attention mechanism and temporal convolutional networks (TCNs), called MCHA-TFCN. The model employs a dual-channel hybrid attention mechanism, integrating self-attention and channel attention to extract spatiotemporal features from multi-channel inputs. It uses causal dilated convolutions in TCNs to capture long-term dependencies and incorporates enhanced residual structures for global feature fusion, effectively extracting high-level spatiotemporal degradation information. The experimental results on the PHM2012 dataset show that MCHA-TFCN achieves excellent performance, with an average Root-Mean-Square Error (RMSE) of 0.091, significantly outperforming existing methods like the DANN and CNN-LSTM.

An image fusion algorithm based on image clustering theory

An image fusion algorithm based on image clustering theory

Quality inspection and error correction of fork-ear type wing-fuselage docking assembly based on multi-camera stereo vision

Abstract During the automatic docking assembly of aircraft wing-fuselage, using monocular camera or dual-camera to monitor the docking stage of the fork-ear will result in an incomplete identification of the fork-ear pose-position and an inaccurate description of the deviation in the intersection holes’ position coordinates. To address this, a quality inspection and error correction method is proposed for the fork-ear docking assembly based on multi-camera stereo vision. Initially, a multi-camera stereo vision detection system is established to inspect the quality of fork-ear docking assembly. Subsequently, a spatial position solution mathematical model of the fork-ear feature points is developed, and a spatial pose determination mathematical model of fork-ear is established by utilised the elliptical cone. Finally, an enhanced artificial fish swarm particle filter algorithm is proposed to track and estimate the coordinate of the fork-ear feature points. An adaptive weighted fusion algorithm is employed to fuse the detection data from the multi-camera and the laser tracker, and a wing pose-position fine-tuning error correction model is constructed. Experimental results demonstrate that the method enhances the effect of the assembly quality inspection and effectively improves the wing-fuselage docking assembly accuracy of the fork-ear type aircraft.

Fusion algorithms on identifying vacant parking spots using vision-based approach

<div class="page" title="Page 1"><div class="layoutArea"><div class="column"><p>In densely populated cities, parking space scarcity results in issues like traffic congestion and difficulty finding parking spots. Recent advancements in computer vision have introduced methods to address parking lot management challenges. The availability of public image datasets and rapid growth in deep learning technology has led to vision-based parking management studies, offering advantages over sensor-based systems in comprehensive area coverage, cost reduction, and additional functionalities. This study presents an innovative fusion algorithm that integrates object detection with occupancy state algorithms to accurately identify vacant parking spaces. The employment of the YOLOv7 framework for vehicle instance segmentation, combined with three occupancy algorithms Euclidean distance (ED), intersection over reference (IoR), and intersection over union (IoU) are compared to determine the occupancy state of observed areas. The proposed method is evaluated using the CNRPark-EXT dataset, and its performance is compared with state-of-the-art methods. As a result, the proposed approach demonstrates robustness under varying conditions. It outperforms existing methods in terms of system evaluation performance, achieving accuracies of 98.88%, 97.99%, and 90.04% for ED, IoR, and IoU, respectively. This fusion detection method enhances adaptability and addresses occlusions, emphasizing YOLOv7’s advantages and accurate shape approximation for slot annotation. This study contributes valuable insights for effective parking management systems and has potential usage in the real-world implementation of intelligent transportation systems.</p></div></div></div>

Integrating multi-omics data reveals neuroblastoma subtypes in the tumor microenvironment

Neuroblastoma (NB) is a severe pediatric tumor originating from the developing sympathetic nervous system, characterized by diverse clinical outcomes, including spontaneous regression and aggressive metastasis. This variability suggests the existence of different NB subtypes, necessitating accurate classification for effective targeted treatment. In this study, we employed the similarity network fusion (SNF) algorithm and identified three NB subtypes, including mesenchymal-like (MES), MYCN-like (MYCN), and neurogenic-like (Neurogenic). The MES subtype exhibited the highest activation of immune-related pathways. The MYCN subtype demonstrated the worst prognosis, with enrichment in cell growth and proliferation pathways. Conversely, the Neurogenic subtype showed the best prognosis, with enrichment in sympathetic nervous system development processes. Through single-cell RNA sequencing (scRNA-seq) analysis, we examined the tumor microenvironments of these distinct NB subtypes, revealing divergent differentiation trajectories for adrenergic cells within the MYCN and Neurogenic subtypes. We also identified a significant presence of naïve T cells in the MES subtype, as well as mesenchymal cell subtypes associated with the unique plasticity observed in both the MES and MYCN subtypes. Drug sensitivity prediction analysis suggested that the MES subtype may respond favorably to MEK inhibitors, while the MYCN subtype may be susceptible to Bcl-2 inhibitors. Our integrative multi-omics approach enabled precise stratification of NB into biologically distinct subtypes, potentially facilitating the development of subtype-specific therapeutic strategies for improved patient management and survival outcomes.

Differential multimodal fusion algorithm for remote sensing object detection through multi-branch feature extraction

Differential multimodal fusion algorithm for remote sensing object detection through multi-branch feature extraction

Multi-spectral Remote Sensing Image Fusion Method Based on Gradient Moment Matching

Image fusion is a popular research direction in the field of computer vision. Traditional image fusion algorithms can achieve good results in fusing grayscale images, but it is difficult to achieve ideal results in processing multi-spectral images. To address the shortcomings of multi-spectral image fusion, this study proposes a low computational complexity and low latency multi-spectral image fusion model by utilizing a multi-step degree moment matching algorithm and a generative adversarial network for fusion. Through experiments, it was found that the F1 score of the GAN-MMN model on the TinyPerson dataset was 89.79%, with an average recall rate of 89.76%. The GAN-MMN performance was higher than that of the control model. Meanwhile, the GAN-MMN model also exhibited superior performance in high-frequency feature extraction and time delay compared to the control model. According to the experimental results, the proposed multi-spectral remote sensing image fusion model had a high feature extraction effect, and its recall rate and F1 score were better than the control model, so the research model had a certain progressiveness. The proposal of this model gives a new approach for the processing of multi-spectral remote sensing images, effectively promoting the development of the computer vision industry.

Evaluation of lower-body gait kinematics on outdoor surfaces using wearable sensors.

The effects of outdoor surfaces on gait are unclear due to difficulties associated with motion tracking outside laboratories. Today, inertial measurement unit (IMU) systems can be deployed to understand the biomechanical adaptations required to navigate real-world environments successfully. This study used IMUs devices to identify lower-limb kinematic adaptations while walking on outdoor surfaces. We hypothesize that gait adaptations between surface types will present as differences in lower-limb joint angles. Thirty able-bodied adults performed walking trials with IMUs on the lower back, thighs, and shanks. Outdoor walking surfaces were flat and even (flateven) (0° grade cement), cobblestone, grass, slope up, slope down, stairs up, and stairs down. A complementary-based sensor fusion algorithm was used to compute hip and knee joint flexion-extension angles, and data were normalized to 100% of the gait cycle based on foot-strike events. Flateven walking was compared against all other surfaces. Two-sample one-dimensional statistical parametric mapping (1d-SPM) t-tests were used to identify differences between angles (α≤0.05). Significant differences in joint angles were identified when grass, slope up, slope down, stairs up, and stairs down walking were compared with flateven (p≤0.005). Moreover, differences were found between slope and stair conditions (p≤0.004). No significant differences were noted between flateven and cobblestone. This study demonstrates that gait adaptations driven by differences in surface types can be observed using IMU sensors in an outdoor setting.

Pix2HDR - A Pixel-Wise Acquisition and Deep Learning-Based Synthesis Approach for High-Speed HDR Videos.

Accurately capturing dynamic scenes with wide-ranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera's frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds - both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.

Dynamic calibration and multi-view fusion of active vision tracking system for surgical instrument

Abstract In recent years, surgical instrument tracking has gained significant attention in the field of surgical navigation. An often-overlooked problem lies in the instrument’s irregular movement conflicting with the tracking system’s fixed viewpoint, leading to line-of-sight (LOS) occlusion, marker occlusion, and sub-optimal accuracy. The active vision tracking system (which actively adjusts the camera viewpoint) is applied to address this issue. In this paper, two algorithms of multi-view fusion and dynamic calibration are proposed in multi-camera active vision to allow each camera to adjust its viewpoint independently. These algorithms are integrated into the parallel computing of the frontend and backend to implement the positioning of the instrument from multiple perspectives and real-time calibration of multiple cameras. Simulations and experiments are carried out to test the accuracy and robustness of the proposed tracking system. Results reveal that our system achieves a closer trajectory (0.12 mm) to the ground truth compared to a stereo camera tracking system (0.32 mm). The proposed tracking system is able to keep track of the instruments within the positioning volume at different surgical phases, ensuring consistent navigation and improving positioning stability and accuracy.

Research and Application Analysis of Global Path Planning Method Based on Radar and Vision in Robot SLAM

In recent years, the production scale of robots has become larger and larger, involving all aspects of society. Many industries are beginning to introduce robots and use them to better development and production. At the same time, autonomous driving is also a promising research direction, which is expected to reach 8 million autonomous vehicles by 2030, and the passenger car market will account for about 13% of total passenger miles. Autonomous driving will also bring many benefits to people, such as freeing up an average of 50 minutes of extra time for drivers per day, which can reduce accidents by more than 90%. Whether it is a robot or a self-driving car, the most important and critical goal is to ensure that the target task can be completed safely and quickly. Simultaneous localization and mapping (SLAM) technology can better help robots achieve this goal. In this paper, laser SLAM, visual SLAM and laser-visual SLAM algorithms are introduced, and then some related applications are shown. Finally, the future development prospect of SLAM should pay more attention to multi-sensor fusion algorithm.

Bio-inspired multi-dimensional deep fusion learning for predicting dynamical aerospace propulsion systems

Rapid and precise forecasting of dynamical systems is critical to ensuring safe aerospace missions. Previous forecasting research has primarily concentrated on global trend analysis using full-scale inputs. However, time series arising from real-world applications such as aerospace propulsion, exhibit a distinct dynamical periodicity over a limited timeframe. Here we develop a deep learning model, TimeWaves, to capture both global trends and local variations, through 3D spectrum-oriented interval extraction from an integrated viewpoint of biological perceptions. Specifically, a shared parameter fusion algorithm is employed to effectively integrate Fourier and Wavelet analyses, providing full and sliced 1D sequences to form 2D tensors that can be seamlessly processed by parameter-efficient inception blocks. Additionally, a dual-way learning workflow using TwinBlock, inspired by the cooperative behavior of visual cells, is implemented to enhance perception of dynamical multi-scale features at a reduced computational cost. TimeWaves demonstrates reliable and robust performance in predicting rocket combustion instability, a key challenge in the aerospace industry.

Design of an Autonomous Orchard Navigation System Based on Multi-Sensor Fusion

To address the limitations of traditional GNSS-based navigation systems in orchard environments, we propose a multi-sensor fusion-based autonomous navigation method for orchards. A crawler-type agricultural platform was used as a test vehicle, and an autonomous orchard navigation system was constructed using a 2D LiDAR, a dynamic electronic compass, and an encoder. The proposed system first filters LiDAR point cloud data and uses the DBSCAN–ratio–threshold method to process data and identify clusters of tree trunks. By matching the center coordinates of trunk clusters with a fruit tree distribution map, the platform’s positional measurements are determined. An extended Kalman filter fusion algorithm is then employed to obtain a posterior estimate of the platform’s position and pose. Experimental results demonstrate that in localization accuracy tests and navigation tests, the proposed system provides high navigation accuracy and robustness, making it suitable for autonomous walking operations in orchard environments.

Multi-scale feature fusion algorithm for underwater remote object detection in forward-looking sonar images

Multi-scale feature fusion algorithm for underwater remote object detection in forward-looking sonar images

Security analysis and adaptive false data injection against multi-sensor fusion localization for autonomous driving

Multi-sensor Fusion (MSF) algorithms are critical components in modern autonomous driving systems, particularly in localization and AI-powered perception modules, which play a vital role in ensuring vehicle safety. The Error-State Kalman Filter (ESKF), specifically employed for localization fusion, is widely recognized for its robustness and accuracy in MSF implementations. While existing studies have demonstrated the vulnerability of ESKF to sensor spoofing attacks, these works have primarily focused on a black-box implementation, leading to an insufficient security analysis. Specifically, due to the lack of theoretical guidance in previous methods, these studies have consistently relied on exponential functions to fit attack sequences across all scenarios. As a result, the attacker had to explore an extensive parameter space to identify effective attack sequences, lacking the ability to adaptively generate optimal ones. This paper aims to fill this crucial gap by conducting a thorough security analysis of the ESKF model and presenting a simple approach for modeling injection errors in these systems. By utilizing this error modeling, we introduce a new attack strategy that employs constrained optimization to reduce the energy needed to reach the same deviation target, guaranteeing that the attack is both efficient and effective. This method increases the stealthiness of the attack, making it harder to detect. Unlike previous methods, our approach can dynamically produce nearly perfect injection signals without requiring multiple attempts to find the best parameter combination in different scenarios. Through extensive simulations and real-world experiments, we demonstrate the superiority of our method compared to state-of-the-art attack strategies. Our results indicate that our approach requires significantly less injection energy to achieve the same deviation target. Additionally, we validate the practical applicability and impact of our method through end-to-end testing on an AI-powered autonomous driving system.

A Review of Multi-Sensor Fusion Techniques for Indoor Mobile Robot Navigation

The application of multi-sensor fusion technology in indoor mobile robot navigation and localization has been increasingly gaining attention, especially in complex indoor environments where achieving high-precision autonomous navigation poses a significant challenge. This review summarizes various sensor fusion methods, including the combination of Light Detection and Ranging (LiDAR), Inertial Measurement Unit (IMU), Ultra-Wideband (UWB), and others, with a focus on discussing the progress of fusion algorithms such as Extended Kalman Filter (EKF) and Adaptive Monte Carlo Localization (AMCL) in improving navigation accuracy and stability. In addition, this paper explores the application of visual Simultaneous Localization and Mapping (SLAM) methods incorporating deep learning in indoor robot navigation. Finally, the main challenges currently faced by multi-sensor fusion technology in robot autonomous navigation are analyzed, and future research directions are proposed, aiming to provide valuable references for researchers in the field.

Intelligent Fault Diagnosis of Inter-Turn Short Circuit Faults in PMSMs for Agricultural Machinery Based on Data Fusion and Bayesian Optimization

The permanent magnet synchronous motor (PMSM) plays an important role in the power system of agricultural machinery. Inter-turn short circuit (ITSC) faults are among the most common failures in PMSMs, and early diagnosis of these faults is crucial for enhancing the safety and reliability of motor operation. In this article, a multi-source data-fusion algorithm based on convolutional neural networks (CNNs) has been proposed for the early fault diagnosis of ITSCs. The contributions of this paper can be summarized in three main aspects. Firstly, synchronizing data from different signals extracted by different devices presents a significant challenge. To address this, a signal synchronization method based on maximum cross-correlation is proposed to construct a synchronized dataset of current and vibration signals. Secondly, applying a traditional CNN to the data fusion of different signals is challenging. To solve this problem, a multi-stream high-level feature fusion algorithm based on a channel attention mechanism is proposed. Thirdly, to tackle the issue of hyperparameter tuning in deep learning models, a hyperparameter optimization method based on Bayesian optimization is proposed. Experiments are conducted based on the derived early-stage ITSC fault-severity indicator, validating the effectiveness of the proposed fault-diagnosis algorithm.

Remote sensing-based agricultural drought mapping in Northern Jordan using Landsat and MODIS data

Monitoring agricultural drought in a semi-arid environment is critical, especially during the growing season, as it negatively impacts vegetation health and crop yield. This study aimed to monitor the spatiotemporal variation of agricultural drought in Northern Jordan using Landsat-8 and Moderate Resolution Imaging Spectroradiometer (MODIS) satellite data. The Spatio-Temporal Image Fusion Model (STI-FM) was used to produce synthetic Landsat images with a high spatiotemporal resolution (30 m / 8 days) by utilizing a pair of successive MODIS images at two points in time (time-1 and time-2) and one Landsat-8 image at time-1. Agricultural drought was mapped and monitored using spectral indices namely, Vegetation Condition Index (VCI), Temperature Condition Index (TCI), and Vegetation Health Index (VHI). Additionally, the Standard Precipitation Index (SPI) -based meteorological rainfall data was used to validate the accuracy of the drought maps. The results revealed significant spatiotemporal variations in drought conditions, with April 2020 showing the least dry conditions, while 2019 was identified as the driest year. Validation through SPI indicated high accuracy, with kappa values ranging from 0.70 to 0.85 and overall accuracy ranging from 80 % to 92 %. Furthermore, the STI-FM data fusion algorithm effectively generated high-resolution Landsat-8 images, demonstrating a strong correlation between original and synthetic images for the red and NIR spectral bands (0.93 and 0.84, respectively). These findings highlight the effectiveness of integrating STI-FM with spectral indices and SPI for accurate and high-resolution agricultural drought monitoring, which can support improved water resource management and agricultural planning in semi-arid regions.

Synthesis of pseudo-PET/CT fusion images in radiotherapy based on a new transformer model.

PET/CT and planning CT are commonly used medical images in radiotherapy for esophageal and nasopharyngeal cancer. However, repeated scans will expose patients to additional radiation doses and also introduce registration errors. This multimodal treatment approach is expected to be further improved. A new Transformer model is proposed to obtain pseudo-PET/CT fusion images for esophageal and nasopharyngeal cancer radiotherapy. The data of 129 cases of esophageal cancer and 141 cases of nasopharyngeal cancer were retrospectively selected for training, validation, and testing. PET and CT images are used as input. Based on the Transformer model with a "focus-disperse" attention mechanism and multi-consistency loss constraints, the feature information in two images is effectively captured. This ultimately results in the synthesis of pseudo-PET/CT fusion images with enhanced tumor region imaging. During the testing phase, the accuracy of pseudo-PET/CT fusion images was verified in anatomy and dosimetry, and two prospective cases were selected for further dose verification. In terms of anatomical verification, the PET/CT fusion image obtained using the wavelet fusion algorithm was used as the ground truth image after correction by clinicians. The evaluation metrics, including peak signal-to-noise ratio, structural similarity index, mean absolute error, and normalized root mean square error, between the pseudo-fused images obtained based on the proposed model and ground truth, are represented by means (standard deviation). They are 37.82 (1.57), 95.23 (2.60), 29.70 (2.49), and 9.48 (0.32), respectively. These numerical values outperform those of the state-of-the-art deep learning comparative models. In terms of dosimetry validation, based on a 3%/2mm gamma analysis, the average passing rates of global and tumor regions between the pseudo-fused images (with a PET/CT weight ratio of 2:8) and the planning CT images are 97.2% and 95.5%, respectively. These numerical outcomes are superior to those of pseudo-PET/CT fusion images with other weight ratios. This pseudo-PET/CT fusion images obtained based on the proposed model hold promise as a new modality in the radiotherapy for esophageal and nasopharyngeal cancer.