Fusion Algorithm Research Articles

Method for enhanced gesture recognition under low light conditions based on wearable mechanoluminescence sensors

Gestures, as one of the key modes of human–computer interaction, are currently identified and processed mainly through two methods: machine vision imaging and wearable stress/strain sensing.However, the former has high requirements for lighting and background.This paper innovatively proposes a feature fusion algorithm that combines mechanoluminescent sensors for gesture recognition under low-light conditions. Unlike existing methods, this algorithm creates a system that integrates mechanoluminescent sensors for visual gesture recognition. By triggering the sensor’s luminescence through mechanical stress, it provides additional light source information to enhance image quality in low-light environments.For this purpose, this study employed the fluorescent emission properties of ZnS:Cu to fabricate a mechanoluminescence sensor. Its mechanoluminescence response characteristics were measured. Based on these findings, an algorithm for the fusion of gesture information features was developed for low illumination backgrounds. This algorithm decomposes the gesture visual images through YCbCr, extracting the HOG features of the Cr channel (Cr-HOG). Simultaneously, feature fusion is performed using the pooling layer (VGG16-Conv5-pool) following the convolutional layer (Conv5-3) in the VGG16 model. A one-to-one composite SVM classifier is constructed to complete the training and testing of the gesture recognition model. Ultimately, the feasibility of this enhanced gesture recognition method was validated through unmanned vehicle control experiment. Experimental results demonstrated that in environments with an illuminance level below 10 LUX measured by a light meter, the average recognition rate of the wearable mechanoluminescence sensor feature fusion algorithm reached 95.82 %. This is 32.24 % higher than the recognition rate of the single Cr-HOG feature classification and 28.27 % higher than the single VGG16-Conv5-pool feature classification. Compared with feature extraction using ResNet-50 and DenseNet-121 networks, the recognition rate for classifying gestures under low-light conditions is approximately 30 % higher.Compared with other image enhancement recognition methods, the recognition can be increased by up to 30.88 %.

Unstructured environment navigation and trajectory calibration technology based on multi-data fusion

In order to effectively improve the accuracy and efficiency of unstructured environment navigation and trajectory calibration, a multi data fusion based unstructured environment navigation and trajectory calibration method was studied. Described the basic principles of the Strapdown Inertial Navigation System and Doppler Log. After introducing DVL velocity information and relative position information of underwater robots into SINS, the Kalman filter fusion algorithm was used to obtain the system's state equation and measurement equation. The navigation parameter error and device error parameters of the system were selected as the state variables. The difference between SINS output speed and DVL measurement speed, as well as the difference between SINS output position and relative position, are used as the speed and position measurements of the measurement system. Error estimation is obtained through indirect filtering, and the accuracy of navigation and positioning is improved through output correction. The experimental results show that the proposed method has a good calibration effect and can effectively improve the accuracy and efficiency of trajectory calibration in unstructured environments.

Dynamic path planning fusion algorithm with improved A* algorithm and dynamic window approach

Dynamic path planning fusion algorithm with improved A* algorithm and dynamic window approach

Research on On-Line Monitoring of Grinding Wheel Wear Based on Multi-Sensor Fusion.

The state of a grinding wheel directly affects the surface quality of the workpiece. The monitoring of grinding wheel wear state can allow one to efficiently identify grinding wheel wear information and to timely and effectively trim the grinding wheel. At present, on-line monitoring technology using specific sensor signals can detect abnormal grinding wheel wear in a timely manner. However, due to the non-linearity and complexity of the grinding wheel wear process, as well as the interference and noise of the sensor signal, the accuracy and reliability of on-line monitoring technology still need to be improved. In this paper, an intelligent monitoring system based on multi-sensor fusion is established, and this system can be used for precise grinding wheel wear monitoring. The proposed system focuses on titanium alloy, a typical difficult-to-process aerospace material, and addresses the issue of low on-line monitoring accuracy found in traditional single-sensor systems. Additionally, a multi-eigenvalue fusion algorithm based on an improved support vector machine (SVM) is proposed. In this study, the mean square value of the wavelet packet decomposition coefficient of the acoustic emission signal, the grinding force ratio of the force signal, and the effective value of the vibration signal were taken as inputs for the improved support vector machine, and the recognition strategy was adjusted using the entropy weight evaluation method. A high-precision grinding machine was used to carry out multiple sets of grinding wheel wear experiments. After being processed by the multi-sensor integrated precision grinding wheel wear intelligent monitoring system, the collected signals can accurately reflect the grinding wheel wear state, and the monitoring accuracy can reach more than 92%.

Spatial correction and Deblurring fusion algorithm for vehicle license plate images based on deep learning

Spatial correction and Deblurring fusion algorithm for vehicle license plate images based on deep learning

Bilinear Perceptual Fusion Algorithm Based on Brain Functional and Structural Data for ASD Diagnosis and Regions of Interest Identification.

Autism spectrum disorder (ASD) is a serious mental disorder with a complex pathogenesis mechanism and variable presentation among individuals. Although many deep learning algorithms have been used to diagnose ASD, most of them focus on a single modality of data, resulting in limited information extraction and poor stability. In this paper, we propose a bilinear perceptual fusion (BPF) algorithm that leverages data from multiple modalities. In our algorithm, different schemes are used to extract features according to the characteristics of functional and structural data. Through bilinear operations, the associations between the functional and structural features of each region of interest (ROI) are captured. Then the associations are used to integrate the feature representation. Graph convolutional neural networks (GCNs) can effectively utilize topology and node features in brain network analysis. Therefore, we design a deep learning framework called BPF-GCN and conduct experiments on publicly available ASD dataset. The results show that the classification accuracy of BPF-GCN reached 82.35%, surpassing existing methods. This demonstrates the superiority of its classification performance, and the framework can extract ROIs related to ASD. Our work provides a valuable reference for the timely diagnosis and treatment of ASD.

A Fusion Tracking Algorithm for Electro-Optical Theodolite Based on the Three-State Transition Model.

This study presents a novel approach to address the autonomous stable tracking issue in electro-optical theodolite operating in closed-loop mode. The proposed methodology includes a multi-sensor adaptive weighted fusion algorithm and a fusion tracking algorithm based on a three-state transition model. A refined recursive formula for error covariance estimation is developed by integrating attenuation factors and least squares extrapolation. This formula is employed to formulate a multi-sensor weighted fusion algorithm that utilizes error covariance estimation. By assigning weighted coefficients to calculate the residual of the newly introduced error term and defining the sensor's unique states based on these coefficients, a fusion tracking algorithm grounded on the three-state transition model is introduced. In cases of interference or sensor failure, the algorithm either computes the weighted fusion value of the multi-sensor measurement or triggers autonomous sensor switching to ensure the autonomous and stable measurement of the theodolite. Experimental results indicate that when a specific sensor is affected by interference or the off-target amount cannot be extracted, the algorithm can swiftly switch to an alternative sensor. This capability facilitates the precise and consistent generation of data, thereby ensuring the stable operation of the tracking system. Furthermore, the algorithm demonstrates robustness across various measurement scenarios.

Sound speed prediction of seafloor sediments in the South Yellow Sea based on BP-AdaBoost model

Based on the sound speed and physical property measurements of seafloor sediments obtained in the central part of the South Yellow Sea, a multi-parameter prediction model for the sound speed of seafloor sediments based on seven parameters, including density, porosity, water content, liquid limit, plasticity index, compression coefficient, and median grain size, was developed by using the BP-AdaBoost fusion algorithm. The results show that the model, configured with 7 input layer neurons, 10 hidden layer neurons, 1 output node, a learning rate of 0.1, and 300 training iterations, achieved a correlation coefficient of 0.962. The mean absolute error (MAE) of the predicted sound speed was 10.19 m/s, and the mean relative error (MAPE) was 0.66%. These results are better than those of single-parameter and dual-parameter prediction equations and other machine learning models. The BP-AdaBoost model prediction method of sound speed of seafloor sediments established in this paper is better than the single-parameter and dual-parameter prediction equations and other prediction models, and it can provide a new way for the prediction of sound speed of seafloor sediments in the study area.

Domain adaptation hyperspectral image fusion based on spatial-spectral domain separation

ABSTRACT Hyperspectral datasets captured by airborne are one of the important data sources in the field of hyperspectral image fusion. However, the limited number of data samples often hinders the optimal performance of deep learning-based methods on these datasets. Domain adaptation is expected to alleviate the issue of data scarcity. However, there are two issues that need to be addressed in applying domain adaptation to hyperspectral image fusion, which are the heterogeneity and spatial degradation diversity of hyperspectral data. In this paper, we propose a domain adaptation hyperspectral image fusion network based on spatial-spectral domain separation. The model, constructed with three-dimensional convolutional layers, effectively addresses the challenge of hyperspectral data heterogeneity. Compared with 2D convolution, 3D convolution is able to consider both spatial and spectral dimensions and extract spatial-spectral features efficiently. Based on this, we design a domain-separation architecture to extract domain-invariant and domain-private features in the spatial-spectral domain. The architecture achieves domain adaptation by extracting domain-invariant features from both datasets to obtain the prior knowledge in the source dataset and apply it to the target dataset. Additionally, the spatial degradation of different datasets varies due to distinct shooting conditions during the acquisition. Therefore, the fusion algorithms should be robust to various degradation scenarios. To tackle this challenge, we design the Observation Module to predict degradation information and the Degradation Information Modulation Module to apply it to the input, thereby enhancing the network’s robustness. Experiments on various datasets demonstrate that our method is qualitatively and quantitatively superior to state-of-the-art methods.

Design and Experiment of an Independent Leg-Type Chassis Vehicle Attitude Adjustment System

In response to the current low work efficiency of soil ridge-working machinery, as well as its poor stability, passability, and adaptability, this paper designs an independent leg-type working platform that can autonomously adjust its vehicle attitude through LiDAR scanning in a soil ridge-working environment. The platform, in terms of its mechanism and structural design, adopts dual parallelogram mechanisms, dual lead screw mechanisms, and independent column leg mechanisms, with a maximum adjustable ground clearance of 107 mm and a maximum wheelbase adjustment of 150 mm. A gyroscope is mounted at the center of the platform for attitude adjustment, ensuring the accurate data collection of the ultrasonic ranging module. Moreover, the platform adopts an adaptive adjustment method based on vehicle attitude and soil ridge shape parameters, obtaining soil ridge parameters through LiDAR and combining ultrasonic ranging module data with stepper motor pulse signals to obtain the absolute vehicle attitude parameters, using first and second linear regression methods to adjust the vehicle attitude and other working parameters. A prototype was also created, and the test data from the soil obtained through experiments show that, after leveling with the gyroscope leveling algorithm, the average value of the pitch angle is up to 0.6154°, and the average value of the roll angle is up to 0.9989°, with the maximum variance of the pitch angle being 0.0474° and the maximum variance of the tilt angle being 0.1320°. After the ultrasonic ranging module data are filtered by the Kalman filter, the maximum variance is 0.0304, and after applying the final fusion algorithm, the maximum variance is only 0.0085. The LiDAR measurement width value deviates from the actual width value by no more than 1.0 cm, and the LiDAR measurement height value deviates from the actual height value by no more than 1.0 cm. The platform’s actual adjusted width deviates from the actual soil ridge width by no more than 2.0 cm, and the platform’s actual adjusted height deviates from the actual soil ridge height by no more than 1.2 cm. This platform can improve the passability, adaptability, and stability of agricultural machinery in soil ridge work and provide technical references for subsequent related research.

Point Cloud Densification Algorithm for Multiple Cameras and Lidars Data Fusion.

Fusing data from many sources helps to achieve improved analysis and results. In this work, we present a new algorithm to fuse data from multiple cameras with data from multiple lidars. This algorithm was developed to increase the sensitivity and specificity of autonomous vehicle perception systems, where the most accurate sensors measuring the vehicle's surroundings are cameras and lidar devices. Perception systems based on data from one type of sensor do not use complete information and have lower quality. The camera provides two-dimensional images; lidar produces three-dimensional point clouds. We developed a method for matching pixels on a pair of stereoscopic images using dynamic programming inspired by an algorithm to match sequences of amino acids used in bioinformatics. We improve the quality of the basic algorithm using additional data from edge detectors. Furthermore, we also improve the algorithm performance by reducing the size of matched pixels determined by available car speeds. We perform point cloud densification in the final step of our method, fusing lidar output data with stereo vision output. We implemented our algorithm in C++ with Python API, and we provided the open-source library named Stereo PCD. This library very efficiently fuses data from multiple cameras and multiple lidars. In the article, we present the results of our approach to benchmark databases in terms of quality and performance. We compare our algorithm with other popular methods.

Sensing Skin Technology for Fatigue Crack Monitoring of Steel Bridges: Laboratory Development, Field Validation, and Future Directions

A significant number of steel bridges are vulnerable to fatigue cracks, and the timely discovery of damage is critical in ensuring safety and continuous operations. Despite various crack monitoring methods available in the field of structural health monitoring (SHM) to empower real-time feedback, few commercially-available technologies are applicable to the task of discovering new cracks because of the highly localized nature of the sensors. The authors have developed a sensing skin constituted from soft elastomeric capacitors (SECs). An SEC is a large-area strain gauge that transduces strain into a measurable change in capacitance. It can be easily deployed over large surfaces, and thus can be used to discover new fatigue cracks. The technology has been developed and characterized in a laboratory environment over the last decade. It has been recently deployed in the field on a bridge located in Kansas, USA. The aim of this paper is to present and discuss technological updates that were necessary to enable field deployment, with the objective of supporting the field deployment of the cSEC and other SHM technologies. In particular, it reviews the following SEC technology modifications: 1) corrugation of the surface of the dielectric, creating a corrugated SEC or cSEC, to improve sensing performance; 2) development of a dedicated wireless data acquisition system; and 3) improvement of the data fusion algorithm to account for higher signal contamination in the field. After this review, challenges in conducting field deployment are examined. Lastly, a discussion is provided on a path to commercialization.

Effect of sensor noise characteristics and calibration errors on the choice of IMU-sensor fusion algorithms

This paper focuses on accurate and precise orientation estimation with consumer-grade MEMS-IMUs for ‘slow’ orientation change and ‘short’-time applications. A simulation platform is developed to predict a suitable algorithm for a MEMS-IMU of known noise specifications, improving similar works. Experimentally measured noise characteristics of two commercial grade IMUs (MPU9250 and BNO055) are used in the simulation platform to generate simulated data and evaluate some popular orientation estimation algorithms along with two new Kalman filter-based algorithms. Real experiments are conducted with the same IMUs using an electromagnetic tracker as reference sensor. The output orientation results for two new improved algorithms are compared with other algorithms in simulations and real experiments. We show that the choice of the ‘best’ algorithm varies with the noise characteristics of individual sensors within the sensor module. The two new best-performing algorithms tested achieve<1˚ RMS angle error for the two low-cost consumer-grade IMUs.

Multi-Source Image Fusion Based on BEMD and Region Sharpness Guidance Region Overlapping Algorithm

Multi-focal image and multi-modal image fusion technology can fully take advantage of different sensors or different times, retaining the image feature information and improving the image quality. A multi-source image fusion algorithm based on bidimensional empirical mode decomposition (BEMD) and a region sharpness-guided region overlapping algorithm are studied in this article. Firstly, source images are decomposed into multi-layer bidimensional intrinsic mode functions (BIMFs) and residuals from high-frequency layer to low-frequency layer by BEMD. Gaussian bidimensional intrinsic mode functions (GBIMFs) are obtained by applying Gaussian filtering operated on BIMF and calculating the sharpness value of segmented regions using an improved weighted operator based on the Tenengrad function, which is the key to comparison selection and fusion. Then, the GBIMFs and residuals selected by sharpness comparison strategy are fused by the region overlapping method, and the stacked layers are weighted to construct the final fusion image. Finally, based on qualitative evaluation and quantitative evaluation indicators, the proposed algorithm is compared with six typical image fusion algorithms. The comparison results show that the proposed algorithm can effectively capture the feature information of images in different states and reduce the redundant information.

NPSFF-Net: Enhanced Building Segmentation in Remote Sensing Images via Novel Pseudo-Siamese Feature Fusion

Building segmentation has extensive research value and application prospects in high-resolution remote sensing image (HRSI) processing. However, complex architectural contexts, varied building morphologies, and non-building occlusions make building segmentation challenging. Compared with traditional methods, deep learning-based methods present certain advantages in terms of accuracy and intelligence. At present, the most popular option is to first apply a single neural network to encode an HRSI, then perform a decoding process through up-sampling or using a transposed convolution operation, and then finally obtain the segmented building image with the help of a loss function. Although effective, this approach not only tends to lead to a loss of detail information, but also fails to fully utilize the contextual features. As an alternative, we propose a novel network called NPSFF-Net. First, using an improved pseudo-Siamese network composed of ResNet-34 and ResNet-50, two sets of deep semantic features of buildings are extracted with the support of transfer learning, and four encoded features at different scales are obtained after fusion. Then, information from the deepest encoded feature is enriched using a feature enhancement module, and the resolutions are recovered via the operations of skip connections and transposed convolutions. Finally, the discriminative features of buildings are obtained using the designed feature fusion algorithm, and the optimal segmentation model is obtained by fitting a cross-entropy loss function. Our method obtained intersection-over-union values of 89.45% for the Aerial Imagery Dataset, 71.88% for the Massachusetts Buildings Dataset, and 68.72% for the Satellite Dataset I.

Detection and Tracking of Low-Frame-Rate Water Surface Dynamic Multi-Target Based on the YOLOv7-DeepSORT Fusion Algorithm

This study aims to address the problem in tracking technology in which targeted cruising ships or submarines sailing near the water surface are tracked at low frame rates or with some frames missing in the video image, so that the tracked targets have a large gap between frames, leading to a decrease in tracking accuracy and inefficiency. Thus, in this study, we proposed a water surface dynamic multi-target tracking algorithm based on the fusion of YOLOv7 and DeepSORT. The algorithm first introduces the super-resolution reconstruction network. The network can eliminate the interference of clouds and waves in images to improve the quality of tracking target images and clarify the target characteristics in the image. Then, the shuffle attention module is introduced into YOLOv7 to enhance the feature extraction ability of the target features in the recognition network. Finally, Euclidean distance matching is introduced into the cascade matching of the DeepSORT algorithm to replace the distance matching of IOU to improve the target tracking accuracy. Simulation results showed that the algorithm proposed in this study has a good tracking effect, with an improvement of 9.4% in the improved YOLOv7 model relative to the mAP50-95 value and an improvement of 13.1% in the tracking accuracy in the DeepSORT tracking network compared with the SORT tracking accuracy.

A screw algorithm for congenital C2-3 fusion with high-riding vertebral arteries: feasibilities and clinical outcomes of five different fixation techniques.

To propose a screw algorithm and investigate the anatomical feasibilities and clinical outcomes of five distinct fixation methods for C2-3 fused vertebra with high-ridding vertebral arteries (VA) (HRVA) when the C2 pedicle screw placement is unfeasible. Thirty surgical patients with congenital C2-3 fusion, HRVA, and atlantoaxial dislocation (AAD) were included. We designed a algorithm for alternative screw implantation into C2-3 fused vertebrae, including C2 pedicle screw with in-out-in (passing VA groove) technique (in-out-in screw), subfacetal screw, translaminar screw, lateral mass screw, C3 pedicle screw. VA diameter and position, C2 and C3 pedicles, superior facets, fused lamina, and fused lateral mass dimensions were evaluated for screw implantation indication. Implant failure, reduction loss, implant placement accuracy were investigated by computed tomography. A total of 5 VAs were identified as distant VAs; a total of 2 VAs were categorized as occlusive VAs. Sufficient dimension of lateral mass and lamina provided the broadest indications for screw implantation, while the distant or occlusive VA provided the most limited indications for in-out-in screw. The indications of five alternative methods ranged from narrowest to widest as follows: in-out-in screw, C3 pedicle screw, subfacetal screw, translaminar screw, lateral mass screw. The translaminar screws and the lateral mass screws increased the probability of implant failure. All patients who received in-out-in screws, C3 pedicle screws, and subfacetal screws achieved fusion. The accuracy ranged from lowest to highest as follows: C3 pedicle screw, lateral mass screw, in-out-in screw, subfacetal screw, translaminar screw. No translaminar screws deviated. The algorithm proved to be a valuable tool for screw selection in cases of C2-3 fused vertebrae with HRVAs. The subfacetal screw, boasting broad indications, a high fusion rate, and exceptional accuracy, stood as the primary preferred alternative.

Advanced Optical‐Thermal Integrated Flexible Tactile Sensor for High‐Fine Recognition of Liquid Property in Non‐Contact Mode

AbstractFlexible sensors have been applied to accurately identify the basic features of various solid objects. However, a significant obstacle is the accurate identification of liquids, due to it is impossible to exert force directly on those. Here, an optical‐thermal flexible tactile sensor is designed for liquid recognition, which integrates PEDOT:PSS/SrTiO3 thermoelectric foam and ZnS‐CaZnOS mechanoluminescent (ML) film coated with thermoplastic polyurethane (TPU), enabling the simultaneous detection of thermal and optical changes with low interference, via its exceptional mechanical stimuli‐caused light and superior thermoelectric voltage, respectively. In addition, the TPU coating imparts the device with excellent hydrophobic and oleophobic properties, enhancing its durability in complex conditions. This flexible sensor, possessing high stability under thermal and mechanical cycles, can distinguish different pure solvents and turbid suspensions, achieving a high‐fine recognition accuracy of 95% and 97.5%, respectively, with the aid of the fusion algorithm of multimodal sensory data based on the thermal and optical outputs. This capability of the device can underscore the practical utility of humanoid robots to identify hazardous liquids in some scenarios like safety inspection and kitchen condiments, which addresses the critical gaps in current sensory technologies by providing a robust, sensitive, and multifunctional strategy for accurate liquid recognition.

Failure rate analysis and maintenance plan optimization method for civil aircraft parts based on data fusion

In the face of data scarcity in the optimization of maintenance strategies for civil aircraft, traditional failure data-driven methods are encountering challenges owing to the increasing reliability of aircraft design. This study addresses this issue by presenting a novel combined data fusion algorithm, which serves to enhance the accuracy and reliability of failure rate analysis for a specific aircraft model by integrating historical failure data from similar models as supplementary information. Through a comprehensive analysis of two different maintenance projects, this study illustrates the application process of the algorithm. Building upon the analysis results, this paper introduces the innovative equal integral value method as a replacement for the conventional equal interval method in the context of maintenance schedule optimization. The Monte Carlo simulation example validates that the equivalent essential value method surpasses the traditional method by over 20% in terms of inspection efficiency ratio. This discovery indicates that the equal critical value method not only upholds maintenance efficiency but also substantially decreases workload and maintenance costs. The findings of this study open up novel perspectives for airlines grappling with data scarcity, offer fresh strategies for the optimization of aviation maintenance practices, and chart a new course toward achieving more efficient and cost-effective maintenance schedule optimization through refined data analysis.

Convolutional neural networks-based multi-sensor data fusion analysis for transmission lines

Abstract With the development of the power system, accurate assessment and fault diagnosis of the health status of transmission lines have become increasingly important. However, the standard data fusion algorithms do not fully exploit the correlation and weight disparity among multiple sensors. This study proposes a technique for analyzing transmission lines using a convolutional neural network-based approach to address this issue. By fusing data collected from multiple sensors in different directions, it can more accurately capture key information about transmission lines and conduct comprehensive analysis to evaluate their health status.