Multi-sensor Fusion Algorithm Research Articles

Interference fading suppression with fault-tolerant Kalman filter in phase-sensitive OTDR

A multi-sensor information fusion algorithm based on fault-tolerant Kalman filter is proposed in phase-sensitive optical time-domain reflectometer (Φ-OTDR) system, for achieving fading-free distributed vibration sensing. Firstly, a fault-tolerant dual-core complementary array model is designed. The Rayleigh scattering signal denoising, and vibration existence judgment of localization points are carried out to obtain the differentiated frequency demodulation results of the sensing points of the dual-core fiber array. Then a fault-tolerant control strategy is used to determine the sensor weight coefficients and vibration judgment coefficients during data fusion processing, and array data fusion is carried out based on time series data using Kalman filter to realize error value identification and filling. The advantage of this method is the combination of redundant data in a complementary way to improve the system stability. The frequency response ranges from 10 Hz to 2400 Hz and the localization accuracy is 98.33%. The influence of key parameters on the frequency demodulation performance of fault-tolerant Kalman filter is discussed, and a standard deviation of 14.6 Hz and an average error of 7.6 Hz are obtained. The demodulation frequency data matrix obtained by the classical demodulation method has a demodulation error probability of 89.18%, which proves the widespread existence of demodulation errors in vibration signals. The fusion error of demodulation frequency is reduced to 0.25 Hz, the frequency demodulation accuracy reaches 100%, and the demodulation error caused by interference attenuation can be completely eliminated. This system based on fault-tolerant Kalman filter has the characteristics of simple multiplexing structure, interference fading resistance and stable demodulation performance.

End-to-end multi-sensor fusion method based on deep reinforcement learning in UASNs

Underwater acoustic sensor networks (UASNs) have attracted considerable attention and are extensively employed for tracking underwater targets. However, due to the complex dynamics of underwater tracking environments, establishing accurate correlations between sensors remains elusive and challenging to precisely model. Traditional multi-sensor fusion methods for handling unknown correlations often face issues related to dependence on prior knowledge and a lack of flexibility. In response, we propose a novel end-to-end multi-sensor fusion algorithm rooted in deep reinforcement learning (DRL). To address the challenge of unknown correlations among sensors in UASNs, we formulate the multi-sensor fusion strategy as a Markov Decision Process (MDP) within the framework of reinforcement learning. The Proximal Policy Optimization (PPO) method is implemented to tackle this MDP without scalability limitations, aiming to derive an optimal sensor scheduling policy for UASNs. To enhance practical applicability, we adopt a mock data approach for algorithm training, eliminating the necessity for ground truth information on non-cooperative targets. Finally, both simulation and real-world experimental results demonstrate the superior performance of the proposed algorithm. It attains an improvement of at least 15% in tracking accuracy and exhibits a notable enhancement in stability when contrasted with traditional algorithms.

Multi-sensor information fusion localization of rare-earth suspended permanent magnet maglev trains based on adaptive Kalman algorithm.

Since the positioning accuracy of sensors degrades due to noise and environmental interference when a single sensor is used to localize a suspended rare-earth permanent magnetically levitated train, a multi-sensor information fusion method using multiple sensors and self-correcting weighting is proposed for permanent magnetic levitated train localization. A decay memory factor is introduced to reduce the weight of the influence of historical measurement data on the fusion estimation, thus enhancing the robustness of the fusion algorithm. The Kalman filtering results suffer from inaccuracy when process noise is present in the system. In this paper, we use a covariance adaptive scheme that replaces the prediction step of the Kalman filter with covariance. It uses the covariance adaptive scheme to search the posterior sequence online and reconstruct the prior error covariance. Since the process noise covariance is not used in the new adaptive scheme, the negative impact of the mismatch noise statistics is greatly reduced. Simulation and experimental results show that the use of multi-sensor information fusion and covariance adaptive Kalman algorithm has significant advantages in terms of adaptability, accuracy and simplicity.

Research on multi-sensor-based online inspection technology for electric vehicle charging pile transfer type equipment

Abstract This paper performs data fusion based on a multi-sensor fusion algorithm, which measures the sensors and calculates the distance between the data to obtain the measurement weights when participating in the fusion. The sensor observation model is created using an extended Kalman filter, and the nonlinear state transition function is linearized using the covariance matrix. Accurately estimating sensor inter-cluster data is necessary to achieve the scalability of online detection technology for charging piles. The results show that the disconnection time of the contactor of the charging pile transfer type equipment is 1.153s after the simulated charging pile output over-voltage in the disconnection time detection. The online detection efficiency can be improved by using multiple sensors, the method analysis can be intuitive, and the charging service capability of the electric vehicle charging pile can be effectively improved.

Multisensor Fusion SLAM Research Based on Improved RBPF-SLAM Algorithm

Simultaneous localization and map construction (SLAM) technology provides the foundation for indoor robots to realize autonomous path planning. The Rao-Blackwellized particle filtering (RBPF) algorithm is widely used to obtain information and perform map construction in unknown environments. This paper proposes a multisensor fusion algorithm that improves the RBPF-SLAM algorithm by addressing the issues of particle distribution errors and degradation. To achieve this, the extended Kalman filter (EKF) is first adopted to effectively fuse odometry and inertial navigation (inertial measurement unit (IMU)) data as the initial positional information. Then, a high-precision light detection and ranging (LIDAR) observation model is integrated to calculate the proposed distribution, and a threshold is introduced to determine the number of valid particles to simplify the resampling step. Finally, raster map construction experiments were carried out on both the Gazebo simulation environment and Robot Operating System (ROS)-based mobile robot Keepbot platform in different scenarios. The experimental results demonstrate that the optimized RBPF-SLAM algorithm generates a clearer and more complete map outline, which can effectively improve the accuracy of robot pose estimation, acquire a reliable 2D raster map with a reduced number of particles, and greatly reduces the computational effort.

Localization of underground pipe jacking machinery: A reliable, real-time and robust INS/OD solution

Accurate positioning is important for guidance control in pipeline construction, but traditional approaches fail in GPS-denied environments. This paper addresses the underground localization problem for the pipe jacking machine by presenting an Inertial Navigation System (INS)/ODometry (OD) scheme. It is a loosely coupled navigation system that combines an INS and an OD to provide reliable, real-time, and robust location estimation in an enclosed and restricted space. The combination is achieved by a multi-sensor fusion algorithm to avoid the divergence of standalone INS. Compared with the traditional localization technologies, the advantage of the INS/OD scheme is that there is no performance degradation or service failure despite severe blockage of the thick soil layer due to the selection of dead reckoning-based sensors. Moreover, an Unbiased Finite Impulse Response (UFIR) filter serves as the fusion mechanism to improve the robustness against a prior knowledge of noise, considering unknown and time-varying noise statistics in practice. The INS/OD scheme is verified through a numerical simulation and employed on-site in a real underground sewage pipeline construction project. Despite the construction site being more than 10 meters deep underground, the INS/OD scheme continues to perform reliably and robustly in real-time, significantly benefiting the engineering guidance enterprise.

A modified federated Student’s t-based variational adaptive Kalman filter for multi-sensor information fusion

To overcome the problem of poor performance of multi-sensor fusion algorithm in linear systems with time-varying noise and outliers, a modified federated Student’s t-based variational robust adaptive Kalman filter (MFSTVRAKF) is proposed in this article. In this algorithm, an improved Student’s t-based variational robust adaptive Kalman filter (ISTVRAKF) is developed for adaptively mitigating the effects of time-varying noise and outliers of each local sensor, which improves the estimation accuracy of the algorithm. It reorganizes and derives a simplified variational filter to approximate the prior distribution of the measurement noise covariance via the Student’s t distribution to eliminate outliers. Subsequently, an adaptive factor is introduced into ISTVRAKF for adjusting the weights of prediction and measurement to further exclude outliers. Moreover, a modified adaptive information sharing factor is devised to adjust the multi-sensor fusion weights in accordance with the performance of local filter, which improves the fusion accuracy of the MFSTVRAKF algorithm. Simulations exhibit that the proposed algorithm is capable of resisting the effects of time-varying noise and outliers and achieving higher accuracy compared to other filtering algorithms and fusion algorithms.

Adaptive Residual Useful Life Prediction for the Insulated-Gate Bipolar Transistors with Pulse-Width Modulation Based on Multiple Modes and Transfer Learning

Currently, residual useful life (RUL) prediction models for insulated-gate bipolar transistors (IGBT) do not focus on the multi-modal characteristics caused by the pulse-width modulation (PWM). To fill this gap, the Markovian stochastic process is proposed to model the mode transition process, due to the memoryless properties of the grid operation. For the estimation of the mode transition probabilities, transfer learning is utilized between different control signals. With the continuous mode switching, fractional Weibull motion (fWm) of multiple modes is established to model the stochasticity of the multi-modal IGBT degradation. The drift and diffusion coefficients are adaptively updated in the proposed RUL prediction model. In the case study, two sets of the real thermal-accelerated IGBT aging data are used. Different degradation modes are extracted from the meta degradation data, and then fused to be a complex health indicator (CHI) via a multi-sensor fusion algorithm. The RUL prediction model based on the fWm of multiple modes can reach a maximum relative prediction error of 2.96% and a mean relative prediction error of 1.78%. The proposed RUL prediction model with better accuracy can reduce the losses of the power grid caused by the unexpected IGBT failures.

Vision-IMU multi-sensor fusion semantic topological map based on RatSLAM

The simultaneous localization and mapping (SLAM) method based on the brain-space cognitive model can improve the localization accuracy of the robot through memory. The traditional method has poor robustness in complex environments such as intense motion, weak texture, and illumination changes, and lack of understanding of the environment. Our work is based on RatSLAM. To improve the robustness, we propose a multi-sensor fusion algorithm, which integrates a camera with an IMU. To improve the understanding of the environment, we use the Yolo to extract the semantic information of objects and store it in the topological nodes and construct a 2D topology map. Yolo-v4 is used to replace Yolo-v3 to improve the accuracy of semantic tag extraction. We validated our approach in the pubic dataset and real environments. The mean RMSE APE is reduced by 51.716% compared to RatSLAM. Also, our method can build a semantic topological map of the environment.

Neural network-based multisensor image fusion algorithm

Aiming at the problems of high mean square error and low fusion efficiency of existing fusion algorithms, a neural network-based multi-sensor image fusion algorithm is proposed. The fusion algorithm based on depth-separable convolution neural network (CNN) is determined by analyzing the quality evaluation and fusion methods of multi-sensor images, and summarizing the fusion rules. It is found that the integrity of image information acquisition is 97%, the mean square error is 4, and the fusion time is 2 s. Therefore, the algorithm has a good image fusion effect.

State assessment of 110–220 kV intelligent substation based on multisensor fusion algorithm control and image vision

With the development of modern information technology, intelligent substation technology has been widely used, which greatly promotes the development of power grid. The information integration platform of intelligent substations realizes panoramic data collection and data sharing of substations. With the continuous development of the economy and society, it is crucial to improve the power regulation ability of intelligent substations, make scientific use of intelligent technology, and optimize the patrol level of intelligent substation patrol robots to better meet the operation needs of substations. In the actual substation construction, the flexibility of equipment utilization in intelligent substations should be fully considered. In addition, the number of substations should be controlled to improve the utilization efficiency of patrol robots. This paper mainly summarizes the monitoring technology of intelligent substations, the positioning technology of inspection robots, and the multi-sensor control technology to lay a foundation for the follow-up inspection work of intelligent substations.

Fault diagnosis of the hydraulic valve using a novel semi-supervised learning method based on multi-sensor information fusion

Hydraulic systems are usually applied in large and complex engineering fields. For hydraulic systems or components in operation, it is difficult to obtain fault data with fault labels due to the high engineering cost. Therefore, a semi-supervised learning (SSL) method based on multi-sensor information fusion is proposed to obtain valuable pseudo label data to diagnose faults of the hydraulic directional valve in operation. In this method, the classification model is trained from a small amount of data with fault labels, thus generating pseudo labels for a large amounts of unmarked data. The contribution of this article is that a multi-sensor fusion algorithm is designed to obtain pseudo labels with high confidence, and an adaptive threshold model similar to generative countermeasure network is designed to intelligently generate thresholds for selecting pseudo labels instead of human intervention. Theoretical and experimental results show that the multi-sensor information fusion algorithm can obtain high confidence pseudo tags, the adaptive threshold model can screen effective pseudo tag samples by generating appropriate thresholds for accelerating the convergence of the classification model. In the hydraulic valve fault diagnostic test, after five iterations, the average diagnosis accuracy of this method can reach 99.72% and 99.00% respectively for different types of hydraulic valves in different engineering fields. This provides a new idea for developing intelligent hydraulic directional valve with self fault diagnosis function.

Research on site selection of agricultural internet of things nodes based on rapid terrain sampling

With the development of Internet of Things (IoT) technology, modern agriculture is moving in the direction of 4.0. The Agricultural IoT is inseparable from wireless communication. However, in traditional agricultural IoT router and gateway site selection, the influence of the actual terrain on transmission loss is not considered, which results in node power wastage and increased maintenance costs. Based on a multi-sensor fusion algorithm, a fast terrain sampler is designed in this study to collect point-cloud data of the experimental site terrain. A reasonable objective function is then designed under the premise of consideration of the electromagnetic wave free-space and diffraction losses, and the locations of the routers and gateway are optimized based on k-means and particle swarm optimization (PSO) algorithm. Simulations show that the running time of the PSO algorithm is very sensitive to the changes in the execution parameters, and the improved PSO algorithm converges faster than the genetic algorithm (GA) in all three initialization methods. After collecting field terrain data, five interpolation methods were compared, and the nearest-neighbor algorithm is used to obtain the terrain model. On-site collection of received signal-strength indication (RSSI) shows that the communication quality of the optimal point selected via this algorithm is significantly higher than those of nearby points. At the same time, it is proved that the RSSI data has serious discontinuities, so the traditional gradient descent method is not suitable for solving the objective function in this work. Therefore, the algorithm used herein is of great significance for the site selection of agricultural Internet of Things nodes. However, since there are many ways to calculate the diffraction loss, the objective function of this work still needs more correction studies. The tool proposed in this work can obtain a 3D model of farmland faster than traditional surveying and mapping methods. With future research, the 3D model may be applied to soil moisture analysis, rainfall and moisture direction inversion, and plant light exposure prediction.

Dynamic Load Regulation of Robots with Multi-sensor Fusion

Multi-sensor fusion technology has been extensively deployed for the autonomous navigation and control of robots. Theoretically, by increasing the number of sensors, more effective data on the surrounding environment and obstacle information can be obtained. However, in practical applications, a massive number of sensors will not only increase the cost of using sensors in the system but also add to the difficulty and cost of data processing, leading to unnecessary resource consumption. To solve this problem, first, the robot navigation area is divided according to the importance of the navigation area and the characteristics of sensor parameters. The navigation non-core area adopts the single lidar sensor, while the core area adopts the multi-sensor fusion of “lidar +” to reduce the number of sensors used in the fusion and improve the performance of multi-sensor fusion after the division. Then, in consideration of the higher requirement for computing capacity of the device imposed by deep learning, the multi-sensor fusion algorithm is introduced into the dynamic load regulation mechanism to solve the problem from the algorithm level so that the outperformance of deep learning in feature extraction and other aspects can be utilized to reduce the performance requirements of the device and the overall power consumption of the system. Finally, by building the ROS experimental platform, it is verified that the robot can use autonomous navigation in complex and diversified home scenes.

A multisensor fusion algorithm of indoor localization using derivative Euclidean distance and the weighted extended Kalman filter

PurposeAt present, smartphones are embedded with accelerometers, gyroscopes, magnetometers and WiFi sensors. Most researchers have delved into the use of these sensors for localization. However, there are still many problems in reducing fingerprint mismatching and fusing these positioning data. The purpose of this paper is to improve positioning accuracy by reducing fingerprint mismatching and designing a weighted fusion algorithm.Design/methodology/approachFor the problem of magnetic mismatching caused by singularity fingerprint, derivative Euclidean distance uses adjacent fingerprints to eliminate the influence of singularity fingerprint. To improve the positioning accuracy and robustness of the indoor navigation system, a weighted extended Kalman filter uses a weighted factor to fuse multisensor data.FindingsThe scenes of the teaching building, study room and office building are selected to collect data to test the algorithm’s performance. Experiments show that the average positioning accuracies of the teaching building, study room and office building are 1.41 m, 1.17 m, and 1.77 m, respectively.Originality/valueThe algorithm proposed in this paper effectively reduces fingerprint mismatching and improve positioning accuracy by adding a weighted factor. It provides a feasible solution for indoor positioning.

Reliability Analysis of Multisensor Fusion Technology in Disaster Nursing Application

The harm and impact of various natural disasters are becoming more and more obvious. Due to the high suddenness of natural disasters and the low efficiency of information sharing, our stress system will still not be able to respond to disasters in time. Therefore, facing the current complex natural environment and the diversity of disasters, we urgently need a practical and comprehensive online collaborative natural disaster protection system. Therefore, this paper proposes a disaster protection application system based on a multisensor information fusion algorithm. Firstly, the software platform of the natural disaster protection system based on multisensor is constructed; the hardware platform of multisensor disaster acquisition is designed. An information fusion algorithm based on Kalman joint filter is proposed. The main problems of joint filter are (1) improving the filtering accuracy, (2) improving the fault-tolerant performance of the filter, and (3) optimizing the fusion algorithm and reducing the amount of calculation and data communication, so as to facilitate real-time implementation of the algorithm. Combined with DS evidence theory, BP neural network, and support vector mechanism, a comprehensive diagnosis model is established. The model uses the preliminary diagnosis results of the BP neural network and support vector machine to carry out the BPA of DS evidence theory, which avoids the blindness caused by human operation. The simulation results show that the comprehensive diagnosis model can significantly improve the prediction accuracy of disasters, solve the uncertainty and fuzziness of diagnosis results of a single diagnosis model, and provide a new method for disaster protection. The results of the BP neural network in the disaster nursing system are fused at the decision level, the results show that the distribution probability corresponding to the fusion disaster category is greater than the threshold value of 0.8, and the diagnosis result can be given immediately. The diagnosis result is completely consistent with the actual disaster category.

Multilayer Low-Cost Sensor Local-Global Filtering Fusion Integrated Navigation of Small UAV

Aimed at improving the nonlinear integrated navigation solution performance of multiple low-cost sensors fusion, this paper presents a multilayer loosely-coupled, local-global, and step-optimized MF5DCKF (Multisensor Federated fifth-degree Cubature Kalman filter) state estimation algorithm for the small unmanned aerial vehicle (UAV). This method establishes a multilayer nonlinear integrated navigation model composed of the nonlinear attitude and heading reference system (AHRS) error model, strapdown inertial navigation system/global positioning system (SINS/GPS) error model, and strapdown inertial navigation system/barometer (SINS/BARO) error model to enhance the robustness and richness of the navigation module. Further, based on the above navigation models, a loosely-coupled error state fusion frame is designed to obtain the local convergent state vector. Simultaneously, a three-layer fifth-degree Cubature Kalman filter is proposed to improve the local state estimation accuracy. Subsequently, to optimize the estimated local state, this paper presents a novel distributed MF5DCKF scheme fusing the local state vector to calculate the global optimal state parameters in a step-optimized process. The experimental flight test results show that the proposed algorithm achieves a higher state solution accuracy and a better convergent performance compared with some conventional multisensor fusion algorithms. The new algorithm framework can provide applicability and reliability for the small UAV during the flight.

Judgment method of grasping stability for dexterous hand based on force balance theorem and Monte Carlo method

PurposeMulti-sensor fusion in robotic dexterous hands is a hot research field. However, there is little research on multi-sensor fusion rules. This study aims to introduce a multi-sensor fusion algorithm using a motor force sensor, film pressure sensor, temperature sensor and angle sensor, which can form a consistent interpretation of grasp stability by sensor fusion without multi-dimensional force/torque sensors.Design/methodology/approachThis algorithm is based on the three-finger force balance theorem, which provides a judgment method for the unknown force direction. Moreover, the Monte Carlo method calculates the grasping ability and judges the grasping stability under a certain confidence interval using probability and statistics. Based on three fingers, the situation of four- and five-fingered dexterous hand has been expanded. Moreover, an experimental platform was built using dexterous hands, and a grasping experiment was conducted to confirm the proposed algorithm. The grasping experiment uses three fingers and five fingers to grasp different objects, use the introduced method to judge the grasping stability and calculate the accuracy of the judgment according to the actual grasping situation.FindingsThe multi-sensor fusion algorithms are universal and can perform multi-sensor fusion for multi-finger rigid, flexible and rigid-soft coupled dexterous hands. The three-finger balance theorem and Monte Carlo method can better replace the discrimination method using multi-dimensional force/torque sensors.Originality/valueA new multi-sensor fusion algorithm is proposed and verified. According to the experiments, the accuracy of grasping judgment is more than 85%, which proves that the method is feasible.

Real-Time Localization and Mapping Utilizing Multi-Sensor Fusion and Visual–IMU–Wheel Odometry for Agricultural Robots in Unstructured, Dynamic and GPS-Denied Greenhouse Environments

Autonomous navigation in greenhouses requires agricultural robots to localize and generate a globally consistent map of surroundings in real-time. However, accurate and robust localization and mapping are still challenging for agricultural robots due to the unstructured, dynamic and GPS-denied environmental conditions. In this study, a state-of-the-art real-time localization and mapping system was presented to achieve precise pose estimation and dense three-dimensional (3D) point cloud mapping in complex greenhouses by utilizing multi-sensor fusion and Visual–IMU–Wheel odometry. In this method, measurements from wheel odometry, an inertial measurement unit (IMU) and a tightly coupled visual–inertial odometry (VIO) are integrated into a loosely coupled framework based on the Extended Kalman Filter (EKF) to obtain a more accurate state estimation of the robot. In the multi-sensor fusion algorithm, the pose estimations from the wheel odometry and IMU are treated as predictions and the localization results from VIO are used as observations to update the state vector. Simultaneously, the dense 3D map of the greenhouse is reconstructed in real-time by employing the modified ORB-SLAM2. The performance of the proposed system was evaluated in modern standard solar greenhouses with harsh environmental conditions. Taking advantage of measurements from individual sensors, our method is robust enough to cope with various challenges, as shown by extensive experiments conducted in the greenhouses and outdoor campus environment. Additionally, the results show that our proposed framework can improve the localization accuracy of the visual–inertial odometry, demonstrating the satisfactory capability of the proposed approach and highlighting its promising applications in autonomous navigation of agricultural robots.

Research on risk assessment method of steel catenary riser based on information fusion

The Steel Catenary Riser (SCR) is an essential component connecting the subsea production system and the offshore floating platform and SCR malfunction leads to disastrous consequences. In order to ensure operation safety, the operating company monitors SCR motion response in real time by installing sensors. This paper aims to develop a multi-sensor information fusion algorithm for SCR risk assessment. The algorithm generates basic probability assignment (BPA) function based on known information. Then, the intuitionistic fuzzy multi-attribute decision matrix is constructed by BPA to evaluate the dynamic reliability of the sensor. Finally, the focal element credibility is calculated to reduce the conflict between evidences, and the improved Desert-Smarandache (DSm) theory fusion rule is utilized to evaluate the SCR risk through decision fusion. The feasibility of this method is verified by the case study. The corresponding risk control strategy is also provided.