Multi-sensor Data Fusion System Research Articles

Predicting aircraft maintenance timetofailure using the interval gradient regression tree algorithm

The operation and maintenance of modern aircraft multi-sensor data fusion systems generate vast amounts of numerical and symbolic data. Learning useful and non-trivial insights from this data may lead to considerable savings, and detection and reduction of the number of faults, as result increasing the overall level of aircraft safety. Several machine learning techniquesexist to learn from big amounts of data. However, the use of thesetechniques to infer the desired readable and accurate interval regression tree models from the data obtained during theoperation and maintenance of aircraft is extremely challenging. Difficulties that need to be addressed include: data warehouse collection and preprocessing, data labeling, machine learning model readability, setup, evaluation and maintenance. This paper presents the Interval Gradient Prediction Tree algorithm INGPRET, which addresses these issues. As shown by our empirical evaluation of a real aircraft multi-sensor data set, the INGPRET algorithm provides better readability and similar performance in comparison to other regression tree machine learning algorithms.

Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS.

The perception module plays an important role in vehicles equipped with advanced driver-assistance systems (ADAS). This paper presents a multi-sensor data fusion system based on the polarization color stereo camera and the forward-looking light detection and ranging (LiDAR), which achieves the multiple target detection, recognition, and data fusion. The You Only Look Once v4 (YOLOv4) network is utilized to achieve object detection and recognition on the color images. The depth images are obtained from the rectified left and right images based on the principle of the epipolar constraints, then the obstacles are detected from the depth images using the MeanShift algorithm. The pixel-level polarization images are extracted from the raw polarization-grey images, then the water hazards are detected successfully. The PointPillars network is employed to detect the objects from the point cloud. The calibration and synchronization between the sensors are accomplished. The experiment results show that the data fusion enriches the detection results, provides high-dimensional perceptual information and extends the effective detection range. Meanwhile, the detection results are stable under diverse range and illumination conditions.

Multisensor Data Fusion System for Wushu Sanda Teaching in Higher Education Institutions

Classical data fusion methods can be roughly divided into two categories: one is based on random methods, such as weighted average, least squares, Kalman filtering, Bayesian estimation, DS evidence inference, and production rules; the other is based on artificial intelligence. Intelligent methods include fuzzy logic reasoning, artificial neural networks, and support vector machine methods. This paper introduces, analyzes, and summarizes the advantages and disadvantages of the above methods, as well as the level of use of these methods, and then introduces the current trend of data fusion technology toward the mixed development of multiple methods. This article is aimed at carrying out a research on the innovation of Wushu Sanda teaching mode in colleges and universities based on multisensor data fusion technology. This article puts forward the innovative theory of youth sports training in Sanda in the new era, combined with the current status of physical education in Chinese colleges and universities, and analyzes the practicality of the theory and the effect of system integration. This article accelerates reforms and collects data from four aspects: the policy system at the macro level, the integrated governance system at the meso level, the teacher service system at the micro level, and the digital industry system at the functional innovation level, and proposes innovative strategies for youth sports training and teaching in the new era. The results of the study show that people with 5.99 are more interested in the World Cup, accounting for the largest proportion, while the number of people who are interested in martial arts is the least, only 1.63. The innovative theory of youth sports training in Sanda in the new era improves the training effect of students by 60% compared with ordinary college Sanda training. It can to a large extent protect students from reluctant exercises that do not receive physical education in traditional teaching. Physiologically, it has unique advantages for the development of students. The teaching innovation model of this article can effectively solve the problems of physical education organization difficulty.

Exploring the role of sports APP in (campus fitness) intelligent solutions using data fusion algorithm and internet of things

The purposes are to study how the multi-sensor Internet of Things (IoT) data fusion algorithm calculates in data fusion systems and improves the systems' fusion efficiency. An improved Weighted Least-Squares (WLS) algorithm is proposed for IoT data fusion, and how it processes massive data in a multi-sensor system is studied. Accordingly, a multi-sensor system for sports fitness is designed based on a data fusion algorithm. First, the purposes and demand for sports Application (APP) are analysed, to understand the problems and necessary improvements of such APP. Then, the implementation of the IoT and the classification of the data fusion algorithm of the IoT are explored. The WLS method is selected through comparison, and its implementation process and the data processing process are analysed and explained. Finally, the sports fitness system based on the IoT data fusion algorithm is designed and analysed. The results show that the wireless communication of a multi-sensor data fusion system is feasible and reliable. The variances calculated through the WLS method and the Arithmetic Mean (AM) method in data fusion are compared. The former value is about one-thousandth of the latter value, indicating that the data fusion based on WLS is advantageous over the traditional data fusion.

Flatfish Measurement Performance Improvement Based on Multi-sensor Data Fusion

In this study, a multi-sensor data fusion system using a load cell and vision sensor was considered in the development of a flatfish classifier for systematic fish management in aquaculture. In the single-sensor measurement method, each sensor has disadvantages. A load cell shows high performance in the measurement of adult fish, but the measurement of fry is affected significantly due to water weight (water weight disturbance). A vision sensor shows high performance in the measurement of fry, but the movement of fish (movement disturbance) affects the accurate measurement of adult fish. Therefore, in this study, these disturbances were compensated for using a datafusion algorithm, of which the performance was evaluated by a comparison between single sensor measurements and multi-sensor data fusion results.

Enhanced mapping of a smallholder crop farming landscape through image fusion and model stacking

ABSTRACT Globally, Smallholder farming systems (SFS) are recognized as one of the most important pillars of rural economic development and poverty alleviation because of their contribution to food security. However, support for this agricultural sector is hampered by lack of reliable information on the distributions and acreage of smallholder fields. This information is essential in not only monitoring food security and informing markets but also in guiding the determination of levels of support required from government by individual farmers. There is urgent need for robust techniques that can be used to cost-effectively and time-efficiently map smallholder crop fields especially in Sub-Saharan Africa and Asia. This study attempts to do this by using an approach in which optical and Synthetic Aperture Radar (SAR) data are systematically combined and classified using Extreme Gradient Boosting (Xgboost). We also investigated model stacking as another technique to improve classification accuracy. We combined Xgboost with Random Forest (RF), Support Vector Machine (SVM), Artificial Neural Networks (ANN), and Naïve Bayes (NB). The combined use of multi-temporal Sentinel-2 bands, spectral indices, and Sentinel-1 produced better results than exclusive use of optical data (α = 0.95, p = 0.0005). Furthermore, stacking of classification algorithms based on model comparisons achieved higher accuracy than stacking the algorithms indiscriminately (α = 0.95, p = 0.0100). Through systematic fusion of SAR and optical data and hyper-parameter tuning of Xgboost, we achieved a maximum classification accuracy of 97.71%, while achieving a maximum accuracy of 96.06% through model stacking. This highlights the importance of multi-sensor data fusion and multi-classifier systems when mapping fragmented agricultural landscapes.

Multi-Sensor Data Fusion for Remaining Useful Life Prediction of Machining Tools by IABC-BPNN in Dry Milling Operations.

Inefficient remaining useful life (RUL) estimation may cause unpredictable failures and unscheduled maintenance of machining tools. Multi-sensor data fusion will improve the RUL prediction reliability by fusing more sensor information related to the machining process of tools. In this paper, a multi-sensor data fusion system for online RUL prediction of machining tools is proposed. The system integrates multi-sensor signal collection, signal preprocess by a complementary ensemble empirical mode decomposition, feature extraction in time domain, frequency domain and time-frequency domain by such methods as statistical analysis, power spectrum density analysis and Hilbert-Huang transform, feature selection by a Light Gradient Boosting Machine method, feature fusion by a tool wear prediction model based on back propagation neural network optimized by improved artificial bee colony (IABC-BPNN) algorithm, and the online RUL prediction model by a polynomial curve fitting method. An example is used to verify whether if the prediction performance of the proposed system is stable and reliable, and the results show that it is superior to its rivals.

Real-Time Onboard 3D State Estimation of an Unmanned Aerial Vehicle in Multi-Environments Using Multi-Sensor Data Fusion.

The question of how to estimate the state of an unmanned aerial vehicle (UAV) in real time in multi-environments remains a challenge. Although the global navigation satellite system (GNSS) has been widely applied, drones cannot perform position estimation when a GNSS signal is not available or the GNSS is disturbed. In this paper, the problem of state estimation in multi-environments is solved by employing an Extended Kalman Filter (EKF) algorithm to fuse the data from multiple heterogeneous sensors (MHS), including an inertial measurement unit (IMU), a magnetometer, a barometer, a GNSS receiver, an optical flow sensor (OFS), Light Detection and Ranging (LiDAR), and an RGB-D camera. Finally, the robustness and effectiveness of the multi-sensor data fusion system based on the EKF algorithm are verified by field flights in unstructured, indoor, outdoor, and indoor and outdoor transition scenarios.

Multi-Sensor Data Fusion for Real-Time Surface Quality Control in Automated Machining Systems.

Multi-sensor data fusion systems entail the optimization of a wide range of parameters related to the selection of sensors, signal feature extraction methods, and predictive modeling techniques. The monitoring of automated machining systems enables the intelligent supervision of the production process by detecting malfunctions, and providing real-time information for continuous process optimization, and production line decision-making. Monitoring technologies are essential for the reduction of production times and costs, and an improvement in product quality, discarding the need for post-process quality controls. In this paper, a multi-sensor data fusion system for the real-time surface quality control based on cutting force, vibration, and acoustic emission signals was assessed. A total of four signal processing methods were analyzed: time direct analysis (TDA), power spectral density (PSD), singular spectrum analysis (SSA), and wavelet packet transform (WPT). Owing to the nonlinear and stochastic nature of the process, two predictive modeling techniques, multiple regression and artificial neural networks, were evaluated to correlate signal parametric characterization with surface quality. The results showed a high correlation of surface finish with cutting force and vibration signals. The signal processing methods based on signal decomposition in a combined time and frequency domain (SSA and WPT) exhibited better signal feature extraction, detecting excitation frequency ranges correlated to surface finish. The artificial neural network model obtained the highest predictive power, with better behavior for the whole data range. The proposed on-line multi-sensor data fusion provided significant improvements for in-process quality control, with excellent predictive power, reliability, and response times.

Evaluating the Reliability Coefficient of a Sensor Based on the Training Data Within the Framework of Evidence Theory

When a multi-sensor data fusion system is used to handle a classification problem, it is necessary to incorporate the reliability coefficients of all the sensors into the fusion process. Within the framework of evidence theory, this paper proposes a new method for evaluating the reliability coefficient of a sensor based on the training data. In this method, the distance between power-set-distribution betting commitments is used to quantify the dissimilarity between the sensor reading and the reality, which can be served as a one-sided discounting factor. Then, the optimization approach is put forward to obtain an all-sided discounting factor from plenty of one-sided discounting factors. The advantages of the proposed method are analyzed comparatively. Numerical examples are also presented to demonstrate its performance by comparing it with other supervised evaluation methods.

The Research of Multi-sensor Data Fusion Technology

Multi-source information from multi-sensors is very important to command information system. Processed multi-source information can offer useful decision information to commander. On the basis of explaining concept, function and advantage of multi-sensor data fusion, the common model of the data fusion was introduced. Multi-sensor data fusion technologies and system structure of multi-sensor data fusion were analyzed.

Effective World Modeling: Multisensor Data Fusion Methodology for Automated Driving.

The number of perception sensors on automated vehicles increases due to the increasing number of advanced driver assistance system functions and their increasing complexity. Furthermore, fail-safe systems require redundancy, thereby increasing the number of sensors even further. A one-size-fits-all multisensor data fusion architecture is not realistic due to the enormous diversity in vehicles, sensors and applications. As an alternative, this work presents a methodology that can be used to effectively come up with an implementation to build a consistent model of a vehicle’s surroundings. The methodology is accompanied by a software architecture. This combination minimizes the effort required to update the multisensor data fusion system whenever sensors or applications are added or replaced. A series of real-world experiments involving different sensors and algorithms demonstrates the methodology and the software architecture.

An Application Case Study on Multi-sensor Data fusion System for Intelligent Process Monitoring

Multi-sensor data fusion is a technology to enable combining information from several sources in order to form a unified picture. Focusing on the indirect method, an attempt was made to build up a multi-sensor data fusion system to monitor the condition of grinding wheels with force signals and the acoustic emission (AE) signals. An artificial immune algorithm based multi-signals processing method was presented in this paper. The intelligent monitoring system is capable of incremental supervised learning of grinding conditions and quickly pattern recognition, and can continually improve the monitoring precision. The application case indicates that the accuracy of condition identification is about 87%, and able to meet the industrial need on the whole.

Distributed, multi-user, multi-application, and multi-sensor data fusion over named data networks

Named Data Networking (NDN) routes data based on their application-layer content names enabling location independence, in-network caching, and enhanced security. A proof-of-concept solution is presented that demonstrates the applicability of NDN for multi-user, multi-application, and multi-sensor data-fusion systems. The system consists of a collaborative network of weather radars name data based on their geographic location and weather feature (e.g., reflectivity of clouds and wind velocity). This enables end users to specify an area of interest for a particular weather feature while being oblivious to the placement of radars and associated computing facilities. Conversely, the data-fusion system can also use its knowledge about the underlying system to decide the best sensing and data processing strategies. Such sensor-independent names also enhance resilience, enable processing data close to the source, and benefit from NDN features such as in-network caching and duplicate query suppression, consequently reducing the bandwidth requirements of the entire data-fusion system. The solution is implemented as an overlaid NDN enabling the benefits of both the NDN and overlay networks. Simulation-based analysis using reflectivity data from an actual weather event showed 84% reduction in peak bandwidth consumption of radars and 95% reduction in peak query resolution latency.

Robust sensor bias estimation for ill-conditioned scenarios

Sensor bias estimation is an inherent problem in multi-sensor data fusion systems. Classical methods such as the Generalized Least Squares (GLS) method can have numerical problems with ill-conditioned sets which are common in practical applications. This paper describes an azimuth-GLS method that provides a solution to the ill-conditioning problem while maintaining reasonable accuracy compared with the classical GLS method. The mean square error is given for both methods as a criterion to determine when to use this azimuth-GLS method. Furthermore, the separation boundary between the azimuth-GLS favorable region and that of the GLS method is explicitly plotted. Extensive simulations show that the azimuth-GLS approach is preferable in most scenarios.

A new nearest-neighbor association approach based on fuzzy clustering

The main components of tracking in multisensor data fusion systems are estimation and data association. This paper proposes a new nearest-neighbor fuzzy association approach for multitarget tracking in a cluttered environment. In the proposed approach, fuzzy clustering is used to generate a likelihood measure in place of the traditional Mahalanobis distance. First, measurements-to-tracks associations are computed jointly across all targets and all validated measurements using fuzzy clustering, then nearest-neighbor correlation is performed based on fuzzy correlation weights. For a given target, the validated measurement that has the maximum fuzzy correlation weight is used for updating the state of the target. The proposed approach determines the association between the measurements and the tracks based on a single correlation matrix, thus it highly reduces the computational complexity compared to the joint probabilistic data association filter and the conventional fuzzy logic data association approaches. The performance of the proposed approach is evaluated using Monte Carlo simulations and compared to that of the nearest-neighbor association with Mahalanobis distance, conventional fuzzy logic data association approaches, and joint probabilistic data association filter. The results show that the proposed approach achieves better performance compared to the nearest-neighbor association with Mahalanobis distance and the conventional fuzzy logic data association approaches. The results also show that the performance of the proposed approach is not far from the performance of the joint probabilistic data association filter. Suggested steps for the proposed approach to be utilized to counter against Electronic Counter Measures techniques are also presented.

The Design of Component Level Software Architecture for Multi-Sensor Data Fusion System

Through investigating the architecture of multi-sensor data fusion system, a universal software architecture which can be used in distributed multi-radar fusion systems is proposed in this paper. In this architecture, functionally independent modules will be extracted from system, it will be standardized as components by object-oriented techniques, and the interface specifications of all the modules are also specified. Aiming at the different objects, system flow and modules can be framed quickly by selecting or expanding proper components based on different scenes feature and object properties. The architecture improves software reusability and scalability, and is propitious to increase software development efficiency.

On the sequential track correlation algorithm in a multisensor data fusion system

In this paper, sequential track correlation algorithm in a multisensor data fusion system is presented. It is well known that the state estimates obtained from a Kalman filter have correlated errors in time. While the innovations are white, this does not carry over to the state estimation errors. It should also be pointed out that the use of a sliding window for track-to-track association with the (appropriate) caveat that the distribution of the sum of chi-square variables over the window is only approximately chi-square distributed.

Composite filtering with feedback information

Abstract The optimal fusion solution with feedback information for a hybrid multisensor data fusion system is presented. In this system, a part of sensors process their data locally to produce local tracks, and another part of sensors only provide detection reports These tracks and detection reports are communicated to a central site where track fusion and composite filtering are performed. The comparative results on the simulations suggest the feedback information from the center can greatly improve the tracking performance of the local node.

New track correlation algorithms in a multisensor data fusion system

In order to resolve the problem of track-to-track association in a distributed multisensor situation, this paper presents independent and dependent sequential track correlation algorithms based on Singer's and Bar-Shalom's algorithms. Based on sequential track correlation algorithm, the restricted and attenuation memory track correlation algorithms and sequential classic assignment rules are proposed. In this paper, these algorithms are described in detail. Then, the track correlation mass and multivalency processing methods are discussed as well. Finally, simulations are designed to compare the correlation performance of these algorithms with that of Singer's and Bar-Shalom's algorithms. The simulation results show that the performance of these algorithms proposed here is much better than that of the classical methods under the environments of dense targets, interfering, noise, track cross, and so on. Under the above situations, their correct correlation ratio is improved about 69 percent over the classical methods