Multi-source Fusion Research Articles

Real-time crash risk prediction in freeway tunnels considering features interaction and unobserved heterogeneity: A two-stage deep learning modeling framework

Real-time prediction of crash risk is an effective method for enhancing traffic safety, but it is not fully explored in freeway tunnels. A two-stage deep learning modeling framework comprising a preliminary exploration stage and a prediction and analysis stage is proposed for real-time crash risk prediction in freeway tunnels. A random parameters logit model with heterogeneity in means and variances is used in the preliminary exploration stage to investigate the unobserved heterogeneity and influence mechanism of precursors on real-time crash risk. In the prediction and analysis stage, a random deep and cross network model considering feature interactions and unobserved heterogeneities is developed to predict and analyze real-time crash risk, which is interpreted by the shapley additive explanations approach. The multi-source fusion dataset, collected from the Caltrans performance measurement system and the weather information website, is used to validate the proposed framework for exploring real-time crash risk in freeway tunnels. Results reveal that: (1) the random parameters logit model with heterogeneity in means and variances outperforms the traditional logit model in terms of the model fitting, providing a reference for deep learning modeling that may be able to improve model performance by addressing heterogeneity; (2) the important crash precursors such as the average difference in speed between detectors of tunnel entrance and exit are discovered based on the marginal effect analysis of the random parameters logit model with heterogeneity in means and variances; (3) the random deep and cross network model yields the best prediction performance compared to its counterparts (some other data-driven models), demonstrating the superior performance of deep learning models for real-time risk prediction tasks. It also indicates that considering feature interaction and heterogeneity in deep learning modeling can improve prediction performance; and (4) the important precursors found in the random deep and cross network model using the shapley additive explanations approach are close to those discovered in the statistical model, indicating that the proposed deep learning model can capture the similar effects of precursors as the statistical models, and the precursor interactions and heterogeneities also can be observed by the shapley additive explanations approach.

Safety Evaluation of Reinforced Concrete Structures Using Multi-Source Fusion Uncertainty Cloud Inference and Experimental Study.

Structural damage detection and safety evaluations have emerged as a core driving force in structural health monitoring (SHM). Focusing on the multi-source monitoring data in sensing systems and the uncertainty caused by initial defects and monitoring errors, in this study, we develop a comprehensive method for evaluating structural safety, named multi-source fusion uncertainty cloud inference (MFUCI), that focuses on characterizing the relationship between condition indexes and structural performance in order to quantify the structural health status. Firstly, based on cloud theory, the cloud numerical characteristics of the condition index cloud drops are used to establish the qualitative rule base. Next, the proposed multi-source fusion generator yields a multi-source joint certainty degree, which is then transformed into cloud drops with certainty degree information. Lastly, a quantitative structural health evaluation is performed through precision processing. This study focuses on the numerical simulation of an RC frame at the structural level and an RC T-beam damage test at the component level, based on the stiffness degradation process. The results show that the proposed method is effective at evaluating the health of components and structures in a quantitative manner. It demonstrates reliability and robustness by incorporating uncertainty information through noise immunity and cross-domain inference, outperforming baseline models such as Bayesian neural network (BNN) in uncertainty estimations and LSTM in point estimations.

Research on localization and navigation methods for intelligent terminal devices oriented to multi-source fusion technology

Abstract In this paper, GPS, WLAN, and INS are used as positioning information data sources, and multi-source fusion technology is used to build an intelligent positioning navigation device. The feature vectors of the original data are observed using the federal Kalman filter algorithm, and the information is continuously updated to assign weights to achieve the optimization of the positioning information. The three-dimensional information of the defined position is measured using the CNN algorithm, the faulty data in the filter is detected and removed, and the detected data enters the main filter instead of the faulty data for fusion. To prove the effectiveness of the device, experimental analysis is performed, and the results show that the device provides diverse, intelligent services with up to 95% localization accuracy and stronger signals.

SBAS InSAR Subsidence Monitoring for Mining Areas Based on Levelling Constraints

SBAS-InSAR technology has been widely researched and applied in the large-scale monitoring of mining areas, but due to the influence of spatial and temporal discorrelation, atmospheric delay and noise on the deformation results, as well as the requirements on image quantity, quality and spatial continuity, the SBAS-InSAR technology inferior to the levelling monitoring results in terms of measurement accuracy and reliability. Taking the Yuncheng coal mine as the research area, and after using SBAS InSAR to monitor the subsidence, the levelling monitoring data was used as the constraint to construct the subsidence error surface by polynomial surface fitting, and then the SBAS-InSAR subsidence monitoring results were be corrected. The experimental results show that the settlement difference is better than ±50mm/a, which effectively improves the overall accuracy of SBAS-InSAR settlement monitoring. The experimental results verify the feasibility and effectiveness of using the level monitoring results to correct the deformation monitoring results of SBAS InSAR, and provide effective reference information for improving the accuracy of the deformation monitoring results by multi-source fusion processing.

A reliability assessment methodology of system-in-package based virtual qualification

System-in-Package (SiP) is a cutting-edge packaging technique that is flourishing in the semiconductor sector. Physical testing, which can be time-consuming and lead to design modification iterations delaying product release or delivery, has traditionally been used to assess the reliability of SiPs. Simulation methods that mimic physical tests can be used to shorten product development cycles, increase reliability, and lower development costs. This paper introduces a SiP reliability assessment methodology based on virtual qualification. A life prediction methodology of SiP based on physics-of-failure (PoF) is studied in conjunction with multi-source fusion life information of typical structures. Based on this, a comprehensive reliability assessment methodology of SiP is formed. The information input includes material types, structure size, and environmental load conditions. The finite element analysis (FEA) is used to analyze the typical structures. The life distribution of SiP components is obtained through the theory of the cumulative damage and competitive failure model. Through multi-source information fusion, the SiP comprehensive reliability assessment methodology is determined. Finally, a case study is done to determine the accuracy of the approach.

A multi-source information fusion layer counting method for penetration fuze based on TCN-LSTM

When employing penetration ammunition to strike multi-story buildings, the detection methods using acceleration sensors suffer from signal aliasing, while magnetic detection methods are susceptible to interference from ferromagnetic materials, thereby posing challenges in accurately determining the number of layers. To address this issue, this research proposes a layer counting method for penetration fuze that incorporates multi-source information fusion, utilizing both the temporal convolutional network (TCN) and the long short-term memory (LSTM) recurrent network. By leveraging the strengths of these two network structures, the method extracts temporal and high-dimensional features from the multi-source physical field during the penetration process, establishing a relationship between the multi-source physical field and the distance between the fuze and the target plate. A simulation model is developed to simulate the overload and magnetic field of a projectile penetrating multiple layers of target plates, capturing the multi-source physical field signals and their patterns during the penetration process. The analysis reveals that the proposed multi-source fusion layer counting method reduces errors by 60% and 50% compared to single overload layer counting and single magnetic anomaly signal layer counting, respectively. The model's predictive performance is evaluated under various operating conditions, including different ratios of added noise to random sample positions, penetration speeds, and spacing between target plates. The maximum errors in fuze penetration time predicted by the three modes are 0.08 ms, 0.12 ms, and 0.16 ms, respectively, confirming the robustness of the proposed model. Moreover, the model's predictions indicate that the fitting degree for large interlayer spacings is superior to that for small interlayer spacings due to the influence of stress waves.

Intelligent control system for industrial robots based on multi-source data fusion

Abstract Industrialization has advanced quickly, bringing intelligent production and manufacturing into people’s daily lives, but it has also created a number of issues with the ability of intelligent control systems for industrial robots. As a result, a study has been conducted on the use of multi-source data fusion methods in the mechanical industry. First, the research analyzes and discusses the existing research at home and abroad. Then, a robot intelligent control system based on multi-source fusion method is proposed, which combines multi-source data fusion with principal component analysis to better fuse data of multiple control periods; In the process, the experimental results are dynamically evaluated, and the performance of the proposed method is compared with other fusion methods. The results of the study showed that the confidence values and recognition correctness of the intelligent control system under the proposed method were superior compared to the Yu, Murphy, and Deng methods. Applying the method to the comparison of real-time and historical data values, it is found that the predicted data under the proposed method fits better with the actual data values, and the fit can be as high as 0.9945. The dynamic evaluation analysis of single and multi-factor in the simulation stage demonstrates that the control ability in the training samples of 0–100 is often better than the actual results, and the best evaluation results may be obtained at the sample size of 50 per batch. The aforementioned findings demonstrated that the multi-data fusion method that was suggested had a high degree of viability and accuracy for the intelligent control system of industrial robots and could offer a fresh line of enquiry for the advancement and development of the mechanical industrialization field.

A graph optimization approach to range-based relative location

With the development of multi-robot cluster system, the research of multi-node relative pose estimation is in the ascendant. However, for the environment without GNSS or fixed anchors, the problem of relative pose estimation is still challenging. In this paper, a general algorithm and system framework based on graph optimization algorithm is proposed to fuse inter- and intra-robot measurements, as well as mobility models with sliding windows for relative attitude estimation. In order to verify the feasibility and universality of the algorithm, the algorithm has used to design a relative positioning system based on UWB positioning with a “single-node multi-tags” system structure, which can simultaneously obtain reliable relative positions and directions. The design of multi-source fusion can further improve the accuracy and robustness of relative positioning. Experiment results shows that in the harsh environment without GNSS and fixed anchor points, the relative pose estimation accuracy of this system can reach centi-meter level, which may be the best potential choice for relative positioning of multi robot cluster in complex environment.

Multi-level autonomous integrity monitoring method for multi-source PNT resilient fusion navigation

For the integrity monitoring of a multi-source PNT (Positioning, Navigation, and Timing) resilient fusion navigation system, a theoretical framework of multi-level autonomous integrity monitoring is proposed. According to the mode of multi-source fusion navigation, the framework adopts the top-down logic structure and establishes the navigation source fault detection model based on the multi-combination separation residual method to detect and isolate the fault source at the system level and subsystem level. For isolated non-redundant navigation sources, the system level recovery verification model is used. For the isolated multi-redundant navigation sources, the sensor fault detection model optimized with the dimension-expanding matrix is used to detect and isolate the fault sensors, and the isolated fault sensors are verified in real-time. Finally, according to the fault detection and verification results at each level, the observed information in the fusion navigation solution is dynamically adjusted. On this basis, the integrity risk dynamic monitoring tree is established to calculate the Protection Level (PL) and evaluate the integrity of the multi-source integrated navigation system. The autonomous integrity monitoring method proposed in this paper is tested using a multi-source navigation system integrated with Inertial Navigation System (INS), Global Navigation Satellite System (GNSS), Long Baseline Location (LBL), and Ultra Short Baseline Location (USBL). The test results show that the proposed method can effectively isolate the fault source within 5 s, and can quickly detect multiple faulty sensors, ensuring that the positioning accuracy of the fusion navigation system is within 5 m, effectively improving the resilience and reliability of the multi-source fusion navigation system.

Research on the Mobile Robot Map-Building Algorithm Based on Multi-Source Fusion

In this paper, the mobile robot position fusion algorithm is inaccurate. There is a delay, and the map-construction accuracy is not high; an improvement method is proposed. First, the Cartographer algorithm is optimized. Radius filtering is used for data processing after voxel filtering. In contrast, the idea of multi-sensor fusion is used to fuse the processed IMU data information. This improved method improves the efficiency of the algorithm and the accuracy of the positional pose fusion. We verify the effect of the algorithm applied to the environment map, respectively, in the experimental building promenade environment and the teaching building hall environment, and analyze and compare the effect of map construction before and after the improvement; the experiment proves that in the experimental building promenade environment, the absolute error of measuring and analyzing the obstacles reduces by 0.06 m, and the relative error decreases by 1.63%; in the teaching building hall environment, the absolute error of measuring and analyzing the longest side of the map decreases by 1.121 m and the relative error decreased by 5.52%. In addition, during the experimental operation, the CPU occupancy of the optimized algorithm is around 59.5%. In contrast, the CPU occupancy of the original algorithm is 67% on average, and sometimes it will soar to 75%. The experimental results prove that the algorithm in this paper significantly improves performance in all aspects when constructing real-time environment maps.

Variational Auto-Encoder Reconstruction Networks for Classification of Hyperspectral and LiDAR Data

The classification of remote sensing scene objects has been the subject of extensive studies in recent years due to the quick advancement of earth observation and remote sensing technology. More concentration, Hyperspectral images, and LiDAR data are complementary, and their combined use for data fusion can better mine the multi-dimensional features of ground objects in remote sensing scenes, which can effectively improve the classification accuracy and reliability of ground objects in remote sensing scenes. Single-modal remote sensing data frequently cannot fully meet the needs of ground feature classification due to the increasingly complex types of ground features. In order to solve this problem, we have developed two distinct multi-source fusion classification approaches using LiDAR and hyperspectral data and deep learning techniques. They are the reconstructed multi-layer perceptron network based on the variational auto-Encoder (encoding-decoding form) and the two-stream input convolutional neural network based on the cross-channel reconstruction mechanism. These two approaches can help us find more effective and deeper feature extraction methods and feature fusion methods in this research direction and design training. We adopted the network architecture model used in this area of research and used experimental data to demonstrate the effectiveness and superiority of the suggested network model.

Enhanced crop classification through integrated optical and SAR data: a deep learning approach for multi-source image fusion

ABSTRACT Agricultural crop mapping has advanced over the last decades due to improved approaches and the increased availability of image datasets at various spatial and temporal resolutions. Considering the spatial and temporal dynamics of different crops during a growing season, multi-temporal classification frameworks are well-suited for mapping crops at large scales. Addressing the challenges posed by imbalanced class distribution, our approach combines the strengths of different deep learning models in an ensemble learning framework, enabling more accurate and robust classification by capitalizing on their complementary capabilities. This research aims to enhance the crop classification of maize, soybean, and wheat in Bei’an County, Northeast China, by developing a novel deep learning architecture that combines a three-dimensional convolutional neural network (3D-CNN) with a variant of convolutional recurrent neural networks (ConvRNN). The proposed method integrates multi-temporal Sentinel-1 polarimetric features with Sentinel-2 surface reflectance data for multi-source fusion and achieves an overall accuracy of 91.7%, a Kappa coefficient of 85.7%, and F1 scores of 93.7%, 92.2%, and 90.9% for maize, soybean, and wheat, respectively. Our proposed model is also compared with alternative data augmentation techniques, maintaining the highest mean F1 score (87.7%). The best performer was weakly supervised with ten per cent of ground truth data collected in Bei’an in 2017 and used to produce an annual crop map for measuring the model’s generalizability. The model learning reliability of the proposed method is interpreted through the visualization of model soft outputs and saliency maps.

Multisource fusion of exogenous inputs based NARXs neural network for vehicle speed prediction between urban road intersections

The economy and safety of passages through the urban road intersection environment is an important research topic in the field of intelligent transportation systems, but vehicle speed prediction as its subtopic is still under-researched, and its prediction accuracy is unsatisfactory. Therefore, a model for vehicle speed prediction based on the nonlinear autoregressive model with multisource exogenous inputs (NARXs) neural network is proposed. The model combines the human-vehicle-road model with the NARXs neural network to perform speed prediction between urban road intersections. First, multisource features, including the variables of driving behavior characteristics, vehicle responses, and road conditions, are extracted to construct the human-vehicle-road model. Then, the model is introduced into the NARXs neural network. Finally, the advantages of the proposed model are verified from two perspectives by evaluation indices such as mean absolute error ( MAE), mean absolute percentage error ( MAPE), root mean square error ( RMSE), Theil index ( Theil ic), and goodness-of-fit ( R2) compared with several other models. On the one hand, the analysis results show that the proposed model has higher prediction accuracy than the other comparative models for different prediction durations and has the best performance in 30 s duration backward prediction. On the other hand, the curves of each evaluation index of the proposed model are horizontal, which indicates that the prediction performance of the model hardly varies with the length of the training dataset. These positive results demonstrate the higher accuracy and outstanding characteristics of the proposed model in the subject of vehicle speed prediction.

A Multi-Source Intelligent Fusion Assessment Method for Dynamic Construction Risk of Subway Deep Foundation Pit: A Case Study

The construction of a subway deep foundation pit is complex and risky, thus multiple safety risk factors bring great challenges to evaluating the safety status accurately. Advanced monitoring technology equipment could obtain a large number of monitoring data, and how integrating complex and diversified monitoring data to assess the safety risk of foundation pits has become a new problem. Therefore, an intelligent multi-source fusion assessment model is proposed. This model is mainly used for solving risk probability distribution, deep learning, and intelligent prediction of monitoring indicators, and then evaluating safety status by fusing various parameters of multiple indicators. Thus, based on the data of deep learning and the measured multivariate data, the dynamic risk during foundation pit construction can be obtained. Moreover, a typical case study was performed through monitoring and carrying out the risk assessment which is located at the Martyrs’ Lingyuan Station of Jinnan Metro Line R2, China. In this case, the PSO-SVM and LSTM models are used to predict the deformation trend, and the monitoring data is reliable with high precision. After multi-index fusion model calculation, the results show that the foundation pit structure is in a safe state, and the evaluation situation is basically consistent with the site. Consequently, the prediction of the new multi-source intelligent fusion risk assessment method is convincing.

Autonomous Multi-Floor Localization Based on Smartphone-Integrated Sensors and Pedestrian Indoor Network

Autonomous localization without local wireless facilities is proven as an efficient way for realizing location-based services in complex urban environments. The precision of the current map-matching algorithms is subject to the poor ability of integrated sensor-based trajectory estimation and the efficient combination of pedestrian motion information and the pedestrian indoor network. This paper proposes an autonomous multi-floor localization framework based on smartphone-integrated sensors and pedestrian network matching (ML-ISNM). A robust data and model dual-driven pedestrian trajectory estimator is proposed for accurate integrated sensor-based positioning under different handheld modes and disturbed environments. A bi-directional long short-term memory (Bi-LSTM) network is further applied for floor identification using extracted environmental features and pedestrian motion features, and further combined with the indoor network matching algorithm for acquiring accurate location and floor observations. In the multi-source fusion procedure, an error ellipse-enhanced unscented Kalman filter is developed for the intelligent combination of a trajectory estimator, human motion constraints, and the extracted pedestrian network. Comprehensive experiments indicate that the presented ML-ISNM achieves autonomous and accurate multi-floor positioning performance in complex and large-scale urban buildings. The final evaluated average localization error was lower than 1.13 m without the assistance of wireless facilities or a navigation database.

PCB surface defect fast detection method based on attention and multi-source fusion

PCB surface defect fast detection method based on attention and multi-source fusion

A Novel Dynamic Fusion Approach Using Information Entropy for Interval-Valued Ordered Datasets

Information fusion is capable of fusing and transforming information originated from multiple sources into an integrated representation. As an important representative of information, interval-valued ordered data aims at characterizing inaccurate and ambiguous information. However, existing methods are inappropriate when applied to the multi-source fusion for them. In addition, it is inevitable that in daily-life the sources and attributes of multi-source information systems may update at the same time. Hence there is a need to fuse data as efficiently as possible. Inspired by these deficiencies, we pay attention to the effective and efficient fusion of multi-source interval-valued ordered data in varieties of cases. First, the concepts of fuzzy dominance relation and dominance classes based on it are put forward between any two samples of interval-valued ordered information systems. Second, we define the fuzzy dominating and dominated conditional entropy. Then the fusion model is established and it is multi-source interval-valued ordered data oriented. Furthermore, it is true that there are numerous real-life applications related to concurrent change of both sources and attributes. Consequently, we design four incremental mechanisms and algorithms for fusing multi-source interval-valued data on the ground of the static condition. Eventually, a series of experiments is carried out on twelve datasets to verify that the proposed fusion approach outperforms other comparative methods on efficacy. Meanwhile, our incremental fusion algorithms are efficient compared with the static one for updating multi-source interval-valued data when sources and attributes are in the simultaneous variation.

Research on refined prediction of coastal wind power based on dynamic downscale and deep learning prediction

With the expansion of the installed capacity of offshore wind power, its impact on the safe operation of the power grid is increasingly obvious, and improving the prediction accuracy of offshore wind power power has become a key problem to be solved in the industry. The improvement of prediction accuracy, on the one hand, can facilitate the operation control of power grid, on the other hand, can also significantly improve the economic benefits of wind power plants. Based on power downscaling and deep learning methods, this paper carries out localized short-term and ultra-short-term power prediction modeling and optimization technologies for coastal offshore wind power. Comprehensive statistical prediction method, artificial intelligence prediction method, multi-model integration method and other combined prediction model, multi-source fusion information, to realize the refined prediction of Zhejiang offshore wind power, and improve the overall prediction accuracy. The research method is generally applicable and can guide the output prediction of the same type of offshore wind power.

Research on mechanical fault diagnosis based on MADS evidence fusion theory

In machine intelligence fault diagnostic and health status decision-making systems, rich, complex, and fuzzy feature information cannot facilitate fault decision-making merely on a single data source. This requires utilizing the heterogeneity of information gathered from multiple sources to diminish the system’s uncertainty and improve the accuracy of decision-making. In this work, a novel neural network-based multi-source fusion classification model is proposed to diagnose the pump mechanical faults. The multi-head attention Dempster–Shafer (D–S) evidence fusion (MADS) system extends the model’s ability to focus on rich features. Furthermore, the uncertain values throwing mechanism can effectively eliminate samples from uncertain categories and increase the model’s ability to distinguish diagnostic results with low confidence. Compared with a single sensor, our multi-sensor joint decision based on seven sensors considerably improved the fault diagnostic accuracy of MADS system, which has increased by at least 12.34%. Experimental validation demonstrates that utilizing the improved combination rules provided for multi-source evidence fusion fault diagnosis can significantly improve the efficacy of conventional D–S fusion and reduce the probability of misjudgment; combining the multi-head attention mechanism can dramatically increase the precision of model fault diagnosis. The proposed method has the potential to substantially accelerate research in the field of multi-source sensor joint fault diagnosis.

Integrating optical imaging techniques for a novel approach to evaluate Siberian wild rye seed maturity.

Advances in optical imaging technology using rapid and non-destructive methods have led to improvements in the efficiency of seed quality detection. Accurately timing the harvest is crucial for maximizing the yield of higher-quality Siberian wild rye seeds by minimizing excessive shattering during harvesting. This research applied integrated optical imaging techniques and machine learning algorithms to develop different models for classifying Siberian wild rye seeds based on different maturity stages and grain positions. The multi-source fusion of morphological, multispectral, and autofluorescence data provided more comprehensive information but also increases the performance requirements of the equipment. Therefore, we employed three filtering algorithms, namely minimal joint mutual information maximization (JMIM), information gain, and Gini impurity, and set up two control methods (feature union and no-filtering) to assess the impact of retaining only 20% of the features on the model performance. Both JMIM and information gain revealed autofluorescence and morphological features (CIELab A, CIELab B, hue and saturation), with these two filtering algorithms showing shorter run times. Furthermore, a strong correlation was observed between shoot length and morphological and autofluorescence spectral features. Machine learning models based on linear discriminant analysis (LDA), random forests (RF) and support vector machines (SVM) showed high performance (>0.78 accuracies) in classifying seeds at different maturity stages. Furthermore, it was found that there was considerable variation in the different grain positions at the maturity stage, and the K-means approach was used to improve the model performance by 5.8%-9.24%. In conclusion, our study demonstrated that feature filtering algorithms combined with machine learning algorithms offer high performance and low cost in identifying seed maturity stages and that the application of k-means techniques for inconsistent maturity improves classification accuracy. Therefore, this technique could be employed classification of seed maturity and superior physiological quality for Siberian wild rye seeds.