Multi-source Information Fusion Research Articles

In-bed human pose estimation using multi-source information fusion for health monitoring in real-world scenarios

In the field of 3D human pose estimation for in-bed health monitoring in real-world scenarios, challenging problems arise due to varying conditions of illumination and depth ambiguity. To address these issues, we followed the principle of 2D-to-3D pipeline and developed a novel 3D skeletons tracking approach based on multi-source image fusion. The utilization of thermal images allows us to estimate the 2D pose, as they are unaffected by changes in illumination and can be used even in complete darkness. Furthermore, depth images are employed to infer the pose depth, benefiting from their excellent depth resolution. During the 2D pose estimation phase, in order to exploit both spatial and temporal information, we have proposed a module for 2D pose estimation based on Graph Convolutional Network and Transformer network. To train this module, we have created a dataset using an independently developed automatic annotation method. In the subsequent 3D pose estimation phase, the typical 2D-to-3D lifting approaches entail inferring a 3D human pose from an intermediate 2D pose estimated by a deep network, but the task is an ill-problem due to the lack of depth resolution in thermal images. In our study, we overcome this challenge by inferring the pose depth through fusion with depth images, which offer superior depth resolution but are subject to limitations posed by illumination and texture conditions. We found that, (i) the developed automatic annotation method yields accurate pose annotations for training 2D pose estimation based on infrared thermal images; (ii) on the proposed dataset, our 2D pose estimation framework achieved a mean error of below 4 pixels and outperformed state-of-the-art methods; (iii) it achieved precise 3D pose estimation via fusion with depth images; and (iv) future research should consider applying the outlined approach to support the development of novel human pose models for health monitoring.

A novel belief Tanimoto coefficient with its applications in multisource information fusion

A novel belief Tanimoto coefficient with its applications in multisource information fusion

Tensor product graph diffusion based on nonlinear fusion of multi-source information to predict circRNA-disease associations

Circular RNA (circRNA), a non-coding RNA, has been shown in recent years via numerous studies to be more than just a "by-product" of gene transcription. It also participates in the regulation of many diseases and influences various biological functions in the human body. Its unique properties and biological functions indicate that circRNA may become new biomarker and therapeutic target, thus, it is crucial for modern medicine to identify potentially relevant circRNAs. This paper proposed novel computational model SNFTPGd-CDA for circRNA-disease associations prediction with the CircR2Disease v2.0 as dataset, which first constructs multiple similarity networks according to multi-source information related to circRNA and disease, and uses similarity network fusion (SNF) to nonlinearly fuse multi-source similarity networks. Then tensor product graph diffusion (TPGd) is employed to diffuse the similarity in the tensor product graph of the fused similarity network into the context of other nodes, which in turn constructs a more reliable similarity network as the feature matrix. Finally, the cascaded forest model is employed for circRNA-disease associations prediction. An AUC value of 0.9937 and an accuracy of 96.92 % were obtained with five-fold cross-validation. This result indicates that SNFTPGd-CDA achieves more accurate performance than previous methods, and can support validation of circRNA-disease associations.

Two-Level Integrity-Monitoring Method for Multi-Source Information Fusion Navigation

To address the issue of integrity monitoring for a multi-source information fusion navigation system, a theoretical framework of two-level integrity monitoring is proposed. Firstly, at the system level, a system-integrity-monitoring method based on the Kalman filter weighted least-squares form is established to detect and isolate faulty navigation sources. Secondly, at the sensor level, considering the redundancy of the faulty navigation sources, this paper presents the design of a multi-mode comprehensive fault-detection method for non-redundant navigation sources. Additionally, an extended-dimension matrix optimized sensor-fault detection and verification method for redundant navigation sources is proposed. Finally, integrity risk allocation criteria are established based on the effectiveness of navigation sources to calculate the system protection level and trigger integrity alarms. The two-level integrity-monitoring method was tested on a multi-source information fusion navigation system integrated with an inertial navigation system (INS), global positioning system (GPS), BeiDou satellite navigation system (BDS), Doppler velocity log (DVL), barometric altimeter (BA), and terrain-aided navigation (TAN). Test results demonstrate that the proposed method can effectively isolate the faulty navigation source within 10 s. Furthermore, it can detect the faulty sensors within the faulty navigation sources, thereby enhancing the reliability and robustness of the multi-source information fusion navigation system.

Engine remaining useful life prediction based on PSO optimized multi-layer long short-term memory and multi-source information fusion

Engine as the core component of mechanical equipment, its operating state directly affects whether the equipment can operate normally. Predicting the engine remaining useful life (RUL) can monitor the health of the engine in real time and formulate a timely and reasonable maintenance plan. Aiming at the engine monitoring data with various and long time span, we propose a direct prediction method of engine RUL based on particle swarm optimization (PSO) optimized multi-layer Long Short-Term Memory (LSTM) in this paper. Firstly, the monitoring data that can well reflect the engine degradation trend is screened out, and the samples are constructed through a sliding time window. Then, a multi-layer LSTM model is constructed to mine the deep-seated features of the samples for predicting the engine RUL. Finally, the hyperparameters of the multi-layer LSTM model are optimized automatically by the PSO algorithm to optimize the performance of the model. The effectiveness of this method is verified by NASA data set. RMSE, MAE and the scoring function are used as evaluation indexes. RMSE and score of the prediction results are 12.35 and 284.1, respectively. It has higher prediction accuracy compared with traditional deep learning and machine learning methods.

Risk assessment approach for tunnel collapse based on improved multi-source evidence information fusion

Risk assessment approach for tunnel collapse based on improved multi-source evidence information fusion

Research on rapid quality identification method of Panax notoginseng powder based on artificial intelligence sensory technology and multi-source information fusion technology

Panax notoginseng powder (PNP) has high medicinal value and is widely used in the medical and health food industries. However, the adulteration of PNP in the market has dramatically reduced its efficacy. Therefore, this study intends to use artificial intelligence sensory (AIS) and multi-source information fusion (MIF) technology to try to establish a quality evaluation system for different grades of PNP and adulterated Panax notoginseng powder (AD-PNP). The highest accuracy rate reached 100% in identifying PNP grade and adulteration. In the prediction of adulteration ratio and total saponin content, the optimal determination coefficients of the test set were 0.9965 and 0.9948, respectively, and the root mean square errors were 0.0109 and 0.0123, respectively. Therefore, the grade identification method of PNP and the evaluation system of AD-PNP based on AIS and MIF technology can rapidly and accurately evaluate the quality of PNP.

A Practical Exploration of the Integration of Civics into Student Education and Management in Colleges and Universities Empowered by Information Integration Technology in the Context of Three-Whole Parenting

Abstract Internet information technology has brought challenges to the integration of college and university Civics in student education and management work, and at the same time, it also provides an opportunity for the integration of information technology. In this paper, on the basis of two kinds of nodes of basic Petri nets, the library and change, we constructed the workflow of identification Petri nets oriented to the education and management of students in the Civic Government of colleges and universities. The workflow engine is designed as the center of gravity of the reference model, consisting of six basic elements of student management. In the information fusion workflow, a multi-source information fusion method based on CNN and LSTM is proposed to further integrate Civics into student education management. The response time of the student management action flow is shorter than the retrieval time based on the original mechanism, and the average response time is reduced by 57.4 ms when the workflow reaches 5000. The fusion of Civic Politics and education management in the context of three-whole-parenting optimizes the management work method.

Higher Order Fractal Belief Rényi Divergence With Its Applications in Pattern Classification.

Information can be quantified and expressed by uncertainty, and improving the decision level of uncertain information is vital in modeling and processing uncertain information. Dempster-Shafer evidence theory can model and process uncertain information effectively. However, the Dempster combination rule may provide counter-intuitive results when dealing with highly conflicting information, leading to a decline in decision level. Thus, measuring conflict is significant in the improvement of decision level. Motivated by this issue, this paper proposes a novel method to measure the discrepancy between bodies of evidence. First, the model of dynamic fractal probability transformation is proposed to effectively obtain more information about the non-specificity of basic belief assignments (BBAs). Then, we propose the higher-order fractal belief Rényi divergence (HOFBReD). HOFBReD can effectively measure the discrepancy between BBAs. Moreover, it is the first belief Rényi divergence that can measure the discrepancy between BBAs with dynamic fractal probability transformation. HoFBReD has several properties in terms of probability transformation as well as measurement. When the dynamic fractal probability transformation ends, HoFBReD is equivalent to measuring the Rényi divergence between the pignistic probability transformations of BBAs. When the BBAs degenerate to the probability distributions, HoFBReD will also degenerate to or be related to several well-known divergences. In addition, based on HoFBReD, a novel multisource information fusion algorithm is proposed. A pattern classification experiment with real-world datasets is presented to compare the proposed algorithm with other methods. The experiment results indicate that the proposed algorithm has a higher average pattern recognition accuracy with all datasets than other methods. The proposed discrepancy measurement method and multisource information algorithm contribute to the improvement of decision level.

Multisource Information Fusion for Autoformer: Soft Sensor Modeling of FeO Content in Iron Ore Sintering Process

As a key thermal-state indicator of the iron ore sintering process, the content of ferrous oxide (FeO) in the finished sinter is directly related to product quality. Based on the massive data of sintering process, the data-driven soft sensor model provides a good choice for real-time FeO content detection. However, the complex characteristics of the data, including dynamics, nonlinearity, and multi-source heterogeneity, are still the main obstacles to improving the modeling accuracy. To solve this problem, a multi-source information fusion Autoformer (MIF-Autoformer) model is introduced in this paper. Firstly, feature-level information fusion and data-level information fusion are implemented based on the MIF strategy. Then, the comprehensive information of the sintering process is fed to the downstream Autoformer model in a serial manner, which not only improves the information capacity, but also provides additional prior information about the FeO content grade. This is helpful for Autoformer to capture the complex temporal distributions in the sintering process. Finally, the proposed model is applied to a real sintering plant. Experimental results show that the hybrid image features provided by the MIF strategy have a general optimization effect on different competitive models, and MIF-Autoformer exhibits the lowest prediction error on the test set.

Multisource information fusion considering the weight of focal element’s beliefs: a Gaussian kernel similarity approach

Similarity has been extensively utilized to measure the degree of conflicts between evidences in multisource information fusion. The existent works, however, assumed that the contribution of each focal element’s belief to the similarity measure is the same, and the influence of the weights of focal element’s belief is not considered, which is unreasonable. This article proposes a new Gaussian kernel similarity approach to measure the similarity between evidences. The proposed Gaussian kernel similarity coefficient can effectively take account of the weights of focal element’s beliefs. In addition, it possesses some preferable properties, such as, bounded, consistent, and symmetrical. A multisource information fusion method based on the Gaussian kernel similarity coefficient is, therefore, investigated. The developed method mainly contains three steps: (1) the Gaussian kernel similarity coefficient, as a connection, is leveraged to calculate the weight of evidences based on the weight of focal element’s beliefs; (2) the initial evidences are, thereby, modified based on the weight of evidence via the weight-average method; and (3) the final multisource information fusion can be achieved by the Dempster’s combination rule using the modified evidences. An illustrative example with single-element subset and an application with multi-element subset are presented, and it is verified that the proposed method is effective in dealing with conflicting evidences.

On-line defect recognition of MIG lap welding for stainless steel sheet based on weld image and CMT voltage: Feature fusion and attention weights visualization

At present, sensor recognition technology has become a research hotspot in welding process monitoring. However, most of the existing defect recognition methods are based on single-sensor information, which hinders the further improvement of recognition accuracy and weakens the anti-interference ability under complex conditions. To overcome the shortcomings of single-sensor recognition technology, multi-source information fusion technology can exploit the advantages of different types of sensors to achieve more comprehensive and accurate monitoring of the welding process. However, given the process of fusion of different information, the feature mining mechanism of the deep learning model is still under-explained, and the direct complementarity of the different information remains unclear. In this paper, we propose a hybrid model that uses the fusion of molten pool image and voltage signals to quickly and accurately identify three types of defects in lap MIG welding: burn-through, weld bias, and uneven width. Firstly, a synchronized trigger device is used to acquire molten pool images and weld voltage information, and the voltage data is sliced and matched with the molten pool image data. Secondly, the Short-Time Fourier Transform (STFT) is applied to explore the frequency variation components in the voltage data, and the resulting time-frequency matrix is transformed into a time spectrogram. Finally, the molten pool image and the time spectrogram are simultaneously fed into the hybrid CNN model for online monitoring of welding defect states. The test results show that the average accuracy of the hybrid CNN model in detecting five different welding states reaches 98.87 %, which is significantly higher than three other models based on single sensor information. The visualization of the confusion matrix and Grad-CAM shows that the hybrid CNN model based on multi-source heterogeneous information has a higher accuracy for detecting the transition states that are difficult to detect in welding, and has a more reasonable mechanism for allocating attention, so it can achieve a more accurate and balanced recognition effect in the whole welding manufacturing process. These studies can provide guidance for quality monitoring of welding processes based on multi-source sensor technology, which can be extended to the field of arc additive monitoring.

A novel information fusion method using improved entropy measure in multi-source incomplete interval-valued datasets

Multi-source data is a comprehensive data type that combines multiple sources of information or datasets. Compared to point-valued data, interval-valued data provides a more accurate representation of the uncertainty and variability associated with objects. In practical situations, data obtained from multiple sources may contain missing values for various reasons. Therefore, it is essential to develop multi-source information fusion technology in order to achieve information fusion or information extraction from multi-source incomplete data. This paper aims to explore the information fusion problem of multi-source incomplete interval-valued datasets. The primary contributions of this study involve utilizing the principle of statistical distribution and KL divergence to establish a metric for measuring the similarity between intervals. Firstly, this approach helps to reduce the problem of disregarding internal information within interval values, which can result in the loss of valuable information. Secondly, we establish an interval fuzzy similarity relation based on the mentioned concept of similarity among interval values. Moreover, we investigate the uncertainty measurement of incomplete interval-valued decision datasets and design an emerging information entropy fusion method. Finally, we comprehensively evaluate the effectiveness of the proposed method. Experimental results indicate that the proposed approach has advantage over the maximum, minimum, mean, and information entropy fusion method based on tolerance relationship. In addition, the distance metric used in this article can improve the fusion classification effect compared to several common interval-valued distance measures.

Optimal Core Acquisition and Remanufacturing Decisions With Discrete Core Quality Grades

The acquisition of used products (cores) is essential for leveraging the advantages of remanufacturing. Due to limited historical information about cores, it is difficult for the remanufacturers to accurately assess the quality of cores before disassembly and inspection. In this article, we investigate the joint acquisition and remanufacturing problem with a discrete empirical distribution. Under both deterministic and stochastic demands, two integer programming models are developed to obtain operational rules for making optimal decisions. Since the discrete empirical distribution is assumed to be the nominal distribution or asymptotic distribution of the true quality, the error caused by the difference between the estimated discrete empirical distribution and the true distribution cannot be neglected. Thus, we evaluate the error using the Phi-divergence measure, and the corresponding effects of this difference on the resultant acquisition and remanufacturing rules are analyzed. Through numerical examples, we observe that optimal acquisition and remanufacturing decisions are negatively affected by the difference between the estimated distribution and the true distribution, even if the estimated distribution can pass a hypothesis test. Furthermore, to overcome suboptimal outcomes, a multisource information fusion approach is proposed, which can utilize quality information collected from multiple sources to make accurate acquisition and remanufacturing decisions.

Tomato Production Prediction Based on Deep Learning Algorithm-Cascade-PSPNET and Bayes

Accounting the problem of small sample size in tomato yield statistics, a dual-fusion prediction analysis model based on Bayes theory and deep learning algorithm Cascade-PSPNET is proposed. The tomato yield prediction is conducted based on remote sensing image time series analysis and multi-source information fusion theory with Bayesian theory and credibility weighting. First, the area of tomato planting is analyzed through semantic segmentation algorithms of remote sensing images during the tomato planting process. The yield is predicted by variables such as planting area fluctuation, disaster cycle fluctuation, etc. Through analyzing the reduced yield affected by disasters in different time periods of remote sensing images during planting process, the value chain of tomato industry is calculated by comprehensively analyzing price-value system, inflation coefficient, and unit area yield. At the same time, the annual patent application volume is used to predict the change of tomato yield year by year according to Bayesian theory, and the relationship between annual patent application volume and tomato yield year by year under different confidence levels is analyzed. The results show that it is feasible to use the Bayes method with semantic segmentation algorithms of remote sensing images to predict tomato yield. Next, the experiment of fusion prediction between two prediction models in the same target area will be carried out for verification.

Incorporating stock prices and text for stock movement prediction based on information fusion

Forecasting stock market via historical financial data is an important issue for market participants because even if the prediction accuracy is only slightly improved, better trading decisions can be made. Historical financial data has evolved from the initial single text or stock price to the fusion of multisource information. However, how to adopt a method that adaptively fuses numerical data and text so that the prediction model can learn time series information in parallel remains a challenging problem. In this paper, we propose a collaborative attention Transformer fusion model for stock movement prediction (CoATSMP), including parallel extraction of text and prices features, parameter-level fusion and a joint feature processing module, that can successfully deeply fuse text and stock prices in view of the soft fusion method. The experiments show that (1) the proposed approach outperforms the baselines, (2) the soft fusion method proposed in this paper has better modeling performance under the CoATSMP framework, which brings greater improvement in the prediction performance, (3) models containing prices and text are better than those using only one data source, and (4) quantitative analysis of experimental results indicates that text plays a relatively more critical role in the CoATSMP framework. Real simulation trading shows that the trading strategy based on CoATSMP can significantly improve profits; thus, the model has practical application value.

A study on the construction of an online interactive teaching platform for English in colleges and universities based on the fusion of multi-source and multi-objective statistical information

Abstract This paper firstly combines English teaching in colleges and universities and big data technology and proposes the information technology of English teaching in colleges and universities based on the fusion of multi-source and multi-objective statistical information and conducts research on the traditional teaching optimization algorithm and multi-task teaching optimization algorithm to promote English teaching in colleges and universities. Secondly, in the “teaching” phase, teachers and teaching assistants teach according to the difference between their average level and the class average to increase the value of each student’s factor and increase individual exploration and group diversity in the “learning” phase. Then, the English teaching platform was designed and studied regarding system structure and platform function, simulated and analyzed. The results show that the efficiency of the multitasking optimization algorithm is increased by 21.85% compared with the traditional teaching optimization algorithm for the number of concurrent users from 10 to 100 and 16.46% when the number of concurrent users is 100, which has a positive effect on students’ English learning efficiency.

Online Dynamic Modelling for Digital Twin Enabled Sintering Systems: An Iterative Update Data-Driven Method

The sintering process is a crucial thermochemical process in the blast furnace iron-making system. Tumble strength (TS), as a vital performance to assess sinter quality, is difficult to monitor due to the lack of timely measurement. Constructing a data-driven model for TS is an alternative for monitoring TS. However, the time-varying dynamic sintering process makes the task of modelling challenging. And the data are incomplete and insufficient in practice for modelling since there are unknown time delays in the system and lack actual TS value. The digital twin (DT) technique is a powerful tool to simulate the system dynamics with the real-time interaction between physical processes and virtual agents in cyberspace. This paper introduces a DT-enabled equivalent of the sintering system and proposes online data-driven modelling for TS monitoring. The time delay in the system is estimated for variable sequence alignment based on a modified maximum information coefficient method. The data used for modelling is enriched based on a multi-source information fusion technique. An adaptive update method is proposed to deal with the time-varying dynamics. The iterative forgetting factor-based algorithm is designed for the support vector regression method and guarantees a fast computational speed. Implementation and validation of the model on a DT-enabled sintering system show the efficiency of the proposed method. The accuracy of TS monitoring reaches 99.6% by analysis of 3 months’ data.

Research on the expression and creativity of traditional ethnic cultural elements in clothing design based on the fusion of multi-source and multi-objective statistical information

Abstract The use of multi-source, multi-objective statistical information fusion is utilized in this paper to identify garment design targets, and the correlation is judged based on garment data. Some parameters are pre-set to generate linear garment descriptions for multi-source, multi-objective information fusion and find the adaptability of ethnic elements under random coefficients using element information relationships. The clothing width vector of the radial basis function is calculated by clustering ethnic culture element data using centroid information. The results show that the innovation rate of ethnic elements of Miao batik reaches the highest 0.57, the average adaptation value in the garment design is increased by 4.92%, and the ethnic-cultural elements are embodied 280 times, which indicates that the proposed method can promote the traditional excellent culture.

Conjunctive and disjunctive combination rules in random permutation set theory: A layer-2 belief structure perspective

In uncertainty reasoning, conjunctive and disjunctive combination rules are the core tools for information updates. As an extension of evidential reasoning, random permutation set reasoning models uncertain information based on ordered focal sets. Existing combination rules in random permutation set theory, orthogonal sums, do not satisfy the commutativity of one of the original distributions and overly obey its order information. In this paper, the random permutation set theory is interpreted as an refined extension of Dempster–Shafer theory, and a layer-2 belief structure is proposed to describe the permutation event space. Compared with the traditional belief structure, the proposed structure can model both symbolic and numerical uncertainty. Based on the above, the conjunctive and disjunctive combination rules in Dempster–Shafer theory are extended to random permutation set theory. Through properties analysis and simulation demonstration, we demonstrate that the proposed methods can not only resolve the counter-intuitive results of orthogonal sums, but make full use of order information in distributions as well. In addition, we also extend the product space operations and discounting methods based on the proposed methods, and give a general framework of multi-source information fusion under the random permutation set theory.