Network Data Fusion Research Articles

Fault diagnosis of rolling bearing based on multimodal data fusion and deep belief network

Aiming at raw vibration signal of rolling bearing with long time series, a fault diagnosis model based on multimodal data fusion and deep belief network is proposed in this paper. First, multimodal data composed of artificial features and model features can be obtained by time-frequency domain analysis and unsupervised learning based on restricted Boltzmann machine (RBM). Second, canonical correlation analysis method is used to extract the typical feature pairs from the multimodal data to realize the feature-level multimodal data fusion. Third, deep belief network is applied to extract deep feature mapping between typical feature pairs and fault types. After greedy layer-wise pre-training and fine-tuning, it is available to achieve the trained model for fault diagnosis of rolling bearing. Typical rolling bearing datasets are used to testify the effectiveness of the proposed method. It is verified that the robustness and accuracy of the proposed method are superior to common methods.

Feedback Convolutional Network for Intelligent Data Fusion Based on Near-Infrared Collaborative IoT Technology

Near-infrared (NIR) data containing spectral response information for detecting target composition are sparsely implied in spectral frequency sequence. Spectral feature information should be extracted using computer-oriented chemometric methods. An Internet of Things (IoT) framework constructed with NIR calibration platform needs some advanced algorithm architectures to realize intelligent analysis. A feedback convolutional neural network (CNN) architecture, including three repeated segments of convolution, pooling, and flattening, is designed in this article for multiple extraction of spectral features from one-dimensional NIR data. An error-feedback iteration mechanism is proposed in the model training process to optimize convolution filters of each segment. Multisegment features are fused successively to ease the sparse information issue. Fusion data are further used to train the calibration models with a parametric-scaling fully connected network to determine the suitable numbers of hidden and output nodes. The adaptive network structure has the advantage of obtaining optimal prediction results from fused feature data. The proposed feedback CNN architecture based on feature information fusion is applied to the NIR rapid quantitative detection of selenium content in paddy rice samples. Experimental results showed that the fusion of multisegment features can enhance the ability of spectral information extraction. The optimal model based on fused feature data performs better than models based on separate feature data of each segment. The feedback convolutional network for information fusion can be applied in the NIR collaborative IoT framework for rapid detection spectroscopy to ensure high-confidence NIR analysis in the artificial intelligence performance of IoT.

ANN-based automated scaffold builder activity recognition through wearable EMG and IMU sensors

Construction worker activity recognition is essential for worker performance and safety assessment. With the development of wearable sensing technologies, many researchers developed kinematic sensor-based worker activity recognition methods with considerable accuracy. However, the limitations of the previous studies remain at the challenge of using smartphones for practical implementation, fewer classified activities, and limited recognized motions and body parts. This study proposes an ANN-based automated construction worker activity recognition method that can recognize complex construction activities. The proposed methodology discusses data acquisition, data fusion, and artificial neural network (ANN) model development. A case study of scaffold builder activities was investigated to validate the proposed methodology's feasibility and evaluate its performance compared to other existing methods. The results show that the proposed model can recognize fifteen scaffold builder activities with an accuracy of 94% with 0.94 weighted precision, recall, and F1 Score.

Internet-of-Things-Enabled Data Fusion Method for Sleep Healthcare Applications.

The Internet of Medical Things (IoMT) aims to exploit the Internet-of-Things (IoT) techniques to provide better medical treatment scheme for patients with smart, automatic, timely, and emotion-aware clinical services. One of the IoMT instances is applying IoT techniques to sleep-aware smartphones or wearable devices’ applications to provide better sleep healthcare services. As we all know, sleep is vital to our daily health. What is more, studies have shown a strong relationship between sleep difficulties and various diseases such as COVID-19. Therefore, leveraging IoT techniques to develop a longer lifetime sleep healthcare IoMT system, with a tradeoff between data transferring/processing speed and battery energy efficiency, to provide longer time services for bad sleep condition persons, especially the COVID-19 patients or survivors, is a meaningful research topic. In this study, we propose an IoT-enabled sleep data fusion networks (SDFN) module with a star topology Bluetooth network to fuse data of sleep-aware applications. A machine learning model is built to detect sleep events through an audio signal. We design two data reprocessing mechanisms running on our IoT devices to alleviate the data jam problem and save the IoT devices’ battery energy. The experiments manifest that the presented module and mechanisms can save the energy of the system and alleviate the data jam problem of the device.

Multi-Sensor Complex Network Data Fusion Under the Condition of Uncertainty of Coupling Occurrence Probability

Based on the time and location information of nodes, multi-sensor complex networks can achieve high-speed and low-delay exchange of control and sensor data between sensor nodes to ensure the accuracy and real-time performance of the entire detection and control system. However, complex multi-sensor networks face limitations in terms of computing, storage, and network resources. After grouping the sensor data of nodes, this paper employs an adaptive weighted data fusion method to complete the data fusion processing in the network. This approach can significantly reduce the data redundancy in the wireless sensor network, save a large amount of storage resources, and lower the network bandwidth occupation, with high efficiency and good scalability. The least square (LS) fitting is performed by two sets of temperature and humidity data obtained by weighted fusion at the cluster head node. Experimental results indicate that the fusion of two-dimensional data transmitted to the base station or control center can further diminish the amount of data transmission. In addition, after the adaptive weighted data fusion of the two-dimensional data at the cluster head node, the energy consumption of the cluster head node without the fusion and the cluster head node after the fusion is compared. The results demonstrate that the energy consumption of the cluster head node is notably abridged, the energy of the cluster head node is saved to a greater extent, and the life cycle of the network is prolonged.

DeepIDA: Predicting Isoform-Disease Associations by Data Fusion and Deep Neural Networks.

Alternative splicing produces different isoforms from the same gene locus, it is an important mechanism for regulating gene expression and proteome diversity. Although the prediction of gene(ncRNA)-disease associations has been extensively studied, few (or no) computational solutions have been proposed for the prediction of isoform-disease association (IDA) at a large scale, mainly due to the lack of disease annotations of isoforms. However, increasing evidences confirm the associations between diseases and isoforms, which can more precisely uncover the pathology of complex diseases. Therefore, it is highly desirable to predict IDAs. To bridge this gap, we propose a deep neural network based solution (DeepIDA) to fuse multi-type genomics and transcriptomics data to predict IDAs. Particularly, DeepIDA uses gene-isoform relations to dispatch gene-disease associations to isoforms. In addition, it utilizes two DNN sub-networks with different structures to capture nucleotide and expression features of isoforms, Gene Ontology data and miRNA target data, respectively. After that, these two sub-networks are merged in a dense layer to predict IDAs. The experimental results on public datasets show that DeepIDA can effectively predict IDAs with AUPRC (area under the precision-recall curve) of 0.9141, macro F-measure of 0.9155, G-mean of 0.9278 and balanced accuracy of 0.9303 across 732 diseases, which are much higher than those of competitive methods. Further study on sixteen isoform-disease association cases again corroborates the superiority of DeepIDA. The code of DeepIDA is available at http://mlda.swu.edu.cn/codes.php?name=DeepIDA.

Neural Network Optimization and Data Fusion Recognition Method for Intelligent Mechanical Fault Diagnosis

With the improvement of mechanical equipment complexity and automation level, the importance of mechanical equipment fault diagnosis is more and more prominent, and the choice of appropriate diagnosis method is crucial to the accuracy of the diagnosis results. Wavelet analysis and neural network technology, as the hot spot and frontier of research, are also important research contents in the development of intelligent diagnosis of mechanical fault. Data fusion can process multisource information to obtain more accurate and reliable methods. At the same time, because of its good nonlinearity, adaptability, and fault tolerance, neural network has become the preferred method of mechanical fault diagnosis. This paper first describes the research content and significance of fault diagnosis technology and introduces the main methods and steps of fault diagnosis, and through the introduction of mechanical fault vibration signals, vibration signals were analyzed in time domain and frequency domain. Secondly, the definition and classification of data I fusion and RBF neural network are introduced in detail and compared with BP neural network. Because the prediction accuracy of the RBF network is higher than that of the BP neural network and the training time of the RBF network is obviously shorter than that of the BP network, the RBF network has significant advantages over diagnostic errors. In this paper, six valve signals were collected under normal conditions and errors, and by analyzing and comparing different theoretical foundations, the 4‐second network crisis time was effectively reduced, which provided the basis for teaching monitoring.

Application of data fusion based on deep belief network in air quality monitoring

Abstract The successive development of social industrialization and modernization has caused a great many of environmental problems. Due to the imbalance between the environment and rapid development, the problem of urban air quality has become more and more prominent, so air quality monitoring has become particularly important. In air quality monitoring, unexpected situations such as damage to the monitoring equipment and the relocation of the station building will occur, resulting in the lack of monitoring data. For this lack of monitoring data, the missing values of the monitoring data are supplemented by the data fusion method using deep belief network (DBN). It can provide relevant researchers with reference data for further analysis and research on air quality. Compared with BP neural network for data fusion, DBN method is closer to the actual monitoring value.

Retracted] Personalized Marketing Recommendation System of New Media Short Video Based on Deep Neural Network Data Fusion

With the rapid development of mobile Internet, short video has become another darling after traditional webcast in recent years. How to make full use of short video for effective marketing has become a hot issue that academia and industry are paying close attention to. This article is mainly aimed at exploring practical new media through in‐depth research and exploration of the specific implementation methods and strategies of short video marketing in social media, based on the advantages and characteristic models of short video marketing in social media. The strategy of short video marketing in social media, and the use of highly in‐depth neural network analysis technology for the personalized marketing recommendation system of new media short videos, so as to better promote the use of social media short videos by enterprises or individuals. We have to learn from marketing activities. The experimental results of this article show that when the data volume reaches 80%, the performance of the VRBCH algorithm steadily improves, so the performance of the main F of the VRBCH algorithm is still relatively ideal when the data volume changes. Due to the high dilution of the experimental data set, the amount of data in the VRBCH algorithm has increased sharply by 30% to 35%, but the purchase rate of the marketing recommendation system is as high as 98%. Therefore, the system has high feasibility.

3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network

Facial expression has long been recognized as containing meaningful nonverbal affective cues for decoding human emotions. Recently, multimodal 2-D + 3-D fusion method has shown significant potential in facial expression recognition (FER) due to its fine-grained face descriptions in various spatial channels. However, current work mainly relies on feature- or even score-level fusion to find emotion cues spread in different channels and may miss key information due to lack of focus. To this end, we propose an attention-based multichannel data fusion network (AMDFN) to better preserve and find such key facial cues. More specifically, we first map a 3-D face scan into multichannel images and then fuse them in a ResNet18 backbone to get layered emotion features. Second, we leverage a layer attention model to explore the dependencies between features of different layers to learn discriminative affective cues for effective emotion recognition. Our comprehensive experiments on two widely used datasets (i.e., Facescape and Bosphorus) have verified the performance of our approach compared to several state-of-the-art rivals.

Intelligent Perception System of Big Data Decision in Cross‐Border e‐Commerce Based on Data Fusion

There is a gap between supply and demand in cross‐border e‐trading platforms. The so‐called supply and demand gap refers to the existing good companies that cannot meet the needs of buyers. Cross‐border e‐commerce, the buyer’s purchase demand is often included in the buyer’s behavior, such as searching for goods by pressing buttons, clicking price, category and other means, and on‐site delivery time. It is important for CBEC to analyze the buyer’s needs to protect the procurement and provide the seller with reference to supply, so as to solve the supply and demand gap between the buyer and the seller. This paper mainly studies the CBEC big data decision intelligent perception system based on data fusion. It is the same as the innovation process of better application of data fusion to the e‐commerce business model. At the same time, the key technologies of data extraction, conversion, data warehouse, and front‐end display in the system construction are analyzed and designed. This paper mainly uses data fusion algorithm, data fusion network model, quaternion method, big data decision intelligent sensing system framework design experiment, and CBEC user experiment to study the CBEC big data decision intelligent sensing system based on data fusion, and also with the development of CBEC, I hope more people can participate in CBEC practice. The results show that with the increase of data volume of the CBEC platform from 2016 to 2020, the construction of the CBEC platform business of big data production intelligent perception system considers the situation of users from seven aspects. With the increase of consumption, domestic consumers are increasingly pursuing a better life, cross‐border import e‐commerce ushered in a great development era.

A Fast Vacuum Arc Detection Method Based on the Neural Network Data Fusion Algorithm for the High-Voltage DC Power Supply of Vacuum Tubes

Vacuum arc is one of the most important failure factors of the vacuum tubes. The amount of delivered energy from the high-voltage dc power supply to the vacuum tube is an important issue during the vacuum arc in the tube. Vacuum arc acts as a short-circuit fault (SCF) at the power supply output. The majority of converters use a single current sensor to measure only the converter output current for detecting the SCF. However, the sensor may provide unreliable data because of the noise effect. Application of a low-pass filter reduces the noise effect. Regarding the delay of the low-pass filter, the interval of arc detection increases and more energy is delivered to the tube. In this article, a fast vacuum arc diagnosis system is proposed based on the neural network data fusion algorithm. The proposed scheme consists of two sensors: the conventional current sensor and the existing voltage sensor used for controlling and monitoring. The data of these two sensors are combined by neural networks to diagnose the vacuum arc and to reject false alarms in noisy environments. The proposed method has a fast response time to the real SCFs while it remains inactive against the false data which are generated by noise. Simulations and experimental tests are carried out to evaluate the proposed scheme.

A trust-based fuzzy neural network for smart data fusion in internet of things

Internet of Things (IoT) devices generates a vast amount of data from extensive applications. Maintaining the sensed data with low energy consumption, delay time, and adaptive coverage fraction rate proportionally influences the storage capacity. To maintain a trade-off between above-listed factors, we proposed an Elfes Sugeno Fuzzy and Trust-based Neural Networks (ESF-TNN) approach enables 3-algorithms. First, Elfes Probability Sensing (EPS) Model addresses the coverage fraction of each IoT sensor. Second, Sugeno Fuzzy Processing model regulates the energy consumption by proportionately distributing data to nodes without the defuzzification process. Third, Trust-based Neural Data Storage algorithm enriches an adequate data storage capacity by considering the average classification ratio while processing regenerated data packets to pertain each interaction information via Trust Mechanism. Simulation results show that our proposed method effectively covers the monitored area with 15 Joules of energy consumption and 1-ms delay time along with sufficient storage capacity.

A Shape-Constrained Neural Data Fusion Network for Health Index Construction and Residual Life Prediction.

With the rapid development of sensor technologies, multisensor signals are now readily available for health condition monitoring and remaining useful life (RUL) prediction. To fully utilize these signals for a better health condition assessment and RUL prediction, health indices are often constructed through various data fusion techniques. Nevertheless, most of the existing methods fuse signals linearly, which may not be sufficient to characterize the health status for RUL prediction. To address this issue and improve the predictability, this article proposes a novel nonlinear data fusion approach, namely, a shape-constrained neural data fusion network for health index construction. Especially, a neural network-based structure is employed, and a novel loss function is formulated by simultaneously considering the monotonicity and curvature of the constructed health index and its variability at the failure time. A tailored adaptive moment estimation algorithm (Adam) is proposed for model parameter estimation. The effectiveness of the proposed method is demonstrated and compared through a case study using the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) data set.

Bearing fault diagnosis based on vibro-acoustic data fusion and 1D-CNN network

Bearing fault diagnosis is an important part of rotating machinery maintenance. Existing diagnosis methods based on single-modal signals not only have unsatisfactory accuracy, but also bear the inherent risk of being misguided by single-modal signal noise. A new method is put forward that fuses multi-modal sensor signals, i.e. the data collected by an accelerometer and a microphone, to realize more accurate and robust bearing-fault diagnosis. The proposed method extracts features from raw vibration signals and acoustic signals, and fuses them using the 1D-CNN-based networks. Extensive experimental results obtained on ten groups of bearings are used to evaluate the performance of the proposed method. By analyzing the loss function and accuracy rate under different SNRs, it is empirically found that the proposed method achieves higher rate of diagnosis accuracy than the algorithms based on a single-modal sensor. Moreover, a visualization analysis is also conducted to investigate the inner mechanism of the proposed 1D-CNN-based method.

Modification of colorimetric method based digital soil test kit for determination of macronutrients in oil palm plantation

It is the need of time that oil palm farmers must perform the spatially planned soil analysis to know about the fertilizer sufficient and deficient zones of land. Colorimetric method is a suitable and fast solution of soil analysis for NPK determination using the digital soil test kit. NPK determination procedure with a digital soil test kit was undefined for oil palm. Furthermore, the digital soil test kit determines the passage of light through an opaque medium of soil solution with a specified reagent. Therefore, environmental light may interfere leading to wrong results of NPK measurement. Likewise, this equipment was non-incorporable with the controller of any VRT fertilizer applicator. In this research, these issues were addressed and the NPK measurement procedure was defined for oil palm plantation by modifying the ‘soil to water’ ratio in sample soil solution with an optimum environmental light range of 18-23 W/m2. ‘Soil to water’ ratios were found for nitrogen, phosphorus and potassium as 0.31 to 5.00, 1.00 to 5.00 and 4.50 to 5.00, respectively to fit the requirement of NPK for oil palm in the prescribed range of the equipment. Validation study of modified digital soil test kit showed that 91.7% N, 89.6% P and 93.8% K results of modified digital soil test kit were matched with analytical laboratory method. Thus, the reliability of NPK results using digital soil test kit was enhanced, making the kit incorporable with the controller of variable rate fertilizer applicator through remote monitoring based data acquisition system. The outcome of this research can be used in the development of an IoT network data fusion for dynamic assessment of the NPK variation in the soil and nutrient management in oil palm plantations. Keywords: digital soil test kit, variable rate fertilizer applicator, oil palm, NPK measurement, data acquisition system, colorimetric method DOI: 10.25165/j.ijabe.20201304.5694 Citation: Yamin M, Ismail W I W, Kassim M S M, Aziz S B A, Akbar F N, Shamshiri R R, et al. Modification of colorimetric method based digital soil test kit for determination of macronutrients in oil palm plantation. Int J Agric & Biol Eng, 2020; 13(4): 188–197.

Improving Search Quality in Crowdsourced Bib Number Tagging Systems Using Data Fusion

Today, crowd computing is successfully applied for many information processing problems in a variety of domains. One of the most acute issues with crowd-powered systems is the quality of results (as humans can make errors). Therefore, a number of methods have been proposed to process the results obtained from the crowd in order to compensate human errors. Most of the existing methods of processing contributions are constructed based on a (natural) assumption that the only information available is unreliable data obtained from the crowd. However, in some cases, additional information is available, and it can be utilized in order to improve the overall quality of the result. The paper describes a crowd computing application for community tagging of running race photos. It presents a utility analysis to identify situations in which community photo tagging is a reasonable choice. It also proposes a data fusion model making use of runners’ location information recorded in their Global Positioning System (GPS) tracks. Field experiments with the applications show that community-based tagging can collect enough contributors to process photosets from medium-sized running events. Simulation results confirm, that the use of data fusion in processing the results of crowd computing is a promising technique, and the use of probabilistic graphical models (e.g., Bayesian networks) for data fusion allows one to smoothly increase the quality of the results with an increase of the available information.

Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors

Gesture recognition using machine-learning methods is valuable in the development of advanced cybernetics, robotics and healthcare systems, and typically relies on images or videos. To improve recognition accuracy, such visual data can be combined with data from other sensors, but this approach, which is termed data fusion, is limited by the quality of the sensor data and the incompatibility of the datasets. Here, we report a bioinspired data fusion architecture that can perform human gesture recognition by integrating visual data with somatosensory data from skin-like stretchable strain sensors made from single-walled carbon nanotubes. The learning architecture uses a convolutional neural network for visual processing and then implements a sparse neural network for sensor data fusion and recognition at the feature level. Our approach can achieve a recognition accuracy of 100% and maintain recognition accuracy in non-ideal conditions where images are noisy and under- or over-exposed. We also show that our architecture can be used for robot navigation via hand gestures, with an error of 1.7% under normal illumination and 3.3% in the dark. A bioinspired machine-learning architecture can combine visual data with data from stretchable strain sensors to achieve human gesture recognition with high accuracy in complex environments.

Automatic crack recognition for concrete bridges using a fully convolutional neural network and naive Bayes data fusion based on a visual detection system

Regular inspections of bridge substructures are very important for evaluating bridge health, since early detection and assessment offer the best chances of bridge repair. However, the traditional inspection methods of checking defects with visual features cannot meet engineering needs sufficiently. Although deep-learning methods have recently demonstrated a remarkable improvement in image classification and recognition, there are still difficulties, such as the countless parameters and large model training sets needed bythese methods. In this paper, we propose a novel crack extraction algorithm for automatic segmentation of cracks and noise using multi-layer features extracted from a fully convolutional network and a naive Bayes data fusion (NB-FCN) model. The bridge images in both the training and testing datasets are taken using an in-house designed high-precision image acquisition device, called Bridge Substructure Detection 10 (BSD-10). BSD-10 is applied to collect 7200 images from ten existing bridges under different illuminants and distances. After gathering the crack datasets, the crack and noise models of the NB-FCN are trained, respectively, with multiple iterations. Next, the skeleton and continuous boundary of a crack are recognized. Then the crack length and width are calculated using electronic distance measurement to verify the error rate of the proposed method. Compared to up-to-date machine-learning-based algorithms, i.e. the crack tree algorithm, the random structured forests algorithm, the relatively competitive convolutional neural networks algorithm, and the fusion convolutional neural network algorithm, the significant superiority of the NB-FCN algorithm in terms of recognition accuracy, computation time, and error rates is illustrated based on different types of crack images of handwriting, peel off, water stains and repair traces. The NB-FCN algorithm is verified with 7200 datasets of bridge substructures collected from 20 in-service bridges under various circumstances. In general, the recognition results show that the proposed algorithm demonstrates a remarkable performance compared to other recent algorithms.

The Analysis of Electronic Circuit Fault Diagnosis Based on Neural Network Data Fusion Algorithm

Symmetries play very important roles in the dynamics of electrical systems. The relevant electronic circuits with fault diagnostics, including the optimized neural network algorithm model, are designed on the basis of symmetry principles. In order to improve the efficiency of the circuit pressure test, a circuit pressure function equivalent compression test method based on the parallel neural network algorithm is proposed. For the implementation stage of the circuit pressure test, the improved modified node algorithm (MNA) is used to build an optimization model, and the circuit network is converted into an ordinary differential equation for the circuit pressure function equivalent compression test. The test aims to minimize flux. Then, backpropagation (BP) neural network algorithm data fusion is introduced to optimize the minimum flux model of the cyclic pressure functional equivalent compression test. Finally, a simulation experiment is carried out to verify the effectiveness of the algorithm in the accuracy and efficiency of the pressure test. The results show that the improved BP neural network improves the data fusion accuracy and shortens the sample training time; compared with the uncompressed algorithm, the running time of the proposed algorithm is greatly reduced and the execution efficiency is high; compared with the vascular pressure test method, there is no significant difference in the convergence accuracy and it is at a level of 10−5. Since the parallel computing problem is not considered in either of the two-pulse tube pressure test methods, the convergence time of the algorithm increases exponentially with the increase in the number of parallel threads. However, the algorithm in this research considers the problem of parallel execution and uses a quad-core processor, with no significant change in computing time and high computing efficiency. Therefore, BP neural network data fusion can be used for the fault diagnosis of electronic circuits, with a high operating efficiency and good development prospects.