Multi-source Heterogeneous Data Research Articles

Data Recognition for Multi-Source Heterogeneous Experimental Detection in Cloud Edge Collaboratives

This article proposes a multisource heterogeneous experimental data recognition method based on CRNN and DBNet in a cloud-edge collaborative environment in an attempt to address the issues of low efficiency and a high error rate that come with traditional manual data detection and recognition. Firstly, a recognition architecture for experimental detection data of intelligent measurement systems is designed based on a cloud-edge collaborative environment to improve the efficiency of data processing. Then, the improved DBNet network is used in the edge computing center to detect the text, and the correction module is used to correct the deviation of the detected text to ensure the standardization of the text. Finally, in the central cloud, the end-to-end indefinite length character recognition (CRNN) algorithm is used to analyze and identify the text order, rules, and other information of the image after the correction is completed, extract the test detection data, and convert the detection data into row data according to the two-dimensional table structure, and conduct structured storage and management through the relational database. An experimental analysis of the proposed method is conducted based on the deep learning framework, and results show that its accuracy rate and recall rate are close to 96% and 94%, respectively, with an average accuracy of 95.09%. This fully demonstrates the proposed method is effective and, therefore, applicable to power equipment experimental detection data recognition.

Design of a Sensor-Technology-Augmented Gait and Balance Monitoring System for Community-Dwelling Older Adults in Hong Kong: A Pilot Feasibility Study.

Routine assessments of gait and balance have been recognized as an effective approach for preventing falls by issuing early warnings and implementing appropriate interventions. However, current limited public healthcare resources cannot meet the demand for continuous monitoring of deteriorations in gait and balance. The objective of this study was to develop and evaluate the feasibility of a prototype surrogate system driven by sensor technology and multi-sourced heterogeneous data analytics, for gait and balance assessment and monitoring. The system was designed to analyze users' multi-mode data streams collected via inertial sensors and a depth camera while performing a 3-m timed up and go test, a five-times-sit-to-stand test, and a Romberg test, for predicting scores on clinical measurements by physiotherapists. Generalized regression of sensor data was conducted to build prediction models for gait and balance estimations. Demographic correlations with user acceptance behaviors were analyzed using ordinal logistic regression. Forty-four older adults (38 females) were recruited in this pilot study (mean age = 78.5 years, standard deviation [SD] = 6.2 years). The participants perceived that using the system for their gait and balance monitoring was a good idea (mean = 5.45, SD = 0.76) and easy (mean = 4.95, SD = 1.09), and that the system is useful in improving their health (mean = 5.32, SD = 0.83), is trustworthy (mean = 5.04, SD = 0.88), and has a good fit between task and technology (mean = 4.97, SD = 0.84). In general, the participants showed a positive intention to use the proposed system in their gait and balance management (mean = 5.22, SD = 1.10). Demographic correlations with user acceptance are discussed. This study provides preliminary evidence supporting the feasibility of using a sensor-technology-augmented system to manage the gait and balance of community-dwelling older adults. The intervention is validated as being acceptable, viable, and valuable.

Fault diagnosis strategy of industrial process based on multi-source heterogeneous information and deep learning

With the progress of technology, modern industrial monitoring data not only includes traditional process data but also includes video data. In order to make full use of these multi-source heterogeneous data to achieve better accuracy of fault classification, feature extraction and feature fusion of different modalities is vital important. For feature extraction, because the contribution rate of process data at different times is different to fault classification, in order to obtain the temporal features that have a great impact on fault classification, the Batch Normalization Long Short-Term Memory based on Self-Attention mechanism (BN-LSTM-SA) is used for feature extraction of process data. Since temporal and spatial features are important indicators of video classification, the Two Stream Shifted Windows 3D Convolution Transformer (TSSCT) model is used to extract temporal and spatial features. Cross Multi-head Self-Attention(CMSA) model is designed for feature fusion, which fully utilize the correlation and complementarity of multi-source heterogeneous information and can reasonably allocate the weight of different modalities. Experiments show that our proposed method is superior to the unimodal model or the existing multimodal fusion model and prove the effectiveness of the proposed method.

A distributed multi-vehicle pursuit scheme: generative multi-adversarial reinforcement learning

Multi-vehicle pursuit (MVP) is one of the most challenging problems for intelligent traffic management systems due to multi-source heterogeneous data and its mission nature. While many reinforcement learning (RL) algorithms have shown promising abilities for MVP in structured grid-pattern roads, their lack of dynamic and effective traffic awareness limits pursuing efficiency. The sparse reward of pursuing tasks still hinders the optimization of these RL algorithms. Therefore, this paper proposes a distributed generative multi-adversarial RL for MVP (DGMARL-MVP) in urban traffic scenes. In DGMARL-MVP, a generative multi-adversarial network is designed to improve the Bellman equation by generating the potential dense reward, thereby properly guiding strategy optimization of distributed multi-agent RL. Moreover, a graph neural network-based intersecting cognition is proposed to extract integrated features of traffic situations and relationships among agents from multi-source heterogeneous data. These integrated and comprehensive traffic features are used to assist RL decision-making and improve pursuing efficiency. Extensive experimental results show that the DGMARL-MVP can reduce the pursuit time by 5.47% compared with proximal policy optimization and improve the pursuing average success rate up to 85.67%. Codes are open-sourced in Github.

Developing a cloud evidence method for dynamic early warning of tunnel construction safety risk in undersea environment

Traditional methods have limitations in achieving precise predictions of risk occurrence at an exact future time and have difficulties transforming between qualitative and quantitative indicators and handling multi-source heterogeneous risk data. This study quantifies and analyzes the multi-source construction safety risks classified into the categories of man, machine, material, method and environment (4M1E), and presents a cloud evidence method that integrates wavelet de-noising algorithm, cloud model, and Dempster-Shafer (D-S) evidence theory. A real-time risk prediction and warning is provided using this method after the fusion of multi-source uncertain information and the transformation between qualitative and quantitative indicators, enabling the timely detection of potential risks for project managers. This method analyzing “uncertainty” with “certainty” is verified by an undersea tunnel construction project. The result shows that this method is effective in early warning risks two days before their actual occurrence, providing reference significance for risk early warning of the tunnel construction project.

A Collaborative Emergency Drill System for Urban Tunnels Using BIM and an Agent-Based Model

With the rapid development of smart cities, the refined management of urban highway tunnels has put forward higher requirements for the emergency disposal ability of operation and maintenance personnel. This paper proposed a collaborative emergency drill system for urban tunnels using building information modeling (BIM) and an agent-based model. The objectives of this paper are as follows: (1) To help address the challenge of multi-person collaborative intelligent drills in complex emergency scenarios, this system constructed an emergency collaborative drill model and a virtual emergency scenario description method based on trait-based objects (TBOs). (2) To help address the challenge of the organization and integration of multi-source heterogeneous data in complex emergency scenarios, the system established an emergency scenario generation method through lightweight BIM data, standard emergency plan documents, and virtual emergency scenario data. The system was successfully applied to the Hongmei South Road Tunnel in Shanghai, China. The feasibility of the proposed system provided practical help for tunnel emergency management and was extended to other urban tunnels in Shanghai.

Spatio-Temporal Knowledge Graph-Based Research on Agro-Meteorological Disaster Monitoring

Currently, there is a wealth of data and expert knowledge available on monitoring agro-meteorological disasters. However, there is still a lack of technical means to organically integrate and analyze heterogeneous data sources in a collaborative manner. This paper proposes a method for monitoring agro-meteorological disasters based on a spatio-temporal knowledge graph. It employs a semantic ontology framework to achieve the organic fusion of multi-source heterogeneous data, including remote sensing data, meteorological data, farmland data, crop information, etc. And it formalizes expert knowledge and computational models into knowledge inference rules, thereby enabling monitoring, early warning, and disaster analysis of agricultural crops within the observed area. The experimental area for this research is the wheat planting region in three counties in Henan Province. The method is tested using simulation monitoring, early warning, and impact calculation of the past two occurrences of dry hot wind disasters. The experimental results demonstrate that the proposed method can provide more specific and accurate warning information and post-disaster analysis results compared to raw records. The statistical results of NDVI decline also validate the correlation between the severity of wheat damage caused by dry hot winds and the intensity and duration of their occurrences. Regarding remote sensing data, this paper proposes a method that directly incorporates remote sensing data into spatio-temporal knowledge inference calculations. By integrating remote sensing data into the regular monitoring process, the advantages of remote sensing data granted by continuous observation are utilized. This approach represents a beneficial attempt to organically integrate remote sensing and meteorological data for monitoring, early warning, and evaluation analysis of agro-meteorological disasters.

Road network pixelization: A traffic flow imputation method based on image restoration techniques

The complete traffic network data is crucial for accurate traffic data prediction in intelligent transportation systems. Inspired by the success of Generative Adversarial Networks (GANs) in image restoration, this study proposes a traffic flow imputation method by employing image restoration techniques. First, we propose a trajectory data representation method called Trajectory2Matrix, that converts the trajectory data into a two-dimensional spatiotemporal relation feature map. Consequently, the data imputation scale and scope are increased. Second, a spatiotemporal feature map generation module based on a graph convolutional network is designed to optimize the GAN generator, thus utilizing its advantages for non-Euclidean data and dynamic spatiotemporal correlation. Finally, a heterogeneous multisource data fusion module based on a channel attention mechanism is proposed to merge dynamic/static external attributes and multimode characteristics in time. The proposed method improved imputation accuracy and robustness for multitype missing data patterns, especially in high missing rate situations.

A Blockchain-Based Traceability Model for Grain and Oil Food Supply Chain.

The structure of the grain-and-oil-food-supply chain has the characteristics of complexity, cross-regionality, a long cycle, and numerous participants, making it difficult to maintain the safety of supply. In recent years, some phenomena have emerged in the field of grain procurement and sale, such as topping the new with the old, rotating grains, the pressure of grades and prices, and counterfeit oil food, which have seriously threatened grain-and-oil-food security. Blockchain technology has the advantage of decentralization and non-tampering Therefore, this study analyzes the characteristics of traceability data in the grain-and-oil-food-supply chain, and presents a blockchain-based traceability model for the grain-and-oil-food-supply chain. Firstly, a new method combining blockchain and machine learning is proposed to enhance the authenticity and reliability of blockchain source data by constructing anomalous data-processing models. In addition, a lightweight blockchain-storage method and a data-recovery mechanism are proposed to reduce the pressure on supply-chain-data storage and improve fault tolerance. The results indicate that the average query delay of public data is 0.42 s, the average query delay of private data is 0.88 s, and the average data-recovery delay is 1.2 s. Finally, a blockchain-based grain-and-oil-food-supply-chain traceability system is designed and built using Hyperledger Fabric. Compared with the existing grain-and-oil-food-supply chain, the model constructed achieves multi-source heterogeneous data uploading, lightweight storage, data recovery, and traceability in the supply chain, which are of great significance for ensuring the safety of grain-and-oil food in China.

A Job Recommendation Method Based on Attention Layer Scoring Characteristics and Tensor Decomposition

In the field of job recruitment, a classic recommendation system consists of users, positions, and user ratings on positions. Its key task is to predict the unknown rating data of users on positions and then recommend positions that users are interested in. However, traditional recommendation methods only rely on user rating data for jobs and provide recommendation services for recruiters and candidates through simple information matching. This simple recommendation strategy not only causes a lot of information waste but also cannot effectively utilize the multi-source heterogeneous data information in the field of job recruitment. Therefore, this paper proposes a job recommendation model based on users’ attention levels and tensor decomposition for specific recruitment positions. This model puts forward assumptions based on browsing time for the special behaviors and habits of users in the field of job recruitment, defines corresponding label values for different interactive behaviors, and establishes a grading method based on the attention of job seekers, thus constructing a three-dimensional tensor of “job seeker user-position-attention layered”. Then, a recommendation model is constructed by decomposing the three-dimensional tensor. The effectiveness of the model is verified by comparative experiments with other recommendation algorithms.

Multi-Modal Spatio-Temporal Knowledge Graph of Ship Management

In modern maritime activities, the quality of ship communication directly impacts the safety, efficiency, and economic viability of ship operations. Therefore, predicting and analyzing ship communication status has become a crucial task to ensure the smooth operation of ships. Currently, ship communication status analysis heavily relies on large-scale, multi-source heterogeneous data with spatio-temporal and multi-modal features, which presents challenges for ship communication quality prediction tasks. To address this issue, this paper constructs a multi-modal spatio-temporal ontology and a multi-modal spatio-temporal knowledge graph for ship communication, guided by existing ontologies and domain knowledge. This approach effectively integrates multi-modal spatio-temporal data, providing support for subsequent efficient data analysis and applications. Taking the scenario of fishing vessel communication activities as an example, the query tasks for ship communication knowledge are successfully performed using a graph database, and we combine the spatio-temporal knowledge graph with graph convolutional neural network technology to achieve real-time communication quality prediction for fishing vessels, further validating the practical value of the multi-modal spatio-temporal knowledge graph.

Multi-source heterogeneous data fusion model based on fuzzy mathematics

Sensors as the sensing end of intelligent control can be used to collect various data instead of human beings. In the context of technological development, the variety of sensors leads to multiple and structurally unequal data sources, and fusion of these data becomes a problem for consideration. The study constructs an intuitionistic fuzzy transformation method to handle data with various attributes with the help of fuzzy mathematical concepts, which characterizes the data based on the hesitancy and ideal solutions under Gaussian distribution. Simulations of classical classification data show that the intuitionistic fuzzy transformation method can effectively differentiate the affiliation of data points in the dataset, and the results of 800 simulations show that the qualitative accuracy of the algorithm can reach 89%, while the causes of abnormal data are explored and it is found that the attributes of the dataset based on Gaussian distribution are too close to each other as the cause of misclassification; the algorithm is also optimized from multi-dimensional considerations, and a An optimization operator based on the distance method of superior and inferior solutions was constructed and simulated for several optimization paths. The results show that the study uses an optimization scheme that is significantly better than the existing fuzzy operator, and 800 times can improve the accuracy rate up to 95.23%, which is 14.01% higher than that of a single attribute. This indicates that the intuitionistic fuzzy algorithm of this study has some rationality and is able to fuse the data of multiple attributes of the sensor for determination and provide the necessary basis for decision making.

Construction and application of knowledge graph for grid dispatch fault handling based on pre-trained model

With the construction of new power systems, the power grid has become extremely large, with an increasing proportion of new energy and AC/DC hybrid connections. The dynamic characteristics and fault patterns of the power grid are complex; additionally, power grid control is difficult, operation risks are high, and the task of fault handling is arduous. Traditional power-grid fault handling relies primarily on human experience. The difference in and lack of knowledge reserve of control personnel restrict the accuracy and timeliness of fault handling. Therefore, this mode of operation is no longer suitable for the requirements of new systems. Based on the multi-source heterogeneous data of power grid dispatch, this paper proposes a joint entity–relationship extraction method for power-grid dispatch fault processing based on a pre-trained model, constructs a knowledge graph of power-grid dispatch fault processing and designs, and develops a fault-processing auxiliary decision-making system based on the knowledge graph. It was applied to study a provincial dispatch control center, and it effectively improved the accident processing ability and intelligent level of accident management and control of the power grid.

An integrated risk assessment model for the multi-perspective vulnerability of distribution networks under multi-source heterogeneous data distributions

The reliability of distribution network transmission corridors are vulnerable to different risk factors in both internal operation parameters and surrounding environmental conditions, and thus pinpoint evaluations of these impacting factors can further enhance their safety. To this end, this paper proposes an integrated assessment method, that is the multi-dimensional assessment with integrated fuzzy association (MDATFA). In this model, healthiness and importance indices of transmission lines are both defined to reflect the self-robustness and potential damages, respectively, and therefore to enable a two-dimensional assessment to incorporate the multi-source heterogeneous inputs. Firstly, healthiness indices for environmental characteristics are analyzed. On the one hand, discrete input features are processed by the established association rule mining model with rare variables (ARMrv) to mine and diagnose rarely occurred elements, and their healthiness contribution values can then be calculated. On the other hand, continuous features are handled by the designed hierarchical probabilistic fuzzy systems (HPFS). The weight assignments of these two expert models are adjusted by a gating network. Secondly, the importance indices for electrical characteristics are calculated. Under the condition that electrical signal constraints are satisfied, the amount of load loss and lost users can be solved by the built distribution network fault recovery model which is according to heuristic rule and second-order cone (HRSO). Finally, the validity and robustness of this operational condition assessment model is verified by an empirical analysis in an actual 39-bus power distribution system.

A multimodal dual-fusion entity extraction model for large and complex devices

In the context of large and complex devices with a multi-source heterogeneous data environment, the extraction of network device configuration related entity information from diverse modalities of the Internet of Things data is a crucial and fundamental step towards establishing a domain knowledge graph for global Zero Touch Provisioning of network devices. In this paper, we present a novel multimodal dual-fusion entity extraction model that serves as a foundation for an intelligent and efficient network device configuration process. Firstly, the multimodal data is encoded, followed by the ViLBERT pre-training model to obtain more feature information of entities for multimodal front-end fusion. Next, the attention weights of each modal feature are learned through a multilayer neural network classifier and probabilistic graphical model, facilitating multimodal back-end fusion and reducing information redundancy. Finally, entity recognition is accomplished by employing a cohesive memory module that extracts the essential parameters for device configuration. The simulation results demonstrate that the proposed model performs exceptionally well on the MSCOCO2017 public dataset and the SFZK-Dev data network device dataset, with F1 values of the comprehensive evaluation index of model quality at 94.65% and 96.94%, respectively, indicating high stability. Additionally, the link prediction metrics HITS@1 achieved accuracy levels of 58.21% and 68.04%.

Retracted: Intelligent Prediction Model of CET-4 Passing Rate Based on Multisource Heterogeneous Data Mining

Retracted: Intelligent Prediction Model of CET-4 Passing Rate Based on Multisource Heterogeneous Data Mining

WDCIP: spatio-temporal AI-driven disease control intelligent platform for combating COVID-19 pandemic

ABSTRACT The outbreak and subsequent recurring waves of COVID −19 pose threats on the emergency management and people’s daily life, while the large-scale spatio-temporal epidemiological data have sure come in handy in epidemic surveillance. Nonetheless, some challenges remain to be addressed in terms of multi-source heterogeneous data fusion, deep mining, and comprehensive applications. The Spatio-Temporal Artificial Intelligence (STAI) technology, which focuses on integrating spatial related time-series data, artificial intelligence models, and digital tools to provide intelligent computing platforms and applications, opens up new opportunities for scientific epidemic control. To this end, we leverage STAI and long-term experience in location-based intelligent services in the work. Specifically, we devise and develop a STAI-driven digital infrastructure, namely, WAYZ Disease Control Intelligent Platform (WDCIP), which consists of a systematic framework for building pipelines from automatic spatio-temporal data collection, processing to AI-based analysis and inference implementation for providing appropriate applications serving various epidemic scenarios. According to the platform implementation logic, our work can be performed and summarized from three aspects: (1) a STAI-driven integrated system; (2) a hybrid GNN-based approach for hierarchical risk assessment (as the core algorithm of WDCIP); and (3) comprehensive applications for social epidemic containment. This work makes a pivotal contribution to facilitating the aggregation and full utilization of spatio-temporal epidemic data from multiple sources, where the real-time human mobility data generated by high-precision mobile positioning plays a vital role in sensing the spread of the epidemic. So far, WDCIP has accumulated more than 200 million users who have been served in life convenience and decision-making during the pandemic.

Creating Historical Building Models by Deep Fusion of Multi-Source Heterogeneous Data Using Residual 3D Convolutional Neural Network

ABSTRACT There are a large number of historical buildings in the world, but many of them are seriously damaged. A fundamental task to revitalize them is to record their details and redraw them, but it is a highly inefficient task. This paper aimed to create three-dimensional (3D) models automatically for those damaged historical buildings while respecting the 1964 Venice Charter, the ICOMOS recommendations, and local ordinances. After reviewing the advantages and disadvantages of various measurement tools, we implemented a machine learning algorithm, the residual 3D convolutional neural network (CNN) method, to combine heterogeneous data from multiple sources including a 3D laser scanner, an unmanned aerial vehicle, a total station, and a camera, in a historical building information model (HBIM) to increase its accuracy. Those data, preprocessed by four steps including data registration, reducing noises, filtering data, and data integration, were trained and fused in a 3D CNN model, which consisted of one input layer, five 3D convolutional and pooling layers, and two connected layers. A Shanghai historical building conservation project was taken as a pilot case to verify its productivity and accuracy in developing the HBIM, on which the architectural features were thoroughly recorded, and the damaged details were recovered.

Federated Generative Model on Multi-Source Heterogeneous Data in IoT

The study of generative models is a promising branch of deep learning techniques, which has been successfully applied to different scenarios, such as Artificial Intelligence and the Internet of Things. While in most of the existing works, the generative models are realized as a centralized structure, raising the threats of security and privacy and the overburden of communication costs. Rare efforts have been committed to investigating distributed generative models, especially when the training data comes from multiple heterogeneous sources under realistic IoT settings. In this paper, to handle this challenging problem, we design a federated generative model framework that can learn a powerful generator for the hierarchical IoT systems. Particularly, our generative model framework can solve the problem of distributed data generation on multi-source heterogeneous data in two scenarios, i.e., feature related scenario and label related scenario. In addition, in our federated generative models, we develop a synchronous and an asynchronous updating methods to satisfy different application requirements. Extensive experiments on a simulated dataset and multiple real datasets are conducted to evaluate the data generation performance of our proposed generative models through comparison with the state-of-the-arts.

Understanding passenger travel choice behaviours under train delays in urban rail transits: a data-driven approach

The analysis of passenger travel choice behaviours under train delays has become a crucial topic in research on urban rail transit operation management. In this paper, we focus on analysing travel choices of affected regular passengers under train delays by utilizing the data collected through an automatic fare collection (AFC) system along with train delay log records. Along this line, we propose a data-driven four-stage framework for studying regular passengers’ responses under delays, consisting of data profiling, regular passenger screening and travel patterns extraction, affected regular passenger identification, and affected passenger behaviour prediction modelling. Using a real-world case of the Shenzhen Metro in China, we conduct extensive experiments for method validation and feature insights analysis. The proposed framework could provide a microscopic view of passenger travel behaviours under train delays for fine prediction and exhibit a possibility for multi-source heterogeneous data mining in passenger behaviour analysis and train delay-related tasks.