Multi-source Heterogeneous Data Research Articles

Graph Feature Fusion-Driven Fault Diagnosis of Complex Process Industrial System Based on Multivariate Heterogeneous Data

The stable operation of the process industrial system, which is integrated with various complex equipment, is the premise of production, which requires the condition monitoring and diagnosis of the system. Recently, the continuous development of deep learning (DL) has promoted the research of intelligent diagnosis in process industry systems, and the sensor system layout has provided sufficient data foundation for this task. However, these DL-driven approaches have had some shortcomings: (1) the output signals of heterogeneous sensing systems existing in process industry systems are often high-dimensional coupled and (2) the fault diagnosis model built from pure data lacks systematic process knowledge, resulting in inaccurate fitting. To solve these problems, a graph feature fusion-driven fault diagnosis of complex process industry systems is proposed in this paper. First, according to the system’s prior knowledge and data characteristics, the original multisource heterogeneous data are divided into two categories. On this basis, the two kinds of data are converted to physical space graphs (PSG) and process knowledge graphs (PKG), respectively, according to the physical space layout and reaction mechanism of the system. Second, the node features and system spatial features of the subgraphs are extracted by the graph convolutional neural network at the same time, and the fault representation information of the subgraph is mined. Finally, the attention mechanism is used to fuse the learned subgraph features getting the global-graph representation for fault diagnosis. Two publicly available process chemistry datasets validate the effectiveness of the proposed method.

How to measure and control indoor air quality based on intelligent digital twin platforms: A case study in China

In order to address the current disparity between indoor air quality assessment, prediction, and control, this study proposes an intelligent management approach for indoor air quality based on digital twin platforms. By collecting and fusing multi-level, and multi-dimensional data at the physical level, a comprehensive data integration and analysis platform is established in the digital realm, encompassing the ‘Sense-Integrate-Retrieve-Analyze-Feedback’ framework. Various digital technologies, such as the Internet of Things (IoT), Building Information Modeling (BIM), and machine learning, are utilized to integrate and visualize multi-source heterogeneous data through a lightweight web-based interface so that data retrieval, rapid prediction, and intelligent control are feasible based on various backend algorithm models. Compared to traditional measurement methods, this system offers broader spatiotemporal coverage, increased mobility, and less disruption to physical infrastructure, which underscores the sophistication of monitoring technology, data integration, and environmental management. The proposed intelligent platform has been validated in a traditional Chinese dwelling, and practical results demonstrate that most air quality problems occur in the early morning of winter while this system can swiftly predict and issue control signals for adjustments in occupied spaces exceeding predefined healthy thresholds. Field measurements confirmed that most air quality problems can be solved within 30 min after their appearance. By eliminating data silos through real-time synchronization and remote intelligent control via the digital platform, it bridges the gap between problem identification by managers and problem resolution by end-users, ensuring timely responses to indoor air quality concerns.

Study on Spatio-Temporal Patterns of Commuting under Adverse Weather Events: Case Study of Typhoon In-Fa

This study focuses on the main urban area of Yangzhou City and conducts a quantitative comparative analysis of traffic accessibility during normal weather and extreme precipitation conditions (typhoon) based on GPS trajectories of buses. From both temporal and spatial dimensions, it comprehensively examines the impact of extreme precipitation on bus travel speed, travel time, and the commuting range of residents in the main urban area of Yangzhou City. (1) Through the mining and analysis of multi-source heterogeneous big data (bus GPS trajectory data, bus network data, rainfall remote sensing data, and road network data), it is found that the rainstorm weather greatly affects the average speed and travel time of buses. In addition, when the intensity of heavy rainfall increases (decreases), the average bus speed and travel time exhibit varying degrees of spatio-temporal change. During the morning and evening rush hour commuting period of rainstorm weather, there are obvious differences in the accessibility change in each typical traffic community in the main urban area of Yangzhou city. In total, 90% of the overall accessibility change value is concentrated around −5 min~5 min, and the change range is concentrated around −25~10%. (2) To extract the four primary traffic districts (Lotus Pond, Slender West Lake, Jinghua City, and Wanda Plaza), we collected Points of Interest (POI) data from Amap and Baidu heat map, and a combination analysis of the employment–residence ratio model and proximity methods was employed. The result show that the rainstorm weather superimposed on the morning peak hour has different degrees of impact on the average speed of the above-mentioned traffic zones, with the most obvious impact on the Lotus Pond and the smallest impact on Wanda Plaza. Under the rainstorm weather, the traffic commute in the main urban area of Yangzhou in the morning and evening peak hour is basically normal. The results of this paper can help to quantify the impact of typhoon-rainstorm weather events on traffic commuting in order to provide a scientific basis for the traffic management department to effectively prevent traffic jams, ensure the reliability of the road network, and allow the traffic management department to more effectively manage urban traffic.

FusionQuery: On-demand Fusion Queries over Multi-source Heterogeneous Data

Centralised data management systems (e.g., data lakes) support queries over multi-source heterogeneous data. However, the query results from multiple sources commonly involve between-source conflicts, which makes query results unreliable and confusing and degrades the usability of centralised data management systems. Therefore, resolving the between-sourced conflicts is one of the most important problems for centralised data management systems. To solve it, many batch data fusion-based methods have been proposed, which require traversing all the data in the centralised data management systems and cause scalability and flexibility issues. To address these issues, this paper explores the problem of on-demand fusion queries, where the between-sourced conflicts are solved with only the query-related data; moreover, we propose an efficient on-demand fusion query framework, FusionQuery, which consists of a query stage and a fusion stage. In the query stage, we frame the heterogeneous data query problem as a knowledge graph matching problem and present a line graph-based method to accelerate it. In the fusion stage, we develop an Expectation Maximization-style algorithm to iteratively updates data veracity and source trustworthiness. Furthermore, we design an incremental estimation method of source trustworthiness to address the lack of sufficient observations. Extensive experiments on two real-world datasets demonstrate that FusionQuery outperforms state-of-the-art data fusion methods in terms of both effectiveness and efficiency.

Multi-source heterogeneous data access management framework and key technologies for electric power Internet of Things

The power Internet of Things (IoT) is a significant trend in technology and a requirement for national strategic development. With the deepening digital transformation of the power grid, China’s power system has initially built a power IoT architecture comprising a perception, network, and platform application layer. However, owing to the structural complexity of the power system, the construction of the power IoT continues to face problems such as complex access management of massive heterogeneous equipment, diverse IoT protocol access methods, high concurrency of network communications, and weak data security protection. To address these issues, this study optimizes the existing architecture of the power IoT and designs an integrated management framework for the access of multi-source heterogeneous data in the power IoT, comprising cloud, pipe, edge, and terminal parts. It further reviews and analyzes the key technologies involved in the power IoT, such as the unified management of the physical model, high concurrent access, multi-protocol access, multi-source heterogeneous data storage management, and data security control, to provide a more flexible, efficient, secure, and easy-to-use solution for multi-source heterogeneous data access in the power IoT.

Research on Multi-Source Heterogeneous Big Data Fusion Method Based on Feature Level

With the development of research on multi-modal data fusion and its combination with online data management, the application of multi-modal big data fusion in information management systems is more and more extensive. How to integrate multi-modal big data effectively is the key technology to building an efficient information management system. In this paper, based on the combination of a multi-support vector machine and convolution neural network, the feature-level data fusion of multi-source heterogeneous big data is implemented, and it is applied to the real data set to test the relevant model. Experimental results show that this method can not only realize heterogeneous integration of big data, but also has high accuracy and reliability.

A Bayesian Deep Learning-Based Probabilistic Risk Assessment and Early-Warning Model for Power Systems Considering Meteorological Conditions

The ongoing process of decarbonizing the power system and the frequent occurrence of extreme weather have led to a significant increase in operating uncertainty and greatly reduced the system controllability. An effective risk assessment and early-warning tool will significantly assist system operators in monitoring and controlling power systems. However, existing methods mainly rely on simplified analytical conditional probability models to describe the component failure probability under different operating conditions and are not suitable for low-probability risk assessment. Inspired by the idea of Bayesian statistics, a Bayesian deep learning-based probabilistic risk assessment and early-warning (BDL-PRAEW) model considering meteorological conditions is proposed in this paper. A new Bayesian neural network (BNN) is proposed which efficiently utilizes expert experience and domain knowledge as prior to model the contingency probability. And a hybrid neural network is developed to rationally utilize the multi-source heterogeneous data and comprehensively analyze the historical and forecast information. Finally, a novel risk assessment and early-warning model for high-impact, low-probability (HILP) extreme events is proposed, which can predict the risk for the next n time-steps. The proposed method is tested on a modified IEEE 118-bus system and a real-world provincial grid, and the results verify its effectiveness.

Self-Supervised Multi-Source Heterogeneous Data Fusion Using Encode and Decode Attention for Intelligent Medical Device Communication Analysis

Self-Supervised Multi-Source Heterogeneous Data Fusion Using Encode and Decode Attention for Intelligent Medical Device Communication Analysis

A Job Recommendation Model Based on a Two-Layer Attention Mechanism

In the field of job recruitment, traditional recommendation methods only rely on users’ rating data of positions for information matching. This simple strategy has problems such as low utilization of multi-source heterogeneous data and difficulty in mining relevant information between recruiters and applicants. Therefore, this paper proposes a recurrent neural network model based on a two-layer attention mechanism. The model first improves the entity representation of recruiters and applicants through user behavior, company-related knowledge and other information. The entities and their combinations are then mapped to the vector space using one-hot and TransR methods, and a recurrent neural network with a two-layer attention mechanism is used to obtain their potential interests from the click sequence, and then a recommendation list is generated. The experimental results show that this model achieves better results than previous models.

Autonomous Data Association and Intelligent Information Discovery Based on Multimodal Fusion Technology

The effective association of multimodal data is the basis of massive multi-source heterogeneous data sharing in the era of big data. How to realize data autonomous association between massive multimodal databases and the automatic intelligent screening of valuable information from associated data, so as to provide a reliable data source for artificial intelligence (AI), is an urgent problem to be solved. In this paper, a data autonomous association method based on the organizational structure of data cells is proposed, including transaction abstraction based on information nucleuses, symmetric and asymmetric data association based on strategies and data pipes, and information generation based on big data. To screen meaningful data associations, an information-driven intelligent information discovery method and a task-driven intelligent information discovery method are proposed. The former screens meaningful data associations by training the reward and punishment model to simulate the manual scoring of data associations. The latter is task-oriented and screens meaningful data associations by training the reward and punishment model to simulate the manual ranking of data associations related to the task requests. Through the above work, autonomous data association and intelligent information discovery are effectively realized based on multimodal fusion technology, which provides a novel data source mining approach using multimodal data sharing and intelligent information discovery.

Assessing landslide susceptibility based on the random forest model and multi-source heterogeneous data

Landslides pose significant threats to human lives and property. The usefulness of mapping susceptibility in predicting these events, by providing early warning and implementing preventive measures, cannot be overstated. This study relies on the well-established grid management mechanism in Dengfeng and utilizes historical data collected from field surveys. Combining multi-source heterogeneous data obtained by aerospace technology, and using random forest model and feature selection, a landslide sensitivity assessment system based on space-ground collaboration was constructed. The study categorized areas according to their susceptibility to landslides using probabilistic forecasts. Specifically, regions were classified into four susceptibility categories: very low (landslide probabilities below 0.26), low (0.26 to 0.38), moderate (0.38 to 0.50), and high (above 0.50). This resulted in a comprehensive landslide susceptibility map that was rigorously analyzed and assessed. The findings indicate that: (1) high, moderate, low, and very low susceptibility areas for landslides in Dengfeng are 127.36, 527.49, 385.67, and 184.03 km2, respectively, accounting for 10.40 %, 43.08 %, 31.49 %, and 15.03 % of the total area. (2) The spatial distribution of landslides in Dengfeng exhibits a “multi-core clustering-radiate distribution” pattern, from the peripheral areas to the center. (3) The spatial distribution of high, moderate, low, and very low susceptibility areas for landslides is primarily influenced by factors such as distance from faults, elevation, and distance from rivers. The study has constructed a landslide sensitivity assessment system based on space-ground collaboration that can provide replicable research methods, provide scientific support for disaster reduction and prevention work, and reduce casualties and property losses caused by landslides.

Traffic Accident Risk Prediction of Tunnel Based on Multi-Source Heterogeneous Data Fusion

Traffic Accident Risk Prediction of Tunnel Based on Multi-Source Heterogeneous Data Fusion

Structured stochastic models based on multi-source heterogeneous data for predicting internal electricity load of non-residential buildings

Electricity load patterns in buildings are increasingly diverse and inherently present building-by-building and temporal variations. This study develops structured stochastic models to capture individual and temporal variations in non-residential building groups (e.g., education, office, culture, and retail buildings). Hourly electricity load data of non-residential buildings, categorized by building use, are sourced from two open datasets (Ministry of Land, Infrastructure and Transport and Korea Electric Power Corporation). First, heterogeneous public datasets are processed and utilized to provide both detailed building information and high-resolution electricity load data. Second, a structured set of four stochastic models are formulated using linear regression and multivariate normal distribution models. Model 1 utilizes the beta distribution to derive daily average electricity load per unit area for each building use. Model 2 utilizes the burr distribution to determine the daily variation of electricity load from the daily average. Models 3 and 4 utilize a multivariate normal distribution to represent the hourly average electricity load profile and its time variation, respectively. The coefficient of variation of the root mean square error (CVRMSE) for the structured probabilistic models ranges from 5.2 % to 14.8 % regarding five key load characterization variables. CVRMSEs for near-peak load, near-base load, rise time, high-load duration, and fall time are 8.5–13.9, 9.5–14.7, 7.3–14.8, 6.4–13.9, and 5.2–11.4 %, respectively. The validation results demonstrate that the proposed probabilistic models capture actual variations in measurements. Structured statistical models improve prediction performance by distinctly reflecting individual and temporal variations in electricity loads.

Quantification of micro-damage evolution process in ceramics through extensive analysis of multi-source heterogeneous data

Due to the randomness of hard and brittle material damage evolution, assessing damage and interpreting mechanisms is challenging. This study focuses on analyzing material damage mechanisms by employing extensive multi-source heterogeneous data to extract quantitative information. A machine vision-based method for detecting and assessing defects in ceramics was proposed, achieving 98 % accuracy. Leveraging this method in scratching experiments, the progression of damage and micro-failure mechanisms in Si3N4 was investigated through the analysis of defect area distribution, force and acoustical signals. Results revealed that the distribution of defect areas conforms to a three-parameter Weibull distribution. At scratch depths of 0.01, 0.02 mm, approximately 90 % of damage accumulated within 1e2–1e4 μm2; conversely, 0.03–0.06 mm led to a decrease to 30 – 40 %. Extensive crack propagation occurs within 0.02–0.03 mm, resulting in the intersecting formation of spalling blocks. Damage evolution occurs in three stages: shallow and small cracks with scattered spalling blocks in the first stage, deep and long cracks with continuous and irregularly shaped spalling blocks in the second stage, and nearly no cracks with continuous and regularly shaped blocks in the third stage. This study not only proposes an efficient method to detect and evaluate ceramic defects, but also deeply understands the evolution process of material damage mechanism. Considering defect distribution and statistical trends, this research not only provide guidance for the optimization of machining parameters, especially in the scene of specific precision and high removal rate. At the same time, the damage evolution and failure mechanism of Si3N4 provide useful experimental basis and data support for deepening the understanding of material behavior during ultra-precision grinding.

Selection and prediction of metro station sites based on spatial data and random forest: a study of Lanzhou, China

Urban economic development, congestion relief, and traffic efficiency are all greatly impacted by the thoughtful planning of urban metro station layout. with the urban area of Lanzhou as an example, the suitability of the station locations of the built metro stations of the rail transit lines 1 and 2 in the study area have been evaluated using multi-source heterogeneous spatial data through data collection, feature matrix construction, the use of random forest and K-fold cross-validation, among other methods. The average Gini reduction value was used to examine the contribution rate of each feature indicator based on the examination of model truthfulness. According to the study's findings: (1) K-fold cross-validation was applied to test the random forest model that was built using the built metro stations and particular factors. The average accuracy of the tests and out-of-bag data (OOB) of tenfold cross-validation were 89.62% and 91.285%, respectively. Additionally, the AUC area under the ROC curve was 0.9823, indicating that this time, from the perspective of the natural environment, traffic location, and social factors The 19 elements selected from the views of the urban function structure, social economics, and natural environment are closely associated to the locations of the metro station in the research region, and the prediction the findings are more reliable; (2) It becomes apparent that more than half of the built station sites display excellent agreement with the predicted sites in terms of geographical location by superimposing the built metro station sites with the prediction results and tally up their cumulative prediction probability values within the 300 m buffering zone; (3) Based on the contribution rate of each indicator to the model, transport facilities, companies, population density, night lighting, science, education and culture, residential communities, and road network density are identified as the primary influential factors, each accounting for over 6.6%. Subsequently, land use, elevation, and slope are found to have relatively lower contributions. The results of the research provided important information for the local metro's best location selection and planning.

A deep learning model for multi-modal spatio-temporal irradiance forecast

Short-term spatio-temporal solar irradiance forecasting plays a pivotal role in scheduling and dispatching energy for distributed energy systems. Fluctuations in cloud cover can be monitored via satellite cloud imagery, which directly impacts irradiance. However, integrating and fusing multi-source heterogeneous data, such as satellite cloud images and ground monitoring data from distributed stations, remains challenging. Here, a spatio-temporal irradiance forecast model is proposed based on multi-modal deep learning model to predict global horizontal irradiance 30 min ahead. To address the feature extraction of heterogeneous data, a dual-channel structure consisting of a time-series processing block and a satellite cloud image processing block is developed to enable parallel processing of multi-modal features. In order To tightly couple cloud images and historical time series at the feature level, maximum mean discrepancy of these two feature is used to help the fusion of heterogeneous data. Furthermore, a self-attention mechanism is employed to construct adaptive inter-region information weights to enhance spatio-temporal representation ability. The evaluation of the method is conducted on open-access datasets from six locations in Jiangsu Province, China. Experimental results demonstrate that the proposed model efficiently utilizes heterogeneous data to improve prediction accuracy under various conditions and enhances model robustness, reducing RMSE by 2.8%–20.58%. Meanwhile, the proposed end-to-end model reduces training and deployment costs for real-world use.

The Embodiment and Innovation of Digital Twin Platform in Modern Interior Environment Design

Abstract In this paper, the digital twin platform for the indoor environment is constructed by combining digital twin technology and modern indoor environment elements to innovate the modern indoor environment design method. The digital twin platform is designed for modern indoor environments in two aspects: data acquisition and 3D model visualization for indoor environments. The indoor environment data are collected, cleaned and quasi-exchanged using sensors, the collected multi-source heterogeneous data of the indoor environment are fused by the time alignment method, and the 3D model of the indoor environment is driven by the design of the 3D model’s operations of translation, rotation and scaling. On this basis, the performance of the indoor environment digital twin platform is analyzed, and the data acquisition method and driving effects of the 3D model are explored. The results show that the data transmission measurement delay is within 20ms, the display delay is within 70ms, the transmission frames per second are basically stabilized at about 200FPS, 100FPS, 60FPS, the accuracy reaches 0.9 in the case of multiple data acquisition, and the fusion speed is about 3.4m/s, and the success rate of the driving operation of the overall three-dimensional model of the indoor environment is all greater than 0.96.

Research on the Digital Twin System of the Centring Process for High-Precision Lens

In the manufacturing process of optical lenses, the lack of monitoring and detection of the central processing process leads to low processing efficiency and difficulty in ensuring product consistency. We propose a digital twin system for alignment processing to address this issue. The system adopts a hierarchical architecture based on the digital twin five-dimensional model, aiming to improve the fidelity and interactivity of the virtual model of the centring lathe by combining dimension-driven virtual models with integrated data and physical models of the turning mechanism. We have successfully achieved the semantic and physical fusion of multi-source heterogeneous data during centring processing using information models and OPC UA-based data interaction methods. In addition, we adopted the VMD-GRU method based on feature fusion for real-time monitoring of critical components of the centring lathe. Finally, we validated the feasibility and effectiveness of the digital twin system for the central lathe through development examples. The application of this system is expected to promote the digital and intelligent development of high-precision optical component processing, providing references including references for related manufacturing fields. In summary, we propose a digital monitoring and detection system for the centring process of optical lens manufacturing. The application of this system will help improve product consistency and processing efficiency while reducing risks and costs in the manufacturing process.

Establishing a dynamic and static knowledge model of the manufacturing cell management system: An active push approach

The workshop operation management (WOM) system contains a large amount of multi-source heterogeneous data, but these data often lack effective correlation, posing challenges in extracting valuable information for achieving unified system modelling at the knowledge level. Due to the lack of knowledge active support in the actual WOM system, managers could only passively rely on their own past experiences. To bridge this gap, the knowledge active push method for optimal operation management of the manufacturing cell (OOMMC) is studied based on knowledge graph (KG). Firstly, a dynamic knowledge model is established according to the knowledge application scenario of manufacturing cell, in which the Local Tangent Space Alignment (LTSA) + k-means algorithm is used for active abnormal warning. Secondly, a static knowledge model based on user needs using feature vocabulary and naive Bayesian classifier is established, where managers can obtain knowledge from the KG based on professional noun keywords or simple query questions. These two models complement and promote each other: the dynamic knowledge model using a service-based collaborative filtering method to push the nearest knowledges in the KG to users, effectively inspiring them to retrieve more relevant knowledge. While the static knowledge model evaluates the effectiveness of previously pushed knowledge by setting the knowledge demand index and providing feedback to the system to ensure accuracy of knowledge push. Finally, an active push simulation system is designed and developed to verify the feasibility and effectiveness of the proposed architecture and key enabling technologies.

Design and Implementation of Power Big Data Platform

In recent years, with the advancement of the national construction of smart grid and the development of power grid enterprise integration system, the traditional power data platform has some defect, such as insufficient scalability, repeated implementation of two sets of logic of off-line data warehouse and real-time data warehouse, highly correlated storage of data warehouse during computation, and difficulty in platform migration, which restrict the deeper mining and analysis of power big data. Therefore, this paper adopts a technical architecture incorporating distributed calculating, micro-service, flow-batch integration and lake-warehouse integration, combining the current popular Web front-end and back-end technologies such as Vue, SpringCloud and Flask, and utilizing big data components such as Hadoop, Flink, Hudi and Kafka as well as Docker containerization technology. We explore and build a power grid system operation oriented big data platform that assembles real-time computing, multi-source heterogeneous data storage, and develop APIs for authority management, information management, data analysis and visualization, machine learning and other data supporting services that assist power enterprises in managing electrical equipment and users’ power consumption.With the help of this platform, power enterprises can intelligently track the equipment status, view the offline and real-time visual charts to analyze the power consumption by users, so as to have better basis in the power dispatching, maintenance and scheme adjustment of electrical market.