Multi-source Heterogeneous Data Research Articles

Research on Lithology Identification based on Machine Learning

Machine learning has great potential in lithology identification. Through supervised learning, unsupervised learning, semi-supervised learning, deep learning and other methods, features can be automatically extracted from complex seismic data and logging data to achieve efficient and accurate lithology classification. These methods not only improve the accuracy and efficiency of lithology identification, but also reduce the workload of geologists, allowing them to focus on higher-level analysis and decision making. Despite significant progress, machine learning still faces many challenges in lithology identification. First, data quality and quantity limitations remain a major problem, especially in certain areas where obtaining high-quality seismic and logging data is difficult. Second, the training and interpretability of complex models also need to be addressed, especially because the "black box" nature of deep learning models makes it difficult for geologists to understand their internal mechanisms and predictions. In addition, the generalization ability and overfitting of models, as well as the need for real-time data processing in practical applications, are also urgent challenges to be solved. To address these challenges, this paper proposes several solutions and future directions. Data enhancement and synthesis techniques can extend existing data sets and improve the robustness and accuracy of models. The development of interpretative models and visualization tools helps geologists understand and trust the decision-making process of models. Multi-source data fusion technology can effectively use multi-source heterogeneous data such as seismic data, well logging data and geological map to improve model performance. Online learning and transfer learning technologies can update models in real time, improve the adaptability and generalization ability of models, and develop more accurate and interpretable models.

A multi-source heterogeneous data fusion method for intelligent systems in the Internet of Things

The advent of the Internet of Things (IoT) has revolutionized the field of intelligent system development by providing an extensive amount of data from IoT devices, essential for the management of these systems and the creation of innovative services. This data covers various aspects, including creation at the physical layer, transmission through the network layer, and processing within the application layer. This study presents a groundbreaking approach to amalgamating multi-source and varied data within intelligent systems leveraging IoT technology. Our approach seeks to optimize the integration of diverse datasets by examining the correlations between different data types using a novel mixed information gain strategy, leading to effective data fusion. It capitalizes on the computational and storage capacities of systems for seamless integration and augments the analysis of information, thereby improving the useability of data in intelligent systems. Simulation tests confirm the superiority of our method, demonstrating a remarkable improvement in performance in the fusion of dynamic, multi-source heterogeneous data compared to conventional techniques.

Fault Classification of Rolling Bearings Based on Multisource Heterogeneous Data

Fault Classification of Rolling Bearings Based on Multisource Heterogeneous Data

Application Effect Evaluation of Intelligent Community Service Platform in Grassroots Governance

The purpose of this study is to design an intelligent community service platform applied in grassroots governance. First of all, this paper uses a distributed framework to build a service platform to provide data services for the intelligent community. Secondly, intelligent machine learning models are used to accelerate network training for community data classification tasks. At the same time, the high-precision association mechanism based on multi-source heterogeneous big data is introduced to store different types of topic data into the corresponding topic database, so as to realize the fusion processing of structured, semi-structured and unstructured data. Four kinds of account information are set up, including platform administrators, community administrators, community service units and community residents, to meet the diverse needs of community residents. The simulation test shows that the intelligent community service platform passes the function test, and the maximum change of response time is less than 1%, which proves the feasibility of applying the intelligent community service platform in grass-roots governance.

Multi source data security protection of smart grid based on edge computing

In order to cope with the continuous development of information technology, the data volume of edge devices in the power supply network is increasing rapidly, and the higher requirements for real-time data processing and transmission bandwidth are put forward, the author proposes the research on security protection of multi-source data in smart grid based on edge computing. Established a distribution network safety risk map, calculated and analyzed safety risks through potential functions, and obtained node potential values and key node data; The intelligent grid multi-source heterogeneous data monitoring model based on edge computing can realize the timely perception and real-time response of distribution network faults, shorten the outage time, and improve the power supply reliability and user satisfaction of the distribution network. The results indicate that: Taking a distribution network line in a certain area as an example, the highest potential value is node 9, followed by nodes 10 and 3. This indicates that in order to ensure the reliable operation of the distribution network and avoid power outages, it is necessary to improve the repair capacity, the proportion of old equipment, and the power supply radius exceeding the standard. ConclusionThe effectiveness of potential function in risk assessment of edge computing is verified, which can provide theoretical guidance for distribution network fault diagnosis.

Forecasting Hand-Foot-and-Mouth Disease in Qingdao City Based on CNN-BiLSTM-Attention Model

Purpose:A deep learning model CNN-BiLSTM-Attention was proposed, which realized the prediction of the outbreak rule and trend of HFMD in Qingdao, and could provide scientific basis for the relevant departments to formulate preventive measures..Methods:Based on the data of temperature, humidity and HFMD in Qingdao city, a multi-source heterogeneous data set was constructed and a long-term time series prediction model was proposed based on attention mechanism, which can predict the future incidence trend of HFMD. Then, ablation experiments were performed to verify the importance of each module, and comparison experiments with infectious disease dynamic model (SEIIeQR) and statistical model (ARIMA, SARIMA) proved the accuracy of the proposed model..Results:The ablation experiments demonstrated that the CNN module,Attention module,and BiLSTM module could effectively enhance the model's performance.The comparative experiment results revealed the superiority of the deep learning model in predicting HFMD outbreaks in Qingdao City.Conclusion:The multi-source heterogeneous dataset incorporating average temperature and relative humidity data significantly improved the model's accuracy.The CNN-BiLSTM-Attention model outperforms traditional methods in predicting HFMD incidence in Qingdao City,thereby assisting relevant departments in Qingdao to adopt scientific preventive measures against HFMD.

Research on the Construction of a Digital Twin System for the Long-Term Service Monitoring of Port Terminals

Structural damage is a prevalent issue in long-term operations of harbor terminals. Addressing the lack of transparency in terminal infrastructure components, the limited integration of sensor monitoring data, and the insufficient support for feedback on service performance, we propose a novel digital twin system construction methodology tailored for the long-term monitoring of port terminals. This study elaborates on the organization and processing of foundational geospatial data, sensor monitoring information, and oceanic hydrometeorological data essential for constructing a digital twin of the terminal. By mapping relationships between physical and virtual spaces, we developed comprehensive dynamic and static models of terminal facilities. Employing a “particle model” approach, we visually represented oceanic and meteorological elements. Additionally, we developed a multi-source heterogeneous data fusion model to facilitate the rapid creation of data indexes for harbor elements under high concurrency conditions, effectively addressing performance issues related to scene-rendering visualization and real-time sensor data storage efficiency. Experimental validation demonstrates that this method enables the rapid construction of digital twin systems for port terminals and supports practical application in business scenarios. Data analysis and comparison confirm the feasibility of the proposed method, providing an effective approach for the long-term monitoring of port terminal operations.

Automatic generation of system model diagrams driven by multi-source heterogeneous data

The demand for functionality, performance, safety, and reliability in complex product domains is continuously increasing, leading to growing product complexity. Traditional document-based systems engineering approaches face numerous challenges in terms of communication efficiency, traceability, and maintenance when dealing with complex product system design. Consequently, Model-Based Systems Engineering (MBSE) has been widely adopted. To improve the efficiency of system modelling, an effective method is needed to automate the process by reusing existing design resources. This study proposes a knowledge graph-based approach for the automatic generation of system model diagrams, consisting of three main steps. Firstly, a system model graph is constructed using an existing system model repository. Then, the required modelling elements are extracted from design documents, and reusable model elements are obtained based on the system model graph. Lastly, a mapping relationship is established between system model elements extracted from multi-source heterogeneous data and SysML metamodel, enabling the automatic generation of system model diagrams based on SysML views. This approach effectively leverages the information from the existing system model repository and achieves automated mapping conversion from multi-source heterogeneous data to the SysML system model. A communication satellite case study is presented to demonstrate the capability of this method.

Automatic construction and optimization method of enterprise data asset knowledge graph based on graph attention network

Traditional knowledge graph construction methods often rely on a large amount of human intervention and specialized knowledge, which seems inefficient and error-prone when dealing with massive, multi-source, and heterogeneous data assets of enterprises. To address this issue, a Multi Neighborhood Awareness Network (MNAN) model was proposed, which combines a multi language pre training model based on Bidirectional Encoder Representations from Transformers (BERT) and a Graph Convolutional Neural Network (GCN). By introducing an improved attention mechanism to deeply explore the neighborhood characteristics of entities, the accuracy and efficiency of entity matching are enhanced. In addition, to improve the integrity of the knowledge graph, a dynamic graph attention network (RDGAT) model based on graph attention network fusion relationship features is proposed. Through dynamic attention mechanism and relationship fusion strategy, deep learning of relationship features is carried out to optimize the completion performance of the knowledge graph spectrum. The performance test results show that the Hits@1 metric of the multi-neighborhood perceptual network model is close to 90% as the proportion of aligned entity seeds increases to 50%; the average inverse rank of the dynamic graph attention network on the test dataset improves by 6.3%. The experimental results show that the research-proposed knowledge graph fusion and complementation model exhibits significant effectiveness in automatically identifying and integrating large-scale, multi-source heterogeneous data at the enterprise level, and can significantly improve the automation level and accuracy of knowledge graph construction. The research method has achieved significant results in the data asset knowledge graph of petroleum enterprises and other fields, providing strong support for applications such as intelligent search, recommendation systems, and semantic analysis.

A quick response data collection and management system for digital twin production line based on cloud-fog-edge computing collaboration

Background A production line is the basic unit of smart factories and smart manufacturing. However, owing to the development of the industrial Internet of Things, sensors, and other technologies, more data are being collected, leading to a data explosion, and the heterogeneous nature of multiple sources makes it difficult to manage data in a unified manner. Methods A production line data collection, storage, and management system based on cloud-fog-edge computing collaboration and a digital twin was designed. Multi-source heterogeneous data were collected and transmitted based on the OPC UA, and an information model of the production line was established. Modules for data mapping, publishing, and receiving were developed to achieve unified data collection and transmission. The data storage and management platform was constructed by front-end and back-end separation technologies. Results The developed data collection and management system was verified for functionality and performance on a digital twin production line. Functional tests show that the system has the functions of data acquisition and transmission, device addition and viewing, device data querying and downloading, data and model visualization, and user rights setting. The average time for edge data collection and transmission is 183.6ms. The average response time of the cloud for fog requests is less than 1s. This shows that the system can satisfy the real-time requirements of a digital twin production line. Conclusions The proposed system is real-time and stable, providing support for big data and virtual-reality interaction in digital twins.

Toward Few-Label Vertical Federated Learning

Federated Learning (FL) provides a novel paradigm for privacy-preserving machine learning, enabling multiple clients to collaborate on model training without sharing private data. To handle multi-source heterogeneous data, Vertical Federated Learning (VFL) has been extensively investigated. However, in the context of VFL, the label information tends to be kept in one authoritative client and is very limited. This poses two challenges for model training in the VFL scenario. On the one hand, a small number of labels cannot guarantee to train a well VFL model with informative network parameters, resulting in unclear boundaries for classification decisions. On the other hand, the large amount of unlabeled data is dominant and should not be discounted, and it is worthwhile to focus on how to leverage them to improve representation modeling capabilities. To address the preceding two challenges, we first introduce supervised contrastive loss to enhance the intra-class aggregation and inter-class estrangement, which is to deeply explore label information and improve the effectiveness of downstream classification tasks. Then, for unlabeled data, we introduce a pseudo-label-guided consistency mechanism to induce the classification results coherent across clients, which allows the representations learned by local networks to absorb the knowledge from other clients, and alleviates the disagreement between different clients for classification tasks. We conduct sufficient experiments on four commonly used datasets, and the experimental results demonstrate that our method is superior to the state-of-the-art methods, especially in the low-label rate scenario, and the improvement becomes more significant.

A Review of Multi-source Heterogeneous Twin Data Processing Methods in Traffic Scenes

As the complexity of intelligent public transportation systems continues to increase and the degree of interconnection of equipment at all levels gradually deepens, building a digital twin system will generate a large amount of multi-source heterogeneous data. Effective processing and in-depth mining of data can provide real-time and intelligent decision-making optimization for the construction of digital twin systems, thereby efficiently handling complex emergencies. This paper conducts a systematic review on the processing methods and technologies of multi-source heterogeneous twin data in traffic dynamic scenarios. First, the content and classification of multi-source heterogeneous twin data in the operation process of intelligent bus systems are clarified; secondly, the multi-source The data processing methods and technologies applied in each stage of data collection, data storage, data fusion and data analysis in heterogeneous twin data processing are analyzed, and the advantages, disadvantages and applications of various methods and technologies are analyzed; finally, corresponding to the multiple This paper summarizes the methods and technologies for processing multi-source heterogeneous twin data, and points out the challenges and development trends faced by the current multi-source heterogeneous twin data processing methods and technologies.

Multi-source heterogeneous blockchain data quality assessment model for enterprise business activities.

Blockchain-based applications are becoming more and more widespread in business operations. In view of the shortcomings of existing enterprise blockchain evaluation methods, this paper proposes a multi-source heterogeneous blockchain data quality evaluation model for enterprise business activities, so as to achieve efficient evaluation of business activity information consistency, credibility and value. This paper proposes a multi-source heterogeneous blockchain data quality assessment method for enterprise business activities, aiming at the problems that most of the data in enterprise business activities come from different data sources, information representation is inconsistent, information ambiguity between the same block chain is serious, and it is difficult to evaluate the consistency, credibility and value of information. The method firstly proposes an entity information representation method based on the Representation learning for fusing entity category information (CEKGRL) model, which introduces the triad structure of related entities in blockchain, then associates them with enterprise business activity categories, and carries out similarity calculation through contextual information to achieve blockchain information consistency assessment. After that, a trustworthiness characterization method is proposed based on information sources, information comments, and information contents, to obtain the trustworthiness assessment of the business. Finally, based on the information trustworthiness characterization, a value assessment method is introduced to assess the total value of business activity information in the blockchain, and a blockchain quality assessment model is constructed. The experimental results show that the proposed model has great advantages over existing methods in assessing inter-block consistency, intra-block activity information trustworthiness and the value of blockchain.

APT-MMF: An advanced persistent threat actor attribution method based on multimodal and multilevel feature fusion

Threat actor attribution is a crucial defense strategy for combating advanced persistent threats (APTs). Cyber threat intelligence (CTI), which involves analyzing multisource heterogeneous data from APTs, plays an important role in APT actor attribution. The current attribution methods extract features from different CTI perspectives and employ machine learning models to classify CTI reports according to their threat actors. However, these methods usually extract only one kind of feature and ignore heterogeneous information, especially the attributes and relations of indicators of compromise (IOCs), which form the core of CTI. To address these problems, we propose an APT actor attribution method based on multimodal and multilevel feature fusion (APT-MMF). First, we leverage a heterogeneous attributed graph to characterize APT reports and their IOC information. Then, we extract and fuse multimodal features, including attribute type features, natural language text features and topological relationship features, to construct comprehensive node representations. Furthermore, we design multilevel heterogeneous graph attention networks to learn the deep hidden features of APT report nodes; these networks integrate IOC type-level, metapath-based neighbor node-level, and metapath semantic-level attention. Utilizing multisource threat intelligence, we construct a heterogeneous attributed graph dataset and various variant datasets for verification purposes. Extensive experimental results show that our method not only outperforms the existing methods, but also demonstrates its considerable robustness to incomplete and noise information and good explainability for attribution analysis tasks.

Unpacking the effects of R&D strategy proximity on inter-firm knowledge flow: evidence from the global 3D printing industry

ABSTRACT The determinants of inter-organisational knowledge flow have shifted from bilateral and static factors to network and dynamic factors. Therefore, this paper aims to explore the impact mechanism of firms’ dynamic behaviours on knowledge flow in a multiplex inter-firm strategy proximity network. The paper proposes a conceptual model to examine how the different research and development (R&D) dynamic strategies impact the inter-firm knowledge flow from a network-to-network lens, considering the moderating effect of the firm size heterogeneity. To test the hypotheses, we conducted an empirical study on 150 listed firms in the global 3D printing industry from 2005 to 2016 using multi-source heterogeneous data and Quadratic Assignment Procedure (QAP) regression model. Results show that the proximities of the R&D internationalisation strategy and R&D domain strategy are positively associated with the inter-firm knowledge flow. Firm size heterogeneity plays a moderating role, and these effects vary at different stages of industrialisation.

The consistent fuzzy suitability assessment of forest land resources with multi-source heterogeneous data.

In view of the suitability assessment of forest land resources, a consistent fuzzy assessment method with heterogeneous information is proposed. Firstly, some formulas for transforming large-scale real data and interval data into fuzzy numbers are provided. To derive the unified representation of multi-granularity linguistic assessment information, a fuzzy quantitative transformation for multi-granularity uncertain linguistic information is proposed. The proofs of the desirable properties and some normalized formulas for the trapezoidal fuzzy numbers are presented simultaneously. Next, the objective weight of each assessment indicator is further determined by calculating the Jaccard-Cosine similarity between the trapezoidal fuzzy numbers. Moreover, the trapezoidal fuzzy numbers corresponding to the comprehensive assessment values of each alternative are obtained. The alternatives are effectively ranked according to the distance from the centroid of the trapezoidal fuzzy number to the origin. Finally, based on the proposed consistent fuzzy assessment method, the suitability assessment of forest land resources is achieved under a multi-source heterogeneous data setting.

Semi-supervised urban haze pollution prediction based on multi-source heterogeneous data

Particulate matter (PM) is defined by the Texas Commission on Environmental Quality (TCEQ) as “a mixture of solid particles and liquid droplets found in the air”. These particles vary widely in size. Those particles that are less than 2.5 μm in aerodynamic diameter are known as Particulate Matter 2.5 or PM2.5. Urban haze pollution represented by PM2.5 is becoming serious, so air pollution monitoring is very important. However, due to high cost, the number of air monitoring stations is limited. Our work focuses on integrating multi-source heterogeneous data of Nanchang, China, which includes Taxi track, human mobility, Road networks, Points of Interest (POIs), Meteorology (e.g., temperature, dew point, humidity, wind speed, wind direction, atmospheric pressure, weather activity, weather conditions) and PM2.5 forecast data of air monitoring stations. This research presents an innovative approach to air quality prediction by integrating the above data sets from various sources and utilizing diverse architectures in Nanchang City, China. So for that, semi-supervised learning techniques will be used, namely collaborative training algorithm Co-Training (Co-T), who further adjusting algorithm Tri-Training (Tri-T). The objective is to accurately estimate haze pollution by integrating and using these multi-source heterogeneous data. We achieved this for the first time by employing a semi-supervised co-training strategy to accurately estimate pollution levels after applying the U-air system to environmental data. In particular, the algorithm of U-Air system is reproduced on these highly diverse heterogeneous data of Nanchang City, and the semi-supervised learning Co-T and Tri-T are used to conduct more detailed urban haze pollution prediction. Compared with Co-T, which train time classifier (TC) and subspace classifier (SC) respectively from the separated spatio-temporal perspective, the Tri-T is more accurate with a and faster because of its testing accuracy up to 85.62 %. The forecast results also present the potential of the city multi-source heterogeneous data and the effectiveness of the semi-supervised learning. We hope that this synthesis will motivate atmospheric environmental officials, scientists, and environmentalists in China to explore machine learning technology for controlling the discharge of pollutants and environmental management.

Fault diagnosis of axial piston pump based on multi-source subdomain adaptation and sensor data fusion

The axial piston pump is the core component in hydraulic systems. Its condition monitoring and fault diagnosis are crucial to ensure the safe and reliable operation of hydraulic systems. However, most of the existing fault diagnosis methods for axial piston pumps use the same working condition data. In actual operation, axial piston pump often experience varying loads, and the collected data is typically correlated but has different distributions. Therefore, a transfer learning method of multi-source subdomain adaptation and sensor fusion (MSASF) is proposed for fault diagnosis of axial piston pump. The proposed MSASF has three modules, a shared feature extraction module, a domain-specific feature extraction module and an output decision module. Firstly, the adaptive weighted fusion of multi-sensor data features is realized by the shared feature extraction module and the common features of multi-source heterogeneous data are extracted. Secondly, in the domain-specific feature extraction module, multi-branch network is used to extract features of each pair of source and target domains and the local maximum mean difference is utilized to align the sub-domain distribution of each pair of source and target domains. Finally, in the output decision module, the distribution distance between each pair of source domain and target domain is calculated using maximum mean discrepancy to obtain its weighted score. Combined with the classification output of each source domain, the final diagnosis decision is made. A dataset was constructed for the axial piston pump design fault experiment, and four sets of transfer tasks were designed to compare with those of seven classic methods. The experimental results showed that the proposed MSASF method exhibits a superior domain adaptation effect and fault diagnosis performance.

An interoperable software system to store, associate, visualize, and publish global open science data of earth surface system

Multi-source heterogeneous, multi-modal, and multi-type open scientific data (e.g., thematic sharing sites, metadata, journal articles, etc.) on earth surface systems (EES) provide important data sources for knowledge mining, discovery, and accurate recommendations, and also pose increasing challenges, resulting in the need to develop appropriate tools to address these challenges and support decision-making. This paper constructs an interoperable software system to store, visualize, and publish open science data of ESS. Utilizing an open scientific data catalogue repository encompassing EES information as foundational input and employing an integrated modeling methodology, this system endeavors to synthesize heterogeneous surface data of diverse linguistic, sourced, and typological origins. The objective is to facilitate multidimensional data retrieval and precise data auto-recommendation, thereby fostering the dissemination of scientific data and facilitating value-added services within EES domain. The tool may be used by stakeholders including researchers, data analysts, policymakers and national authorities to support decision-making on questions ranging from locating the location of open data related to the topic, to discovering high-quality data, selecting the data with the better overall evaluation. Along with a description of the system/platform design process, its structure, and the constituent models, key results are presented relating to the user interface, and several application examples. Software systems can help modelers to use the best features of a single software tool to answer open scientific data-related questions that seek to discovery, use, comparison or synthesis within or across topics of ESS.

Multi-source heterogeneous data fusion prediction technique for the utility tunnel fire detection

Diverse and complex fire environment in modern utility tunnels with multiple uncertainties make fire detection difficult to be achieved accurately. This study aims to develop an intelligent fire detection technique to address the difficulty. In the technique, initially, a lightweight image segmentation method is proposed for initial estimation of the fire source location. Then, the multi-source heterogeneous data fusion fire detection is implemented for fire source localization and ceiling temperature distribution prediction based on Gauss model and the improved multi-particle swarm optimization (MPSO) algorithm. Additionally, the results of the case study support the ability of the intelligent fire detection technique through compared with the experiment results and the previous methods, which can be used to achieve precise and stable fire source localization and ceiling temperature prediction in the utility tunnel fire.