Heterogeneous Data Sources Research Articles

An NLP-based framework for early identification of design reliability issues from heterogeneous automotive lifecycle data

Natural Language Processing is increasingly used in different areas of design and product development with varied objectives, from enhancing productivity to embedding resilience into systems. In this paper, we introduce a framework that draws on NLP algorithms and expert knowledge for the automotive engineering domain, to extract actionable insight for system reliability improvement from data available from the operational phase of the system. Specifically, we are looking at the systematic exploration and exploitation of automotive heterogeneous data sources, including both closed-source (such as warranty records) and open-source (e.g., social networks, chatrooms, recall systems) data, to extract and classify information about faults, with predictive capability for early detection of issues. We present a preliminary NLP-based framework for enhancing system knowledge representation to increase the effectiveness and robustness of information extraction from data, and discuss the temporal alignment of data sources and insight to improve prediction ability. We demonstrate the effectiveness of the proposed framework using real-world automotive data in a recall study for a vehicle lighting system and a particular manufacturer: four recall campaigns were identified leading to corrective actions by the warranty experts.

Research on optimization of e-commerce supply chain logistics service model based on multi-source data fusion

Abstract Inefficiencies in supply chain logistics management, particularly in certain regions, often result in “broken chain” incidents that significantly impact the efficiency of e-commerce operations and degrade consumer experiences. This paper addresses the necessity of optimizing the logistics service model by integrating 12 heterogeneous data sources related to warehousing, transportation, processing, and sorting. We propose a site selection-inventory-path e-commerce logistics optimization model. For the first time, the model incorporates the Dempster-Shafer (D-S) evidence algorithm to quantitatively evaluate the support relationships among various pieces of evidence by adjusting the weight coefficients accordingly. This adjustment is further refined through the application of the Nash equilibrium in the evidence combination process. The optimized logistics model is then solved using a fusion of the model and an intuitive fuzzy set approach. A case study demonstrates that the optimized logistic supply capacity consistently outperformed the non-optimized scenarios, with statistical increases of 1.64%, 1.72%, 2.15%, 3.73%, 2.71%, 2.46%, 3.59%, 2.17%, and 3.98%. Furthermore, by refining the logistics model, the gap in logistics performance was narrowed by at least 5.13% and by as much as 12.9% across successive evaluations, significantly alleviating issues related to enterprise capacity constraints. This study offers novel insights and methodologies for enhancing the efficiency of regional e-commerce supply chain logistics, boosting the competitiveness of logistics enterprises, and elevating the consumer experience in e-commerce settings.

Combination of Process Mining and Causal Discovery Generated Graph Models for Comprehensive Process Modeling

The extraction of process knowledge and its use for modeling and subsequent analysis is a very valuable approach to optimizing processes. Process Mining (PM) methods are widely used knowledge extraction techniques based on process knowledge recorded in event logs. One shortcoming is that PM approaches cannot easily access data in non-event log format, and secondly, that models are generated using directly-follows relations, which in the case of processes with concurrent activities means that process transitions may not be modeled correctly, and the resulting models do not correspond to reality as well as being unnecessarily complex. In addition, increasing digitization in manufacturing enables an ever greater collection of various data sources throughout the process with varying levels of information. Besides PM, in recent years great progress has been made in learning causal structures via causal discovery (CD) as well as in explicitly using additional expert knowledge. We argue that CD and expert knowledge are valuable additions to existing PM based approaches, as they allow knowledge extraction and modeling on different process levels. To combine these levels of information, this paper uses a manufacturing use case to show heterogeneous data sources and expert knowledge are used to create hybrid models. The results show that the combination of data-driven modeling methods with expert knowledge help to compensate for the weaknesses of the individual methods and achieve better overall results.

Data Lake Management System based on Topic Modeling

In an environment full of competitiveness, data is a valuable asset for any company looking to grow. It represents a real competitive economic and strategic lever. The most reputable companies are not only concerned with collecting data from heterogeneous data sources, but also with analyzing and transforming these datasets into better decision-making. In this context, the data lake continues to be a powerful solution for storing large amounts of data and providing data analytics for decision support. In this paper, we examine the intelligent data lake management system that addresses the drawbacks of traditional business intelligence, which is no longer capable of handling data-driven demands. Data lakes are highly suitable for analyzing data from a variety of sources, particularly when data cleaning is time-consuming. However, ingesting heterogeneous data sources without any schema represents a major issue, and a data lake can easily turn into a data swamp. In this study, we implement the LDA topic model for managing the storage, processing, analysis, and visualization of big data. To assess the usefulness of our proposal, we evaluated its performance based on the topic coherence metric. The results of these experiments showed our approach to be more accurate on the tested datasets

Classification of Urban Surface Elements by Combining Multisource Data and Ontology

The rapid pace of urbanization and increasing demands for urban functionalities have led to diversification and complexity in the types of urban surface elements. The conventional approach of relying solely on remote sensing imagery for urban surface element extraction faces emerging challenges. Data-driven techniques, including deep learning and machine learning, necessitate a substantial number of annotated samples as prerequisites. In response, our study proposes a knowledge-driven approach that integrates multisource data with ontology to achieve precise urban surface element extraction. Within this framework, components from the EIONET Action Group on Land Monitoring in Europe matrix serve as ontology primitives, forming a shared vocabulary. The semantics of surface elements are deconstructed using these primitives, enabling the creation of specific descriptions for various types of urban surface elements by combining these primitives. Our approach integrates multitemporal high-resolution remote sensing data, network big data, and other heterogeneous data sources. It segments high-resolution images into individual patches, and for each unit, urban surface element classification is accomplished through semantic rule-based inference. We conducted experiments in two regions with varying levels of urban scene complexity, achieving overall accuracies of 93.03% and 97.35%, respectively. Through this knowledge-driven approach, our proposed method significantly enhances the classification performance of urban surface elements in complex scenes, even in the absence of sample data, thereby presenting a novel approach to urban surface element extraction.

Assessing conceptual comparability of single-item survey instruments with a mixed-methods approach

An increasing number of research projects and infrastructure services involve pooling data across different survey programs. Creating a homogenous integrated dataset from heterogeneous source data is the domain of ex-post harmonization. The harmonization process involves various considerations. However, chief among them is whether two survey measurement instruments have captured the same concept. This issue of conceptual comparability is a fundamental precondition for pooling different source variables to form a harmonized target variable. Our paper explores this issue with a mixed-methods approach. On the one hand, we use psychometric latent variable modeling by presenting several single-item wordings for social trust to respondents and then performing factor analytic procedures. On the other hand, we complement and contrast these quantitative findings with qualitative findings gained with an open-ended web probe. The combined approach gave valuable insights into the conceptual comparability of the eleven social-trust-related single-item wordings. For example, we find that negative, distrust-related wordings and positive, trust-related wordings should not be pooled into an integrated variable. However, the paper will also illustrate and discuss why it is easier to disprove conceptual comparability than fully prove it.

Human_SNP_TATAdb: a database of SNPs that statistically significantly change the affinity of the TATA-binding protein to human gene promoters: genome-wide analysis and use cases

Human_SNP_TATAdb: a database of SNPs that statistically significantly change the affinity of the TATA-binding protein to human gene promoters: genome-wide analysis and use cases

A communication platform demonstrator for new generation railway traffic management systems: Testing and validation

Current rail traffic management and control systems cannot be easily upgraded to the new needs and challenges of modern railway systems because they do not offer interoperable data structures and standardized communication interfaces. To meet this need, the Horizon 2020 Shift2Rail OPTIMA project has developed a communication platform for testing and validating the new generation of traffic management systems (TMS), whose main innovative features are the interoperability of the data structures used, standardization of communications, continuous access to real-time and persistent data from heterogeneous data sources, modularity of components and scalability of the platform. This paper presents the main components, their functions and characteristics, then describes the testing and validation of the platform, even when federated with other innovative TMS modules developed in separate projects. The successful validation of the system has confirmed the achievement of the objectives set and allowed a new set of objectives to be defined for the reference platform for the railway TMS/Traffic Control systems.

A semantics-enabled approach for personalised Data Lake exploration

The increasing availability of Big Data is changing the way data exploration for Business Intelligence is performed, due to the volume, velocity and uncontrolled variety of data on which exploration relies. In particular, data exploration is required in Data Lakes that have been proposed to host heterogeneous data sources, given their flexibility to cope with cumbersome properties of Big Data. However, as data grows, new methods and techniques are required for extracting value and knowledge from data stored within Data Lakes, aggregating data into indicators according to multiple analysis dimensions, to enable a large number of users with different roles and competencies to capitalise on available information. In this paper, we propose PERSEUS (PERSonalised Exploration by User Support), a computer-aided approach for data exploration on top of a Data Lake, structured over three phases: (1) the construction of a semantic metadata catalog on top of the Data Lake, leveraging tools and metrics to ease the annotation of the Data Lake metadata; (2) modelling of indicators and analysis dimensions, guided by an openly available Multi-Dimensional Ontology to enable conformance checking of indicators and let users explore Data Lake contents; (3) enrichment of the definition of indicators with personalisation aspects, based on users’ profiles and preferences, to make easier and more usable the exploration of data for a large number of users. Results of an experimental evaluation in the Smart City domain are presented with the aim of demonstrating the feasibility of the approach.

An Aggregated Shape Similarity Index: A Case Study of Comparing the Footprints of OpenStreetMap and INSPIRE Buildings

The mutual identification of spatial objects is a fundamental issue when updating geographic data with other data sets. Representations of spatial objects in different sources may not have the same identifiers, which would unambiguously assign them to each other. Intersections of spatial objects can be used for this purpose, but this does not allow for the detection of possible changes and their quantification. The aim of this paper is to propose a simple, applicable procedure for calculating the shape similarity measure, which should be able to efficiently identify different representations of spatial objects in two data sources, even though they may be changed or generalised. The main result is the aggregated index of shape similarity and instructions for its calculation and implementation. The shape similarity index is based on the calculation of the set similarity, the distance of the boundaries, and the differences in the area, perimeter, and number of the vertices of areal spatial objects. In the case study, the footprints of the building complexes in Dúbravka (part of the city of Bratislava, the capital of Slovakia) are compared using data from OpenStreetMap and INSPIRE (Infrastructure for Spatial Information in Europe) Buildings. A contribution to the quality check of the OpenStreetMap data is then a secondary result. The proposed method can be effectively used in the semi-automatic integration of heterogeneous data sources, updating the data source with other spatial data, or in their quality control.

Automated Knowledge Integration from Heterogeneous Data Sources Using Text Analytics: A Case Study of COVID-19

Gathering information from multiple data sources takes a long time to collect, analyze, and classify. Furthermore, if the data sources have different data structures, the merged data structure must support such heterogeneity. In addition, semantic of data must also be considered. This paper proposes automated knowledge integration from heterogeneous data sources using ontology engineering combined with text analytics. Text stemming is used to preprocess data. Part-of-speech (POS) tagging, Universal Dependencies (UD), and text similarity measurement called cosine similarity are used to analyze and integrate data. Our work focuses on five COVID- 19 knowledge scopes: COVID-19, coronaviruses, diseases, pandemics, and vaccines. For evaluation, six ontologies were constructed with six different cosine similarity values ranging from 0.5 to 1.0. Each constructed ontology has COVID-19 related and non-COVID-19 data in a ratio of 70 to 30. The six constructed ontologies were evaluated for consistency with the original data. Using cosine similarity with 0.6, precision, recall, and F1-score are 0.82, 0.71, and 0.76, respectively, and the constructed ontology is optimal, containing the highest amount of relevant COVID-19 information for this case study.

A data-driven global observatory addressing worldwide challenges through text mining and complex data visualisation.

Observing the world on a global scale can help us understand better the context of problems that engage us all. In this paper, we propose a data-driven global observatory methodology that puts together the different perspectives of media, science, statistics and sensing over heterogeneous data sources and text mining algorithms. We also discuss the implementation of this global observatory in the context of epidemic intelligence, monitoring the impact of the COVID-19 pandemic, and in the context of climate change, with a specific focus on water resource management. Moreover, we discuss the value of this global solution in local contexts and priorities, based on the exchange with stakeholders in municipalities, utilities and governmental institutions.

CONNECTED: leveraging digital twins and personal knowledge graphs in healthcare digitalization.

Healthcare has always been a strategic domain in which innovative technologies can be applied to increase the effectiveness of services and patient care quality. Recent advancements have been made in the adoption of Digital Twins (DTs) and Personal Knowledge Graphs (PKGs) in this field. Despite this, their introduction has been hindered by the complex nature of the context itself which leads to many challenges both technical and organizational. In this article, we reviewed the literature about these technologies and their integrations, identifying the most critical requirements for clinical platforms. These latter have been used to design CONNECTED (COmpreheNsive and staNdardized hEalth-Care plaTforms to collEct and harmonize clinical Data), a conceptual framework aimed at defining guidelines to overcome the crucial issues related to the development of healthcare applications. It is structured in a multi-layer shape, in which heterogeneous data sources are first integrated, then standardized, and finally used to realize general-purpose DTs of patients backed by PKGs and accessible through dedicated APIs. These DTs will be the foundation on which smart applications can be built.

NetMUG: a novel network-guided multi-view clustering workflow for dissecting genetic and facial heterogeneity.

Introduction: Multi-view data offer advantages over single-view data for characterizing individuals, which is crucial in precision medicine toward personalized prevention, diagnosis, or treatment follow-up. Methods: Here, we develop a network-guided multi-view clustering framework named netMUG to identify actionable subgroups of individuals. This pipeline first adopts sparse multiple canonical correlation analysis to select multi-view features possibly informed by extraneous data, which are then used to construct individual-specific networks (ISNs). Finally, the individual subtypes are automatically derived by hierarchical clustering on these network representations. Results: We applied netMUG to a dataset containing genomic data and facial images to obtain BMI-informed multi-view strata and showed how it could be used for a refined obesity characterization. Benchmark analysis of netMUG on synthetic data with known strata of individuals indicated its superior performance compared with both baseline and benchmark methods for multi-view clustering. The clustering derived from netMUG achieved an adjusted Rand index of 1 with respect to the synthesized true labels. In addition, the real-data analysis revealed subgroups strongly linked to BMI and genetic and facial determinants of these subgroups. Discussion: netMUG provides a powerful strategy, exploiting individual-specific networks to identify meaningful and actionable strata. Moreover, the implementation is easy to generalize to accommodate heterogeneous data sources or highlight data structures.

AUTOMATIC GENERATION OF ROUTING GRAPHS FOR INDOOR-OUTDOOR TRANSITIONAL SPACE TO SUPPORT SEAMLESS NAVIGATION

Abstract. With the fast development of urbanization, the complexity of built environments has dramatically increased, driving a need for assistance in seamless indoor-outdoor navigation. This requires integration of spatial information of indoor and outdoor environments from heterogeneous data sources. While outdoor road network data is largely available from many sources (such as OpenStreetMap), indoor spatial information is either inexistent or is inconsistently represented using several different standards. Among these standards, IndoorGML is a well-developed standard with the focus on indoor location-based services. This standard has already been accepted by Open Geospatial Consortium (OGC) and is now under active development. Although in IndoorGML some mechanisms have been defined to enable integration of indoor and outdoor networks, there is still a lack of concrete guidelines for determination of indoor-outdoor connections. It also lacks solid scientific foundations and efficient tools to extract the connecting nodes and edges that link indoor and outdoor spaces. To address this gap, in this study we focus on the connection of indoor and outdoor spaces and aim to provide a tool, which can automatically construct navigation graphs of the indoor-outdoor transitional space to support seamless integration of indoor-outdoor navigation. To this end, voxel-based modeling approaches are used to model the connecting space between indoor and outdoor environments. Based on Python, we develop the intended tool, which can generate voxel models from point clouds, identify navigable space by taking into account the characteristics of agents (such as pedestrians, wheelchairs, and vehicles), and automatically build navigation graphs linking IndoorGML networks with outdoor street networks. It is expected that the methodology and tools developed from this project will benefit the IndoorGML ecosystem and greatly advance the capability of IndoorGML in representing navigable space to support location-based services.

Knowledge graph‐based multimodal neural networks for smart‐grid defect detection

AbstractThe application of artificial intelligence in smart grid operations, particularly for defect detection, has seen remarkable progress. However, effectively leveraging multi‐modal data sources to enhance defect detection performance remains challenging. This paper presents a novel Knowledge Graph‐based Multimodal Neural Network (KGMNN) approach for robust and efficient defect detection in smart grids. Our proposed method first leverages multiple heterogeneous data sources, such as structural data, temporal sensor readings, and maintenance records, creating a comprehensive representation of the grid system. The system is then modeled as a knowledge graph, with nodes representing various entities and edges signifying their interconnections and interactions. This paper introduces a novel neural network architecture that effectively leverages the knowledge graph structure. This architecture incorporates multi‐modal fusion techniques, enabling the model to process and integrate diverse data sources. It can recognize intricate patterns indicative of potential defects that are often undetectable in individual data modes. Experimental results on several real‐world smart grid datasets demonstrate that the KGMNN significantly outperforms state‐of‐the‐art techniques in terms of precision, recall, and computational efficiency. Furthermore, the KGMNN's ability to provide explainable predictions, aided by the inherent interpretability of the knowledge graph structure, is of significant practical value in the field. This research paves the way for a new generation of AI‐powered defect detection in smart grids, facilitating enhanced grid reliability and operational efficiency.

Using Data Fabric Architecture to Create Personalized Visual Analytics Systems in the Field of Digital Medicine

The paper describes an innovative approach of applying data factory technologies, which have found their successful practical application to create new generations of corporate data warehouses, are proposed to be used to develop personalized visual analytics environments. The approach provides access to digital data of a particular person from both heterogeneous distributed external and local sources not on the principles of their consolidation, but on the principles of virtual integration. The application of the described approach is demonstrated by the example of the development of visual analytics tools in the field of digital medicine. The emphasis is on the technological aspects of applying the concept of data factories for the development of personalized visual analytics services, means of semantic integration of heterogeneous data sources and their adaptation not only to heterogeneous structured and unstructured data sources, but also to the semantic content of various subject areas.

Using retrieval-augmented generation to elevate low-code developer skills

This article proposes applying retrieval-augmented generation (RAG) to improve the skills of low-code developers by augmenting large language models with up-to-date domain-specific knowledge. As low-code development requires combining multiple systems into a final product, developers must consult several sources of documentation and various articles, videos, and forum threads. Such a process may be time-consuming, prompting the use of an LLM for the authoritative answer. However, LLMs often lack knowledge of low-code platforms, leading to hallucinations and superficial responses. RAG utilizes the benefits of LLMs on relevant information, suggesting a presumption that it may be effectively applied in low-code development. Heterogeneous data sources concerning low-code systems are converted to a text representation, split into logical chunks, and stored in a vector database. During the exploitation of the model, cosine similarity is used to retrieve top-K documents and concatenate them with user query, using the produced text as a prompt to an LLM. The results support the hypothesis that RAG models outperform standard LLMs in knowledge retrieval in this domain

Unsupervised machine learning for risk stratification and identification of relevant subgroups of ascending aorta dimensions using cardiac CT and clinical data

The potential of precision population health lies in its capacity to utilize robust patient data for customized prevention and care targeted at specific groups. Machine learning has the potential to automatically identify clinically relevant subgroups of individuals, considering heterogeneous data sources. This study aimed to assess whether unsupervised machine learning (UML) techniques could interpret different clinical data to uncover clinically significant subgroups of patients suspected of coronary artery disease and identify different ranges of aorta dimensions in the different identified subgroups. We employed a random forest-based cluster analysis, utilizing 14 variables from 1170 (717 men/453 women) participants. The unsupervised clustering approach successfully identified four distinct subgroups of individuals with specific clinical characteristics, and this allows us to interpret and assess different ranges of aorta dimensions for each cluster. By employing flexible UML algorithms, we can effectively process heterogeneous patient data and gain deeper insights into clinical interpretation and risk assessment.

Clustering pipeline for vehicle behavior in smart villages

Smart cities and villages present a plethora of opportunities for fusing and managing multi-source data. However, in the analysis of mobility patterns, the use of only one data source (i.e., road sensors) without considering other contextual data sources, limits the understanding of the process. To address this gap, we propose a pipeline that integrates multiple data sources, providing valuable information for pattern extraction, mainly based on vehicle mobility behavior and provenance. Our research also highlights the critical role of selecting the appropriate normalization algorithm to scale input features from heterogeneous data sources, which has not received sufficient attention in the literature. We conducted our analysis using data from four License Plate Recognition (LPR) cameras, spanning nine months, and incorporating several databases that include provenance, gross income, and holiday information, resulting in a dataset of over 50,000 vehicles. Using this data and our clustering pipeline, we identified various traffic patterns among residents and visitors in a rural touristic area. Our findings assist data analysts in choosing algorithms for analyzing heterogeneous datasets. Moreover, policymakers could use our results to adjust the resources, such as new parking zones.