Automated Data Integration Research Articles

Discrete-time survival analysis in the critically ill: a deep learning approach using heterogeneous data

Prediction of survival for patients in intensive care units (ICUs) has been subject to intense research. However, no models exist that embrace the multiverse of data in ICUs. It is an open question whether deep learning methods using automated data integration with minimal pre-processing of mixed data domains such as free text, medical history and high-frequency data can provide discrete-time survival estimates for individual ICU patients. We trained a deep learning model on data from patients admitted to ten ICUs in the Capital Region of Denmark and the Region of Southern Denmark between 2011 and 2018. Inspired by natural language processing we mapped the electronic patient record data to an embedded representation and fed the data to a recurrent neural network with a multi-label output layer representing the chance of survival at different follow-up times. We evaluated the performance using the time-dependent concordance index. In addition, we quantified and visualized the drivers of survival predictions using the SHAP methodology. We included 37,355 admissions of 29,417 patients in our study. Our deep learning models outperformed traditional Cox proportional-hazard models with concordance index in the ranges 0.72–0.73, 0.71–0.72, 0.71, and 0.69–0.70, for models applied at baseline 0, 24, 48, and 72 h, respectively. Deep learning models based on a combination of entity embeddings and survival modelling is a feasible approach to obtain individualized survival estimates in data-rich settings such as the ICU. The interpretable nature of the models enables us to understand the impact of the different data domains.

Matching Large Biomedical Ontologies Using Symbolic Regression Using Symbolic Regression

The problem of ontology matching consists of finding the semantic correspondences between two ontologies that, although belonging to the same domain, have been developed separately. Ontology matching methods are of great importance today since they allow us to find the pivot points from which an automatic data integration process can be established. Unlike the most recent developments based on deep learning, this study presents our research efforts on the development of novel methods for ontology matching that are accurate and interpretable at the same time. For this purpose, we rely on a symbolic regression model (implemented via genetic programming) that has been specifically trained to find the mathematical expression that can solve the ground truth provided by experts accurately. Moreover, our approach offers the possibility of being understood by a human operator and helping the processor to consume as little energy as possible. The experimental evaluation results that we have achieved using several benchmark datasets seem to show that our approach could be promising.

Automatic Integration of Clinical and Genetic Data Using cBioPortal.

Precision medicine seeks to improve the prevention, diagnosis and treatment of patients based on genetic characteristics unique to each person. In oncology, therapeutic decisions have been established based on the genomic characteristics of each patient's tumor. Data integration is key for the successful implementation of precision medicine since it is necessary for both studying a large volume of data from different sources and working with an interdisciplinary and translational vision. In this work, a bioinformatic process was successfully implemented that allows the integration of patients' genomic data, from two molecular biology laboratories, with their clinical data provided by their electronic medical records. For this, the REDCap data capture software, the cBioPortal visualization and analysis software, and a computer tool developed to automate the processing and annotation of the information in REDCap were used to be included in cBioPortal, for the "Map of Tumor Genomic Actionability of Argentina" project.

THE INTEGRATION OF THE MULTI-TEMPORAL CONSERVATION WORKS AND NON-INVASIVE MEASUREMENTS

Abstract. Heritage sites are affected by many factors that cause degradation and decay over time, such as soil deformation, changes in weather conditions, or excess moisture, which can cause damages to historical sites. For this reason, restoration works are planned and carried out to preserve the object's existing condition, renovate and repair the damaged parts, or restore the original condition of the object. The Rescue Project was undertaken in the Blessed Ladislas’ chapel in St. Anna's Church in Warsaw, Poland. The main goal of the planned work was to restore the original rococo character of the dome paintings. The aim of this article was to present the integration of the multi-temporal conservation work and non-invasive measurements for changes analysis during conservation works. For this purpose, point clouds acquired with a Z+F 5006h ground-based scanner and images captured via Structure-from-Motion and MultiView Stereo method implemented in Agisoft Metashape software were used. The proposed conservation analysis method was a multi-stage process that involved: (1) automatic data integration using a feature-based approach based on 2D detectors and point clouds converted to spherical images, (2) generalisation and harmonisation of source data, and (3) analysis of deviations between point clouds. Having registered point clouds with each other, it was possible to analyse changes that occurred during conservation work. Calculated distances between both point clouds show the localisation of cracks and damages and enable measure of the size and depth of crevices on the dome wall.

Semantic Integration of Heterogeneous Data Sources Using Ontology-Based Domain Knowledge Modeling for Early Detection of COVID-19.

The enormous outbreak of biomedical knowledge, the aim of reducing computation and processing costs and the widespread availability of internet connection have created a profuse amount of electronic data. Such data are stored across the globe in various data sources that are semantically, structurally and syntactically different. This decentralized nature of biomedical data has made it difficult to obtain a unified view of the data. Data integration plays a crucial role in enhancing access to heterogeneous data making the retrieval easier and faster. A variety of ontology, machine learning, deep learning and fuzzy logic-based solutions are being developed for heterogeneous data integration. The proposed model concentrates on the automatic ontology-based data integration method that can be effectively deployed and used in the healthcare domain. The proposed model is divided into three phases. The first phase includes the automatic mapping of data and generation of local ontology across heterogeneous data sources, the second phase combines the local ontology models developed in the first phase to create a root global schema mapping and the third phase queries diverse databases to retrieve semantically analogous records. The model is created based on the medical records, chest X-ray details and COVID-19 symptom questionnaire data of various patients distributed across three data sources (SQL, mongodb and excel). Based on the data, the patients who have moderate/higher risk of developing serious illness from COVID-19 are retrieved.

National high-resolution land cover and land use information system

ABSTRACT In the last few years, the Spanish Land Cover and Land Use Information System (SIOSE) is undergoing a methodological and productive evolution focused on achieving a high level of geometric, thematic, and temporal detail. This methodological advance is based on the reuse and integration of official reference information on land cover and land use, such as cadastre (1:500–1:5000), agricultural identification systems (1:5000), forest maps (1:25,000), or LiDAR data (0.5 p/m2). This article presents the work carried out by the Instituto Geográfico Nacional (Spanish National Geographic Institute), in coordination with the rest of the project partners, in the generation of a high-resolution SIOSE by automatic data integration and visual review. To carry out the latter, an online framework has been designed to facilitate the access to data and to guide reviewers in editing. Resulting integrated data offers a detailed description for all thematic domains on land cover and land use never seen before in Spain at the National level and advances in the land management on many application fields. High-resolution SIOSE provides information at 1:5000 for the majority of classes and ensures a complete thematic content with more than 140 classes structured in an object-oriented information system.

Disease risk analysis for schizophrenia patients by an automatic AHP framework

BackgroundBased on more than 15 million follow-up records of 404,426 patients from Guangdong Mental Health Center over the past 10 years, this study aims to propose a disease risk analysis and prediction model to support chronic disease management and clinical research for schizophrenia patients.MethodsBased on a mental health information and intelligent data processing platform, we design an automatic AHP framework called AutoAHP to analyze and predict the disease risks of schizophrenia patients. Through automatic extraction, transformation and integration of follow-up data in the real world such as demography, treatment, and the disease course, a chronic database of patient status is established. In combination with age-period-cohort, logistic regression and Cox models, we apply the AutoAHP to assess disease risk and implement risk prediction in practice.ResultsA list of essential factors for risk prediction are identified, including annual changes in mental health policy, public support, regional difference, patient gender, compliance, and social function. After the verification of 1,222,038 complete disease course and treatment records of 256,050 patients, the AutoAHP framework achieves a precision of 0.923, a recall of 0.924, and a F1 of 0.923. The model is demonstrated to be superior to general models and has better performance in risk prediction.ConclusionsAiming at the risk assessment of patients with schizophrenia which is influenced by factors, such as time, region and complication, the AutoAHP framework is able to be applied as a model in combination with logistic regression and Cox models to support clinical analysis of disease risk related factors and assist decision-making in chronic disease management.

A semi-automated, high throughput approach for O-glycosylation profiling of in vitro established cancer cell lines by MALDI-FT-ICR MS

The study of protein O-glycosylation is important in biological research as O-glycans have been reported to regulate a multitude of molecular and cell biology processes occurring in cancer. It is known that alterations in O-glycosylation are involved in the development and progression of cancer. Their easy accessibility makes in vitro established cell lines suitable and useful models for studying biological mechanisms in disease. However, the O-glycosylation analysis of large numbers of samples, as required in systems biology and biomarker discovery studies, is often challenging. In the present study, O-glycans from three human colorectal cancer cell lines and two human pancreatic cancer cell lines were released by semi-automated, high throughput reductive β-elimination and analysed using ultrahigh resolution MALDI-FT-ICR MS. Automated data integration and processing was performed using MassyTools, where the analyte was automatically included for relative quantitation based on a range of selection criteria including signal-to-noise ratio, mass error and isotopic pattern quality scores. A total of 126 O-glycan compositions, ranging from a single monosaccharide to large oligosaccharides exhibiting complex glycan motifs, were detected. The use of ultrahigh resolution MALDI-FTICR MS enabled glycan identification and quantitation in the matrix region of the spectrum. This approach has the potential to be used for O-glycosylation analysis of large numbers of samples, such as patient sample cohorts.

Secure and energy-efficient smart building architecture with emerging technology IoT

With the advent of the Internet-of-Things (IoT), it is considered to be one of the latest innovations that offer interesting opportunities for different vertical industries. One of the most relevant IoT technology areas is smart construction. IoT operates in several sectors on a daily basis; implementation includes smart building, smart grids, smart cities, smart houses, physical defense, e-health, asset, and transportation management, but it is not restricted to this. Support from smart IoT buildings is an IoT-level, connected, and cost-effective system. Commercial space has major requirements in terms of comfort, accessibility, security, and energy management. Such requirements can be served organically by IoT-based systems. As the supply of energy has been exhausted and energy demand has risen, there has been a growing focus on energy usage and the maintenance of buildings. With the use of evolving IoT technology, we present a secure and energy-efficient smart building architecture. Every device is known by its unique address, and one of the key web transfer protocols is the Constrained Application Protocol (CoAP). It is an application layer protocol that does not use protected channels for data transfer. Automatic key management, confidentiality, authentication, and data integrity are all features of the Datagram Transport Layer Protection (DTLS). To achieve energy efficiency, we propose a smart construction architecture that, through IoT, manages the performance of all technological systems. The results of the simulation show that the energy consumption is lowered by about 30.86% with the use of the CoAP in the smart building, which is less than the Message Queuing Telemetry Transport case (MQTT). This paper also aims to observe how to integrate the DTLS protocol with the Secure Hash Algorithm (SHA-256) using optimizations from the Certificate Authority (CA) to improve security.

Harmonizing heterogeneous multi-proxy data from lake systems

When performing spatial-temporal investigations of multiple lake systems, geoscientists face the challenge of dealing with complex and heterogeneous data of different types, structure, and format. To support comparability, it is necessary to transform such data into a uniform format that ensures syntactic and semantic comparability. This paper presents a data science approach for transforming research data from different lake sediment cores into a coherent framework. For this purpose, we collected published and unpublished data from paleolimnological investigations of Arctic lake systems. Our approach adapted methods from the database field, such as developing entity-relationship (ER) diagrams, to understand the conceptual structure of the data independently of the source. We demonstrated the feasibility of our approach by transforming our ER diagram into a database schema for PostgreSQL, a popular database management system (DBMS). We validated our approach by conducting a comparative analysis on a set of acquired data, hereby focusing on the comparison of total organic carbon and bromine content in eight selected sediment cores. Still, we encountered serious obstacles in the development of the ER model. Heterogeneous structures within collected data made an automatic data integration impossible. Additionally, we realized that missing error information hampers the development of a conceptual model. Despite the strong initial heterogeneity of the original data, our harmonized dataset leads to comparable datasets, enabling numerical inter-proxy and inter-lake comparison.

A Framework for the Automatic Integration and Diagnosis of Building Energy Consumption Data.

Buildings account for a majority of the primary energy consumption of the human society, therefore, analyses of building energy consumption monitoring data are of significance to the discovery of anomalous energy usage patterns, saving of building utility expenditures, and contribution to the greater environmental protection effort. This paper presents a unified framework for the automatic extraction and integration of building energy consumption data from heterogeneous building management systems, along with building static data from building information models to serve analysis applications. This paper also proposes a diagnosis framework based on density-based clustering and artificial neural network regression using the integrated data to identify anomalous energy usages. The framework and the methods have been implemented and validated from data collected from a multitude of large-scale public buildings across China.

Automated Data Integration of Biodiversity with OLAP and OLTP

Biodiversity is one of emerging issue over decades; many have performed research to map and to document the data over the world. This issue is very important due to the event of extinction have been accelerating happening because of human extinction. Bengkulu, as one of the province lied in one of 19 hotspots in the world, Sundanese, has experienced the degradation of flora and fauna over the case of forest degradation and habitat loss. Although many application and software has been developed to solve the case, the existences of data standardization still become an issue over this problem. In this research, study of data integration had been developed to make the process of biodiversity data acquisition can be more effective and efficient. The system proposed the integration based on OLAP and OLTP that will be connected to IUCN, as one of the biggest center for monitoring the loss of biodiversity all around the world. This application had been built with web based using UML design and followed SDLC to provide the best fit of the need. This research had also succeeded to build the automated integration to show the record of dynamic number over biodiversity existences in Bengkulu. The application had been tested using black box and has the perfect performance (100%) over the testing that can help the monitoring process over biodiversity data.

Automated system for monitoring the development and spread of forest crop pests

An automated monitoring system has been developed for the AFM “Protection” of the development and spread of pests of forestry crops. The system is designed in the form of a local computer network, operates on-line and has the following additional features: automatic integration of daily data required for services of regional departments and the Forestry Agency of the republic. Based on these data, weekly, monthly maps and summary tables will be compiled.
 
 SUMMARY
 In the main directions of sustainable development of forestry in the Republic of Uzbekistan at the present stage for the period up to 2030, it is envisaged to further increase the productivity of forests on the basis of technical re-equipment, the introduction of advanced techniques to increase the efficiency of forestry production, improve the protection of forests from fires and protect them from pests and diseases. According to the forestry agency of the Republic of Uzbekistan, the area of forest plantations is 3.2 million hectares. This is about 7% of the total area of the republic. Of these, 821.0 thousand hectares are forest plantations. This means that only 2% of the area is covered by forests.
 When planning the necessary forest protection measures, it is required to identify complexes of the most important, most dangerous pest species and predict further changes in the number of pest populations. Therefore, when organizing a monitoring system, it is necessary to obtain reliable information about the level of the number of pests and the degree of their threat to crops and plantations. First, when organizing a systematic system of forestry monitoring, the territory should be zoned based on economic and ecological criteria. Secondly, due to the fact that the emergence of foci of mass reproduction of pests is usually confined to regions where critical weather situations are observed, then when designing forestry monitoring such areas should be given priority attention. Thirdly, it is necessary to strongly recommend the organization of stationary objects in all farms, which will annually assess the level of abundance, the qualitative state of the populations of the main species and the state of the growing stand.
 In case of significant damage to forest stands, a detailed survey with an assessment of the dynamics of tree mortality, determination of the qualitative and quantitative characteristics of the dominant pest species and the development of a strategy for forest protection measures should be carried out by specialists-entomologists.

Error-free, automated data integration of exosome cargo protein data with extensive clinical data in an ongoing, multi-omic translational research study.

e16743 Background: Major advances in understanding the biology of cancer have come from genomic analysis of tumor and normal tissue. Integrating extensive patient-related data with deep analysis of omic data is crucial to informing omic data interpretation. Currently, such integrations are a highly manual, asynchronous, and costly process as well as error-prone and time-consuming. To develop new blood assays that may detect very early stage PDAC, a multi-omic investigation with deep clinical annotation is needed. Using pilot data from an on-going study, we test a new platform allowing automated error-free integration of an extensive clinical database with extensive omic data. Methods: Demographic, clinical, family pedigree and pathology data were collected on the Rave EDC platform. Exosomes were purified from 46 plasma samples from 14 controls and 24 PDAC patients and cargo proteins were quantified via SILAC. The Rave Omics platform was used to ingest and integrate clinical and omic data, run quality checks and generate integrated clinical-omic datasets. Data fidelity was validated by systematically computing differences between corresponding values in the source flies with those present in the extracted data object (integrated data). The root mean squared error (RMSE) was calculated for numeric values in each sample. Additional validation was conducted by manual inspection to ascertain data integrity. Results: We demonstrated automatic integration, without human intervention, of a subset of the clinical data and all available SILAC data into an analysis-ready data object. Data transfer was completely faithful, with 100% concordance between the source and the integrated data without loss of features. All proteins (n = 1515) and clinical variables (n = 64) were imported. Their nomenclature and corresponding sample values (n = 69690) and clinical values (n = 2432) matched exactly between datasets. In all samples, the RMSE was exactly zero, indicating no deviation between data sources. Conclusions: We demonstrated that automatic, efficient, and reliable integration of clinical-omic data is achievable during an in-flight PDAC trial. Automatic exploratory analytics supporting biomarker discovery are currently being used to uncover associations between omic and clinical features. The Rave Omics platform is disease-agnostic and we plan to expand to trials of varying size, indication, and completion status where systematic, automated integration of clinical and (multi)omic data is needed.

A multilayer graph model of the internet topology

The application of internet maps for developing more effective routing algorithms, security mechanisms and resilience management for both companies and internet service providers can be advanced through their more detailed structural decomposition. This article carefully integrates traceroute datasets from various large-scale measurement campaigns such as iPlane, CAIDA, Carna, DIMES, RIPE Atlas and RIPE IPv6L into internet graphs of a previously unknown large and thus solid scale. It further works with different well-established abstraction levels such as the IP interface, router, point of presence (PoP), autonomous system (AS) and internet service provider (ISP). Finally, by employing a broad diversity of graph measures, this study creates an exhaustive snapshot of the global internet topology. This work creates a baseline for future internet research. Repeated measurements and automated data integration could enable a better understanding of the internet dynamics.

A multilayer graph model of the internet topology

The application of internet maps for developing more effective routing algorithms, security mechanisms and resilience management for both companies and internet service providers can be advanced through their more detailed structural decomposition. This article carefully integrates traceroute datasets from various large-scale measurement campaigns such as iPlane, CAIDA, Carna, DIMES, RIPE Atlas and RIPE IPv6L into internet graphs of a previously unknown large and thus solid scale. It further works with different well-established abstraction levels such as the IP interface, router, point of presence (PoP), autonomous system (AS) and internet service provider (ISP). Finally, by employing a broad diversity of graph measures, this study creates an exhaustive snapshot of the global internet topology. This work creates a baseline for future internet research. Repeated measurements and automated data integration could enable a better understanding of the internet dynamics.

Usage of the machine learning to organize time series and find anomalies

The subject of the study is the process of collecting, preparing, and searching for anomalies on data from heterogeneous sources. Economic information is naturally heterogeneous and semi-structured or unstructured. This makes pre-processing of input dynamic data an important prerequisite for the detection of significant patterns and knowledge in the subject area, so the topic of research is relevant. Pre-processing of data is several unique problems that have led to the emergence of various algorithms and heuristic methods for solving such pre-processing problems as merging and cleaning and identifying variables. In this work, an algorithm for preprocessing and searching for anomalies using LSTM is formulated, which allows you to consolidate into a single database and structure information by time series from different sources, as well as search for anomalies in an automated mode. A key modification of the preprocessing method proposed by the authors is the technology of automated data integration. The technology proposed by the authors involves the joint use of methods for building a fuzzy time series and machine lexical matching on a thesaurus network, as well as the use of a universal database built using the MIVAR concept. The preprocessing algorithm forms a single data model with the possibility of transforming the periodicity and semantics of the data set and integrating into a single information bank data that can come from various sources.

Automated data integration system at the aircraft factory

The paper discusses the experience of introducing a developed automated system for integrating data directories from various, often unrelated and heterogeneous, information sources at the aircraft manufacturing enterprise Aviastar-SP JSC (Ulyanovsk, Russia). This automated system allows both manual and automatic modes to integrate the databases of directories used at the enterprise. To test the operability on real data, the developed system went through trial exploitation and gave good results.

What happens when decision support systems fail? — the importance of usability on performance in erroneous systems

ABSTRACTWith the advent of artificial intelligence (AI) methods, smart decision support systems (DSSs) are becoming ubiquitous. Such systems help reduce complexity for operators by automating data integration tasks and recommending actions. However, these systems are sometimes flawed. It is not sufficiently understood whether, when and why operators comply with such systems in erroneous or correct cases. We empirically investigate compliance with correct and defective DSSs, the influence of correct and erroneous DSS's on performance and subjective factors related to compliance. In the study, a business game was used as an experimental setting in which 40 users took part. The impact of system correctness on user acceptance, trust, compliance and overall performance was investigated. The results show that the defective system reduces trust in automation (−47%), reduces usefulness (−58%), reduces acceptance (−62%) and reduces overall performance (−32%). Overall, the defective system was less user-friendly (−27%). Nevertheless, users who rated the system's usability higher, outperformed users who rated it lower. Usability is therefore an intermediary that compensates for the negative influence of erroneous DSSs.

Interpretation and automatic integration of geospatial data into the Semantic Web

In the context of disaster management, geospatial information plays a crucial role in the decision-making process to protect and save the population. Gathering a maximum of information from different sources to oversee the current situation is a complex task due to the diversity of data formats and structures. Although several approaches have been designed to integrate data from different sources into an ontology, they mainly require background knowledge of the data. However, non-standard data set schema (NSDS) of relational geospatial data retrieved from e.g. web feature services are not always documented. This lack of background knowledge is a major challenge for automatic semantic data integration. Focusing on this problem, this article presents an automatic approach for geospatial data integration in NSDS. This approach does a schema mapping according to the result of an ontology matching corresponding to a semantic interpretation process. This process is based on geocoding and natural language processing. This article extends work done in a previous publication by an improved unit detection algorithm, data quality and provenance enrichments, the detection of feature clusters. It also presents an improved evaluation process to better assess the performance of this approach compared to a manually created ontology. These experiments have shown the automatic approach obtains an error of semantic interpretation around 10% according to a manual approach.