Big Data Integration Research Articles

Crowdsourcing Based Description of Urban Emergency Events Using Social Media Big Data

Crowdsourcing is a process of acquisition, integration, and analysis of big and heterogeneous data generated by a diversity of sources in urban spaces, such as sensors, devices, vehicles, buildings, and human. Especially, nowadays, no countries, no communities, and no person are immune to urban emergency events. Detection about urban emergency events, e.g., fires, storms, traffic jams is of great importance to protect the security of humans. Recently, social media feeds are rapidly emerging as a novel platform for providing and dissemination of information that is often geographic. The content from social media usually includes references to urban emergency events occurring at, or affecting specific locations. In this paper, in order to detect and describe the real time urban emergency event, the 5W (What, Where, When, Who, and Why) model is proposed. Firstly, users of social media are set as the target of crowd sourcing. Secondly, the spatial and temporal information from the social media are extracted to detect the real time event. Thirdly, a GIS based annotation of the detected urban emergency event is shown. The proposed method is evaluated with extensive case studies based on real urban emergency events. The results show the accuracy and efficiency of the proposed method.

Big Earth Observation Data Integration in Remote Sensing Based on a Distributed Spatial Framework

The arrival of the era of big data for Earth observation (EO) indicates that traditional data management models have been unable to meet the needs of remote sensing data in big data environments. With the launch of the first remote sensing satellite, the volume of remote sensing data has also been increasing, and traditional data storage methods have been unable to ensure the efficient management of large amounts of remote sensing data. Therefore, a professional remote sensing big data integration method is sorely needed. In recent years, the emergence of some new technical methods has provided effective solutions for multi-source remote sensing data integration. This paper proposes a multi-source remote sensing data integration framework based on a distributed management model. In this framework, the multi-source remote sensing data are partitioned by the proposed spatial segmentation indexing (SSI) model through spatial grid segmentation. The designed complete information description system, based on International Organization for Standardization (ISO) 19115, can explain multi-source remote sensing data in detail. Then, the distributed storage method of data based on MongoDB is used to store multi-source remote sensing data. The distributed storage method is physically based on the sharding mechanism of the MongoDB database, and it can provide advantages for the security and performance of the preservation of remote sensing data. Finally, several experiments have been designed to test the performance of this framework in integrating multi-source remote sensing data. The results show that the storage and retrieval performance of the distributed remote sensing data integration framework proposed in this paper is superior. At the same time, the grid level of the SSI model proposed in this paper also has an important impact on the storage efficiency of remote sensing data. Therefore, the remote storage data integration framework, based on distributed storage, can provide new technical support and development prospects for big EO data.

Research on the Innovation of Personalized Education in Colleges and Universities under the Context of Big Data

The advent of the big data era has opened up unprecedented quantifiable dimensions for all walks of life, while also presented many opportunities and challenges. The widespread application of big data in the field of education will inevitably promote the innovation and transformation of the higher education in colleges and universities which introduce new opportunities and challenges to personalized education in colleges and universities. How to promote the in-depth integration of big data and higher education to meet the needs of students' personalized development is one of the hot topics in the theoretical field today. In view of this, this article starts with explaining the connotation of personalized education in colleges and universities, analyzes the opportunities and challenges faced by personalized education in colleges and universities under the context of big data, and proposes the ideas of innovating personalized education in colleges and universities under the comntext of big data.

Leveraging Knowledge Graphs for Big Data Integration: the XI Pipeline

This article gives an overview of recent efforts focusing on integrating heterogeneous data using Knowledge Graphs. I introduce a pipeline consisting of five steps to integrate semi-structured or unstructured content. I discuss some of the key applications of this pipeline through three use-cases, and present the lessons learnt along the way while designing and building data integration systems.

The application of big data to cardiovascular disease: paths to precision medicine.

Advanced phenotyping of cardiovascular diseases has evolved with the application of high-resolution omics screening to populations enrolled in large-scale observational and clinical trials. This strategy has revealed that considerable heterogeneity exists at the genotype, endophenotype, and clinical phenotype levels in cardiovascular diseases, a feature of the most common diseases that has not been elucidated by conventional reductionism. In this discussion, we address genomic context and (endo)phenotypic heterogeneity, and examine commonly encountered cardiovascular diseases to illustrate the genotypic underpinnings of (endo)phenotypic diversity. We highlight the existing challenges in cardiovascular disease genotyping and phenotyping that can be addressed by the integration of big data and interpreted using novel analytical methodologies (network analysis). Precision cardiovascular medicine will only be broadly applied to cardiovascular patients once this comprehensive data set is subjected to unique, integrative analytical strategies that accommodate molecular and clinical heterogeneity rather than ignore or reduce it.

A Product Process Adaptive Design Method Based on Manufacturing-Related Big Data

As digital and smart production methods being applied widely in manufacturing, enterprises should pay more attention to the values of manufacturing-related big data, which is important to the innovation of product process design. This study aims to propose a product process adaptive design method based on manufacturing-related big data. This method is proposed based on the requirement of enterprises for data–business deep integration and it is used for solving the insufficient utilization of manufacturing-related data among enterprises. To this end, a product process adaptive design model “data+ knowledge+ decision” is proposed and the data mining and utilization processes are summarized for the model, namely, multi-source heterogeneous data fusion; data cleaning and preprocessing; data conversion and dimensionality reduction; data mining; data visualization; and design decision. Subsequently, the automobile welding process is used as an example. A prediction model of the relationship between welding parameters and welding defects is established, aiming to improve welding quality and realize welding process adaptive design. This research reveals that manufacturingrelated big data contains rich knowledge and patterns and thus can guide product design decisions and support the product process adaptive design under different manufacturing environments. In the future, to enhance the integration of manufacturing-related big data with product process design, the integration of big data with 5G technology should be promoted and investment should be increased in the development of big data and algorithm design platforms.

Enhanced Ontology Matching for Big Data Integration

ontology matching (OM) is a critical process for many disciplines. It aims at identifying the semantic correspondences among different ontologies that are merged for data integration. Unfortunately, OM still faces challenges, especially, in the big data integration (BDI) area. The high degree of semantic heterogeneity problem prevents the integration of relevant data and increased with large-scale ontologies of BDI. The quality of OM still needs more improvements to cope with BDI applications. So, this paper proposes a semantic OM approach called semantic matcher. It achieves the goals of semantic heterogeneity resolving and the quality improvement. It exploits the semantic similarity based on the word embedding model. The word embedding model provides efficient distribution semantic representation of domain words as vectors based on their context. The applicability of the proposed semantic matcher is evaluated through an experiment. The conference and anatomy gold standard datasets are evaluating the experimental results. Accuracy evaluates the quality through the precision, recall, and F-measure measures. Based on the experimental results evaluation, the proposed semantic matcher is promising and efficient in the semantic OM for BDI.

Innovations from Iran: Resolving quality issues in the integration of administrative and big data in official statistics

In a modernized statistical production process, non-traditional data sources such as ‘big’ data are increasingly being considered either as the main or as a supplementary source for official statistics. Their use however has brought new sorts of challenges: messy datasets, duplicate entries, missing information and misspellings to name a few. In many cases, there is also no unique identifier which can be used to unambiguously identify a record for the purpose of data integration. These challenges can be compounded by a non-English based alphabet like the Persian/Farsi alphabet used in Iran. In this paper, two innovative methods have been elaborated to address such data challenges. More specifically, the application of probabilistic record linkage using an ACSII coding system is an innovative way to deal with both data challenges and lack of unique identifier simultaneously. Moreover, text mining is an innovative way to address categorization and grouping systems that are not suitable for statistical purposes. Both innovative approaches can improve the accuracy and coherency of datasets and for data integration result in higher quality datasets. Results of research undertaken by the authors show the innovations lead to more effective data integration and improve the quality of the resulting official statistics. The innovations have wide applicability especially in non-English alphabet countries.

Research on Strategies to Improve Service Quality of College Teacher Development Center in the Age of “Internet +”

Against the backdrop of integration of big data and Internet technologies with higher education, higher education reforms have raised standards for teachers’ teaching ability and information literacy, hence higher standards for the service quality of college teacher development centers. In this context, this paper analyzes the service quality and problems of private college teacher development centers in the system of “Internet + training”, constructs the service quality gap model of the college teacher development center, and explores the causes for poor service quality in the training process. At last, according to the causes, this paper builds the data platform, develops the training teams (program development team and technical support team), relies the virtual community of “E-home of college teacher development”, optimizes the training mode to propose the strategies to improve private college teacher development center in the age of “Internet+”.

Big data inconsistencies and incompleteness: a literature review

The analysis and integration of big data highlight some issues in the identification and resolution of data inconsistencies and knowledge incompleteness. This paper presents an overview of data inconsistencies and a review of approaches to resolve various levels of data inconsistencies. Moreover, we discuss some issues related to incompleteness and stability of known knowledge over specific time periods, and the implication for the decision-making process. In addition, the use of a Bayesian network model in inconsistency resolution in data analysis and knowledge engineering will also be considered.

Big Data Management and Analytics Metamodel for IoT-Enabled Smart Buildings

Big data management and analytics, in the context of IoT (Internet of Things)-enabled smart buildings, is a challenging task. It is a diffused and complex area of knowledge due to the diversity of IoT devices and the nature of data generated by the IoT devices. Many international bodies have developed metamodels for IoT-enabled ecosystems to allow knowledge sharing. However, these are often narrow in focus and deal with only the IoT aspects without taking into account the management and analytics of big data generated by the IoT devices. Hence, in this article we propose a metamodel for the Integrated Big Data Management and Analytics (IBDMA) framework for IoT-enabled smart buildings. The IBDMA Metamodel can be used to facilitate interoperability between existing big data management and analytics ecosystems deployed in smart buildings or other smart environments. We import the metamodel into a knowledge graph management tool and by considering a case study we validate the metamodel using this tool. The evaluation results demonstrate that IBDMA Metamodel is indeed suitable for its intended purpose.

Guest Editorial: Special Section on Integration of Big Data and Artificial Intelligence for Internet of Things

These redundant data in IoT should be compressed or removed. Furthermore, the unstructured data in IoT plays an important role for analyzing the user behaviors, while transmitting and processing these unstructured data consumption of substantial energy. These unstructured data should be mined or restructured. In addition, the increasing number of users in IoT leads to a fast-growing data in IoT, while the Quality of Service (QoS) of IoT should be maintained regardless of the number of IoT users. Therefore, the data transmission and processing in IoT should be performed in a more intelligent manner. All these observations indicate that the integration of big data and artificial intelligence (AI) for IoT is a good propellant to improve the data transmission and processing in IoT, since big data technology (e.g., data integration, data mining, data prediction) could effectively handle various data while AI technology could further facilitate capturing and structuring big data.

PRAISE-HK: A personalized real-time air quality informatics system for citizen participation in exposure and health risk management

Exposure to air pollutants causes a range of adverse health effects. These harmful effects occur whenever and wherever people come into direct contact with air pollution. Therefore, individual actions that reduce the frequency, duration, and severity of personal contact with air pollution can reduce health risks. We developed a system that empowers the public with personalized information on air quality and exposure health risk. This system, the Personalised Real-Time Air Quality Informatics System for Exposure – Hong Kong (PRAISE-HK, http://praise.ust.hk/), is embodied in an interactive mobile application. PRAISE-HK is based on real-time data on emissions, high resolution urban morphology, meteorology, physical and chemical processes affecting pollutant transport and transformations, extensive measurements of air pollution concentrations in typical locations such as homes, schools, offices, and transportation, and big data integration of sensor monitoring to accurately estimate current and short-term forecasted street-level air quality. The street-level air quality simulation has been validated against reference monitoring data. Ongoing and planned future enhancements to PRAISE-HK include prediction of personal exposure and health response. PRAISE-HK is an example of the use of collective intelligence in a smart city to engage citizens in learning about and managing their own exposure to air pollution.

Biomedical ontologies and their development, management, and applications in and beyond China

Abstract Since the boom of biomedical big data studies, various big data processing technologies have been developed rapidly. As an important form of knowledge representation, ontology has become an important means for the utilization and integration of biomedical big data. The emergence of new technologies for ontology development has resulted in the generation of many biomedical ontologies by many ontology development communities. The Open Biological and Biomedical Ontology Foundry, an academic organization for bio-ontology developers, has provided a set of principles to guide community-based open ontology construction. The Open Biological and Biomedical Ontology Foundry have also built many widely used ontologies, such as Gene Ontology, Human Phenotype Ontology, and Chemical Entities of Biological Interest. Other various ontology repositories have also been created and used to support ontology reuse. Many efficient tools for ontology applications, such as data annotation and terms mapping, have also been developed. High quality ontologies are also being used to develop new methods and tools for biomedical data analysis. The applications of Gene Ontology and Human Phenotype Ontology for data analysis and integration in recent years are reviewed here. To promote the development and applications of biomedical ontologies in China, a research community, OntoChina, was founded recently. OntoChina aims to support the development of reference ontologies, especially bilingual and Chinese translated ontologies. OntoChina also encourages ontology developers to follow the Open Biological and Biomedical Ontology Foundry principles.

Database Resources of the National Genomics Data Center in 2020.

The National Genomics Data Center (NGDC) provides a suite of database resources to support worldwide research activities in both academia and industry. With the rapid advancements in higher-throughput and lower-cost sequencing technologies and accordingly the huge volume of multi-omics data generated at exponential scales and rates, NGDC is continually expanding, updating and enriching its core database resources through big data integration and value-added curation. In the past year, efforts for update have been mainly devoted to BioProject, BioSample, GSA, GWH, GVM, NONCODE, LncBook, EWAS Atlas and IC4R. Newly released resources include three human genome databases (PGG.SNV, PGG.Han and CGVD), eLMSG, EWAS Data Hub, GWAS Atlas, iSheep and PADS Arsenal. In addition, four web services, namely, eGPS Cloud, BIG Search, BIG Submission and BIG SSO, have been significantly improved and enhanced. All of these resources along with their services are publicly accessible at https://bigd.big.ac.cn.

European network staff eXchange for integrAting precision health in the health Care sysTems” project

Abstract Background Precision health aims to prevent and predict illness, maintaining health and quality of life for as long as possible, by drawing on the new technological and data science tools to translate volumes of research and clinical data into information that citizens, patients and doctors can use. Objective The ExACT consortium, funded by the Marie Curie Research and Innovation Staff Exchange (RISE) 2017 - Horizon 2020, is aimed at building a community of academic and non-academic institutions that generates high quality, multidisciplinary collaboration by exchanging knowledge in research and training activities on precision health. Results From 2019 to 2023, 74 secondments are foreseen; staff involved will be trained on precision health research topics unavailable at their home institutions. The research topics include 5 domains: Integration of Big Data and digital solutions into healthcare systems; design and promotion of innovative citizen engagement models; education of healthcare professionals and leadership; HTA in precision health; Ethical-legal, social, organisational and policy issues surrounding precision health. Conclusions Secondees will produce key reports, policy recommendations, scientific papers, and informative materials for citizens, fostering public-private interplay and fostering integration of precision health in the EU health systems, contributing to better health for EU citizens. Key messages Once the secondees are back in their home institution, they will use competences acquired during the secondment to advance the research, and transfer the knowledge to the home organization. Sharing knowledge,building synergies and expertise and encouraging best practices,among top-level institutions,will stimulate translational effort for implementing precision health in EU health system.

Integration of big data in small and medium organizations: Business intelligence and cloud computing

Although it is well known, organizations are becoming increasingly concerned about how they make use of information and thus obtain the greatest benefit from it for competitive purposes and continuous improvement in a globalized world of constant change, large volumes of data produced by organizations turn out to be a key element in decision-making for productive purposes, the multinationals already do it, why not integrate it into small and medium-sized companies? Today small and medium enterprises are still making decisions based on intuition, which do not take into account the information, intuition in most cases does not work as a good guide for decision making, in small and medium enterprises bosses or owners of these manage their companies more by the experience acquired through the years that often do not take into account the technological advances that are being developed to improve their processes.

WOT Enabled Smart Cities integrated Big Data Analytics

WOT Enabled Smart Cities integrated Big Data Analytics

Big data in yeast systems biology.

Systems biology uses computational and mathematical modeling to study complex interactions in a biological system. The yeast Saccharomyces cerevisiae, which has served as both an important model organism and cell factory, has pioneered both the early development of such models and modeling concepts, and the more recent integration of multi-omics big data in these models to elucidate fundamental principles of biology. Here, we review the advancement of big data technologies to gain biological insight in three aspects of yeast systems biology: gene expression dynamics, cellular metabolism and the regulation network between gene expression and metabolism. The role of big data and complementary modeling approaches, including the expansion of genome-scale metabolic models and machine learning methodologies, are discussed as key drivers in the rapid advancement of yeast systems biology.

Big Data Interoperability for E-Governance

The notion of governance is innate to any society in the word. The use of big-data allow governance as a competitive advantage to attract resources, with the emergence of the social web and exponential data generated accordingly. Such social web and exponential data can be analyzed to give the enlightened human beings about the good governance and maintain an intelligent global state of mega cities. The government agencies should collect data from heterogeneous sources. To improve the services and applications, these agencies offer the citizens and businesses, through big data analytics, which is a global opportunity to integrate a large number of public databases together, fraud detection and tax control activities. The impact of big data on these e-government applications strives to facilitate better governance by demonstrating its use in research and practice. Therefore, the current paper zeroes in on an approach based on the interoperability and integration of big data. In our approach, such data is gathered from several separate and heterogeneous government sources.