Data Integration Requirements Research Articles

A framework for corridor-level traffic safety network screening and its implementation using Business Intelligence

Currently, there are both methodological and practical barriers that together preclude the use of theoretically sound approaches for network screening as part of a traffic safety management process. Methodological barriers include, among others, lack of a comprehensive framework for corridor-level network screening. Existing corridor screening methodologies use observed crash frequency as a performance measure. In practice, corridor-level screening is extremely important because traffic safety engineers prefer to deploy countermeasures and provide homogenous conditions throughout corridors to meet drivers’ expectations and avoid confusion. On the other hand, practical barriers that limit the use of sound approaches for traffic safety include (1) significant data integration requirements, (2) a particular data schema is needed to enable analysis using specialized software, (3) time-consuming and intensive processes are involved, (4) substantial technical knowledge is needed, (5) visualization capabilities are limited, and (6) coordination across various data owners is required. This research proposes a systematic methodology for corridor-level network screening. The solution algorithm is implemented within a Business Intelligence (BI) platform to address, to the extent possible, the practical barriers listed above. BI provides methods and mechanisms to integrate and process data, generate advanced analytics, and visualize results by using intuitive and interactive web-based dashboards and maps. Experiments and results illustrate the advantage of using the proposed framework for corridor-level network screening implemented within a BI platform.

Data integrity in regulated bioanalysis: a summary from the European Bioanalysis Forum Workshop in collaboration with the MHRA.

In this conference report, we summarize the main findings and messages from a workshop on 'Data Integrity'. The workshop was held at the 11th European Bioanalysis Forum Open (EBF) Symposium in Barcelona (21-23 November 2018), in collaboration with the Medicines and Health products Regulatory Agency to provide insight and understanding of regulatory data integrity expectations. The workshop highlighted the importance of engaging with software developers to address the gap between industry's data integrity needs and current system software capabilities. Delegates were also made aware of the importance of implementing additional procedural controls to mitigate the risk associated with using systems that do not fully meet data integrity requirements.

A review of geoanalytical databases

Geoanalytical data provide fundamental information according to which the Earth’s resources can be known and exploited to support human life and development. Large amounts of manpower and material and financial resources have been invested to acquire a wealth of geoanalytical data over the past 40 years. However, these data are usually managed by individual researchers and are preserved in an ad hoc manner without metadata that provide the necessary context for interpretation and data integration requirements. In this scenario, fewer data, except for published data, can be reutilized by geological researchers. Many geoanalytical databases have been constructed to collect existing data and to facilitate their use. These databases are useful tools for preserving, managing, and sharing data for geological research, and provide various data repositories to support geological studies. Since these databases are dispersed and diverse, it is difficult for researchers to make full use of them. This contribution provides an introduction on available geoanalytical databases. The database content can be made accessible to researchers, the ways in which this can be done, and the functionalities that can be used are illustrated in detail. Moreover, constraints that have limited the reutilization of geoanalytical data and creation of more advanced geoanalytical databases are discussed.

Data integrity: requirements set in the accreditation standards

The requirements for the competence of laboratories are described in two standards, namely ISO/IEC 17025 and ISO 15189. The former applies to all laboratories except medical laboratories where the latter is the most appropriate. The requirements of both standards include specific references to data and other related terms, their protection and integrity. The recently revised ISO/IEC 17025 introduces additional provisions for the control of data and information management compared to those of the 2005 edition. Both standards include provisions for the use of laboratory information management system(s) for the collection, processing, recording, reporting, storage or retrieval of data. ISO 15189 addresses these issues in more detail in order to meet the particular needs of the sector dealing with personal data of the patients. This article presents the requirements for data integrity and the control of relevant components of the management system set by the standards for laboratories as well as for some other conformity assessment bodies of importance for laboratory work, namely reference materials producers (ISO 17034) and proficiency testing providers (ISO/IEC 17043). Some additional regulatory requirements recently implemented are also discussed.

Consortium Blockchain Based Data Aggregation and Regulation Mechanism for Smart Grid

Flexible regulation of smart grid is vital for grid operation. This paper proposes a smart grid data aggregation and regulation mechanism based on consortium blockchain, and its signcryption algorithm can be applied to multidimensional data acquisition and multiple receivers in the consortium blockchain. In the process of regulation, the control center, the grid operator, and the equipment supplier receive fixed-height blocks from the blockchain and obtain plaintext from the decryption. Each receiver analyzes the multidimensional data and formulates corresponding control policies for individual users. Grid operators implement user power regulation by feedback on smart contracts. The security analysis and performance comparison show that the proposed scheme has advantages in computing and communication costs while meeting security requirements for confidentiality and data integrity.

Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integration in Precision Medicine.

Machine learning (ML) is being ubiquitously incorporated into everyday products such as Internet search, email spam filters, product recommendations, image classification, and speech recognition. New approaches for highly integrated manufacturing and automation such as the Industry 4.0 and the Internet of things are also converging with ML methodologies. Many approaches incorporate complex artificial neural network architectures and are collectively referred to as deep learning (DL) applications. These methods have been shown capable of representing and learning predictable relationships in many diverse forms of data and hold promise for transforming the future of omics research and applications in precision medicine. Omics and electronic health record data pose considerable challenges for DL. This is due to many factors such as low signal to noise, analytical variance, and complex data integration requirements. However, DL models have already been shown capable of both improving the ease of data encoding and predictive model performance over alternative approaches. It may not be surprising that concepts encountered in DL share similarities with those observed in biological message relay systems such as gene, protein, and metabolite networks. This expert review examines the challenges and opportunities for DL at a systems and biological scale for a precision medicine readership.

An E-Commerce Platform for Industrialized Construction Procurement Based on BIM and Linked Data

Industrialized construction has raised the requirements of procurement methods used in the construction industry. The rapid development of e-commerce offers efficient and effective solutions, however the large number of participants in the construction industry means that the data involved are complex, and problems arise related to volume, heterogeneity, and fragmentation. Thus, the sector lags behind others in the adoption of e-commerce. In particular, data integration has become a barrier preventing further development. Traditional e-commerce platform, which considered data integration for common product data, cannot meet the requirements of construction product data integration. This study aimed to build an information-integrated e-commerce platform for industrialized construction procurement (ICP) to overcome some of the shortcomings existing platforms. We proposed a platform based on Building Information Modelling (BIM) and linked data, taking an innovative approach to data integration. It uses industrialized construction technology to support product standardization, BIM to support procurement process, and linked data to connect different data sources. The platform was validated using a case study. With the development of an e-commerce ontology, industrialized construction component information was extracted from BIM models and converted to Resource Description Framework (RDF) format. Related information from different data sources was also converted to RDF format, and Simple Protocol and Resource Description Framework Query Language (SPARQL) queries were implemented. The platform provides a solution for the development of e-commerce platform in the construction industry.

Supporting the design of data integration requirements during the development of data warehouses: a communication theory-based approach

Data warehouses (DW) form the backbone of data integration that is necessary for analytical applications, and play important roles in the information technology landscape of many industries. We introduce an approach for addressing the fundamental problem of semantic heterogeneity in the design of data integration requirements during DW development. In contrast to ontology-driven or schema-matching approaches, which propose the automatic resolution of differences ex-post, our approach addresses the core problem of data integration requirements: understanding and resolving different contextual meanings of data fields. We ground the approach firmly in communication theory and build on practices from agile software development. Besides providing relevant insights for the design of data integration requirements, our findings point to communication theory as a sound underlying foundation for a design theory of information systems development.

Application and research on business intelligence in audit business

With audit data requirements, taking for example the audit data of construction industry, the audit data integration requirements are analyzed and application architecture of business intelligence technology in the audit business is put forward. The modeling process and the populating process of data warehouse are detailed, and the scheme of data pre-processing is set out. The report display methods are proposed, such as proportion chart, tendency chart, index plots and etc. The application model and value of business intelligence technology in audit business are analyzed, which is based on data mining technology such as outlier detection. This research has been applied to National Twelfth Five-Year Plan Science and Technology Support Program and has achieved better analysis and prediction of audit.

Ensuring Long-Term Data Integrity: ETCS Data Integrity Requirements Can Be Fulfilled Even under Unfavorable Conditions with the Proper LTE Mechanisms

The European Train Control System (ETCS) is the leading signaling system for train command and control. In the future, ETCS may be delivered over long-term evolution (LTE) networks. Thus, LTE performance offered to ETCS must be analyzed and confronted with the railway safety requirements. It is especially important to ensure the integrity of the ETCS data, i.e., to protect ETCS data against loss and corruption. In this article, various retransmission mechanisms are considered for providing end-to-end ETCS data integrity in LTE. These mechanisms are validated in simulations, which model worst-case conditions regarding train locations, traffic load, and basestation density. The simulation results show that ETCS data integrity requirements can be fulfilled even under these unfavorable conditions with the proper LTE mechanisms.

Enhanced secure sensor association and key management in wireless body area networks

Body area networks (BANs) have emerged as an enabling technique for e-healthcare systems, which can be used to continuously and remotely monitor patients' health. In BANs, the data of a patient's vital body functions and movements can be collected by small wearable or implantable sensors and sent using short-range wireless communication techniques. Due to the shared wireless medium between the sensors in BANs, it may be possible to have malicious attacks on e-healthcare systems. The security and privacy issues of BANs are becoming more and more important. To provide secure and correct association of a group of sensors with a patient and satisfy the requirements of data confidentiality and integrity in BANs, we propose a novel enhanced secure sensor association and key management protocol based on elliptic curve cryptography and hash chains. The authentication procedure and group key generation are very simple and efficient. Therefore, our protocol can be easily implemented in the power and resource constrained sensor nodes in BANs. From a comparison of results, furthermore, we can conclude that the proposed protocol dramatically reduces the computation and communication cost for the authentication and key derivation compared with previous protocols. We believe that our protocol is attractive in the application of BANs.

PanDA Workload Management System Meta-data Segmentation

Abstract The PanDA (Production and Distributed Analysis) workload management system (WMS) was developed to meet the scale and complexity of LHC distributed computing for the ATLAS experiment. PanDA currently distributes jobs among more than 100,000 cores at well over 120 Grid sites, supercomputing centers, commercial and academic clouds. ATLAS physicists submit more than 1.5 M data processing, simulation and analysis PanDA jobs per day, and the system keeps all meta-information about job submissions and execution events in Oracle RDBMS. The above information is used for monitoring and accounting purposes. One of the most challenging monitoring issues is tracking errors that has occurred during the execution of the jobs. Current meta-data storage technology doesn’t support inner tools for data aggregation, needed to build error summary tables, charts and graphs. Delegating these tasks to the monitor slows down the execution of requests. We will describe a project aimed at optimizing interaction between PanDA front-end and back-end, by meta-data storage segmentation into two parts – operational and archived. Active meta-data are remained in Oracle database (operational part), due to the high requirements for data integrity. Historical (read-only) meta-data used for the system analysis and accounting are exported to NoSQL storage (archived part). New data model based on usage of Cassandra as the NoSQL backend has been designed as a set of query-specific data structures. This allowed to remove most of data preparation workload from PanDA Monitor and improve its scalability and performance. Segmentation and synchronization between operational and archived parts of jobs meta-data is provided by a Hybrid Meta-data Storage Framework (HMSF). PanDA monitor was partly adopted to interact with HMSF. The operational data queries are forwarded to the primary SQL-based repository and the analytic data requests are processed by NoSQL database. The results of performance and scalability tests of HMSF-adopted part of PanDA Monitor shows that presented method of optimization, in conjunction with a properly configured NoSQL database and reasonable data model, provides performance improvements and scalability.

Incorporation of personal single nucleotide polymorphism (SNP) data into a national level electronic health record for disease risk assessment, part 1: an overview of requirements.

BackgroundPersonalized medicine approaches provide opportunities for predictive and preventive medicine. Using genomic, clinical, environmental, and behavioral data, tracking and management of individual wellness is possible. A prolific way to carry this personalized approach into routine practices can be accomplished by integrating clinical interpretations of genomic variations into electronic medical records (EMRs)/electronic health records (EHRs). Today, various central EHR infrastructures have been constituted in many countries of the world including Turkey.ObjectiveThe objective of this study was to concentrate on incorporating the personal single nucleotide polymorphism (SNP) data into the National Health Information System of Turkey (NHIS-T) for disease risk assessment, and evaluate the performance of various predictive models for prostate cancer cases. We present our work as a miniseries containing three parts: (1) an overview of requirements, (2) the incorporation of SNP into the NHIS-T, and (3) an evaluation of SNP incorporated NHIS-T for prostate cancer.MethodsFor the first article of this miniseries, the scientific literature is reviewed and the requirements of SNP data integration into EMRs/EHRs are extracted and presented.ResultsIn the literature, basic requirements of genomic-enabled EMRs/EHRs are listed as incorporating genotype data and its clinical interpretation into EMRs/EHRs, developing accurate and accessible clinicogenomic interpretation resources (knowledge bases), interpreting and reinterpreting of variant data, and immersing of clinicogenomic information into the medical decision processes. In this section, we have analyzed these requirements under the subtitles of terminology standards, interoperability standards, clinicogenomic knowledge bases, defining clinical significance, and clinicogenomic decision support.ConclusionsIn order to integrate structured genotype and phenotype data into any system, there is a need to determine data components, terminology standards, and identifiers of clinicogenomic information. Also, we need to determine interoperability standards to share information between different information systems of stakeholders, and develop decision support capability to interpret genomic variations based on the knowledge bases via different assessment approaches.

PROTECTION OF INFORMATION IN CRITICAL APPLICATION DATA PROCESSING SYSTEMS

The results of researches, allowing raising the level of protection of the automated data processing systems of critical applications and intellectual information systems of enterprises are presented in the article. The mathematical models and results of vulnerability estimation of information systems which have Internet connection through various communication channels are resulted in this work. The system approach to solving problems of information security, proposed in this work provides for the integration of mathematical models of the processing and protection of information. The method of modeling the security policy to provide a highly reliable information processing has been developed. The mathematical models of synthesis of policy safe interaction of information processes, allowing security policy to consider separately the various structural components of network with the ability to its further interlinkages have been developed. Using the new mathematical models of flexible reliability, availability, confidentiality and integrity of information processed, allowing mathematically describe the mechanisms to ensure the availability and confidentiality of the information and take into account the quantitative requirements for data integrity.

Modular protections against non-control data attacks

This paper introduces Yarra, a conservative extension to C to protect applications from non-control data attacks. Yarra programmers specify their data integrity requirements by declaring critical data types and ascribing these critical types to important data structures. Yarra guarantees that such critical data is only written through pointers with the given static type. Any attempt to write to critical data through a pointer with an invalid type (perhaps because of a buffer overrun) is detected dynamically. We formalize Yarra's semantics and prove the soundness of a program logic designed for use with the language. A key contribution is to show that Yarra's semantics are strong enough to support sound local reasoning and the use of a frame rule, even across calls to unknown, unverified code. We evaluate a prototype implementation of a compiler and runtime system for Yarra by using it to harden four common server applications against known non-control data vulnerabilities. We show that Yarra defends against these attacks with only a negligible impact on their end-to-end performance.

A secure remote data integrity checking cloud storage system from threshold encryption

The confidentiality and integrity are the most important issues in cloud storage system. We address the privacy issue of decentralized cloud storage system using threshold cryptography. Once the user uploads his data and deletes the local copy, he might lost the full control of the data. The major challenges of designing this cloud storage system are to provide a better privacy guarantee and ensure the integrity of data at untrusted servers in distributed cloud storage system. To achieve this goal, we propose a threshold encryption scheme and integrate it with a secure decentralized erasure code to form a secure cloud storage system, where the user generates a secret parameter participated in system encryption and decryption of plaintext blocks in the combine process. So our systems can prevent cloud storage servers from modifying data by providing remote data check and meets the requirements of data robustness, confidentiality and integrity.

The Application of IBE Encryption Scheme in Grid Environments

The computing service and security service are separated in current Grid system, but the data transmission of the system has to cross unsafe public channel. This makes the results computing by the Grid system may be stolen or destroyed by artificial or non-artificial factors in transmission process. So the Grid system has secure requirements of confidential communication, data integrity and non-repudiation. For these requirements, this paper puts forward an identity-based encryption scheme and realizes the function of confidentiality, Integrity, Key Update and non-repudiation. To test the security of the proposed scheme, the authors make a detailed analysis in random oracle model and prove the security of the scheme equivalents to the difficulty of the bilinear Diffle-Hellman problem. So our scheme is proved to be correct.

Research on Identity-Based Encryption Scheme for Grid Computing System

Grid system has secure requirements of confidential communication, data integrity and non-repudiation. According to the secure requirements for Grid service, the paper proposed an identity-based encryption scheme for Grid, which can solve a series of problem: the privacy of data transmission, validation of integrity of data, key update after long run time and non-repudiation. The scheme is constructed by bilinear paring on elliptic cures and its security can be reduced to the computational Bilinear Diffie-Hellman assumption. Finally, the authors analyses the security and efficiency of this scheme.

Load Balancing Algorithm of Stream Data Based on Correlation Analysis

With the development of stream processing technology, achieving load balance of resource accessing becomes one of key problems. However, the existing technologies are difficult to balance the system and maintain the data integrity requirements when large data stream arrive. In this paper, combined with the dynamic load balancing algorithm, we propose a load balancing algorithm of stream data based on correlation analysis (SDCA-LBA). The algorithm analyses correlation of stream data through the window feature statistics. On the basis of ensuring the load balancing of the stream data system, we consider the data correlation. The experiment results show that this method can effectively solve the problem of stream data load balancing, maintain the data integrity, and improve the system data processing capabilities.

ENSURING OF INFORMATION PROCESSES’ RELIABILITY AND SECURITY IN CRITICAL APPLICATION DATA PROCESSING SYSTEMS

Summary: The system approach to solving problems of information security, proposed in this work provides for the integration of mathematical models of the processing and protection of information. This model connects invulnerability and flexibility for each of three aspects of security (confidentiality, availability and integrity) of information based on structural unification of these contradictions. The method of modeling the security policy (SP) to provide a highly reliable information processing (HRIP) has been developed. This method differs by using a new problem-based graph-theoretic unit of standard model of the protected automated system for connection flexibility the discretionary model with the principled security of models of the final states of the SP. The mathematical models of synthesis of policy safe interaction of information processes, allowing SP to consider separately the various structural components of network with the ability to its further interlinkages have been developed. Using the new mathematical models of flexible reliability, availability, confidentiality and integrity of information processed, allowing mathematically describe the mechanisms to ensure the availability and confidentiality of the information and take into account the quantitative requirements for data integrity.