Data Integration Requirements Research Articles

Blockchain-Based Solution for Supply Chain Data Integrity

Data plays a crucial role in today's world, guiding business decisions across various computer- assisted activities. Maintaining the integrity of data is vital, as any tampering could have serious consequences for important business decisions. This concern is particularly significant in cloud computing settings, where data owners have limited control over key aspects like physical storage and access control. Blockchain technology has emerged as an intriguing solution to address data integrity concerns. With its inherent properties, blockchain offers promising avenues for ensuring data integrity. However, some challenges need to be overcome, such as low throughput, high latency, and stability issues, which currently limit the practical implementation of blockchain-based solutions. we focus on a case study from the European SUNFISH project, which aims to develop a secure cloud federation platform for the public sector. We examine the specific data integrity requirements in cloud computing environments and identify the research questions that need to be addressed to implement blockchain-based databases effectively. We start by outlining the open research questions and the inherent difficulties associated with addressing them. Then, we propose a preliminary design for a blockchain-based database tailored to cloud computing environments. This design aims to leverage the strengths of blockchain technology while addressing the challenges unique to cloud computing. By addressing these research questions and proposing practical solutions, we aim to pave the way for the adoption of blockchain-based databases in cloud computing environments. This has the potential to enhance data integrity and security, ultimately benefiting organizations operating in the cloud.

The service model of a heterogeneous distributed database for software system scaling

The article explores the topicality of horizontal scaling for software systems that re-quire the use of database management systems (DBMS) to process large volumes of structured, JSON, and geospatial data, while maintaining requirements for ACID proper-ties and data integrity. An analysis of existing products is conducted, outlining their main advantages, disadvantages and identifying issues such as single points of failure, techni-cal limitations, and lack of support for necessary functionality. To solve these issues, a service model of a heterogeneous distributed database is developed, providing a descrip-tion of its characteristics and architecture. Using Kubernetes technology, performance metrics of the database cluster based on the developed model are investigated, demon-strating superior performance compared to the products discussed.

Research on ecosystem detection based on wireless communication network

Ecosystem monitoring is of great significance in environmental protection and sustainable development. Wireless communication networks provide a broad application prospect for ecosystem monitoring, but there is a lack of further research on the strengths and weaknesses of the existing research results and specific application scenarios, as well as on the reliability of data processing. This paper aims to explore the theoretical basis of the reliability of wireless communication networks in ecosystem monitoring. Firstly, the current status, characteristics and challenges of wireless communication networks in ecosystem monitoring are reviewed, the key elements in ecosystem monitoring and the requirements for data integrity and accuracy are analyzed in detail, and the reliability problems of signal interference, node failure and data loss in data transmission are discussed. Secondly, methods and strategies to improve the reliability of ecosystem monitoring based on wireless communication networks are explored. By reviewing and analyzing the existing research results, the advantages and disadvantages of different methods as well as the applicable scenarios are elaborated. Finally, the paper proposes directions and challenges for future research. The exploration of emerging technologies is emphasized. At the same time, issues that need to be addressed, network security, energy efficiency and cost feasibility are pointed out. The significance of this paper in academic and industry practice can further deepen the exploration of the theoretical basis of reliability and combine it with practical application scenarios for verification and optimization.

Enabling Integrity and Compliance Auditing in Blockchain-Based GDPR-Compliant Data Management

The General Data Protection Regulation (GDPR) is a European Union (EU) data protection and privacy law. According to the GDPR, the data on a hosting platform must meet semantic consistency and data integrity requirements. Semantic consistency means that the data operation should comply with the GDPR, while data integrity is meant to ensure that the outsourcing data should be intact. The two terms are not interchangeable. For example, if a cloud service provider migrates data to foreign storage nodes without authorization of the data owner, the data integrity requirement of the GDPR is met but the semantic consistency requirement is not. How to ensure data integrity and compliance is the main challenge for a GDPR-compliant data supervision platform. To achieve this aim, we leverage a blockchain based data management framework to check the data compliance, which can break the black box of the data hosting platform and demonstrate its logic to data owners, allowing for inspection. We propose a new provable data possession (PDP) scheme for the aforementioned framework that can check for semantic consistency and data integrity simultaneously. The verifier does not need to hold any audited data, which can reduce bandwidth usage. The verification result can be regarded as the proof for subsequent data recovery and accountability. Experimental results show higher efficiency of the PDP scheme.

Security of Cyber-Physical Systems of Chemical Manufacturing Industries Based on Blockchain

The traditional manufacturing systems are often enterprise-centric systems, whereas the modern chemical industry is oriented towards industrial chain integration. Enterprise entities present a loosely coupled state at the scale of the industrial chain, with decentralized characteristics. This poses greater challenges and requirements for the industrial safety system. Based on the characteristics of the chemical manufacturing industry and blockchain, the application of the information security of blockchain in the chemical manufacturing industry is studied herein and the cyber-physical systems security architecture model of dual blockchains is proposed. The first-layer blockchain is applied at the system’s core function level to solve security issues at the system level and provide security guarantees for communication, transactions, and billing between users and manufacturers. Meanwhile, the second layer involves the system resource layer, which not only solves the security problem of cross-level platform data interaction, but also enables the point-to-point security of the device-level cyber-physical system to ensure internal equipment communication information security. A domestic commercial concrete manufacturing company’s real production and operation data were used to simulate basic functions such as transaction requests, trade success, and blockchain queries. After multiple tests, results show that its basic blockchain, query response, transaction creation, and block creation functions are all finished within milliseconds, meeting the industrial requirements. Its safety verification can meet the requirements of safety, efficiency, and low latency for production control in chemical industry sites, proving the feasibility of applying the dual blockchain model in the chemical manufacturing industry. Based on data security, privacy, and integrity requirements, the blockchain technology proposed in this article provides a more efficient, transparent, and secure operation and management solution for the chemical industry.

제약산업에서의 데이터 완전성(Data Integrity) 관련 최근 이슈 및 규제 동향

Pharmaceutical manufacturers are responsible for providing safe and effective high-quality products to patients and consumers by assuring that decisions on production and distribution of products are based on accurate, reliable, truthful, and complete data. Nevertheless, in recent years, regulatory authorities have increasingly observed good manufacturing practice (GMP) violations related to data integrity during GMP inspections. This is problematic since ensuring data integrity is a mandatory requirement to assure the safety, efficacy, and quality of drug products. The breaches in data integrity found during inspections have set off the need to publish more detailed guidance documents that describe more clearly the regulatory expectations in GMP-related systems. With new awareness derived from recent regulatory restrictions, industry uses state-of-the-art technology to improve processes and establish systems for detection and mitigation of gaps that affect data integrity in paper and computerized systems. Recently issued guidances by regulatory agencies emphasize the importance of good laboratory and manufacturing documentation practices. The main purpose of regulatory requirements remains confident in the quality and integrity of the data generated to ensure patient safety and product quality. This review summarizes data integrity guidances and risk assessment that facilitate firms to develop robust data integrity management system with both manual processes with paper records and computerized systems.

Data Integrity and Risk

Data Integrity is a critical component of Data lifecycle management. Its importance increases even more in a complex and dynamic landscape. Actions like unauthorized access, unauthorized modifications, data manipulations, audit tampering, data backdating, data falsification, phishing and spoofing are no longer restricted to rogue individuals but in fact also prevalent in systematic organizations and states as well. Therefore, data security requires strong data integrity measures and associated technical controls in place. Without proper customized framework in place, organizations are prone to high risk of financial, reputational, revenue losses, bankruptcies, and legal penalties which we shall discuss further throughout this paper. We will also explore some of the improvised and innovative techniques in product development to better tackle the challenges and requirements of data security and integrity.

Exploring the Data Integrity Ordinance in Pharmaceutical Domain

Published on: October 2022Journal of Young Pharmacists, 2022; 14(4):377-381Review Article | doi:10.5530/jyp.2022.14.76 Authors:Darshil B Shah1,*, Jignesh S Shah2, Dilip G Maheshwari11Department of Quality Assurance, L J Institute of Pharmacy, LJ University, Ahmedabad, Gujarat, INDIA.2Department of Regulatory Affairs, L J Institute of Pharmacy, LJ University, Ahmedabad, Gujarat, INDIA. Abstract:With the revolutionary expedition of technologies being used in pharmaceutical world to collect and process data, the honesty of data is predominant factor and subject to preservation. If the data collection of the products soon to be marketed, is adulterated it would not only be a potential harm to general public but also cause of economical and reputational damage for manufacturers. To secure the value of crucial data, regulatory authorities has been producing guidance regarding the integrity of data. Data integrity guarantees that data is not tampered with be it paper or electronic. Protecting the integrity, whether electronic or handwritten, is essential for making sound business decisions. In attempt of introducing aspects of data, the FDA first developed the ALCOA principles which are still the first line defence of integrity breaches in pharmaceutical companies. With the changing times of technologies, FDA evolved the data integrity principles into requirements and specifications of current good manufacturing practice answering questions directly from manufacturers. To understand the basic yet complex terms and conditions, this article strives to outline the Data Integrity requirements of USFDA specified in its cGMP guidance for data integrity specifications.Keywords: Pharmaceutical Data Integrity, ALCOA, U.S. Food and Drug Administration (FDA).

Secure advance metering infrastructure protocol for smart grid power system enabled by the Internet of Things

The incorporation of Advanced Metering Infrastructure (AMI) into Smart Grid applications has piqued the interest of Internet of Things (IoT) experts, who have aided smart city initiatives. One of the fundamental building components of AMI for information sharing is wireless communication networks. Wireless communication networks are prone to a variety of security threats, which causes issues when developing the AMI system in smart city projects. Confidentiality, integrity, availability, and privacy are the primary criteria of cyber-physical systems such as AMI. To meet the needs of cyber-physical systems, we worked on a wireless communication protocol that solves data transmission dependability and data security while consuming little power and overhead. We proposed a novel Secure and Reliable AMI (SRAMI) protocol in this paper using a lightweight cryptographic technique and trust-based mechanism for reliability for Wireless Sensor Network (WSN)-assisted AMI. The trust-based approach presents a trustworthy data transfer technique between two communicating AMI entities. In this regard, the next-hop is chosen based on trust-evaluation characteristics such as energy expenditure, geographical distance, and bandwidth availability. Furthermore, a lightweight hybrid approach based on Elliptic Curve Cryptography (ECC) is introduced to meet the data integrity and privacy requirements in the AMI network. SRAMI's performance was improved compared to state-of-the-art protocols in terms of throughput, Packet Delivery Ratio (PDR), average energy consumption, latency and communication overhead.

Mapping Late Hokusai Research: Digitizing and Publishing Bilingual Research Data

The initiative “Late Hokusai: Thought, Technique, Society” took place at the British Museum (BM) and SOAS, University of London (2016–2019). As part of its activities, it built a linked-data platform prototype on ResearchSpace. The prototype offers a redesigned process for how museum researchers and users find, research with, discuss and expand bilingual data about early modern Japanese artist Katsushika Hokusai (1760–1849) and instigated a discussion about what a collaborative research platform for the Hokusai research community could look like. While Japanese resource specialists have long recognized the complexity of Japanese script as a challenge for multilingual research and collection platforms, the processes for and results of integrating Japanese source data into bi- or multilingual museum databases remained unsatisfactory.This paper revisits the challenges posed by “non-Latin script” (NLS) in museum databases in the case of the Hokusai research platform at the British Museum, which integrated Japanese and English languages. It localizes the issues arising from working with Japanese source data in the Latin script project environment and accompanies the museum researchers’ tasks regarding the correct input, rendering and display of the source script at each step: 1) object analysis, 2) registering NLS metadata, 3) processing NLS information and 4) visualizing LS and NLS information for general and specialist audiences. After assessing these practices, the paper critically reflects on selected approaches, successes, and shortcomings experienced while creating such a prototype. By sharing its experiences, the project hopes to aid prospective research projects on a similar path regarding project setup and documentation. Furthermore, it advocates the sustainability of research practices according to data reusability parameters.L’initiative « Late Hokusai : Thought Technique and Society » (Hokusai tardif : Pensées techniques et société) a eu lieu au British Museum (BM) et SOAS, l’Université de Londres (2016-2019). Dans le cadre des activités, cette initiative a produit une plateforme prototype de Web des données sur ResearchSpace. Le prototype offre un processus redessiné aidant les chercheurs de musée et les usagers à trouver, à faire de la recherche, à discuter et à étoffer les données bilingues concernant l’artiste Katsushika Hokusai (1760-1849), du début de l’ère moderne japonaise. Cela a déclenché une discussion sur l’apparence possible d’une plateforme de recherche collaborative dédiée à la communauté de recherche sur Hokusai. Tandis que les spécialistes de ressources japonaises reconnaissent depuis longtemps la complexité de l’écriture japonaise comme un défi pour la recherche multilingue et pour les plateformes de collection, les processus et les résultats de l’intégration des données sources japonaises dans des bases de données de musées bi- ou plurilingues demeurent insatisfaisants.Cet article réexamine les défis liés à des « écritures non-latines » (NLS, non-Latin script) dans des bases de données de musée dans le cas de la plateforme de recherche sur Hokusai au British Museum, ce qui a intégré les langues japonaise et anglaise. L’article localise les questions qui se posent durant le travail avec les données sources japonaises dans un environnement de projet en écriture latine et accompagne les tâches des chercheurs de musée concernant l’entrée correcte, le rendu et l’affichage de l’écriture source à chaque étape : 1) les analyses d’objet, 2) les enregistrements de métadonnées NLS, 3) le traitement de l’information NLS et 4) la visualisation de l’information LS (écriture latine, Latin script) et NLS pour des audiences générales et spécialistes. Cet article présentera une évaluation de ces pratiques et, ensuite, considérera de façon critique les approches sélectionnées, les succès et les défauts rencontrés pendant la création d’un tel prototype. En partageant ces expériences, ce projet vise à aider des projets de recherche prospectifs qui se trouvent dans un cas similaire, considérant la configuration de projets et la documentation. En outre, ce projet promeut la viabilité de pratiques de recherche conformément à des paramètres de réutilisation de données.

Management аccounting: What is subject to digital transformation?

The subject of the article are the problems of transformation on the basis of breakthrough digital technologies of activity on information support of management and use of accounting information in managerial decision-making. The authors in the course of the research analyze the current state of management accounting and prove that only a factor information model is able to ensure the implementation of the functions of the actual management of the object, allowing observation based on operational, technical, accounting, statistical and data integration requirements. On the example of the system of productivity indicators (qualitatively defined quantity), it is demonstrated that it is possible to implement such requirements for this factor as the object, accuracy and specificity of the work; and an example of a system of indicators for fixed assets — harmonies the functions performed for the object. The article also substantiates the conclusion that the solution of the problems of digital transformation of speech can be provided by an information system functioning based on the platform using Big Data and cloud technologies — DaaS interacting with active elements of the Internet community. In the course of the study, in generalization of modern concepts of management of economic systems, directions of development of digital technologies and their implementation in the processes of information management, methods of system and comparative analysis were used.

SmartAuction: A blockchain-based secure implementation of private data queries

A data marketplace provides a platform for data trading by bringing together data owners and data consumers with a data broker. Recent advancements in mechanism design of data marketplaces have introduced a number of private data query mechanisms that facilitate the trading of private data. A critical assumption to ensuring that these mechanisms function as intended in a real-world online platform, which is often overlooked in the mechanism design literature, is that they are implemented in a network infrastructure that guarantees secure and fair trade. Such a network infrastructure should allow data storage, communications and transactions to satisfy confidentiality, data integrity, fair brokerage, and privacy requirements. In this paper, we propose SmartAuction, the first blockchain-based secure implementation of a private data query system. SmartAuction utilises a public-key encryption scheme, a digital signature scheme and two smart contracts to facilitate the four tasks of a data trade: data submission, data request, auction and query, and payment and delivery. Using universal composability (UC) framework, we verify that SmartAuction satisfies the desirable properties mentioned above. We also conduct experiment to show that SmartAuction can be implemented in real Ethereum platforms and it has a broad applicability.

WIDE: A witness-based data priority mechanism for vehicular forensics

In this paper, we present a WItness based Data priority mEchanism (WIDE) for vehicles in the vicinity of an accident to facilitate liability decisions. WIDE evaluates the integrity of data generated by these vehicles, called witnesses, in the event of an accident to assure the reliability of data to be used for making liability decisions and ensure that such data are received from credible witnesses. To achieve this, WIDE introduces a two-level integrity assessment to achieve end-to-end integrity by initially ascertaining the integrity of data-producing sensors, and validating that data generated have not been altered on transit by compromised road-side units (RSUs) by executing a practical byzantine fault tolerance (pBFT) protocol to reach consensus on data reliability. Furthermore, WIDE utilises a blockchain based reputation management system (BRMS) to ensure that only data from highly reputable witnesses are utilised as contributing evidence for facilitating liability decisions. Finally, we formally verify the proposed framework against data integrity requirements using the Automated Verification of Internet Security Protocols and Applications (AVISPA) with High-Level Protocol Specification Language (HLPSL). Qualitative arguments show that our proposed framework is secured against identified security attacks and assures the reliability of data utilised for making liability decisions, while quantitative evaluations demonstrate that our proposal is practical for fully autonomous vehicle forensics.

A Cloud-Based Framework for Agricultural Data Integration: A Top-Down-Bottom-Up Approach

In recent years, growing use of information technology (IT) and the big data revolution in agriculture have led many farms to adopt precision agriculture methods and to accumulate large amounts of data. Exploiting these data for effective decision support requires the ability to integrate data from various sources, to conveniently analyze the data, and to infer valuable insights. This paper presents a framework for integrating and analyzing agricultural data from various sources, which leverages cloud-computing, thereby contributing to the scalability, flexibility, affordability, and maintainability of the solution compared to existing solutions. The framework defines a functional infrastructure of cloud-based services that facilitate integration, analysis, and data visualization. These services can be either end-user applications or services intended as a platform for creating new services. To design the framework’s architecture, we applied a top-down and bottom-up approach. Based on the top-down analysis of information collected through interviews, questionnaires, and literature review, we defined an initial architecture of the framework. We then used this initial architecture to develop different applications, and based on the experience and insights gained and new requirements that were faced, the architecture was revised in an iterative process. We demonstrate the application of the framework through several use cases. Each use case represents different data integration requirements and is based on different services of the proposed framework.

Toward a Platform for Structured Data Acquisition in Oncology: A Pilot Study on Prostate Cancer Screening

Introduction: Physicians spend an ever-rising amount of time to collect relevant information from highly variable medical reports and integrate them into the patient’s health condition. Objectives: We compared synoptic reporting based on data elements to narrative reporting in order to evaluate its capabilities to collect and integrate clinical information. Methods: We developed a novel system to align medical reporting to data integration requirements and tested it in prostate cancer screening. We compared expenditure of time, data quality, and user satisfaction for data acquisition, integration, and evaluation. Results: In a total of 26 sessions, 2 urologists, 2 radiologists, and 2 pathologists conducted the diagnostic work-up for prostate cancer screening with both narrative reporting and the novel system. The novel system led to a significantly reduced time for collection and integration of patient information (91%, p < 0.001), reporting in radiology (44%, p < 0.001) and pathology (33%, p = 0.154). The system usage showed a high positive effect on evaluated data quality parameters completeness, format, understandability, as well as user satisfaction. Conclusion: This study provides evidence that synoptic reporting based on data elements is effectively reducing time for collection and integration of patient information. Further research is needed to assess the system’s impact for different patient journeys.

State Analysis of Apple Industry in China

China is the first apple production country in the world, and apple planting industry occupies an important position in China, so through the analysis of the state of apple industry in China. It is of great significance to stabilize the apple commodity market and promote the healthy development of the industry. According to the preliminary analysis of the data, it can be found that with the increase of the year, the apple yield has a significant linear growth trend. Therefore, through comprehensive analysis, this paper establishes a regression model and uses MATLAB to solve. Based on the test and analysis of the model, five criteria are given to judge the merits and demerits of the data, and the requirements of data integrity, that is, whether there is abnormal data culling in the process of model fitting. By analyzing the export volume of fresh apple and apple juice in China and the world, the export scatter model of fresh apple and apple juice in China and the world is established. The scattered point model of apple planting area between China and the world is established. It shows that China’s apple market plays a very important role in and influences international trade. China is the largest apple producer in the world and plays an important role in the international apple production. The influencing factors of fresh apple export include distance, economic development of importing country, international politics, product quality, cost, market environment and brand. In terms of the future change law and forecast, the development level of the apple industry is constantly improving, the supply will continue to exceed the demand in the next few years. And the polarization phenomenon of domestic apple price will be more and more obvious.

Optimizing Behavioral Paradigms to Facilitate Development of New Treatments for Anhedonia and Reward Processing Deficits in Schizophrenia and Major Depressive Disorder: Study Protocol.

Background: Behavioral tasks focusing on different subdomains of reward processing may provide more objective and quantifiable measures of anhedonia and impaired motivation compared with clinical scales. Typically, single tasks are used in relatively small studies to compare cases and controls in one indication, but they are rarely included in larger multisite trials. This is due to limited systematic standardization as well as the challenges of deployment in international studies and stringent adherence to the high regulatory requirements for data integrity. The Reward Task Optimization Consortium (RTOC) was formed to facilitate operational implementation of reward processing tasks, making them suitable for use in future large-scale, international, multisite drug development studies across multiple indications. The RTOC clinical study aims to conduct initial optimization of a set of tasks in patients with major depressive disorder (MDD) or schizophrenia (SZ).Methods: We will conduct a multicenter study across four EU countries. Participants (MDD = 37, SZ = 37, with ≤80 age- and gender-matched healthy volunteers) will attend a study visit comprising screening, self-report and clinically rated assessments of anhedonia and symptom severity, and three reward processing tasks; specifically, the Grip Strength Effort task, the Doors task, and the Reinforcement Learning Working Memory task. The Grip Strength Effort and Doors tasks include simultaneous electroencephalography/event-related potential recordings. Outcomes will be compared using a two-way group design of MDD and SZ with matched controls, respectively. Further analyses will include anhedonia assessment scores as covariates. Planned analyses will assess whether our findings replicate previously published data, and multisite deployment will be evaluated through assessments of quality and conduct. A subset of participants will complete a second visit, to assess test–retest reliability of the task battery.Discussion: This study will evaluate the operational deployment of three reward processing tasks to the regulatory standards required for use in drug development trials. We will explore the potential of these tasks to differentiate patients from controls and to provide a quantitative marker of anhedonia and/or impaired motivation, establishing their usefulness as endpoints in multisite clinical trials. This study should demonstrate where multifaceted reward deficits are similar or divergent across patient populations.Registration: ClinicalTrials.gov (NCT04024371).

The Golem In The Machine: FERPA, Dirty Data, and Digital Distortion in the Education Record

Like its counterpart in the criminal justice system, dirty data—data that is inaccurate, incomplete, and or misleading—in K-12 education records create and catalyze catastrophic life events. The presence of this data in any record suggests a lack of data integrity. The systemic problem of dirty data in education records means the data stewards of those records have failed to meet the data integrity requirements embedded in the Family Educational Rights and Privacy Act (FERPA). FERPA was designed to protect students and their education records from the negative impact of erroneous and misinformation rendered from the “private scribblings” of educators. The legislative history of FERPA indicates that legislators were concerned about the harm to students’ education and structure of opportunities based on misinformation in secret files created and kept in schools. Dirty data created, collected, and processed as accurate and reliable, notwithstanding the disproportionate impact of school discipline, on marginalized students in general, and Black children specifically, is exactly the kind of harm that FERPA was intended to prevent. This Article demonstrates (1) How educational inequities linked to dirty data violates FERPA and (2) How FERPA should be enhanced to prevent dirty data harms at the point of collection and creation. Additionally, this Article outlines the concept of dirty data and data integrity requirements embedded in FERPA and proceeds to examine the phenomenon of dirty data and student harm in historically marginalized students’ education records, starting at the point of creation and collection. While several Articles have examined the failure of FERPA, none of the prior scholarship has analyzed FERPA’s connection to dirty data in the education record related to racial discrimination. This Article introduces a two-step process that would require input validation in the educational record context through (1) substantive content and input validation and (2) a reasonable inference review. Additionally, this Article introduces an accounting of disclosures to law enforcement requirement.

Relational into Non-Relational Database Migration with Multiple-Nested Schema Methods on Academic Data

The rapid development of internet technology has increased the need of data storage and processing technology application. One application is to manage academic data records at educational institutions. Along with massive growth of information, decrement in the traditional database performance is inevitable. Hence, there are many companies choose to migrate to NoSQL, a technology that is able to overcome the traditional database shortcomings. However, the existing SQL to NoSQL migration tools have not been able to represent SQL data relations in NoSQL without limiting query performance. In this paper, a relational database transformation system transforming MySQL into non-relational database MongoDB was developed, using the Multiple Nested Schema method for academic databases. The development began with a transformation scheme design. The transformation scheme was then implemented in the migration process, using PDI/Kettle. The testing was carried out on three aspects, namely query response time, data integrity, and storage requirements. The test results showed that the developed system successfully represented the relationship of SQL data in NoSQL, provided complex query performance 13.32 times faster in the migration database, basic query performance involving SQL transaction tables 28.6 times faster on migration results, and basic performance Queries without involving SQL transaction tables were 3.91 times faster in the migration source. This shows that the theory of the Multiple Nested Schema method, aiming to overcome the poor performance of queries involving many JOIN operations, is proved. In addition, the system is also proven to be able to maintain data integrity in all tested queries. The space performance test results indicated that the migrated database transformed using the Multiple Nested Schema method showed a storage requirement of 10.53 times larger than the migration source database. This is due to the large amount of data redundancy resulting from the transformation process. However, at present, storage performance is not a top priority in data processing technology, so large storage requirements are a consequence of obtaining efficient query performance, which is still considered as the first priority in data processing technology.

A framework for corridor-level traffic safety network screening and its implementation using Business Intelligence

Currently, there are both methodological and practical barriers that together preclude the use of theoretically sound approaches for network screening as part of a traffic safety management process. Methodological barriers include, among others, lack of a comprehensive framework for corridor-level network screening. Existing corridor screening methodologies use observed crash frequency as a performance measure. In practice, corridor-level screening is extremely important because traffic safety engineers prefer to deploy countermeasures and provide homogenous conditions throughout corridors to meet drivers’ expectations and avoid confusion. On the other hand, practical barriers that limit the use of sound approaches for traffic safety include (1) significant data integration requirements, (2) a particular data schema is needed to enable analysis using specialized software, (3) time-consuming and intensive processes are involved, (4) substantial technical knowledge is needed, (5) visualization capabilities are limited, and (6) coordination across various data owners is required. This research proposes a systematic methodology for corridor-level network screening. The solution algorithm is implemented within a Business Intelligence (BI) platform to address, to the extent possible, the practical barriers listed above. BI provides methods and mechanisms to integrate and process data, generate advanced analytics, and visualize results by using intuitive and interactive web-based dashboards and maps. Experiments and results illustrate the advantage of using the proposed framework for corridor-level network screening implemented within a BI platform.