Data Management Architecture Research Articles

Scaling Data Engineering with Advanced Data Management Architecture: A Comparative Analysis of Traditional ETL Tools Against the Latest Unified Platform

Scaling Data Engineering with Advanced Data Management Architecture: A Comparative Analysis of Traditional ETL Tools Against the Latest Unified Platform

Privacy vs convenience: Understanding intention-behavior divergence post-GDPR

The privacy paradox describes a scenario in which individuals express privacy concerns but still share private data online. We explore how the paradox can be understood following the introduction of the European Union's landmark GDPR (General Data Protection Regulation) legislation. Through qualitative interviews with online platform users, we find that individuals are concerned about personal data but remain constrained in their privacy self-management. In this context of limited perceived control, users' privacy attitudes are guided by anticipated value from using the platform and the convenience of privacy protection measures. Our study also highlights the role of peer influence on users' privacy choices, specifically through micro- and macro-network effects. We identify that (1) users move to privacy-protecting platforms to align with their social network, or because of information disseminated within their networks; and (2) users remain on platforms offering minimal privacy protection despite privacy concerns due to presence of their entire peer network. These findings provide a unified view on Privacy Paradox post-GDPR, bringing together a more comprehensive range of influences on individual-level privacy dynamics. Our research underscores the need for policymakers to streamline and standardize data protection measures lest the intentions of GDPR be undermined. We also highlight the need to go beyond a reliance on privacy self-management by better regulating the architecture of data management and enforcing principles of privacy by design and default.

Enrollment Appraisal Using Rule-Based Decision Support System

This study explores the development and evaluation of a web-based enrollment appraisal system utilizing a rule-based decision support system for universities and colleges. Designed for the context of Antique, Philippines, the system leverages a two-tier architecture for efficient processing and secure data management. A user-friendly interface guides applicants through the enrollment process, while a secure server and rule-based engine analyze data against pre-defined eligibility criteria. The Rapid Application Development (RAD) methodology ensured a user-centered design with iterative development cycles incorporating user feedback. Evaluations using McCall's Software Quality Standard and the International Organization for Standardization and International Electrotechnical Commission (ISO/IEC) 25010 standard revealed the system's strengths in technical aspects (security, usability) and overall functionality. User feedback highlighted areas for improvement in interface design, clarity, and user perceptions of data security. The study concludes that the system offers a promising approach for more efficient and transparent enrollment processes. By addressing the feedback received, the system can be further enhanced to improve the user experience and build trust among applicants.

Data management of X bill of material of missile and rocket products based on digital thread

The X bill of material (XBOM) of a missile and rocket product is a comprehensive and basic support document produced by a multi-stakeholder that considers structural composition, data information and interactional processes in different life stages. The XBOM data management of missile and rocket products coordinates the data organization and control in the whole life cycle, eliminates the interactional disconnection and one-way transmission between related parties, business and data in different life stages and provides effective data source support for the full digital model supply. To meet the requirements of organic integration and multi-party collaboration, this paper proposes the XBOM data management method based on digital thread. Firstly, the digital thread framework is analyzed, and the XBOM data management architecture based on the digital thread is proposed and established. Secondly, guided by the hierarchy of "data-interaction-modeling-application", the XBOM data management method for the missile and rocket product are developed, including data organization, data relational analysis, data modeling and data application. Finally, the structural system of a certain model of carrier rocket is studied, and the XBOM data management system based on digital thread is developed and its application is validated.

Reference architecture design for developing data management systems in smart farming

The traditional data management systems prove inadequate to handle the volume, velocity, and variety of the data within farm business processes. Smart farming technologies offer advanced data management systems as a practical solution to these challenges. However, data is complex and originates from many sources; hence many aspects of data must be considered during the data management design of smart farming systems. This study proposes a reference architecture for data management in smart farming, developed through domain analysis and architecture modeling approaches. The domain analysis provides insights into the common and variant features and modules of the smart farming system, resulting in a blueprint representing family features across various smart farming domains. The effectiveness of the proposed reference architecture has been evaluated through two case studies, demonstrating its efficacy in designing data management systems for smart farming. The study found that the percentage of reused modules in the case studies, compared to the provided reference architecture, was 82.6%. The outcomes of this research will pave the way for further exploration in smart farming, particularly addressing data management issues within smart farming systems.

Development of a blockchain-based architecture for the health data management and sharing

Development of a blockchain-based architecture for the health data management and sharing

Development of a blockchain-based architecture for health data management and sharing

Development of a blockchain-based architecture for health data management and sharing

Moduli: A Disaggregated Data Management Architecture for Data-Intensive Workflows

As companies store, process, and analyse bigger and bigger volumes of highly heterogeneous data, novel research and technological challenges are emerging. Traditional and rigid data integration and processing techniques become inadequate for a new class of data-intensive applications. There is a need for new architectural, software, and hardware solutions that are capable of providing dynamic data integration, assuring high data quality, and offering safety and security mechanisms, while facilitating online data analysis. In this context, we propose moduli , a novel disaggregated data management reference architecture for data-intensive applications that organizes data processing in various zones. Working on moduli allowed us also to identify open research and technological challenges.

Legal Challenges for International Fintech Startups

Financial technology (Fintech) startups looking to scale globally often face complex regulations across jurisdictions inhibiting market access and rapid customer adoption. This research investigates legal obstacles frequently encountering international Fintech platforms through qualitative research methods. It aims to highlight recurring concerns like data sovereignty, interoperability, licensing barriers and talent acquisition issues for innovators attempting multi-country service availability. Though results indicate regulatory principles remain largely unharmonized currently, recommendations urge sectoral cooperation advancing unified standards spanning data management, oversight rules and open banking architectures. Phasing proportionate need-based compliances and controlled sandboxes can enable clearer policy signaling benefiting responsibly expanding startups. Effective policy incentives and dispute assistance schemes may ease market entry and sustain trust lacking today. However, truly unraveling Fintech possibilities requires expediting progressive, agile and internationally integrated legal systems protecting consumers while fostering inclusive innovation ecosystems.

Hypermedia-based software architecture enables Test-Driven Development.

Using agile software development practices, develop and evaluate an architecture and implementation for reliable and user-friendly self-service management of bioinformatic data stored in the cloud. Comprehensive Oncology Research Environment (CORE) Browser is a new open-source web application for cancer researchers to manage sequencing data organized in a flexible format in Amazon Simple Storage Service (S3) buckets. It has a microservices- and hypermedia-based architecture, which we integrated with Test-Driven Development (TDD), the iterative writing of computable specifications for how software should work prior to development. Relying on repeating patterns found in hypermedia-based architectures, we hypothesized that hypermedia would permit developing test "templates" that can be parameterized and executed for each microservice, maximizing code coverage while minimizing effort. After one-and-a-half years of development, the CORE Browser backend had 121 test templates and 875 custom tests that were parameterized and executed 3031 times, providing 78% code coverage. Architecting to permit test reuse through a hypermedia approach was a key success factor for our testing efforts. CORE Browser's application of hypermedia and TDD illustrates one way to integrate software engineering methods into data-intensive networked applications. Separating bioinformatic data management from analysis distinguishes this platform from others in bioinformatics and may provide stable data management while permitting analysis methods to advance more rapidly. Software engineering practices are underutilized in informatics. Similar informatics projects will more likely succeed through application of good architecture and automated testing. Our approach is broadly applicable to data management tools involving cloud data storage.

Blockchain-Based Privacy-Preserving Authentication Model Intelligent Transportation Systems

Intelligent Transportation Systems (ITS) have gained popularity due to smart services and applications to facilitate the users on the roads. The increasing growth of users in these networks created new and complex data processing, storage, security, and privacy concerns. These networks are using centralized edge, fog, or cloud architecture for data management. User privacy is compromised in these networks due to the increasing demands and service provider’s services. To ensure the data privacy, the centralized architectures are used without privacy regulations. In this paper, we present a Blockchain-based Privacy-Preserving Authentication (BPPAU) model for ITS networks to ensures users privacy and security. The proposed model provides data storage, data accessing, and processing management by using a blockchain smartcontract system, access control policy and on demand based functions. The proposed model is tested in a simulation environment to check its performance in terms of transaction cost with data size, transaction per second analysis with block time, and computational time analysis with several transactions.

Designing an Efficient Master Data Management (MDM) using Cloud-based Solution

Abstract: Data is a valuable resource that is utilised as a deciding factor for the operations and demands of the organisation. The information is gathered from a number of different databases, rather than just one, in order to form a single cohesive whole. The occurrence of duplicate data in various forms, even when the data is the same, presents a challenge throughout the process. Issues that can be improved by the use of sound data governance. References and master information is one approach used in the management of data. This procedure may serve as the basis for developing master data management solutions on a cloud-based platform. The analysis and creation of new overall data management architectures are the main goals of this study. Consequently, provide a framework for master managing information tools for free software systems.

Data Evolution Governance for Ontology-Based Digital Twin Product Lifecycle Management

Product lifecycle management (PLM) is an effective method for enhancing the market competitiveness of modern manufacturing industries. The digital twin is characterized by a profound integration of physics and information systems, which provides a technical means for integrating multisource information and breaking the time and space barrier of communication at each link of the lifecycle. Currently, however, the application of this technology focuses primarily on the product itself and “service-oriented” application results. There is a lack of focus on twin data and its internal evolutionary mechanisms separately. In the management of global data resources, the benefits of digital twin technology cannot be fully realized. This article applies ontology technology in an innovative manner to the field of the digital twin to increase the reusability of twin data. Initially, a four-layered ontology-based twin data management architecture is presented. Then, a three-dimensional and three-granularity unified evolution model of full lifecycle twin data is proposed, as well as its ontology model. Then, the service mode of data components at each stage of the lifecycle is defined, a knowledge-sharing plane is established in the digital twin, and a data governance method based on ontology reasoning using data components on the shared plane is proposed. The ICandyBox simulation platform is then used to demonstrate the concept of the proposed method, and future research directions are proposed.

GamaBox-One: A Proposed Architecture for Cloud-based Big Data Management Platform for Multipurpose Computation using Hybrid Architecture

The architecture of the data center is the main key for producing a highly functional demand big data management platform for multipurpose uses. Nowadays various technologies have come to provide construction of that purpose, providing several use cases for big data analytics and processing. In this paper, we want to explore possibilities of architecture that had to be built in answer to the multipurpose data center, such as analytical research, scientific simulation, machine learning, deep data learning, and data orchestration. We discover how Hadoop and its element supporter can be used alongside cloud orchestrators such as Terraform or Occopus and container orchestrators such as Kubernetes or Docker Swarm. We also provide possible supporting components that can handle the different jobs in High-Performance Computing and how the system can be secured. Our proposed approach in this research has developed the architecture for cloud-based big data management for multipurpose computation.

ChainSure: Agent free insurance system using blockchain for healthcare 4.0

The health insurance industry requires a secure data management architecture characterized by reliable information processing and fast response. Classical health insurance management system runs on a manually controlled centralized architecture which leverages a vulnerable single point failure and requires frequent human intervention. This makes the model prone to human errors and security attacks. The existing system also lacks the flexibility to choose appropriate health insurance policies for different consumers. To this end, we propose an Ethereum blockchain-based framework, ChainSure, to address this problem. With the power of TOPSIS and smart contracts, ChainSure provides an automated, tamperproof, transparent, scalable system that takes care of all the major functional blocks in a medical insurance environment. We have used the TOPSIS method in this proposed model to help the users to find an insurance policy that best suits their needs. ChainSure works in a decentralized environment and provides an interactive interface for the user. The proposed model is implemented on the Ethereum test network and its performance has been compared empirically with other state-of-art models. ChainSure is found to outperform others in terms of service integrity, latency and cost.

Industry-Oriented System Architecture for Feature-Based Data Management in CNC Machining Processes

Industry-Oriented System Architecture for Feature-Based Data Management in CNC Machining Processes

A Novel Tightly Coupled Information System for Research Data Management

Most research projects are data driven. However, many organizations lack proper information systems (IS) for managing data, that is, planning, collecting, analyzing, storing, archiving, and sharing for use and re-use. Many research institutions have disparate and fragmented data that make it difficult to uphold the FAIR (findable, accessible, interoperable, and reusable) data management principles. At the same time, there is minimal practice of open and reproducible science. To solve these challenges, we designed and implemented an IS architecture for research data management. Through it, we have a centralized platform for research data management. The IS has several software components that are configured and unified to communicate and share data. The software components are, namely, common ontology, data management plan, data collectors, and the data warehouse. Results show that the IS components have gained global traction, 56.3% of the total web hits came from news users, and 259 projects had metadata (and 17 of those also had data resources). Moreover, the IS aligned the institution’s scientific data resources to universal standards such as the FAIR principles of data management and at the same time showcased open data, open science, and reproducible science. Ultimately, the architecture can be adopted by other organizations to manage research data.

Data Management and Network Architecture Effect on Performance Variability in Direct Attenuation Correction via Deep Learning for Cardiac SPECT: A Feasibility Study.

Attenuation correction (AC) is important for accurate interpretation of SPECT myocardial perfusion imaging (MPI). However, it is challenging to perform AC in dedicated cardiac systems not equipped with a transmission imaging capability. Previously, we demonstrated the feasibility of generating attenuation-corrected SPECT images using a deep learning technique (SPECTDL) directly from non-corrected images (SPECTNC). However, we observed performance variability across patients which is an important factor for clinical translation of the technique. In this study, we investigate the feasibility of overcoming the performance variability across patients for the direct AC in SPECT MPI by proposing to develop an advanced network and a data management strategy. To investigate, we compared the accuracy of the SPECTDL for the conventional U-Net and Wasserstein cycle GAN (WCycleGAN) networks. To manage the training data, clustering was applied to a representation of data in the lower-dimensional space, and the training data were chosen based on the similarity of data in this space. Quantitative analysis demonstrated that DL model with an advanced network improves the global performance for the AC task with the limited data. However, the regional results were not improved. The proposed data management strategy demonstrated that the clustered training has potential benefit for effective training.

A privacy-preserving blockchain-based tracing model for virus-infected people in cloud

The outbreak of COVID-19 has exposed the privacy of positive patients to the public, which will lead to violations of users’ rights and even threaten their lives. A privacy-preserving scheme involving virus-infected positive patients is proposed by us. The traditional ciphertext policy attribute-based encryption (CP-ABE) has the features of enhanced plaintext security and fine-grained access control. However, the encryption process requires the high computational performance of the device, which puts a high strain on resource-limited devices. After semi-honest users successfully decrypt the data, they will get the real private data, which will cause serious privacy leakage problems. Traditional cloud-based data management architectures are extremely vulnerable in the face of various cyberattacks. To address the above challenges, a verifiable ABE scheme based on blockchain and local differential privacy is proposed, using LDP to perturb the original data locally to a certain extent to resist collusion attacks, outsourcing encryption and decryption to corresponding service providers to reduce the pressure on mobile terminals, and deploying smart contracts in combination with blockchain for fair execution by all parties to solve the problem of returning wrong search results in a semi-honest cloud server. Detailed security proofs are performed through the defined security goals, which shows that the proposed scheme is indeed privacy-protective. The experimental results show that the scheme is optimized in terms of data accuracy, computational overhead, storage performance, and fairness. In terms of efficiency, it greatly reduces the local load, enhances personal privacy protection, and has high practicality as well as reliability. As far as we know, it is the first case of applying the combination of LDP technology and blockchain to a tracing system, which not only mitigates poisoning attacks on user data, but also improves the accuracy of the data, thus making it easier to identify infected contacts and making a useful contribution to health prevention and control efforts.

An Automated Metadata Generation Method for Data Lake of Industrial WoT Applications

Recent trends in the Web of Things (WoT) have led to data explosion. Data lake (DL), as a flexible on-demand heterogeneous data management architecture, has become a feasible solution in data management. Metadata modeling for DLs is the key basis for smart analysis and processing. However, the varieties in structures and semantics of industrial WoT data hinder metadata modeling and maintenance. Moreover, the lack of textual descriptions and the semantics hidden in value streams make it hard to automatically construct semantic metadata. The dynamic nature of WoT requires on-time evolution on metadata. To overcome these challenges, we propose an automated bottom-up metadata generation approach for DL of WoT applications. Applying a data-driven framework, raw data are notated as linked data and self-organizing map-based online clustering is applied to real timely extract data characteristics. To recognize entities, concepts and relations, semantics-based entity discovery approach from short texts is proposed according to the feature of WoT data. The numerical analysis is performed to find the hidden relations from raw values. Full-dimensional metadata with rich semantic knowledge are finally built. Experiments on a real-world dataset are conducted to verify the effectiveness of methods and a case study on an energy WoT system is provided to demonstrate the feasibility of the approach.