Data Management Solutions Research Articles

A Scalable, Secure, and Efficient Framework for Sharing Electronic Health Records Using Permissioned Blockchain Technology

This paper presents a scalable, secure blockchain-based healthcare system architecture that efficiently manages large patient datasets. DHTs and Skip Lists enable efficient data access, while DPoS and PBFT facilitate parallel transaction processing. Adaptive filters, Radix Trees extended by Merkle Trees, and an immutable blockchain ledger secured by Tendermint consensus ensure data integrity and protection against evolving threats. Threshold Cryptography secures consensus participant selection, and Bulletproofs verify transactions, complying with healthcare regulations. ChaCha20, a symmetric stream cipher, encrypts sensitive data, enhancing performance across devices. ABAC manages access rights, ensuring fine-grained control over data accessibility. This architecture offers a comprehensive, efficient, and secure solution for healthcare data management in blockchain environments.

The dry lab microscopist or prompt microscopist: do we need them?

AbstractIn modern biological microscopy, the explosion of data volume and complexity highlights the urgent need for specialised data management support roles. While traditional microscopy focuses on visual data presentation, the rapid increase in big data acquisition and data mining demands advanced handling and analysis. This gap underscores the need for “dry lab microscopists” or data experts skilled in microscopy data management, software interoperability, and AI-driven solutions. Job markets reflect this demand, pointing to the necessity for dedicated training programs. Integrating these specialists into research institutions is crucial for addressing digital data challenges and maintaining high standards in data integrity and analysis. Their role is essential for advancing research in the data-driven era.

Digital Phenotyping of Mental and Physical Conditions: Remote Monitoring of Patients Through RADAR-Base Platform.

The use of digital biomarkers through remote patient monitoring offers valuable and timely insights into a patient's condition, including aspects such as disease progression and treatment response. This serves as a complementary resource to traditional health care settings leveraging mobile technology to improve scale and lower latency, cost, and burden. Smartphones with embedded and connected sensors have immense potential for improving health care through various apps and mobile health (mHealth) platforms. This capability could enable the development of reliable digital biomarkers from long-term longitudinal data collected remotely from patients. We built an open-source platform, RADAR-base, to support large-scale data collection in remote monitoring studies. RADAR-base is a modern remote data collection platform built around Confluent's Apache Kafka to support scalability, extensibility, security, privacy, and quality of data. It provides support for study design and setup and active (eg, patient-reported outcome measures) and passive (eg, phone sensors, wearable devices, and Internet of Things) remote data collection capabilities with feature generation (eg, behavioral, environmental, and physiological markers). The back end enables secure data transmission and scalable solutions for data storage, management, and data access. The platform has been used to successfully collect longitudinal data for various cohorts in a number of disease areas including multiple sclerosis, depression, epilepsy, attention-deficit/hyperactivity disorder, Alzheimer disease, autism, and lung diseases. Digital biomarkers developed through collected data are providing useful insights into different diseases. RADAR-base offers a contemporary, open-source solution driven by the community for remotely monitoring, collecting data, and digitally characterizing both physical and mental health conditions. Clinicians have the ability to enhance their insight through the use of digital biomarkers, enabling improved prevention, personalization, and early intervention in the context of disease management.

B-031 A new Ready to use Bio-Rad Quality Control to Streamline Daily Workflow

Abstract Background A comprehensive Quality Management System starts with performing a daily Quality Control (Q.C.) process. The daily Q.C. process monitors the Instrument’s optimal performance and detects shifts and trends. However, the existing traditional daily Q.C. process is not only prone to manual errors, but also time consuming, costly and labor intensive. We evaluated a Bio-Rad InteliQTM Load and go Quality Control that comes in a barcoded tube ready to be loaded. We monitored the workflow efficiency gained in comparison to the traditional quality control material that comes in glass vials. Methods We evaluated the new quality control material ready to load on the instrument. This new Q.C. (Bio-Rad InteliQ Assayed Multiqual Control - 3 levels) were processed in parallel with our routine Q.C. (Bio-Rad Assayed Chemistry Lyphocheck vials - 2 levels). We performed the evaluation on Abbott Architect c8000 Chemistry analyzers in our central Laboratory (NGHA-Jeddah). The Q.C. was stored frozen. Before testing it was thawed at room temperature as per the insert instruction. Over a 5 days period, in parallel with our routine quality control, we run the new Q.C. Two times per day for 33 analytes. We monitored the processing time difference between the new and current approaches. We monitored the stability of all the analytes in the tube after Two days and again after Five days. We also monitored the cost effectiveness, labor intensiveness, probability of operator error and the overall improvement in workflow and Turn Around Time. Results Over the evaluation period, we observed the following: - Saving 40 minutes in average per week only for chemistry Q.C (time spent in reconstituting, mixing and aliquoting old Q.C) - 100% reduction in waste of plastic consumables such as pipette and plastic tubes. - Save an average of 1.0 ml dead volume in every 5.0 ml old Q.C. vial (20% of the volume, hence saving 20% of the total Q.C. cost). The dead volume wasted due to aliquoting step using the old Q.C. - 100% Eliminated delays caused by operator error such as wrong reconstitution, inadequate mixing and wrong Q.C. aliquoting. Not needed for the InteliQ. - High score of staff satisfaction. - Stability of the 32 analytes was excellent except that of Co2, which deteriorated by 14% after Two days as indicated in the insert package. Co2 day five result deteriorated by 34%. Conclusions Bio-Rad InteliQ is easy to use. It saves time and cost. When combined with Unity Q.C. data management solution it can streamline Q.C. workflow, eliminate operator errors, reduce T.A.T. and increase the overall laboratory performance. It may also work optimally with Instruments that have on-board fridge for Q.C.

Enhancing Smart Healthcare Networks: Integrating Attribute-Based Encryption for Optimization and Anti-Corruption Mechanisms

This study investigates the feasibility and effectiveness of integrating Attribute-Based Encryption (ABE) into smart healthcare networks, with a particular focus on its role in enhancing anti-corruption mechanisms. The study provides a comprehensive analysis of current vulnerabilities in these networks, identifying potential data security risks. An anti-corruption mechanism is designed to ensure data integrity and reliability. The ABE approach is then empirically compared to other prominent encryption algorithms, such as Identity-Based Encryption, Data Encryption Standard, Advanced Encryption Standard, and Rivest-Shamir-Adleman algorithms. These methods are evaluated based on access latency, data transmission speed, system stability, and anti-corruption capabilities. Experimental results highlight the strengths of the ABE algorithm, demonstrating an average access latency of 31.6 milliseconds, a data transmission speed of 3.56 MB/s, and an average system stability of 98.74%. Furthermore, when integrated into anti-corruption mechanisms, ABE effectively protects against data tampering and misuse, ensuring secure data transmission. Compared to alternative algorithms, ABE offers a more efficient, secure, and stable solution for data management within smart healthcare networks, supported by its robust anti-corruption capabilities. This positions ABE as an optimal choice for safeguarding the integrity and security of healthcare data.

A Comparative Analysis of Microsoft Fabric and Snowflake: Architectural Paradigms and Performance Metrics in Modern Cloud Data Platforms

Abstract: This article presents a comprehensive comparative analysis of two leading cloud data platforms: Microsoft Fabric and Snowflake. As organizations increasingly rely on cloud-based solutions for data management and analytics, understanding the strengths and limitations of these platforms becomes crucial. This article examines the architectural foundations, functional capabilities, performance metrics, and cost considerations of both platforms. We explore Microsoft Fabric's integrated service model within the Azure ecosystem and Snowflake's multi-cluster shared data architecture, assessing their approaches to data integration, storage, analytics, and machine learning integration. Through empirical performance comparisons and an evaluation of scalability mechanisms, we provide insights into the operational efficiencies of each platform. Additionally, we analyze the security features, compliance standards, and pricing models to offer a holistic view of the total cost of ownership. Our findings reveal distinct advantages in Microsoft Fabric's end-to-end integration and Snowflake's performance in multicloud environments, while also highlighting areas for potential improvement in both platforms. This article aims to serve as a valuable resource for organizations navigating the complex landscape of cloud data solutions, offering evidence-based criteria for platform selection aligned with specific business needs and technological ecosystems

HYBRID CLOUD DATABASES FOR BIG DATA ANALYTICS: A REVIEW OF ARCHITECTURE, PERFORMANCE, AND COST EFFICIENCY

The increasing reliance on big data analytics has driven organizations to seek more flexible, scalable, and cost-effective data management solutions, with hybrid cloud databases emerging as a prominent option. Hybrid cloud databases integrate both public and private cloud environments, allowing businesses to balance the scalability and cost advantages of public clouds with the security and control provided by private clouds. This systematic review, conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, examines the architecture, performance, cost efficiency, security, and technological advancements of hybrid cloud databases. The study synthesizes findings from 40 peer-reviewed articles published between 2015 and 2024, focusing on key factors that affect the performance of hybrid cloud databases, such as workload distribution, network latency, and the use of advanced data analytics tools like Hadoop and Spark. The review also explores cost-saving mechanisms, including dynamic resource scaling and pay-as-you-go pricing models, which help organizations reduce infrastructure costs by up to 28%. Additionally, it discusses the security risks and privacy concerns associated with hybrid cloud environments, highlighting the effectiveness of encryption protocols and identity access management systems in mitigating data breaches by as much as 40%. Furthermore, the review identifies emerging trends in the integration of machine learning and artificial intelligence (AI), which have significantly enhanced system automation and resource optimization in hybrid cloud infrastructures, leading to performance improvements of up to 45%.

Application of edge computing and IoT technology in supply chain finance

This study proposes an IoT data management framework based on blockchain and edge computing and conducts detailed simulation experiment design and performance evaluation for the field of supply chain finance. Previous methods often struggled with high latency, limited scalability, and inadequate risk management, and to address these issues, the experimental platform includes traditional centralized systems, distributed systems, blockchain-based systems, and edge computing + blockchain systems. By monitoring indicators such as data processing delay, data transmission delay, data retrieval time, system throughput, and data integrity of each platform, and introducing evaluation formulas for credit risk, operational risk, and market risk, the performance of different systems in processing supply chain financial data is comprehensively analyzed. The experimental results show that the edge computing + blockchain system performs well in data processing efficiency, security, real-time, and system throughput, especially under high load conditions. Our market risk value is reduced to 0.015. This framework improves the efficiency and security of data transmission and effectively reduces credit risk and operational risk, providing an efficient and reliable solution for data management in supply chain finance.

A customizable data management framework for high-repetition-rate high-energy-density science

The high-energy-density (HED) physics community is moving toward a new paradigm of high-repetition-rate (HRR) operation. To fully leverage the scientific power of HRR HED facilities, all of the components of each subsystem (laser, targetry, and performance diagnostics) must be connected and synchronized in a reliable and robust manner while the data acquired are tagged and archived in real time. To this end, GA has begun developing a generalized NoSQL-database framework, the MongoDB repository for information and archiving. An organizational strategy has been developed that shifts HED data organization from a shot-based to a diagnostic-based approach in order to increase archival and retrieval efficiency that lends itself to optimization applications. This work is a first step in pushing HRR HED science toward data management solutions that emphasize machine actionability and aim to stimulate community engagement to define data standards in HED science.

Ichor: A Python library for computational chemistry data management and machine learning force field development.

We present ichor, an open-source Python library that simplifies data management in computational chemistry and streamlines machine learning force field development. Ichor implements many easily extensible file management tools, in addition to a lazy file reading system, allowing efficient management of hundreds of thousands of computational chemistry files. Data from calculations can be readily stored into databases for easy sharing and post-processing. Raw data can be directly processed by ichor to create machine learning-ready datasets. In addition to powerful data-related capabilities, ichor provides interfaces to popular workload management software employed by High Performance Computing clusters, making for effortless submission of thousands of separate calculations with only a single line of Python code. Furthermore, a simple-to-use command line interface has been implemented through a series of menu systems to further increase accessibility and efficiency of common important ichor tasks. Finally, ichor implements general tools for visualization and analysis of datasets and tools for measuring machine-learning model quality both on test set data and in simulations. With the current functionalities, ichor can serve as an end-to-end data procurement, data management, and analysis solution for machine-learning force-field development.

Advancements in Securing Cloud-Stored Data and Managing Sensitive Information: A Comprehensive Review of Ciphertext-Policy Attribute-Based Encryption (CP-ABE) Techniques

Abstract—The exploring comprehensive review of cutting-edge techniques for securing cloud-stored data and managing sensitive information in the context of smart cities through the application of Ciphertext-Policy Attribute-Based Encryption (CP-ABE). It highlights the innovative integration of blockchain technology with CP-ABE, which introduces a decentralized and tamper- resistant key management system, thereby enhancing the overall security framework in cloud environments where data sharing is prevalent. Introducing an online/offline multi-authority CP- ABE scheme, characterized by hidden policies, offers significant advancements in protecting user attributes and access structures, ensuring that sensitive information remains confidential even during encryption and decryption processes. This dual approach not only fortifies security but also optimizes the efficiency of data- sharing mechanisms. Furthermore, the paper delves into imple- menting hidden sensitive policies and keyword search techniques within smart city infrastructures, which are designed to facilitate secure and efficient data retrieval. These techniques ensure that while data remains accessible to authorized users, privacy is rigorously maintained. Collectively, these approaches represent significant strides in bolstering the security and confidentiality of data in both cloud-based and smart city applications, addressing the growing demand for robust and efficient data management solutions in increasingly interconnected environments. Index Terms—Ciphertext-Policy Attribute-Based Encryption (CP-ABE), Blockchain, Key Management, Decentralized Secu- rity, Cloud-Stored Data, Yolo V7, Explainable AI (XAI), On- line/Offline Multi-Authority Scheme, Hidden policies, Privacy Preservation, Data Protection, Secure Data Management

FUTURE TRENDS IN SQL DATABASES AND BIG DATA ANALYTICS: IMPACT OF MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE

This study systematically reviews the integration of machine learning (ML) and artificial intelligence (AI) into SQL databases and big data analytics, highlighting significant advancements and emerging trends. Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, a comprehensive review of 60 selected articles published between 2010 and 2023 was conducted. The findings reveal substantial improvements in query optimization through ML algorithms, which adapt dynamically to changing data patterns, reducing processing times and enhancing performance. Additionally, embedding ML models within SQL databases facilitates real-time predictive analytics, streamlining workflows, and improving the accuracy and speed of predictions. AI-driven security systems provide proactive and real-time threat detection, significantly enhancing data protection. The development of hybrid systems that combine relational and non-relational databases offers versatile and efficient data management solutions, addressing the limitations of traditional systems. This study confirms the evolving role of AI and ML in transforming data management practices and aligns with and extends previous research findings.

Secure environmental, social, and governance (ESG) data management for construction projects using blockchain

Environmental, social, and governance (ESG) considerations are increasingly becoming imperative and obligatory across various industries. The ESG performance within the architecture, engineering, and construction (AEC) industry is under heightened market scrutiny. However, current ESG management in construction is still in its infancy due to two limitations: (1) a deficiency in ESG knowledge, such as indicators pertinent to construction activities, and (2) a lack of data security in ESG management, culminating in inefficient and unreliable environmental management practices. Therefore, this paper employs the Design Science Research Method (DSRM) to introduce a Blockchain-ESG Integrated (BESGI) framework, facilitating traceable ESG data management within construction projects. This framework presents three significant contributions. First, it identifies ten AEC-ESG indicators by analyzing ESG methods. Second, it proposes a mapping approach for AEC-ESG indicators to construction projects for key ESG information access and data source identification. Third, it develops a blockchain-based data management mechanism for traceable ESG data management in the BESGI framework. It validates and evaluates the framework in a construction project in Hong Kong. The results show that the framework is usable and can save labor costs by 20.15 % compared to traditional ESG management. This study offers a secure data management solution for ESG analysis of construction projects.

Barriers to Effective Electronic Records Management in Public Sector Organizations in East Africa

Purpose: The aim of the study was to investigate the barriers to effective electronic records management in public sector organizations in East Africa. Methodology: This study adopted a desk methodology. A desk study research design is commonly known as secondary data collection. This is basically collecting data from existing resources preferably because of its low cost advantage as compared to a field research. Our current study looked into already published studies and reports as the data was easily accessed through online journals and libraries. Findings: Barriers to effective electronic records management (ERM) in East African public sector organizations include inadequate technology infrastructure, limited financial resources, staff training gaps, policy implementation challenges, resistance to change from paper-based systems, data security concerns, and insufficient organizational commitment. Addressing these barriers requires holistic strategies encompassing technological upgrades, enhanced financial support, robust training programs, streamlined policies, and fostering a culture supportive of ERM practices. Unique Contribution to Theory, Practice and Policy: Institutional theory, technology acceptance model (TAM) & resource dependency theory may be used to anchor future studies on barriers to effective electronic records management in public sector organizations in East Africa. Implementing cloud-based solutions for data storage and management can improve accessibility and data integrity, overcoming technological barriers to effective ERM implementation. Depoliticize ERM processes by establishing bipartisan support and clear governance frameworks. Policy reforms should include guidelines for managing political transitions without disrupting ERM continuity

Integrating FAIR Experimental Metadata for Multi-omics Data Analysis

The technological advancements in bio-science research are resulting in the generation of vast amounts of complex and heterogeneous data sets from individual studies. Efficient Research Data Management solutions based on FAIR principles can guide the research groups toward standardizing and packaging the study-specific research results into uniquely identifiable digital objects that are easily traceable and identifiable, leading to knowledge discovery, collaboration, and innovation. However, to explore the inter-dependencies among data sets originating from different study disciplines, it is crucial to deploy a generic data-centric RDM solution that overcomes inherent challenges and helps manage complex data sets. This solution should respect participating groups’ data and security policies while providing an integrated data view. In this paper, we introduce ARC Registry, a cloud-native search & exploration application that integrates experimental metadata from individual research groups that have deployed PLANTdataHUB (FAIR-RDM) solution. The focus is on the parts of PLANTdataHUB and ARC Registry that facilitate near-real-time integration of experimental metadata from standardized digital objects (Annotated Research Contexts) across participating research groups.

Towards Municipal Data Utilities: Experiences Regarding the Development of a Municipal Data Utility for Intra- and Intermunicipal Actors within the German City of Mainz

This paper describes the requirements analysis phase towards the establishment and implementation of a municipal data utility (KDW = Kommunales Datenwerk, German) to facilitate data sharing between intra- and intermunicipal stakeholders. Against the backdrop of increasing digitisation and the growing importance of data-driven decision making in municipal governance, this paper aims to address the pressing need for efficient data management solutions within and across municipalities. Based on a structured self-developed methodology, the authors use a qualitative research approach: the paper examines the experiences and challenges encountered during the requirements phase, the design phase, and the development phase of the KDW. The findings indicate that the establishment of a robust KDW requires (1) extensive stakeholder engagement, (2) clear governance structures, and (3) a robust technical infrastructure. In addition, the study highlights the critical importance of establishing a sound legal framework that addresses data ownership, privacy, security and regulatory compliance. Addressing legal and regulatory barriers to data sharing is paramount to the successful implementation and operation of the KDW. The paper concludes by highlighting the potential benefits of KDWs and outlining future work. The overall methodology, approach, and outcome are validated within the city of Mainz, and the lessons learned are accommodated in the insights presented in the rest of the paper.

A Review on Blockchain Technology

Blockchain technology has emerged as a disruptive force with the potential to revolutionize various industries by offering decentralized, secure, and transparent solutions for data management and transactions. Initially conceptualized as the underlying technology for cryptocurrencies like Bitcoin, blockchain has evolved into a versatile tool applicable across diverse domains such as finance, supply chain management, healthcare, and government services. This paper provides an overview of blockchain technology, its fundamental principles, and key components. It explores the decentralized consensus mechanisms, cryptographic techniques, and data structures that underpin blockchain networks, ensuring immutability, transparency, and integrity of recorded transactions. Furthermore, the paper examines the challenges and opportunities associated with blockchain adoption, including scalability, interoperability, regulatory concerns, and environmental impact. Through a comprehensive review of existing literature and case studies, this research elucidates the potential benefits of blockchain technology, such as increased efficiency, reduced costs, enhanced security, and improved trust among participants. Additionally, it highlights ongoing developments, innovations, and future directions in blockchain research and implementation, emphasizing the need for interdisciplinary collaboration and regulatory frameworks to realize the full potential of this transformative technology. Keywords : IOT, blockchain, cryptocurrency, smart contracts, consensus algorithms, internet of things.

Streamlining Toll Management: A Survey of Business Requirement Solutions

Effective toll control performs a pivotal role in improving traffic flow, decreasing congestion, and enhancing overall transportation performance. However, toll plaza operators frequently face demanding situations in efficaciously handling statistics and monitoring transactional activities. This studies paper proposes a secure, fast, and dependable statistics control answer tailor-made to the precise wishes of toll plaza operators. By implementing this solution, toll plaza proprietors can successfully track income and loss states, examine regular transaction statistics, and optimize traffic control measures. Through the improvement and implementation of this solution, massive enhancements in traffic flow efficiency and congestion reduction may be achieved, ultimately leading to more advantageous transportation systems. This paper offers a practical method to addressing the complexities of toll control, imparting insights into the advantages and impacts of adopting such answers withinside the transportation sector. Keywords-Toll management, Traffic flow, Congestion reduction, Transportation efficiency, Data management solution, Profit tracking, Transaction analysis, Traffic control, Toll plaza operators, Transportation systems

The intelligent prediction and assessment of financial information risk in the cloud computing model

cloud computing (cloud computing) is a kind of distributed computing, referring to the network "cloud" will be a huge data calculation and processing program into countless small programs, and then, through the system composed of multiple servers to process and analyze these small programs to get the results and return to the user. This report explores the intersection of cloud computing and financial information processing, identifying risks and challenges faced by financial institutions in adopting cloud technology. It discusses the need for intelligent solutions to enhance data processing efficiency and accuracy while addressing security and privacy concerns. Drawing on regulatory frameworks, the report proposes policy recommendations to mitigate concentration risks associated with cloud computing in the financial industry. By combining intelligent forecasting and evaluation technologies with cloud computing models, the study aims to provide effective solutions for financial data processing and management, facilitating the industry's transition towards digital transformation.

Smart contract: Revolutionizing transactions in the digital age

Blockchain technology is a revolutionary concept that has transformed various industries, offering decentralized and secure solutions for data management. At its core, a blockchain is a distributed and immutable ledger that records transactions across a network of computers. One of the key features of blockchain is its transparency, as each participant in the network has access to the same information. The application of blockchain technology extends across diverse sectors, with finance being the most promising and well-known application domain. Cryptocurrencies, like Bitcoin and Ethereum, rely on blockchain to facilitate secure and transparent transactions. The decentralized nature of blockchain eliminates the need for intermediaries, reducing transaction costs and increasing efficiency. Beyond finance, blockchain is making significant strides in supply chain management. The emergence of blockchain technology has paved the way for a transformative innovation in the realm of digital transactions: smart contracts. These self-executing contracts encoded in blockchain have garnered significant attention for their potential to revolutionize the way agreements are made, executed, and enforced. This paper explores the concept of smart contracts, delving into the underlying technology, finding applications across various industries, legal implications, and future prospects.