Online Transaction Processing Systems Research Articles

Implementation and evaluation of a decentralized medical data exchange system based on the IOTA Tangle and approximate algorithms for optimal data placement


 
 
 
 
 
 
 
 © Voronin V., Rybalchenko A., Shmatko O., Kolomiitsev O., Tretiak V., Kliuchka Ya., 2023
 
 
 
 
 
 
 
 
 Facilitating the transfer of patient medical information across healthcare providers is crucial for ensuring high-quality care. However, this process encounters obstacles pertaining to privacy, security, and centralised control. This article outlines the creation of a decentralised system for exchanging medical data. The system utilises Distributed Ledger Technology, specifically the IOTA Tangle (open protocol (and network) for data and value transfer). This document outlines the structure and fundamental elements of a system designed for the secure and unalterable storage and transfer of medical records. The system employs masking and encryption methodologies to protect patient confidentiality while enabling healthcare practitioners to access complete information with patient authorization. The IOTA Tangle facilitates transactions without any fees and ensures the verification of data integrity through its Directed Acyclic Graph (DAG) topology. Simulation trials validate the system’s capability to securely communicate medical data on a large scale, while incurring lower resource costs than typical blockchain systems. The system demonstrates the feasibility of a decentralised and self-governing method for efficient and confidential sharing of medical data, utilising Distributed Ledger Technology (DLT). The secure data sharing platform facilitates the implementation of novel care and research frameworks, all the while upholding patient confidentiality and adhering to healthcare ethical standards. Also, in the development of billing Online Transaction Processing (OLTP) systems, which are designed for input, structured storage and processing of information in real time, the use of cloud technology is proposed. The problems that arise and their relevance to the solution of an integer linear programming problem with Boolean variables are shown. Approximate algorithms for optimal data placement and mathematical models for optimizing the structure of a distributed database of a cloud system are proposed, taking into account the limitations on the amount of node memory, the available costs of renting cloud resources, and the number of replicas of fragments of the distributed database.

Proposed framework for enhancing integrity technique using distributed query operation

Database growth and Storage problems are main concern for large and small enterprises nowadays, which significantly influences the efficiency of database applications. Database archiving is one of the solutions available for management. Many issues have been identified as a result of the use of archive databases, including the elimination of inactive data from online transaction processing systems (OLTP), performance and integrity management. The aim of this paper is to propose a framework for storing OLTP and archiving data in a distributed database environment as part of an integrated system. Maintaining the integrity between an OLTP database and an archiving database is crucial, but even more critical is the performance of the required query. The main aspect of the proposed framework is that it will not only ensure data integrity for primary and unique keys between OLTP and archive database, but it will also improve query performance by introducing parallel processing and query execution plans to maintain that integrity.

Learning Graph Databases: Neo4j an overview

As the internet is growing day by day, the amount of data being generated is huge. This data includes structured data and unstructured data. The data along with its relationship with other data makes the most powerful and meaningful information. Maximum data exists in the form of the relationship between different or same objects and the noticeable thing is the relationship between the data is more important than the data itself. These relationships are handled efficiently by Relational databases that store data having structures and which have several records. The important point to be noted here is that these Relational database management Systems use tables with normalization concept. If the amount of data in such tables is huge, then handling such a large amount of data with its relationships is a tedious task. Here, Graph Databases come into picture. Entities and their relationships in relational databases will be reflected with nodes and relationships in graph databases. Graph databases provide very simple data model than databases with Online Transaction Processing systems. Graph databases provide features such as transactional integrity and operational availability. This paper introduces the idea of graph database systems in conjunction with Neo4j encompassing its query features, consistency, transactions, availability, and scaling.

DBPA: A Benchmark for Transactional Database Performance Anomalies

Anomaly diagnosis is vital to the performance of online transaction processing (OLTP) systems. In the meanwhile, machine learning techniques can reason complex relationships beyond human abilities and perform well on such problems. However, they rely on a large number of training samples for anomalies, which are in serious shortage in both industry and academia due to the difficulty of collection. The problem raises the demand of a benchmark for anomaly reproduction and data collection. In this paper, we propose DBPA, a benchmark for transactional database performance anomalies. Specifically, we identify nine common anomalies rooted in the diverse influence factors. For each anomaly, we carefully design a reproduction procedure, which consists with its root cause in real-world databases. With the reproduction procedures, users can easily generate a dataset in a new environment and extend new anomaly types. For compound anomalies, we provide a generation algorithm that allows users to generate compound anomalies data of any possible combinations with existing collected data. We also provide a large dataset of both normal and anomalous monitoring data collected from various environments, facilitating the training of machine learning models and the evaluation of new algorithms for anomaly diagnosis.

Database isolation by scheduling

Transaction isolation is conventionally achieved by restricting access to the physical items in a database. To maximize performance, isolation functionality is often packaged with recovery, I/O, and data access methods in a monolithic transactional storage manager. While this design has historically afforded high performance in online transaction processing systems, industry trends indicate a growing need for a new approach in which intertwined components of the transactional storage manager are disaggregated into modular services. This paper presents a new method to modularize the isolation component. Our work builds on predicate locking, an isolation mechanism that enables this modularization by locking logical rather than physical items in a database. Predicate locking is rarely used as the core isolation mechanism because of its high theoretical complexity and perceived overhead. However, we show that this overhead can be substantially reduced in practice by optimizing for common predicate structures. We present DIBS, a transaction scheduler that employs our predicate locking optimizations to guarantee isolation as a modular service. We evaluate the performance of DIBS as the sole isolation mechanism in a data processing system. In this setting, DIBS scales up to 10.5 million transactions per second on a TATP workload. We also explore how DIBS can be applied to existing database systems to increase transaction throughput. DIBS reduces per-transaction file system writes by 90% on TATP in SQLite, resulting in a 3X improvement in throughput. Finally, DIBS reduces row contention on YCSB in MySQL, providing serializable isolation with a 1.4X improvement in throughput.

Micro-architectural analysis of in-memory OLTP: Revisited

Micro-architectural behavior of traditional disk-based online transaction processing (OLTP) systems has been investigated extensively over the past couple of decades. Results show that traditional OLTP systems mostly under-utilize the available micro-architectural resources. In-memory OLTP systems, on the other hand, process all the data in main-memory and, therefore, can omit the buffer pool. Furthermore, they usually adopt more lightweight concurrency control mechanisms, cache-conscious data structures, and cleaner codebases since they are usually designed from scratch. Hence, we expect significant differences in micro-architectural behavior when running OLTP on platforms optimized for in-memory processing as opposed to disk-based database systems. In particular, we expect that in-memory systems exploit micro-architectural features such as instruction and data caches significantly better than disk-based systems. This paper sheds light on the micro-architectural behavior of in-memory database systems by analyzing and contrasting it to the behavior of disk-based systems when running OLTP workloads. The results show that, despite all the design changes, in-memory OLTP exhibits very similar micro-architectural behavior to disk-based OLTP: more than half of the execution time goes to memory stalls where instruction cache misses or the long-latency data misses from the last-level cache (LLC) are the dominant factors in the overall execution time. Even though ground-up designed in-memory systems can eliminate the instruction cache misses, the reduction in instruction stalls amplifies the impact of LLC data misses. As a result, only 30% of the CPU cycles are used to retire instructions, and 70% of the CPU cycles are wasted to stalls for both traditional disk-based and new generation in-memory OLTP.

Incremental join view maintenance on distributed log-structured storage

Modern database systems desperate for the ability to support highly scalable transactions and efficient queries simultaneously for real-time applications. One solution is to utilize query optimization techniques on the on-line transaction processing (OLTP) systems. The materialized view is considered as a panacea to decrease query latency. However, it also involves the significant cost of maintenance which trades away transaction performance. In this paper, we examine the design space and conclude several design features for the implementation of a view on a distributed log-structured merge-tree (LSM-tree), which is a well-known structure for improving data write performance. As a result, we develop two incremental view maintenance (IVM) approaches on LSM-tree. One avoids join computation in view maintenance transactions. Another with two optimizations is proposed to decouple the view maintenance with the transaction process. Under the asynchronous update, we also provide consistency queries for views. Experiments on TPC-H benchmark show our methods achieve better performance than straightforward methods on different workloads.

Micro-architectural analysis of OLAP

Understanding micro-architectural behavior is important for efficiently using hardware resources. Recent work has shown that in-memory online transaction processing (OLTP) systems severely underutilize their core micro-architecture resources [29]. Whereas, online analytical processing (OLAP) workloads exhibit a completely different computing pattern. OLAP workloads are read-only, bandwidth-intensive, and include various data access patterns. With the rise of column-stores, they run on high-performance engines that are tightly optimized for modern hardware. Consequently, micro-architectural behavior of modern OLAP systems remains unclear. This work presents a micro-architectural analysis of a set of OLAP systems. The results show that traditional commercial OLAP systems suffer from their long instruction footprint, which results in high response times. High-performance columnstores execute tight instruction streams; however, they spend 25 to 82% of their CPU cycles on stalls both for sequential- and random-access-heavy workloads. Concurrent query execution can improve the utilization, but it creates interference in the shared resources, which results in sub-optimal performance.

Equilibrium of Redundancy in Relational Model for Optimized Data Retrieval

Conceptual and relational data models of online transaction processing (OLTP) applications are usually created and maintained following the principle of normalization, which implies avoidance of redundancy. Data retrieval from a disk-based normalized relational database often requires complex and inefficient queries that may cause noticeable performance issues when executed on larger volumes of data. Computer professionals sometimes intentionally trade off the strict normal form to optimize data retrieval queries through error-prone manual tuning and denormalization. We propose a fully automatic optimization approach, based on data redundancy, that relies on a formal cost-benefit model. We prove that finding the optimal level of data redundancy, for given workload statistics, is an NP-Complete optimization problem. A detailed reduction of the problem to binary linear programming is presented in the paper. The proposed optimization approach was evaluated using the TPCE benchmark for OLTP systems. The evaluation has shown that the proposed optimization approach is highly scalable, and that it can be efficiently applied to real-life relational data models.

STRATEGIC HUMAN RESOURCES MANAGEMENT AND BUSINESS INTELLIGENCE SYSTEMS IN THE PURPOSE OF SOCIAL AND ECONOMIC DEVELOPMENT

Interesting knowledge is gained by researching and re-examining how human resources management works in our region - the city. Human resource management in every business system becomes completely different from what it has been. It is no longer understood only as a mere functional activity, but as a new business philosophy to be implemented at all managerial levels. In this century, human resources are not only the key competitive advantage of any organization, but of every country. A holistic, coherent human resource management of the organization in order to ensure the maximum value added and the best positioning for the achievement of organizational strategic goals, in broad terms, leads directly to general socio-economic development and progress. One of the key functions of HRM is to spread the various components of the strategic plan to all employees in the organization. An important function of human resources management can and should be to provide information on the strategic readiness of human capital - whether and to what extent available human resources have the required knowledge, skills and motivation to support and implement an organization’s strategy. There is no efficient strategic management of human resources without the support of information technologies and information systems. The development of information technologies and information systems dramatically changes the management of human resources. The strategic readiness of information capital (IT and systems) is the most meaningful measure of the value of the information capital of the organization. The greatest availability of information capital in human resources management is measured by the level of development of online transaction processing systems, whether it is a module of an ERP system, or that it is HRIS (Human Resources Information Systems). Another technology that has even more strategic significance is BI (Business Intelligence) technology and BI decision support systems. A particular challenge is to examine the state of and opportunities for the development of human resources in profit and non-profit institutions in our narrower region. This scientific work will attempt to obtain the missing information on the content and method of human resources management in the profit and non-profit sector, observed in the context of information systems.

Accelerating in-memory transaction processing using general purpose graphics processing units

High throughput is critical for on-line transaction processing (OLTP) applications with a large amount of users. With massive parallel processing units and high memory bandwidth, GPUs are suitable for accelerating OLTP transactions. However, it is challenge to implement transaction execution on GPUs, due to (1) the branch divergences caused by the single instruction multiple threads (SIMT) execution paradigm, and (2) the lack of fine-grained synchronization mechanism and pointer-based dynamic data structures in the GPU ecosystem.In this paper, we present a high-performance in-memory transaction processing system on GPUs to accelerate OLTP applications, named GPU-TPS. Firstly, we propose a transaction execution model to improve GPU hardware utilization and perform synchronization among transactions. Secondly, we optimize the indexing data structures that used extensively in OLTP systems (i.e., hash table for unordered store, and b＋ tree for ordered store) for fast storing on GPUs.To evaluate GPU-TPS, we apply it to two popular OLTP workloads (SmallBank and TPCC), and compare it with the state-of-the-art hardware transactional memory based CPU OLTP system (DrTM) and a GPU OLTP system (GPUTx). The experimental results show that GPU-TPS outperforms the CPU implementation by 3.8X for SmallBank and by 1.9X for TPCC, and outperforms the GPU implementation by 1.6X for SmallBank and by 1.8X for TPCC.

Workload Aware Incremental Repartitioning of NoSQL for Online Transactional Processing Applications

<p><span lang="EN-US">Numerous applications are deployed on the web with the increasing popularity of internet. The applications include, 1) Banking applications,<br /> 2) Gaming applications, 3) E-commerce web applications. Different applications reply on OLTP (Online Transaction Processing) systems. OLTP systems need to be scalable and require fast response. Today modern web applications generate huge amount of the data which one particular machine and Relational databases cannot handle. The E-Commerce applications are facing the challenge of improving the scalability of the system. Data partitioning technique is used to improve the scalability of the system. The data is distributed among the different machines which results in increasing number of transactions. The work-load aware incremental repartitioning approach is used to balance the load among the partitions and to reduce the number of transactions that are distributed in nature. Hyper Graph Representation technique is used to represent the entire transactional workload in graph form. In this technique, frequently used items are collected and Grouped by using Fuzzy C-means Clustering Algorithm. Tuple Classification and Migration Algorithm is used for mapping clusters to partitions and after that tuples are migrated efficiently.</span></p>

A software cybernetics approach to self-tuning performance of on-line transaction processing systems

We construct two layer fuzzy rules based on the collected data.We develop algorithms from software cybernetics view to reconfigure the performance control parameters.We show self-tuning performance for On-Line Transaction Processing. Self-tuning performance of On-Line Transaction Processing (OLTP) Systems is a challenging and time-consuming task since multiple performance parameters are needed to be automatically configured in Database Management Systems (DBMSs). In this paper, we present a software cybernetics approach to self-tune the performance of DBMSs. A DBMS is designed with an adaptive control based on fuzzy logic such that it has the capability to control objects, i.e., the performance parameters, and update the controller itself, i.e., a set of fuzzy rules in our case. The principles and concepts in software cybernetics are applied to guide the synthesis of software controllers for monitoring and adapting system behaviors. Experimental results for On-Line Transaction Processing using TPC-C, a benchmark of the Transaction Processing Performance Council, show that the proposed method is feasible and effective.

English

The banking sector is the backbone of the entire financial economy of a country. In today’s globalized world, most organizations use online transaction processing systems for transferring money and doing business. Natural or man-made disasters can lead to data loss which in turn can cause millions of dollars of money lost. This study focuses on disaster recovery practices in commercial banks in Sri Lanka. From our preliminary findings, it was concluded that commercial banks only have ad-hoc disaster recovery standards and practices, as there is no standard framework available. Fourteen (14) banks were selected for data collection and relevant authorities were interviewed. The results were translated as qualitative observations to understand the best practices. Similarly, international standards, compliance requirements of the central bank, and existing researches were used to develop a disaster recovery practice framework. The proposed framework was then validated for its efficiency and usefulness among commercial banks and found to be acceptable by the banking industry.

Application of Machine Learning Algorithms for the Query Performance Prediction

This paper analyzes the relationship between the system load/throughput and the query response time in a real Online transaction processing (OLTP) system environment. Although OLTP systems are characterized by short transactions, which normally entail high availability and consistent short response times, the need for operational reporting may jeopardize these objectives. We suggest a new approach to performance prediction for concurrent database workloads, based on the system state vector which consists of 36 attributes. There is no bias to the importance of certain attributes, but the machine learning methods are used to determine which attributes better describe the behavior of the particular database server and how to model that system. During the learning phase, the system's profile is created using multiple reference queries, which are selected to represent frequent business processes. The possibility of the accurate response time prediction may be a foundation for automated decision-making for database (DB) query scheduling. Possible applications of the proposed method include adaptive resource allocation, quality of service (QoS) management or real-time dynamic query scheduling (e.g. estimation of the optimal moment for a complex query execution).

Scalable real-time OLAP on cloud architectures

In contrast to queries for on-line transaction processing (OLTP) systems that typically access only a small portion of a database, OLAP queries may need to aggregate large portions of a database which often leads to performance issues. In this paper we introduce CR-OLAP, a scalable Cloud based Real-time OLAP system based on a new distributed index structure for OLAP, the distributed PDCR tree. CR-OLAP utilizes a scalable cloud infrastructure consisting of multiple commodity servers (processors). That is, with increasing database size, CR-OLAP dynamically increases the number of processors to maintain performance. Our distributed PDCR tree data structure supports multiple dimension hierarchies and efficient query processing on the elaborate dimension hierarchies which are so central to OLAP systems. It is particularly efficient for complex OLAP queries that need to aggregate large portions of the data warehouse, such as “report the total sales in all stores located in California and New York during the months February–May of all years”. We evaluated CR-OLAP on the Amazon EC2 cloud, using the TPC-DS benchmark data set. The tests demonstrate that CR-OLAP scales well with increasing number of processors, even for complex queries. For example, for an Amazon EC2 cloud instance with 16 processors, a data warehouse with 160 million tuples, and a TPC-DS OLAP query stream where each query aggregates between 60% and 95% of the database, CR-OLAP achieved a query latency of below 0.3 s which can be considered a real time response.

Database Analytics: A Reconfigurable-Computing Approach

Businesses are experiencing a growing need for performing real-time analytics on ever-increasing enterprise data. While providing useful business insights and improved market responsiveness, analytics also adds a computational burden on traditional online transaction processing (OLTP) systems and could adversely affect the performance of OLTP workloads. The authors present a highly pipelined, high-throughput query-processing engine on a field-programmable gate array (FPGA) to offload expensive queries for database analytics. The proposed solution provides a mechanism for a database management system (DBMS) to seamlessly harness the FPGA accelerator without requiring any changes in the application or the existing data layout. The authors' system, which uses an off-the-shelf server and a PCIe-attached FPGA card and is integrated into a commercial DBMS platform, achieves an order-of-magnitude speedup on various real-life queries.

Analysis of Data Quality and Performance Issues in Data Warehousing and Business Intelligence

A Data Warehouse is an integral part of those enterprises which want to have a clear business insights from customer and operational data. It includes collection of technologies aimed at enabling the knowledge worker (executive, manager, analyst) to make better and faster decisions. It is expected to present the right information in the right place at the right time with the right cost in order to support the right decision. Over the years ,the practice of Data warehousing proved that the traditional online Transaction Processing (OLTP) systems are not fully appropriate for decision support. From the survey and evaluation of the literature related to Data Warehouse and with consultation and feedback of the data warehouse practitioners working in renowned IT giants ,it has been observed that the fundamental problems arise in populating a warehouse with quality data. . This paper mainly focuses on the study of the issues that hinder the data quality and performance of the Data warehouse and some of the means that may be opted to realize a better performance with respect to accuracy and quality to meet the challenging and dynamic needs of the corporate world.

Next Generation Data Warehouse Design with OLTP and OLAP Systems Sharing same Database

Online Transaction Processing (OLTP) systems have been using the traditional relational Database management system for many years. Online Analytical Processing (OLAP) system for analytical reporting is optimized to aggregate many records which involve more read operations. Hence to enhance the performance in OLAP systems various modeling options like star schema, extended star schema was implemented. But with the latest innovations in hardware technologies like availability of multi core dual processors, availability of 64 bit processors, decreasing cost of main memory and software technology innovations like In-Memory computing, Columnar storage and other with various other new features now there is less concern towards the performance degradation and performance optimization in a data warehouse system. If there is an option to use same database for the OLTP and OLAP systems then the business intelligence system can get the real time data for the analytical query reporting. This document presents the overview of the next generation data warehouse architecture which shares the database along with OLTP and discuss how it is going to use the in memory capability to apply real time analytics.

Building an XML document warehouse

Data Warehouses and OLAP (On Line Analytical Processing) technologies are dedicated to analyzing structured data issued from organizations’ OLTP (On Line Transaction Processing) systems. Furthermore, in order to enhance their decision support systems, these organizations need to explore XML (eXtensible Markup Language) documents as an additional and important source of unstructured data. In this context, this paper addresses the warehousing of document-centric XML documents. More specifically, we propose a two-method approach to build Document Warehouse conceptual schemas. The first method is for the unification of XML document structures; it aims to elaborate a global and generic view for a set of XML documents belonging to the same domain. The second method is for designing multidimensional galaxy schemas for Document Warehouses.