Database Transaction Processing Research Articles

Fast Commitment for Geo-Distributed Transactions via Decentralized Co-Coordinators

In a geo-distributed database, data shards and their respective replicas are deployed in distinct datacenters across multiple regions, enabling regional-level disaster recovery and the ability to serve global users locally. However, transaction processing in geo-distributed databases requires multiple cross-region communications, especially during the commit phase, which can significantly impact system performance. To optimize the performance of geo-distributed transactions, we propose Decentralized Two-phase Commit (D2PC), a new transaction commit protocol aiming to minimize the negative impact of cross-region communication. In D2PC, we employ multiple co-coordinators that perform commit coordination in parallel. Each co-coordinator is responsible for collecting 2PC votes and making a PreCommit decision in its local region. This approach allows for the concurrent invocation of multiple cross-region network round trips, and each region can end its concurrency control locally before replication is complete, thus significantly reducing the chances of blocking and enhancing system concurrency. Moreover, we propose the bypass leader replication reply method, leveraging decentralized co-coordinators to bypass the leader for message transmission, thereby reducing the commit latency. Experimental results have demonstrated that D2PC can reduce commit latency by 43% and improve throughput by up to 2.43 × compared to the geo-distributed transaction processing methods based on 2PC.

Real-Time Memory Data Optimization Mechanism of Edge IoT Agent

With the full development of disk-resident databases (DRDB) in recent years, it is widely used in business and transactional applications. In long-term use, some problems of disk databases are gradually exposed. For applications with high real-time requirements, the performance of using disk database is not satisfactory. In the context of the booming development of the Internet of things, domestic real-time databases have also gradually developed. Still, most of them only support the storage, processing, and analysis of data values with fewer data types, which can not fully meet the current industrial process control system data types, complex sources, fast update speed, and other needs. Facing the business needs of efficient data collection and storage of the Internet of things, this paper optimizes the transaction processing efficiency and data storage performance of the memory database, constructs a lightweight real-time memory database transaction processing and data storage model, realizes a lightweight real-time memory database transaction processing and data storage model, and improves the reliability and efficiency of the database. Through simulation, we proved that the cache hit rate of the cache replacement algorithm proposed in this paper is higher than the traditional LRU (Least Recently Used) algorithm. Using the cache replacement algorithm proposed in this paper can improve the performance of the system cache.

Chiller

Distributed transactions on high-overhead TCP/IP-based networks were conventionally considered to be prohibitively expensive. In fact, the primary goal of existing partitioning schemes is to minimize the number of cross-partition transactions. However, with the new generation of fast RDMAenabled networks, this assumption is no longer valid. In this paper, we first make the case that the new bottleneck which hinders truly scalable transaction processing in modern RDMA-enabled databases is data contention, and that optimizing for data contention leads to different partitioning layouts than optimizing for the number of distributed transactions. We then present Chiller, a new approach to data partitioning and transaction execution, which aims to minimize data contention for both local and distributed transactions.

A Heterogeneous Distributed Computing Model Based on Web Service

Based on the in-depth study of the existing database synchronization model, in order to improve the cross platform ability of the system and facilitate the construction of small and medium-sized enterprise information platform, this paper proposes a heterogeneous distributed computing scheme based on Web service. The scheme uses JMS to realize the message transmission between systems, and uses web service technology to realize cross platform data reading and writing. In the aspect of distributed transaction processing, the two-phase commit protocol is improved to reduce the probability of system deadlock and effectively ensure the consistency of distributed database data. In order to improve the performance of distributed database system, cache technology is introduced, and the way of integrating cache and database transaction processing is proposed, which effectively ensures the validity of cache data. The architecture is oriented to program developers, who can develop efficient and convenient distributed database system on the basis of this architecture. Finally, this architecture is applied to the background management system of mobile express service. The running results show that the architecture can well meet the business requirements of distributed heterogeneous database system synchronization.

Database Transaction Processing with Effective Locking Mechanism for Consistency in Cloud Database

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

ZHyperLink: Low-latency I/O for Db2 on IBM Z and DS8880 storage

zHyperLink is a new low-latency I/O capability for storage area networks (SANs). When using zHyperLink, the IBM z14 processor, along with the DS8880 Storage System (IBM's latest enterprise-class storage for mainframe attachment), delivers up to a 90% reduction in storage I/O service times compared with fiber channel transport technology. zHyperLink I/O service times eliminate the need for an operating system task switch for most read I/O operations, fundamentally improving application response times and changing system scaling behavior. Initial use of zHyperLink on IBM Z, IBM DS8880 storage, and z/OS accelerates the Db2 for z/OS relational database transaction processing information management system hierarchal database and reduces the elapsed time for Virtual Storage Access Method applications. These improvements occur without any application changes. In this paper, we explain the strategic importance of a low-latency I/O capability for the SAN and describe the z14 architecture, hardware, firmware, and software implementation concepts and the preliminary performance results for Db2.

Clay

Transaction processing database management systems (DBMSs) are critical for today's data-intensive applications because they enable an organization to quickly ingest and query new information. Many of these applications exceed the capabilities of a single server, and thus their database has to be deployed in a distributed DBMS. The key factor affecting such a system's performance is how the database is partitioned. If the database is partitioned incorrectly, the number of distributed transactions can be high. These transactions have to synchronize their operations over the network, which is considerably slower and leads to poor performance. Previous work on elastic database repartitioning has focused on a certain class of applications whose database schema can be represented in a hierarchical tree structure. But many applications cannot be partitioned in this manner, and thus are subject to distributed transactions that impede their performance and scalability. In this paper, we present a new on-line partitioning approach, called Clay, that supports both tree-based schemas and more complex "general" schemas with arbitrary foreign key relationships. Clay dynamically creates blocks of tuples to migrate among servers during repartitioning, placing no constraints on the schema but taking care to balance load and reduce the amount of data migrated. Clay achieves this goal by including in each block a set of hot tuples and other tuples co-accessed with these hot tuples. To evaluate our approach, we integrate Clay in a distributed, main-memory DBMS and show that it can generate partitioning schemes that enable the system to achieve up to 15× better throughput and 99% lower latency than existing approaches.

Improving Update-Intensive Workloads on Flash Disks through Exploiting Multi-Chip Parallelism

Solid state drives (SSDs), or flash disks have been considered as ideal storage for various data-intensive workloads, because of the low random access latency and the intra-disk multi-chip parallelism. However, due to inherent nature of flash memories, update-intensive workloads cause the flash disk fragmented, and trigger costly internal activities such as cleaning and wear leveling. We use database transaction processing as a motivating update-intensive workload. Our studies based on a flash disk simulator as well as flash disks show that, these activities result in significant overhead to the I/O response time and system throughput. To resolve the impact of internal activities, we propose dynamic page replications to exploit the multi-chip parallelism on the flash disk. Specifically, we replicate the frequently blocked data pages to improve the data availability even when internal activities block the request. To reduce the overhead of replications, we take advantage of the idle periods in the flash chips for the I/O operations by writes to replicas or reads from replicas, and further develop a prediction model for the decisions on those I/O operations to minimize the interference to normal I/O operations. We evaluate our techniques with three public transaction benchmarks in the simulator as well as on the real flash disks. Our results demonstrate the effectiveness of our replication management on improving I/O response time and system throughput.

Optimizing random write performance of FAST FTL for NAND flash memory

The NAND flash memory has gained its popularity as a storage device for consumer electronics due to its higher performance and lower power consumption. In most of these devices, an FTL (Flash Translation Layer) is adopted to emulate a block device interface to support the conventional disk-based file systems that make the flash management much easier. Among various FTLs, the FAST (Fully-Associative Sector Translation) FTL has shown superior performance, becoming one of the state-of-the-art approaches. However, the FAST FTL performs poorly while dealing with a huge number of small-sized random writes brought by upper applications such as database transaction processing workloads. The two important reasons are the absence of efficient selection schemes for the reclaiming of random log blocks that leads to large overhead of full merges, and the sequential log block scheme which no longer applies to random writes due to the large costs of partial merges. To overcome the above two defects in the presence of random writes, two techniques have been proposed. The first technique reduced full merge costs by adopting a novel random log block selection algorithm, based on the block associativity and the relevant-valid-page-amount of random log blocks as the key block selection criterion. The second technique replaced the sequential log block with a random log block to eliminate the overhead of partial merges. Experimental results showed that our optimizations can outperform FAST FTL significantly in three aspects: erase counts, page migration amount, and response time. The maximum improvement level in these cases could reach up to 66.8%, 98.2%, and 51.0%, respectively.

The Information Management System (IMS) Program Product

The Information Management System (IMS) was one of the first program products offered by IBM. From its inception in a North American Rockwell and IBM joint study in the 1960s, the product has evolved into the preeminent transaction processing database management system (DBMS). Throughout its history, IMS has remained the leader in DBMS technology.

Tolerating byzantine faults in transaction processing systems using commit barrier scheduling

This paper describes the design, implementation, and evaluation of areplication scheme to handle Byzantine faults in transaction processing database systems. The scheme compares answers from queries and updates on multiple replicas which are unmodified, off-the-shelf systems, to provide a single database that is Byzantine fault tolerant. The scheme works when the replicas are homogeneous, but it also allows heterogeneous replication in which replicas come from different vendors. Heterogeneous replicas reduce the impact of bugs and security compromises because they are implemented independently and are thus less likely to suffer correlated failures. The main challenge in designing a replication scheme for transactionprocessing systems is ensuring that the different replicas execute transactions in equivalent serial orders while allowing a high degreeof concurrency. Our scheme meets this goal using a novel concurrency control protocol, commit barrier scheduling (CBS). We have implemented CBS in the context of a replicated SQL database, HRDB(Heterogeneous Replicated DB), which has been tested with unmodified production versions of several commercial and open source databases as replicas. Our experiments show an HRDB configuration that can tolerate one faulty replica has only a modest performance overhead(about 17% for the TPC-C benchmark). HRDB successfully masks several Byzantine faults observed in practice and we have used it to find a new bug in MySQL.

Data warehousing in an undergraduate curriculum

Databases and data warehousing are important technologies for modern enterprises. Whereas a traditional online transaction processing database is designed to support the day-to-day events that occu...

On Optimistic Concurrency Control for Real-Time Database Systems

The performance of database transaction processing system can be profoundly affected by the concurrency control method employed since it is necessary to preserve database integrity in a multi-user environment. In addition to satisfying the consistency requirement as in traditional database system, real-time database systems must also satisfy timing constraints. In this study we present a virtual run policy for the restarted first run optimistic transactions and compare its performance with optimistic concurrency control in firm real-time database system.

The optimum pipeline depth considering both power and performance

The impact of pipeline length on both the power and performance of a microprocessor is explored both by theory and by simulation. A theory is presented for a range of power/performance metrics, BIPS m /W . The theory shows that the more important power is to the metric, the shorter the optimum pipeline length that results. For typical parameters neither BIPS/W nor BIPS 2 /W yield an optimum, i.e., a non-pipelined design is optimal. For BIPS 3 /W the optimum, averaged over all 55 workloads studied, occurs at a 22.5 FO4 design point, a 7 stage pipeline, but this value is highly dependent on the assumed growth in latch count with pipeline depth. As dynamic power grows, the optimal design point shifts to shorter pipelines. Clock gating pushes the optimum to deeper pipelines. Surprisingly, as leakage power grows, the optimum is also found to shift to deeper pipelines. The optimum pipeline depth varies for different classes of workloads: SPEC95 and SPEC2000 integer applications, traditional (legacy) database and on-line transaction processing applications, modern (e. g. web) applications, and floating point applications.

A Survey of Academic and Commercial Approaches to Transaction Support in Mobile Computing Environments

The need for information access and processing in mobile computing environments presents great challenges to database researchers. Apart from advances in wireless communications and device technology, new data management techniques are needed to provide a transparent interaction between fixed and mobile devices in global information systems. A key aspect is to provide a flexible and powerful transaction processing infrastructure that takes into account the peculiarities of mobile computing. Mobile transaction processing shall be able to transparently deal with frequent disconnections and occasional movements of mobile devices. This paper surveys the mobile transaction models proposed in academia and outlines the state-of-the-art of transaction processing in commercial mobile databases.

The optimum pipeline depth for a microprocessor

The impact of pipeline length on the performance of a microprocessor is explored both theoretically and by simulation. An analytical theory is presented that shows two opposing architectural parameters affect the optimal pipeline length: the degree of instruction level parallelism (superscalar) decreases the optimal pipeline length, while the lack of pipeline stalls increases the optimal pipeline length. This theory is tested by analyzing the optimal pipeline length for 35 applications representing three classes of workloads. Trace tapes are collected from SPEC95 and SPEC2000 applications, traditional (legacy) database and on-line transaction processing (OLTP) applications, and modern (e. g. web) applications primarily written in Java and C++. The results show that there is a clear and significant difference in the optimal pipeline length between the SPEC workloads and both the legacy and modern applications. The SPEC applications, written in C, optimize to a shorter pipeline length than the legacy applications, largely written in assembler language, with relatively little overlap in the two distributions. Additionally, the optimal pipeline length distribution for the C++ and Java workloads overlaps with the legacy applications, suggesting similar workload characteristics. These results are explored across a wide range of superscalar processors, both in-order and out-of-order.

Dynamic multiple parity (DMP) disk array for serial transaction processing

The performance of today's database systems is usually limited by the speed of their I/O devices. Fast I/O systems can be built from an array of low cost disks working in parallel. This kind of disk architecture is called RAID (Redundant Arrays of Inexpensive Disks). RAID promises improvement over SLED (Single Large Expensive Disks) in performance, reliability, power consumption, and scalability. However, a general fact about RAID is that the "write" operation is difficult to speedup. In this paper, we propose a new RAID architecture, called Dynamic Multiple Parity (DMP) Disk Array, for serial transaction processing database systems. Serial transaction processing database systems include engineering database systems, fully replicated database systems using a completely centralized algorithm and distributed systems using the conservative timestamp ordering algorithm. DMP Disk Array can significantly increase the I/O throughput by incorporating multiple parity disks. Due to the inherent distributed sparing property, DMP Disk Array can provide normal service to the users under single disk failure condition. Delay and maximum throughput analysis on DMP Disk Array is performed. Results show that, for a typical "write" job proportion of 20 percent, DMP Disk Array can provide nearly 20 percent improvement on I/O throughput over that of RAID level 5 when one extra parity disk is used.

Adapting on-line analytical processing for decision modelling: the interaction of information and decision technologies

The introduction of new technologies and concepts has redefined the relative positioning of information systems (IS) and decision technologies in a corporate context. Corporate IS have been extended to include not only transaction processing databases but also analytical databases, often known as Data Warehouses. On-line analytical processing (OLAP), as introduced by Codd et al. [E.F. Codd, S.B Codd, C.T. Salley, Providing On-Line Analytical Processing to User–Analysts: An IT Mandate, E.F. Codd and Associates, 1993], is capable of capturing the structure of the real world data in the form of multidimensional tables which are known as ` datacubes' by management information systems (MIS) and statistical systems specialists. Manipulation and presentation of such information through multidimensional views and graphical displays provide invaluable support for the decision-maker. We illustrate the natural coupling, which exists between data modelling, symbolic modelling and `What if' analysis phases of a decision support system (DSS). In particular, we explore the power of roll-up and drill-down features of OLAP and show how these translate into aggregation and disagreggation of the underlying decision models. Our approach sets out a paradigm for analysing the data, applying DSS tools and progressing through the information value chain to create organisational knowledge.

Transaction processing in multilevel secure databases with kernelized architecture: Challenges and solutions

Multilevel security poses many challenging problems for transaction processing. The challenges are due to the conflicting requirements imposed by confidentiality, integrity, and availability-the three components of security. We identify these requirements on transaction processing in Multilevel Secure (MLS) database management systems (DBMSs) and survey the efforts of a number of researchers to meet these requirements. While our emphasis is primarily on centralized systems based on kernelized architecture, we briefly overview the research in the distributed MLS DBMSs as well.

Implementation of a real-time database system

A real-time database system is a database system in which transactions have explicit timing constraints such as deadlines. Apart from satisfying the database consistency constraints, transaction executions must also satisfy these timing constraints. The goal of transaction and query processing in real-time databases is to maximise the number of successful transactions in the system. This paper reports on the design, development and performance evaluation of RT-Genesis, a real-time database management system resulting from modifying an existing commercial DBMS, Genesis. RT-Genesis is a relational database management system that accommodates SQL queries and transactions having time constraints. It features time-cognizant algorithms for scheduling, concurrency control and buffer management. The system has been tested and the performance of the different algorithms compared in isolation as well as in combination with different classes of SQL workloads. In addition, a two-phased approach to transaction execution has also been implemented with the goal of exploiting access invariance to improve predictability. This work demonstrates the feasibility of converting non-real-time DBMSs into real-time DBMSs for firm deadline transactions. Besides reporting on how this was achieved and on the performance of the incorporated algorithms, lessons learned from this experience are also discussed.