Transactions In Distributed Systems Research Articles

Scythe: A Low-latency RDMA-enabled Distributed Transaction System for Disaggregated Memory

Scythe: A Low-latency RDMA-enabled Distributed Transaction System for Disaggregated Memory

Fine-Grained Re-Execution for Efficient Batched Commit of Distributed Transactions

Distributed transaction systems incur extensive cross-node communication to execute and commit serializable OLTP transactions. As a result, their performance greatly suffers. Caching data at nodes that execute transactions can cut down remote reads. Batching transactions for validation and persistence can amortize the communication cost during committing. However, caching and batching can significantly increase the likelihood of conflicts, causing expensive aborts. In this paper, we develop Hackwrench to address the challenge of caching and batching. Instead of aborting conflicted transactions, Hackwrench tries to repair them using fine-grained re-execution by tracking the dependencies of operations among a batch of transactions. Tracked dependencies allow Hackwrench to selectively invalidate and re-execute only those operations necessary to "fix" the conflict, which is cheaper than aborting and executing an entire batch of transactions. Evaluations using TPC-C and other micro-benchmarks show that Hackwrench can outperform existing commercial and research systems including FoundationDB, Calvin, COCO, and Sundial under comparable settings.

The Cryptographic Complexity of Anonymous Coins: A Systematic Exploration

The modern financial world has seen a significant rise in the use of cryptocurrencies in recent years, partly due to the convincing lure of anonymity promised by these schemes. Bitcoin, despite being considered as the most widespread among all, is claimed to have significant lapses in relation to its anonymity. Unfortunately, studies have shown that many cryptocurrency transactions can be traced back to their corresponding participants through the analysis of publicly available data, to which the cryptographic community has responded by proposing new constructions with improved anonymity claims. Nevertheless, the absence of a common metric for evaluating the level of anonymity achieved by these schemes has led to numerous disparate ad hoc anonymity definitions, making comparisons difficult. The multitude of these notions also hints at the surprising complexity of the overall anonymity landscape. In this study, we introduce such a common framework to evaluate the nature and extent of anonymity in (crypto) currencies and distributed transaction systems, thereby enabling one to make meaningful comparisons irrespective of their implementation. Accordingly, our work lays the foundation for formalizing security models and terminology across a wide range of anonymity notions referenced in the literature, while showing how “anonymity” itself is a surprisingly nuanced concept, as opposed to existing claims that are drawn upon at a higher level, thus missing out on the elemental factors underpinning anonymity.

Read atomic transactions with prevention of lost updates: ROLA and its formal analysis

Abstract Designers of distributed database systems face the choice between stronger consistency guarantees and better performance. A number of applications only require read atomicity (RA) (either all or none of a transaction’s updates are visible to other transactions) and prevention of lost updates (PLU). Existing distributed transaction systems that meet these requirements also provide additional stronger consistency guarantees (such as causal consistency ), but this comes at the price of lower performance. In this paper we propose a new distributed transaction protocol, ROLA, that targets application scenarios where only RA and PLU are needed. We formally specify ROLA in Maude. We then perform model checking to analyze both the correctness and the performance of ROLA. For correctness, we use standard model checking to analyze ROLA’s satisfaction of RA and PLU. To analyze performance we: (a) perform statistical model checking to analyze key performance properties; and (b) compare these performance results with those obtained by also modeling and analyzing in Maude the well-known protocols Walter and Jessy that also guarantee RA and PLU. Our statistical model checking results show that ROLA outperforms both Walter and Jessy.

Benchmarking "Hello, World!"

As more and more software moves off the desktop and into data centers, and more and more cell phones use server requests as the other half of apps, observation tools for large-scale distributed transaction systems are not keeping up. This makes it tempting to look under the lamppost using simpler tools. You will waste a lot of high-pressure time following that path when you have a sudden complex performance crisis. Instead, know what each tool you use is blind to, know what information you need to understand a performance problem, and then look for tools that can actually observe that information directly.

Fast In-Memory Transaction Processing Using RDMA and HTM

DrTM is a fast in-memory transaction processing system that exploits advanced hardware features such as remote direct memory access (RDMA) and hardware transactional memory (HTM). To achieve high efficiency, it mostly offloads concurrency control such as tracking read/write accesses and conflict detection into HTM in a local machine and leverages the strong consistency between RDMA and HTM to ensure serializability among concurrent transactions across machines. To mitigate the high probability of HTM aborts for large transactions, we design and implement an optimized transaction chopping algorithm to decompose a set of large transactions into smaller pieces such that HTM is only required to protect each piece. We further build an efficient hash table for DrTM by leveraging HTM and RDMA to simplify the design and notably improve the performance. We describe how DrTM supports common database features like read-only transactions and logging for durability. Evaluation using typical OLTP workloads including TPC-C and SmallBank shows that DrTM has better single-node efficiency and scales well on a six-node cluster; it achieves greater than 1.51, 34 and 5.24, 138 million transactions per second for TPC-C and SmallBank on a single node and the cluster, respectively. Such numbers outperform a state-of-the-art single-node system (i.e., Silo) and a distributed transaction system (i.e., Calvin) by at least 1.9X and 29.6X for TPC-C.

Balancing Performance, Accuracy, and Precision for Secure Cloud Transactions

In distributed transactional database systems deployed over cloud servers, entities cooperate to form proofs of authorizations that are justified by collections of certified credentials. These proofs and credentials may be evaluated and collected over extended time periods under the risk of having the underlying authorization policies or the user credentials being in inconsistent states. It therefore becomes possible for policy-based authorization systems to make unsafe decisions that might threaten sensitive resources. In this paper, we highlight the criticality of the problem. We then define the notion of trusted transactions when dealing with proofs of authorization. Accordingly, we propose several increasingly stringent levels of policy consistency constraints, and present different enforcement approaches to guarantee the trustworthiness of transactions executing on cloud servers. We propose a Two-Phase Validation Commit protocol as a solution, which is a modified version of the basic Two-Phase Validation Commit protocols. We finally analyze the different approaches presented using both analytical evaluation of the overheads and simulations to guide the decision makers to which approach to use.

Preserving Data Consistency through Neighbor Replication on Grid Daemon

In modern distributed systems, replication receives particular attention for providing high data availability, fault tolerance and enhance the performance of the system. It is an important mechanism because it enables organizations to provide users with access to current data where and when they need it. However, this way of data organization introduces low data consistency and data coherency as more than one replicated copies need to be updated. Expensive synchronization mechanisms are needed to maintain the consistency and integrity of data among replicas when changes are made by the transactions. In this paper, we present Neighbor Replication on Grid (NRG) daemon in order to manage replication and transactions in distributed system. NRG Transaction Model has been implemented in order to preserve the data consistency and availability. Based on experiment and result, it shows that NRG daemon guarantees consistency and obey serializability through the synchronization approach. Key word: Distributed system, replication, consistency, synchronous, transaction, serializability.

Discovering likely invariants of distributed transaction systems for autonomic system management

Large amount of monitoring data can be collected from distributed systems as the observables to analyze system behaviors. However, without reasonable models to characterize systems, we can hardly interpret such monitoring data effectively for system management. In this paper, a new concept named flow intensity is introduced to measure the intensity with which internal monitoring data reacts to the volume of user requests in distributed transaction systems. We propose a novel approach to automatically model and search relationships between the flow intensities measured at various points across the system. If the modeled relationships hold all the time, they are regarded as invariants of the underlying system. Experimental results from a real system demonstrate that such invariants widely exist in distributed transaction systems. Further we discuss how such invariants can be used to characterize complex systems and support autonomic system management.

Capacity Planning for and Performance Analysis of Large Distributed Transaction Systems

Every transaction system supporting business processes must satisfy some computational performance requirements. If it does not, the system may have a significantly reduced value or become useless. The number of factors that influence performance of distributed transaction systems is so large, that the traditional capacity planning methods appropriate for centralized mainframe systems are not applicable. The most cost-effective method has proven to be predictive modeling. A combination of analytic and simulation modeling is most useful for analyzing and tuning performance of systems that have been designed or are in production. The task of designing a system ab initio given expected workloads and required performance/cost constraints is much harder. In this special issue, the paper by Marc Brittan and Janusz Kowalik describes a method for designing initial systems configurations and capacities. This paper focuses on predictive simulation modeling that is used for detailed performance analysis of existing systems. Such analyses may be motivated by what-if studies or by a desire to improve system's unsatisfactory performance.

The design and implementation of a distributed transaction system based on atomic data types

The complexity of potential interactions among concurrent activities and the multitude of failure modes that can occur in distributed systems make it hard to reason about distributed programs. Transactions provide the programmer with a mechanism that simplifies the development of concurrent and distributed programs. In this paper we present the design and implementation of a distributed transaction system that uses atomic data types to provide synchronization and recovery. Generally speaking, implementing user-defined atomic data types is a difficult task. However, unlike existing systems, our system requires programmers to do very little extra work to make an object atomic. Programmers implement atomic data types as if for a sequential and reliable environment and specify the concurrent semantics of object operations separately in a small, but expressive declarative language. Appropriate synchronization and recovery code for atomic data types is generated automatically by the system according to this information.

Timestamp-based orphan elimination

An orphan in a distributed transaction system is an activity executing on behalf of an aborted transaction. A method is proposed for managing orphans created by crashes and by aborts that ensures that orphans are detected and eliminated in a timely manner, and also prevents them from observing inconsistent states. The method uses timestamps generated at each site. Transactions are assigned timeouts at different sites. These timeouts are related by a global invariant, and they may be adjusted by simple two-phase protocols. The principal advantage of this method is simplicity: it is easy to understand, and to implement, and it can be proved correct. An 'eager' version of this method uses approximately synchronized real-time clocks to ensure that orphans are eliminated within a fixed duration, and a 'lazy' version uses logical clocks to ensure that orphans are eventually eliminated as information propagates through the system. The method is fail-safe: unsynchronized clocks and lost messages may affect performance, but they cannot produce inconsistencies or protect orphans from eventual elimination. Although the method is informally described in terms of two-phase locking, the formal argument shows it is applicable to any concurrency control method that preserved atomicity. >

A model for adaptable systems for transaction processing

Adaptability is an essential tool for managing escalating software costs and to build high-reliability, high-performance systems. Algorithmic adaptability, which supports techniques for switching between classes of schedulers in distributed transaction systems, is modeled. RAID, an experimental system implemented to support experimentation in adaptability, is discussed. Adaptability features in RAID, including algorithmic adaptability, fault tolerance, and implementation techniques for an adaptable server-based design, are modeled.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An approach to efficient distributed transactions

Most distributed systems proposed on the basis of the concept of atomic action or transaction strongly limit parallelism, thus reducing their level of efficiency. In this paper, features of efficiency in a distributed transaction system are investigated. Two mechanisms are proposed in order to enhance potential concurrency both among different transactions and within a single transaction during the commit phase:

An Availability Model for Distributed Transaction Systems

A method is proposed for quantitatively evaluating the availability of a distributed transaction system (DTS). The DTS dynamics can be modeled as a Markov process. The problem of formulating the set of linear homogeneous equations is considered, obtaining the related coefficient matrix, that is, the transition rate matrices of the DTS elements. Such operations can be performed according to the rules of Kronecker algebra. The transition rate matrices are used to calculate the probabilities of the different possible states of the DTS. The availability with respect to a transaction T is computed through its representation by means of a structure graph and a structure vector related to the probabilistic state of the DTS element relevant to the transaction T itself.

Correctness of a distributed transaction system

A distributed transaction system manages information that is dispersed over a number of storage devices. This paper deals with an experimental transaction system designed to satisfy real-time constraints through distributed control of the executions of transactions. Of interest is the correctness of the algorithm for distributed control. Demonstrating the correctness involves showing that the algorithm guarantees the consistency of distributed data, and equally importantly, that every transaction will eventually terminate. Proof of consistency is based on the notion of serializability of transactions while proof of termination is based on the conflict resolution and failure recovery strategies employed by the transaction system.

Global States of a Distributed System

A global state of a distributed transaction system is consistent if no transactions are in progress. A global checkpoint is a transaction which must view a globally consistent system state for correct operation. We present an algorithm for adding global checkpoint transactions to an arbitrary distributed transaction system. The algorithm is nonintrusive in the sense that checkpoint transactions do not interfere with ordinary transactions in progress; however, the checkpoint transactions still produce meaningful results.