Transaction Response Time Research Articles

Capturing, indexing, clustering, and retrieving system history

We present a method for automatically extracting from a running system an indexable signature that distills the essential characteristic from a system state and that can be subjected to automated clustering and similarity-based retrieval to identify when an observed system state is similar to a previously-observed state. This allows operators to identify and quantify the frequency of recurrent problems, to leverage previous diagnostic efforts, and to establish whether problems seen at different installations of the same site are similar or distinct. We show that the naive approach to constructing these signatures based on simply recording the actual ``raw'' values of collected measurements is ineffective, leading us to a more sophisticated approach based on statistical modeling and inference. Our method requires only that the system's metric of merit (such as average transaction response time) as well as a collection of lower-level operational metrics be collected, as is done by existing commercial monitoring tools. Even if the traces have no annotations of prior diagnoses of observed incidents (as is typical), our technique successfully clusters system states corresponding to similar problems, allowing diagnosticians to identify recurring problems and to characterize the ``syndrome'' of a group of problems. We validate our approach on both synthetic traces and several weeks of production traces from a customer-facing geoplexed 24 x 7 system; in the latter case, our approach identified a recurring problem that had required extensive manual diagnosis, and also aided the operators in correcting a previous misdiagnosis of a different problem.

Epidemic algorithms for replicated databases

We present a family of epidemic algorithms for maintaining replicated database systems. The algorithms are based on the causal delivery of log records where each record corresponds to one transaction instead of one operation. The first algorithm in this family is a pessimistic protocol that ensures serializability and guarantees strict executions. Since we expect the epidemic algorithms to be used in environments with low probability of conflicts among transactions, we develop a variant of the pessimistic algorithm which is optimistic in that transactions commit as soon as they terminate locally and inconsistencies are detected asynchronously as the effects of committed transactions propagate through the system. The last member of the family of epidemic algorithms is pessimistic and uses voting with quorums to resolve conflicts and improve transaction response time. A simulation study evaluates the performance of the protocols.

Two-version based concurrency control and recovery in real-time client/server databases

While there has been a significant amount of research in real-time concurrency control, little work has been done in logging and recovery for real-time databases. This paper proposes a two-version approach which considers both real-time concurrency control and recovery. We propose a network-server-based architecture and algorithms which can not only reduce the blocking time of higher-priority transactions and improve the response time of client-side read-only transactions, but also provide a diskless runtime logging mechanism and an efficient and predictable recovery procedure. The performance of the algorithms was verified by a series of simulation experiments by comparing the algorithms with the well-known Priority Ceiling Protocol (PCP), the Read/Write PCP, the New PCP, and the 2-version two-phase locking protocol, for which we have very encouraging results. The schedulability of higher-priority transactions and the response time of client-side read-only transactions were all greatly improved.

Optimistic voting for managing replicated data

An epidemic model gives an efficient approach for transaction processing of replication systems in weakly connected environments. The approach has the advantages of high adaptation, support for low-bandwidth network, and committing updates in an entirely decentralized control fashion. But the previous implementing protocols, like ROWA protocol, quorum protocol, and voting protocol, have a common shortcoming that they are pessimistic in conflict reconciliation, therefore bring high transaction abort rate and reduce system performance dramatically when the workload scales up. In this paper, an optimistic voting protocol, which introduces condition vote and order vote in the voting process of transactions, is proposed. The condition vote and order vote postpone the final decision on conflicting transactions and avoid transaction aborts that are incurred by read-write and write-write conflicts. Experimental results indicate that the optimistic voting protocol decreases abort rate and improves average response time of transactions markedly when compared to other protocols.

Real-time processing in client-server databases

We propose and experimentally evaluate the use of the client-server database paradigm for real-time processing. To date, the study of transaction processing with time constraints has mostly been restricted to centralized or single-node systems. Recently, client-server databases have exploited locality of data accesses in real-world applications to successfully provide reduced transaction response times. Our objective is to investigate the feasibility of real-time processing in a data-shipping client-server database architecture. We compare the efficiency of the proposed architecture with that of a centralized real-time database system. We discuss transaction scheduling issues in the two architectures and propose a new policy for scheduling transactions in the client-server environment. This policy assigns higher priorities to transactions that have a greater potential for successful completion through the use of locally available data. We investigate the effects of client data-access locality and various updating workloads on transaction completion rates. Our experimental results show that real-time client-server databases can provide significant performance gains over their centralized counterparts. These gains become evident when large numbers of clients (more than 40) are attached per server, even in the presence of high data contention.

An analytical tool for predicting the performance of parallel relational databases

The uptake of parallel DBMSs is being hampered by uncertainty about the impact on performance of porting database applications from sequential to parallel systems. The development of tools which aid the system manager or machine vendor could help to reduce this problem. This paper describes an analytical tool which determines the performance characteristics (in terms of throughput, resource utilisation and response time) of relational database transactions executing on particular machine configurations and provides simple graphical visualisations of these to enable users to obtain rapid insight into particular scenarios. The problems of handling different parallel DBMSs are illustrated with reference to three systems – Ingres, Informix and Oracle. A brief description is also given of two different approaches used to confirm the validity of the analytical approach on which the tool is based. Copyright © 1999 John Wiley & Sons, Ltd.

Approximate Estimation of Transaction Response Time

This paper describes a study of different approximation techniques used to predict the response times of database transactions represented as patterns of resource consumption and modelled with non-product-form queueing networks. The techniques are applied to a range of examples. The experiments show that none of the approximation techniques has a consistent advantage over the others for all cases considered. On the other hand, a simple heuristic rule is formulated which provides an acceptable approximation to the average transaction response time for the entire range of examples. The rule specifies a procedure for labelling each queue in a queueing network as either an M/M/1 or an M/G/1 resource. The resulting network can then be solved to obtain the mean response time of individual transactions.

Multiview access protocols for large-scale replication

The article proposes a scalable protocol for replication management in large-scale replicated systems. The protocol organizes sites and data replicas into a tree-structured, hierarchical cluster architecture. The basic idea of the protocol is to accomplish the complex task of updating replicated data with a very large number of replicas by a set of related but independently committed transactions. Each transaction is responsible for updating replicas in exactly one cluster and invoking additional transactions for member clusters. Primary copies (one from each cluster) are updated by a cross-cluster transaction. Then each cluster is independently updated by a separate transaction. This decoupled update propagation process results in possible multiple views of replicated data in a cluster. Compared to other replicated data management protocols, the proposed protocol has several unique advantages. First, thanks to a smaller number of replicas each transaction needs to atomically update in a cluster, the protocol significantly reduces the transaction abort rate, which tends to soar in large transactional systems. Second, the protocol improves user-level transaction response time as top-level update transactions are allowed to commit before all replicas have been updated. Third, read-only queries have the flexibility to see database views of different degrees of consistency and data currency. This ranges from global, most up to date, and consistent views, to local, consistent, but potentially old views, to local, nearest to users but potentially inconsistent views. Fourth, the protocol maintains its scalability by allowing dynamic system reconfiguration as it grows by splitting a cluster into two or more smaller ones. Fifth, autonomy of the clusters is preserved as no specific protocol is required to update replicas within the same cluster. Clusters are, therefore, free to use any valid replication or concurrency control protocols.

Vertical partitioning in database design

When a transaction in a relational database system is processed, transaction response time is likely dominated by the disk access time. By partitioning a relation into fragments, according to the requirement of transactions, a transaction can avoid accessing the useless data. In this paper, an algorithm using the A ∗ technique, which can find the global optimal partition quickly, is presented. Two refinement methods—reduction of the search space and good estimation — are also proposed to improve the performance of the search procedure.

The Effect of Failures on the Performance of Long-Duration Database Transactions

The occurrence of transaction failures is often unavoidable in most database processing environments and such failures can have a particularly detrimental effect on long-duration transactions. In this paper, the impact of failures on the performance of long-duration database transactions is studied, and results for the transaction response time and failure overhead are presented for general failure patterns, which include Poisson failures as a special case. It is found that the failure overhead can be excessive for long-duration transactions and the transaction response time can increase sharply with increased failure rates. The benefits of transaction nesting to arrest performance deterioration is considered and results for quantifying the extent of restructuring improvement are provided. A procedure based on branch and bound techniques for determining the optimal restructuring strategy to minimize total transaction response time is also presented. It is shown that restructuring optimization can bring about significant performance advantages.

The location-based paradigm for replication: Achieving efficiency and availability in distributed systems

Replication techniques for transaction-based distributed systems generally achieve increased availability but with a significant performance penalty. We present a new replication paradigm, the location-based paradigm, which addresses availability and other performance issues. It provides availability similar to quorum-based replication protocols but with transaction-execution delays similar to one-copy systems. The paradigm further exploits replication to improve performance in two instances. First, it takes advantage of local or nearby replicas to further improve the response time of transactions, achieving smaller execution delays than one-copy systems. Second, it takes advantage of replication to facilitate the independent crash recovery of replica sites-a goal which is unattainable in one-copy systems. In addition to the above the location-based paradigm avoids bottlenecks, facilitates load balancing, and minimizes the disruption of service when failures and recoveries occur. In this paper we present the paradigm, a formal proof of correctness, and a detailed simulation study comparing our paradigm to one-copy systems and to other approaches to replication control. >

Buffer analysis for a data sharing environment with skewed data access

Examines the effect of skewed database access on the transaction response time in a multisystem data sharing environment, where each computing node has access to shared data on disks, and has a local buffer of recently accessed granules. Skewness in data access can increase data contention since most accesses go to few data items. For the same reason, it can also increase the buffer hit probability. We quantify the resultant effect on the transaction response time, which depends not only on the various system parameters but also on the concurrency control (CC) protocol. Furthermore, the CC protocol can give rise to rerun transactions that have different buffer hit probabilities. In a multisystem environment, when a data block gets updated by a system, any copies of that block in other systems' local buffers are invalidated. Combining these effects, we find that higher skew does not necessarily lead to worse performance, and that with skewed access, optimistic CC is more robust than pessimistic CC. Examining the buffer hit probability as a function of the buffer size, we find that the effectiveness of additional buffer allocation can be broken down into multiple regions that depend on the access frequency distribution. >

MMDB partial reload

When a system failure occurs in a main memory database (MMDB) system, the primary copy of the database which resides permanently in a volatile memory is lost. Reload of the database from archive memory into main memory is needed to recover this lost copy. Several complete reload algorithms, which are used when main memory is large enough to store the entire database, have been designed for a parallel and multi-processor database environment. However, little research has been done for partial reload which is needed when only a part of the database can fit into main memory. This paper proposes three MMDB partial reload algorithms for a multi-processor main memory database system and presents their performance results with the Least Recently Used paging scheme being used to handle page faults. The simulation experiments show that reload based on frequency of data access yields the best transaction response time and system throughput.

Performance evaluation of an efficient multiple copy update algorithm

A well-known algorithm for updating multiple copies is the Thomas majority consensus algorithm. This algorithm, before performing an update, needs to obtain permission from a majority of the nodes in the system. We study the response-time behavior of a symmetric (each node seeks permission from the same number of other nodes and each node receives requests for update permission from the same number of other nodes) distributed update-synchronization algorithm where nodes need to obtain permission from only O(/spl radic/N) (N being the number of database copies) other nodes before performing an update. The algorithm we use is an adaptation of Maekawa's O(/spl radic/N) distributed mutual exclusion algorithm to multiple-copy update-synchronization. This increase in the efficiency of the update-synchronization algorithm enhances performance in two ways. First, the reduction in transaction service time reduces the response time. Second, for a given arrival rate of transactions, the decrease in response time reduces the number of waiting transactions in the system. This reduces the probability of conflict between transactions. To capture the interaction between the probability of conflict and the transaction response time, we define a new measure called the conflict response-time product. Based on the solution of a queueing model we show that optimizing this measure yields a different and more appropriate choice of system parameters than simply minimizing the mean transaction response time.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

On real-time databases: concurrency control and scheduling

In addition to maintaining database consistency as in conventional databases, real-time database systems must also handle transactions with timing constraints. While transaction response time and throughput are usually used to measure a conventional database system, the percentage of transactions satisfying the deadlines or a time-critical value function is often used to evaluate a real-time database system. Scheduling real-time transactions is far more complex than traditional real-time scheduling in the sense that (1) worst case execution times are typically hard to estimate, since not only CPU but also I/O requirement is involved; and (2) certain aspects of concurrency control may not integrate well with real-time scheduling. In this paper, we first develop a taxonomy of the underlying design space of concurrency control including the various techniques for achieving serializability and improving performance. This taxonomy provides us with a foundation for addressing the real-time issues. We then consider the integration of concurrency control with real-time requirements. The implications of using run policies to better utilize real-time scheduling in a database environment are examined. Finally, as timing constraints may be more important than data consistency in certain hard realtime database applications, we also discuss several approaches that explore the nonserializable semantics of real-time transactions to meet the hard deadlines. >

A performance study of robust distributed load sharing strategies

In this paper, we examine a set of load sharing strategies that are robust to the unreliable state information that is often present in a distributed database system. In this environment, sites must solve the problem of how alternative sites should be selected to process incoming transactions, given that the information on which the decision is based exhibits varying degrees of obsolescence. A set of regression-based adaptive strategies is examined in which a feedback mechanism is used to compensate for obsolete information. Transaction response time under the different adaptive strategies is evaluated, and the reasons for these performance differences discussed. The key characteristic of the best regression strategy is that transaction site affinity is taken into consideration when adjusting for the effect of information obsolescence.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Lock-Based Concurrency Control in Distributed Real-Time Database Systems

A real-time database system (RTDBS) is designed to provide timely response to the transactions of data-intensive applications. The transactions processed in a RTDBS are associated with real-time constraints typically in the form of deadlines. With the current database technology it is extremely difficult to provide schedules guaranteeing transaction deadlines. This difficulty comes from the unpredictability of transaction response times. Efficient resource scheduling algorithms and concurrency control protocols are required to schedule RTDB transactions so as to maximize the number of satisfied deadlines. In this paper, we describe several distributed, lock-based, real-time concurrency control protocols and report on the relative performance of the protocols in a distributed database environment. The protocols are different in the way real-time constraints of transactions are involved in controlling concurrent accesses to shared data. A detailed performance model of a distributed RTDBS was employed in the evaluation of concurrency control protocols.

Mechanisms for managing partial reload in a high performance memory-resident database system

Upon a system failure, a memory-resident database system will lose the portion of its database that resides in a volatile memory. In the paper, it is not assumed that the main memory is large enough to store the entire database. Instead, only a major portion of the database fits into main memory. An efficient partial reload scheme is needed to restore the lost portion of the database quickly without degrading the overall system performance upon a system crash. Page faults may be incurred during the reload process. A partial reload scheme should be capable of handling these page faults as well as database reload. The paper presents three different partial reload algorithms and page replacement schemes that aim at fast transaction response time and high system throughput.

I/O subsystem configurations for ESA: New roles for processor storage

I/O subsystem configurations are dictated by the storage and I/O requirements of the specific applications that use the disk hardware. Treating the latter requirement as a given, however, draws a boundary at the channel interface that is not well-suited to the capabilities of the Enterprise Systems Architecture (ESA). This architecture allows hardware expenditures in the I/O subsystem to be managed, while at the same time improving transaction response time and system throughput capability, by a strategy of processor buffering coupled with storage control cache. The key is to control the aggregate time per transaction spent waiting for physical disk motion. This paper investigates how to think about and accomplish such an objective. A case study, based on data collected at a large Multiple Virtual Storage installation, is used to investigate the potential types and amounts of memory use by individual files, both in storage control cache and in processor buffers. The mechanism of interaction between the two memory types is then examined and modeled so as to develop broad guidelines for how best to deploy an overall memory budget. These guidelines tend to contradict the usual metrics of storage control cache effectiveness, underscoring the need for an adjustment in pre-ESA paradigms.

Simulation of Main Memory Database Recovery

In a main memory database (MMDB), the primary copy of the database may reside permanently in a volatile memory. When a system failure occurs, the database must be reloaded efficiently from archive memory into main memory. This paper presents four different reload schemes and the simulation models constructed to compare the algorithms. Simulation results indicate that the reload scheme based on freguency of data access gives the best overall performance in terms of transaction response time and system throughput.