Scalable hierarchical locking for distributed systems

Abstract

Middleware components are becoming increasingly important as applications share computational resources in distributed environments, such as high-end clusters with ever larger number of processors, computational grids and increasingly large server farms. One of the main challenges in such environments is to achieve scalability of synchronization. In general, concurrency services arbitrate resource requests in distributed systems. But concurrency protocols currently lack scalability. Adding such guarantees enables resource sharing and computing with distributed objects in systems with a large number of nodes. The objective of our work is to enhance middleware services to provide scalability of synchronization and to support state replication in distributed systems. We have designed and implemented a middleware protocol in support of these objectives. Its essence is a peer-to-peer protocol for multi-mode hierarchical locking, which is applicable to transaction-style processing and distributed agreement. We demonstrate high scalability combined with low response times in high-performance cluster environments. Our technical contribution is a novel, fully decentralized, hierarchical locking protocol to enhance concurrency in distributed resource allocation following the specification of general concurrency services for large-scale data and object repositories. Our experiments on an IBM SP show that the number of messages approaches an asymptote at 15 node from which point on the message overhead is in the order of 3–9 messages per request, depending on system parameters. At the same time, response times increase linearly with a proportional increase in requests and, consequently, higher concurrency levels. Specifically, in the range of up to 80 nodes, response times under 10 ms are observed for critical sections that are one 25th the size of non-critical code. The high degree of scalability and responsiveness of our protocol is due in large to a high level of concurrency upon resolving requests combined with dynamic path compression for request propagation paths. Our approach is not only applicable to CORBA, its principles are shown to provide benefits to general distributed concurrency services and transaction models. Besides its technical strengths, our approach is intriguing due to its simplicity and its wide applicability, ranging from large-scale clusters to server-style computing.