Hash Table Design Research Articles

Iceberg Hashing: Optimizing Many Hash-Table Criteria at Once

Despite being one of the oldest data structures in computer science, hash tables continue to be the focus of a great deal of both theoretical and empirical research. A central reason for this is that many of the fundamental properties that one desires from a hash table are difficult to achieve simultaneously; thus many variants offering different trade-offs have been proposed. This article introduces Iceberg hashing, a hash table that simultaneously offers the strongest known guarantees on a large number of core properties. Iceberg hashing supports constant-time operations while improving on the state of the art for space efficiency, cache efficiency, and low failure probability. Iceberg hashing is also the first hash table to support a load factor of up to 1 - o(1) while being stable, meaning that the position where an element is stored only ever changes when resizes occur. In fact, in the setting where keys are Θ (log n ) bits, the space guarantees that Iceberg hashing offers, namely that it uses at most \(\log \binom{|U|}{n} + O(n \log \ \text{log} n)\) bits to store n items from a universe U , matches a lower bound by Demaine et al. that applies to any stable hash table. Iceberg hashing introduces new general-purpose techniques for some of the most basic aspects of hash-table design. Notably, our indirection-free technique for dynamic resizing, which we call waterfall addressing, and our techniques for achieving stability and very-high probability guarantees, can be applied to any hash table that makes use of the front-yard/backyard paradigm for hash table design.

ESH: Design and Implementation of an Optimal Hashing Scheme for Persistent Memory

Recent advancements in memory technology have opened up a wealth of possibilities for innovation in data structures. The emergence of byte-addressable persistent memory (PM) with its impressive capacity and low latency has accelerated the adoption of PM in existing hashing-based indexes. As a result, several new hashing schemes utilizing emulators have been proposed. However, these schemes have proven to be suboptimal, lacking scalability when implemented on real devices. Only a handful of hash table designs have successfully addressed critical properties such as load factor, scalability, efficient memory utilization, and recovery. One of the main challenges in redesigning data structures for an effective hashing scheme in PM is minimizing the overhead associated with dynamic hashing operations in the hash table. To tackle this challenge, this paper introduces ESH, an efficient and scalable hashing scheme that significantly improves memory efficiency, scalability, and overall performance on PM. The ESH scheme maximizes the utilization of the hash table’s available space, thus reducing the frequency of full-table rehashing and improving performance. Consequently, this scheme achieves a high load factor while minimizing the need for rehashing. To evaluate the effectiveness of the ESH scheme, we compare it to widely used dynamic hashing schemes employing similar techniques on Intel Optane® DC persistent memory (DCPMM). The experimental results demonstrate that ESH outperforms CCEH and Dash in terms of data insertion performance, exhibiting a 30% improvement over CCEH and a 4% improvement over Dash. Furthermore, ESH improves the lookup operation by nearly 10% compared to Dash, while achieving a load factor of up to 91%, surpassing its competitors.

Scaling Dynamic Hash Tables on Real Persistent Memory

Byte-addressable persistent memory (PM) brings hash tables the potential of low latency, cheap persistence and instant recovery. The recent advent of Intel Optane DC Persistent Memory Modules (DCPMM) further accelerates this trend. Many new hash table designs have been proposed, but most of them were based on emulation and perform sub-optimally on real PM. They were also piecewise and partial solutions that side-stepped many important properties, in particular good scalability, high load factor and instant recovery.

Dash

Byte-addressable persistent memory (PM) brings hash tables the potential of low latency, cheap persistence and instant recovery. The recent advent of Intel Optane DC Persistent Memory Modules (DCPMM) further accelerates this trend. Many new hash table designs have been proposed, but most of them were based on emulation and perform sub-optimally on real PM. They were also piece-wise and partial solutions that side-step many important properties, in particular good scalability, high load factor and instant recovery. We present Dash, a holistic approach to building dynamic and scalable hash tables on real PM hardware with all the aforementioned properties. Based on Dash, we adapted two popular dynamic hashing schemes (extendible hashing and linear hashing). On a 24-core machine with Intel Optane DCPMM, we show that compared to state-of-the-art, Dash-enabled hash tables can achieve up to ∼3.9× higher performance with up to over 90% load factor and an instant recovery time of 57ms regardless of data size.

Revisiting Hash Table Design for Phase Change Memory

Phase Change Memory (PCM) is emerging as an attractive alternative to Dynamic Random Access Memory (DRAM) in building data-intensive computing systems. PCM offers read/write performance asymmetry that makes it necessary to revisit the design of in-memory applications. In this paper, we focus on in-memory hash tables, a family of data structures with wide applicability. We evaluate several popular hash-table designs to understand their performance under PCM. We find that for write-heavy workloads the designs that achieve best performance for PCMdiffer from the ones that are best for DRAM, and that designs achieving a high load factor also cause a high number of memory writes. Finally, we propose PFHT, a PCM-Friendly Hash Table which presents a cuckoo hashing variant that is tailored to PCM characteristics, and offers a better trade-off between performance, the amount of writes generated, and the expected load factor than any of the existing DRAMbased implementations.

Applications of sequential set partitioning: a set of technical information systems problems

Many problems that appear in different contexts are conceptually similar and so are amenable to solution by a common technique. Three such technical Information Systems (IS) problems are: (1) segmentation of computer programs; (2) attribute discretization for decision tree induction; and (3) design of hash tables in database systems. In this paper we show how each of these seemingly different problems can be formulated as a sequential (set) partitioning problem, and solved using a parametric linear programming (LP) procedure. This approach provides optimal solutions unlike previous solution approaches which were either greedy heuristics or limited to solving only a specific problem situation. Given the likelihood that other applications of the sequential partitioning problem exist in IS, the material presented here could be useful to other researchers in formulating the problem at an appropriate level of abstraction so that available optimal solution approaches can be identified. In addition to providing a common solution method, parametric LP frees the user from having to make premature decisions regarding the number of groups for the partition, and this decision can be delayed post solution.

Optimal design of multiple hash tables for concurrency control

In this paper, we propose the approach of using multiple hash tables for lock requests with different data access patterns to minimize the number of false contentions in a data sharing environment. We first derive some theoretical results on using multiple hash tables. Then, in light of these derivations, a two-step procedure to design multiple hash tables is developed. In the first step, data items are partitioned into a given number of groups. Each group of data items is associated with the use of a hash table in such a way that lock requests to data items in the same group will be hashed into the same hash table. In the second step, given an aggregate hash table size, the hash table size for each individual data group is optimally determined so as to minimize the number of false contentions. Some design examples and remarks on the proposed method are given. It is observed from real database systems that different data sets usually have their distinct data access patterns, thus resulting in an environment where this approach can offer significant performance improvement.