Consistent Hashing Research Articles

Structured Allocation-Based Consistent Hashing With Improved Balancing for Cloud Infrastructure

Consistent hashing has played an indispensable role in cloud infrastructure, although its load balancing performance is not necessarily perfect. Consistent hashing has long remained the most widely used method despite many methods being proposed to improve load balancing because these methods trade off load balancing against consistency, memory usage, lookup performance, and/or fault-tolerance. This article presents Structured Allocation-based Consistent Hashing (SACH), a cloud-optimized consistent hashing algorithm that overcomes the trade-offs by taking advantage of the characteristics of cloud environments: scaling management and auto-healing. Since scaling can be distinguished from failures, SACH applies two different algorithms to update hashing functions: a fast-update algorithm for unmanaged backend failures to satisfy fault-tolerance with quick response and a slow-update algorithm for managed scaling. Hashing functions are initialized or slow-updated considering the characteristics of the fast-update algorithm to satisfy load balancing and the other properties as far as the number of failed backends is kept small by auto-healing. The experimental results show that SACH outperforms existing algorithms in each aspect. SACH will improve the load balancing of cloud infrastructure components, where the trade-offs have prevented the renewal of hashing functions.

Analysis and optimisation of server load balancing-based multi-factor integrated algorithm

In distributed server application service deployment, load balancing is an important factor that affects service performance. However, the biggest factor affecting load balancing is the performance of the server and the performance ratio of the requested service execution. This paper used the application of Service-Oriented Architecture system as the embedding point, based on the analysis and comparison with traditional load balancing algorithms, such as: Random load balancing algorithm, Round-Robin algorithm, Consistent Hash algorithm, etc. This paper is aimed to design a new load factor-based algorithm to solve some defects of traditional load balancing algorithm by improving minimum concurrency number and integrating various factors that affect server performance. This paper proposes an optimisation algorithm that can more fully reflect server load capacity. The experiment results show that the load balancing effect of this algorithm has a better effect than the traditional load balancing algorithm.

An Efficient Organization Method for Large-Scale and Long Time-Series Remote Sensing Data in a Cloud Computing Environment

Historical earth observation (EO) data have played an important role in long-term scientific and environmental monitoring. The effective organization of large-scale and long-term remote-sensing data to achieve efficient retrieval and access has become one of the important issues. However, inherent big data characteristics, such as a large scale, and asymmetrical temporal and spatial distributions, have caused problems with the efficiency of data retrieval and access. Therefore, this study proposes an efficient data organization method for use in a cloud-computing environment that has two aims. First, it addresses the problem of low retrieval efficiency. An asymmetrical index model for the image metadata is constructed that is based on a unified spatio-temporal grid coding; a prepartitioning mechanism under the HBase architecture is established to realize the uniform storage of the metadata with an asymmetrical spatiotemporal distribution and to avoid retrieval efficiency bottlenecks caused by a load imbalance. Second, it addresses low access efficiency. By dividing the remote-sensing image into tiles, a unified spatio-temporal code is established for each tile, and a consistent hash operation is performed; tiles with similar hash values are stored in the same or adjacent Hadoop Distributed File System nodes. In this way, tiles with temporal or spatial correlations can be gathered and stored, and lots of disk seeks can be avoided during retrieval, thereby greatly improving the data access efficiency. Comparative experiments showed that the data organization method can effectively improve the retrieval and access efficiencies of large-scale and long time-series remote-sensing data in a cloud-computing environment.

Analysis and optimisation of server load balancing-based multi-factor integrated algorithm

In distributed server application service deployment, load balancing is an important factor that affects service performance. However, the biggest factor affecting load balancing is the performance of the server and the performance ratio of the requested service execution. This paper used the application of Service-Oriented Architecture system as the embedding point, based on the analysis and comparison with traditional load balancing algorithms, such as: Random load balancing algorithm, Round-Robin algorithm, Consistent Hash algorithm, etc. This paper is aimed to design a new load factor-based algorithm to solve some defects of traditional load balancing algorithm by improving minimum concurrency number and integrating various factors that affect server performance. This paper proposes an optimisation algorithm that can more fully reflect server load capacity. The experiment results show that the load balancing effect of this algorithm has a better effect than the traditional load balancing algorithm.

AnchorHash: A Scalable Consistent Hash

Consistent hashing is a central building block in many networking applications, such as maintaining connection affinity of TCP flows. However, current consistent hashing solutions do not ensure full consistency under arbitrary changes or scale poorly in terms of memory footprint, update time and key lookup complexity. We present AnchorHash, a scalable and fully-consistent hashing algorithm. AnchorHash achieves high key lookup rate, low memory footprint and low update time. We formally establish its strong theoretical guarantees, and present an advanced implementation with a memory footprint of only a few bytes per resource. Moreover, evaluations indicate that AnchorHash scales on a single core to 100 million resources while still achieving a key lookup rate of more than 15 million keys per second.

Asymmetric Supervised Consistent and Specific Hashing for Cross-Modal Retrieval.

Hashing-based techniques have provided attractive solutions to cross-modal similarity search when addressing vast quantities of multimedia data. However, existing cross-modal hashing (CMH) methods face two critical limitations: 1) there is no previous work that simultaneously exploits the consistent or modality-specific information of multi-modal data; 2) the discriminative capabilities of pairwise similarity is usually neglected due to the computational cost and storage overhead. Moreover, to tackle the discrete constraints, relaxation-based strategy is typically adopted to relax the discrete problem to the continuous one, which severely suffers from large quantization errors and leads to sub-optimal solutions. To overcome the above limitations, in this article, we present a novel supervised CMH method, namely Asymmetric Supervised Consistent and Specific Hashing (ASCSH). Specifically, we explicitly decompose the mapping matrices into the consistent and modality-specific ones to sufficiently exploit the intrinsic correlation between different modalities. Meanwhile, a novel discrete asymmetric framework is proposed to fully explore the supervised information, in which the pairwise similarity and semantic labels are jointly formulated to guide the hash code learning process. Unlike existing asymmetric methods, the discrete asymmetric structure developed is capable of solving the binary constraint problem discretely and efficiently without any relaxation. To validate the effectiveness of the proposed approach, extensive experiments on three widely used datasets are conducted and encouraging results demonstrate the superiority of ASCSH over other state-of-the-art CMH methods.

Query Optimization Algorithm of Replication Join Based on Sampling Partition

Aiming at the low efficiency of join query in MapReduce traditional partition join algorithm when data skew, a replication join optimization algorithm based on sampling partition is proposed. According to the sampled statistics of connection attribute data, the algorithm divides the datasets in connection relationship into skewed data subset and non skewed data subset. In order to optimize the query performance, join query processing is carried out on them respectively. For the join queries of non skewed data subsets, the improved consistency hash function is used to partition these subsets, so that the load of data connection query processing of each node is balanced. For the skewed data subset join query, the smaller skewed data subsets are distributed to each node, and the larger skewed data subsets are partitioned according to the non skewed fields. In the Reduce stage, these skewed data subsets are join queried. Experiments show that the algorithm can optimize the join query performance under different data skew rates, and achieve efficient join query processing of large datasets.

Inductive Structure Consistent Hashing via Flexible Semantic Calibration.

Semantic-preserving hashing establishes efficient multimedia retrieval by transferring knowledge from original data to hash codes so that the latter can preserve the underlying visual and semantic similarities. However, it becomes a crucial bottleneck: how to effectively bridge the trilateral domain gaps (i.e., the visual, semantic, and hashing spaces) to further improve the retrieval accuracy. In this article, we propose an inductive structure consistent hashing (ISCH) method, which can interactively coordinate the semantic correlations between the visual feature space, the binary class space, and the discrete hashing space. Specifically, an inductive semantic space is formulated by a simple multilayer stacking class-encoder, which transforms the naive class information into flexible semantic embeddings. Meanwhile, we design a semantic dictionary learning model to facilitate the bilateral visual-semantic bridging and guide the class-encoder toward reliable semantics, which could well alleviate the visual-semantic bias problem. In particular, the visual descriptors and respective semantic class representations are regularized with a coinciding alignment module. In order to generate privileged hash codes, we further explore semantic and prototype binary code learning to jointly quantify the semantic and latent visual representations into unified discrete hash codes. Moreover, an efficient optimization algorithm is developed to address the resulting discrete programming problem. Comprehensive experiments conducted on four large-scale data sets, i.e., CIFAR-10, NUSWIDE, ImageNet, and MSCOCO, demonstrate the superiority of our method over the state-of-the-art alternatives against different evaluation protocols.

Distributed Video Content Caching Policy With Deep Learning Approaches for D2D Communication

In this paper, we develop a novel active video content caching scheme (RCC) based on a recommendation system, and consistent hash for device-to-device(D2D) communication. The RCC scheme is constructed by a cache placement scheme, a consistent hash algorithm, an optimal video segmentation scheme, and an optimal video library segmentation scheme. To begin with, a well-designed cache placement scheme based on a mobile model with helper notes, and video segmentation is proposed to reduce the redundancy of the videos, and save users’ cache. In order to solve the interruption problem caused by segmentation, a consistent hash algorithm is introduced to improve the success probability of D2D communication. According to the recommendation system, all users’ predictive score for all videos can be calculated, which results that users’ most interesting video files rather than popular video files as previous work can be obtained, and cached in advance to improve the hitting probability. Furthermore, an optimal video segmentation scheme, and an optimal video library segmentation scheme are developed to minimize the transmission delay, and maximize the hitting probability respectively. Simulation results show that compared with other traditional caching schemes, the proposed RCC scheme can reach about 70% reduction in outage probability, 40% reduction in system latency, and 10% improvement in hitting probability, all of which can achieve the best performance.

Consistent Hash Algorithm in Distributed Monitoring System

Consistent Hash Algorithm in Distributed Monitoring System

The design of Apache IoTDB distributed framework

Apache Internet of Things Database (IoTDB) is a new open-source timeseries database management system. A distributed data management system not only needs to solve the problem of metadata synchronization between nodes caused by data partition and multiple replicas, but also needs to support efficient query request processing. To solve the problem of metadata synchronization among nodes, we propose a dual-layer granularity metadata management strategy. Based on the consistency hash partitioning method and Raft protocol, we designed a distributed framework that supports both strong consistency query and eventual consistency query. Based on the single-machine version of Apache IoTDB, we carried out the system implementation and experimental test. Compared with the single-level granularity management strategy, the test results showed that the two-level granularity metadata management strategy takes less memory resources and improves the write performance by $5%\sim10%$. Also, the results showed that the read and write performance of the distributed Apache IoTDB increases linearly with the extension of cluster size.

Flexible Discrete Multi-view Hashing with Collective Latent Feature Learning

Multi-view hashing has gained considerable research attention in efficient multimedia studies due to its promising performance on heterogeneous data from various sources. However, its application in discriminative hash codes learning remains challenging as it fails to efficiently capture preferable components from multiple representations. In this work, we propose a novel discriminative multi-view hashing framework, dubbed flexible discrete multi-view hashing, in conjunction with collective latent feature learning by combining multiple views of data and consistent hash codes learning by fusing visual features and flexible semantics. Specifically, an adaptive multi-view analysis dictionary learning model is developed to skillfully combine diverse representations into an established common latent feature space where the complementary properties of different views are well explored based on an automatic multi-view weighting strategy. Moreover, we introduce a collaborative learning scheme to jointly encode the visual and semantic embeddings into an aligned consistent Hamming space, which can effectively mitigate the visual-semantic gap. Particularly, we employ the correntropy induced regularization to improve the robustness of the formulated flexible semantics. An efficient learning algorithm is proposed to solve the optimization problem. Extensive experiments show the state-of-art performance of the proposed method on several benchmark datasets.

Consistent hashing with bounded loads and virtual nodes-based load balancing strategy for proxy cache cluster

Response time and bandwidth occupancy of backbone network are greatly affected by load balancing and cache hit ratio in proxy cache cluster (PCC). A novel load balancing strategy is proposed to improve the homogeneous and heterogeneous PCC performance. By combining with the different performance parameters, performance ratio can be formulated with independent information data fluctuation weighting method, and the virtual nodes (VNs) attached to every cache node (CN) can be generated by a random function. The greatest common divisor m of the number of VNs attached to every CN can be calculated and the hash ring is cut into m arcs, and the VNs are mapped proportionally to arcs of the hash ring by the MD5 function. Requests are assigned to VNs based on the forwarding rule of consistent hashing with bounded loads, and the CN in correspondence with the VNs will be selected to provide services. Simulation results show that the strategy can significantly improve the load balancing in homogeneous and heterogeneous PCC, effectively reduce bandwidth occupancy of backbone network and PCC response time.

Effective peer-to-peer design for supporting range query in Internet of Things applications

The Internet of Things (IoT) refers to the internetworking of diverse devices in an ad-hoc manner to support pervasive applications. IoT devices often generate a wealth of data that ought to be accessible and managed in a distributed manner. Such operational model requires architecture that supports contextual and information centric retrieval of data, and efficient data storage. In this paper, we argue that peer-to-peer (P2P) overlays are well suited for IoT systems. However, existing P2P systems do not efficiently handle queries of data within a range, which is a popular access pattern in IoT applications. Moreover, many of the IoT devices are constrained in their computational resources and consequently the data management model has to cope with such limitation. Existing P2P solutions do not factor the heterogeneity of the involved nodes and assume abundant storage space. This paper opts to fill the technical gap and proposes an effective P2P solution for efficient handling of range queries in IoT (RQIOT). RQIOT employs a data distribution model based on both consistent and order-preserving hashing and introduces a novel scheme for capacity management of the involved devices. The simulation results have confirmed the effectiveness and scalability of RQIOT, and the superiority of its performance over competing approaches.

Contrasting SCSI Disks and Consistent Hashing

Contrasting SCSI Disks and Consistent Hashing

Research of Voice-over-IP Systems for Internet

The implications of ambimorphic archetypes have been far-reaching and pervasive. After years of natural research into consistent hashing, we argue the simulation of public-private key pairs, which embodies the confirmed principles of theory. Such a hypothesis might seem perverse but is derived from known results. Our focus in this paper is not on whether the well-known knowledge-based algorithm for the emulation of checksums by Herbert Simon runs in Θ(n) time, but rather on exploring a semantic tool for harnessing telephony (Swale).

Low powered blockchain consensus protocols based on consistent hash

Current blockchain consensus protocols have a triangle of contradictions in aspects of decentralization, security, and energy consumption, and cannot be synchronously optimized. We describe a design of two new blockchain consensus protocols, called “CHB-consensus” and “CHBD-consensus,” based on a consistent hash algorithm. Honest miners can fairly gain the opportunity to create blocks. They do not consume any extra computational power resources when creating new blocks, and such blocks can obtain the whole blockchain network to confirm consensus with fairness. However, malicious miners have to pay massive computational power resources for attacking the new block creation privilege or double-spending. Blockchain networks formed by CHB-consensus and CHBD-consensus are based on the same security assumption as that in Bitcoin systems, so they save a huge amount of power without sacrificing decentralization or security. We analyze possible attacks and give a rigorous but adjustable validation strategy. CHB-consensus and CHBD-consensus introduce a certification authority (CA) system, which does not have special management or control rights over blockchain networks or data structures, but carries the risk of privacy breaches depending on credibility and reliability of the CA system. Here, we analyze the robustness and energy consumption of CHB-consensus and CHBD-consensus, and demonstrate their advantages through theoretical derivation.

Linear-Time, Client-Server Communication for Consistent Hashing

The implications of robust technology have been far-reaching and persistent. After years of private research into IPv7, we show the development of hierarchical databases. BasicJub, our new heuristic for decentralized theory, is the solution to all of these obstacles[1,3,5]

Analysis of Consistent Hashing

The combination of put in writing flipside cache has studied IPv6, and up to date trends put forward that the construction of hash tables will materialize. Here, we show the amalgamation of Web services, which symbolize the archetypal principles of theory. We motivate a low energy means for exploring the location-identity split, which we call WAVEY.

A Methodology for the Analysis of Consistent Hashing

The synthesis of IPv4 is a practical obstacle. After years of unproven research into the World Wide Web, we demonstrate the synthesis of access points. In our research we use Bayesian methodologies to argue that the much-touted empathic algorithm for the analysis of superblocks by Takahashi et al. [19] is impossible.