Hierarchical Hash Research Articles

Implementation and Evaluation of Lightweight in-Vehicle Security Switch with Hierarchical Hash Table

Implementation and Evaluation of Lightweight in-Vehicle Security Switch with Hierarchical Hash Table

Hierarchical Consensus Hashing for Cross-Modal Retrieval

Cross-modal hashing (CMH) has gained much attention due to its effectiveness and efficiency in facilitating efficient retrieval between different modalities. Whereas, most existing methods unconsciously ignore the hierarchical structural information of the data, and often learn a single-layer hash function to directly transform cross-modal data into common low-dimensional hash codes in one step. This sudden drop of dimension and the huge semantic gap can cause the discriminative information loss. To this end, we adopt a coarse-to-fine progressive mechanism and propose a novel Hierarchical Consensus Cross-Modal Hashing (HCCH). Specifically, to mitigate the loss of important discriminative information, we propose a coarse-to-fine hierarchical hashing scheme that utilizes a two-layer hash function to refine the beneficial discriminative information gradually. And then, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math></inline-formula> -norm is imposed on the layer-wise hash function to alleviate the effects of redundant and corrupted features. Finally, we present consensus learning to effectively encode data into a consensus space in such a progressive way, thereby reducing the semantic gap progressively. Through extensive contrast experiments with some advanced CMH methods, the effectiveness and efficiency of our HCCH method are demonstrated on four benchmark datasets. We have released the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/sunyuan-cs</uri> .

Hierarchical Hashing Learning for Image Set Classification.

With the development of video network, image set classification (ISC) has received a lot of attention and can be used for various practical applications, such as video based recognition, action recognition, and so on. Although the existing ISC methods have obtained promising performance, they often have extreme high complexity. Due to the superiority in storage space and complexity cost, learning to hash becomes a powerful solution scheme. However, existing hashing methods often ignore complex structural information and hierarchical semantics of the original features. They usually adopt a single-layer hashing strategy to transform high-dimensional data into short-length binary codes in one step. This sudden drop of dimension could result in the loss of advantageous discriminative information. In addition, they do not take full advantage of intrinsic semantic knowledge from whole gallery sets. To tackle these problems, in this paper, we propose a novel Hierarchical Hashing Learning (HHL) for ISC. Specifically, a coarse-to-fine hierarchical hashing scheme is proposed that utilizes a two-layer hash function to gradually refine the beneficial discriminative information in a layer-wise fashion. Besides, to alleviate the effects of redundant and corrupted features, we impose the ℓ2,1 norm on the layer-wise hash function. Moreover, we adopt a bidirectional semantic representation with the orthogonal constraint to keep intrinsic semantic information of all samples in whole image sets adequately. Comprehensive experiments demonstrate HHL acquires significant improvements in accuracy and running time. We will release the demo code on https://github.com/sunyuan-cs.

Optimize data-driven multi-agent simulation for COVID-19 transmission

BackgroundMulti-Agent Simulation is an essential technique for exploring complex systems. In research of contagious diseases, it is widely exploited to analyze their spread mechanisms, especially for preventing COVID-19. Nowadays, transmission dynamics and interventions of COVID-19 have been elaborately established by this method, but its computation performance is seldomly concerned. As it usually suffers from inadequate CPU utilization and poor data locality, optimizing the performance is challenging and important for real-time analyzing its spreading.ResultsThis paper explores approaches to optimize multi-agent simulation for COVID-19 disease. The focus of this work is on the algorithm and data structure designs for improving performance, as well as its parallelization strategies. We propose two successive methods to optimize the computation. We construct a case-focused iteration algorithm to improve data locality, and propose a fast data-mapping scheme called hierarchical hash table to accelerate hash operations. As a result, The case-focused method degrades sim 90 % cache references and achieves times 4.3 speedup. Hierarchical hash table can further boost computation speed by 47%. And parallel implementation with 20 threads on CPU achieves times 80 speedup consequently.ConclusionsIn this work, we propose optimizations for multi-agent simulation of COVID-19 transmission from aspects of algorithm and data structure. Benefit from improvement of locality and multi-thread implementation, our methods can significantly accelerate the simulation computation. It is promising in supporting real-time prevention of COVID-19 and other infectious diseases in the future.

Efficient hierarchical hash tree for OpenFlow packet classification with fast updates on GPUs

Packet classification is an important functionality of modern routers/switches, needed in packet forwarding, Quality of Service (QoS), firewall etc. In order to better utilize routers on the Internet, Software Defined Network (SDN) decouples control plane from data plane to fulfill centralized management. Based on OpenFlow standards, packet classification in SDN is designed for multi-field rules which are more complex than traditional 5-tuple rules. In the paper, we propose a novel packet classification algorithm, called hierarchical hash tree (H-HashTree), based on the two IP address fields and the 7 exact-match fields to partition rules into groups. An extended Bloom filter is also proposed to accelerate search process by skipping groups in the hash tree. To further improve the performance, H-HashTree is implemented on GPU. We tested on 100K rules including synthesized rules containing characteristics of ACL, FW, and IPC with different wildcard ratios in exact-match fields, and real OpenFlow rules from Open vSwitch. Compared with the existing state-of-the-art algorithms, CutTSS and TabTree [19][18], H-HashTree achieves the best performance on both search and update speeds. H-HashTree achieves 1.17-13.9 and 2.48-12.7 times faster in search speed and 2.03-6.0 and 1.87-4.53 times faster in rule updates from synthesized rulesets than CutTSS and TabTree, respectively. On the GPU platform, H-HashTree can achieve up to 114 MPPS in search speed and less than 0.04 usec/rule in rule updates.

Hierarchical Preference Hash Network for News Recommendation

Personalized news recommendation is becoming increasingly important for online news platforms to help users alleviate information overload and improve news reading experience. A key problem in news recommendation is learning accurate user representations to capture their interest. However, most existing news recommendation methods usually learn user representation only from their interacted historical news, while ignoring the clustering features among users. Here we proposed a hierarchical user preference hash network to enhance the representation of users' interest. In the hash part, a series of buckets are generated based on users' historical interactions. Users with similar preferences are assigned into the same buckets automatically. We also learn representations of users from their browsed news in history part. And then, a Route Attention is adopted to combine these two parts (history vector and hash vector) and get the more informative user preference vector. As for news representation, a modified transformer with category embedding is exploited to build news semantic representation. By comparing the hierarchical hash network with multiple news recommendation methods and conducting various experiments on the Microsoft News Dataset (MIND) validate the effectiveness of our approach on news recommendation.

SimSim: A Service Discovery Method Preserving Content Similarity and Spatial Similarity in P2P Mobile Cloud

Mobile cloud has become a new computing paradigm such that services are accessible in any place and at any time. Despite its promising prospect, challenges arise due to unreliable channel condition and limited bandwidth in wireless communication, dynamic route establishment due to node mobility, difficulties in associating request to relevant service providers, and complication in service deployment. To ensure the fairness of resource allocation and network load balance, it is necessary to consider strategies for distributing services. In this paper, we propose SimSim, a service discovery scheme based on keywords search which preserves content similarity and spatial similarity. A mapping from a keyword set of services to a bit vector with identical hash is designed to preserve content similarity. The proposed technique applies a hierarchical hash clustering model and investigates the strategies of service deployment and discovery. By mapping the services characterized by keywords to the Gray space, SimSim offers similar services at close geographical proximity. Extensive simulations have been conducted to assess the proposed system.

H-CNN: Spatial Hashing Based CNN for 3D Shape Analysis.

We present a novel spatial hashing based data structure to facilitate 3D shape analysis using convolutional neural networks (CNNs). Our method builds hierarchical hash tables for an input model under different resolutions that leverage the sparse occupancy of 3D shape boundary. Based on this data structure, we design two efficient GPU algorithms namely hash2col and col2hash so that the CNN operations like convolution and pooling can be efficiently parallelized. The perfect spatial hashing is employed as our spatial hashing scheme, which is not only free of hash collision but also nearly minimal so that our data structure is almost of the same size as the raw input. Compared with existing 3D CNN methods, our data structure significantly reduces the memory footprint during the CNN training. As the input geometry features are more compactly packed, CNN operations also run faster with our data structure. The experiment shows that, under the same network structure, our method yields comparable or better benchmark results compared with the state-of-the-art while it has only one-third memory consumption when under high resolutions (i.e., 2563).

Deep Constrained Siamese Hash Coding Network and Load-Balanced Locality-Sensitive Hashing for Near Duplicate Image Detection.

We construct a new efficient near duplicate image detection method using a hierarchical hash code learning neural network and load-balanced locality-sensitive hashing (LSH) indexing. We propose a deep constrained siamese hash coding neural network combined with deep feature learning. Our neural network is able to extract effective features for near duplicate image detection. The extracted features are used to construct a LSH-based index. We propose a load-balanced LSH method to produce load-balanced buckets in the hashing process. The load-balanced LSH significantly reduces the query time. Based on the proposed load-balanced LSH, we design an effective and feasible algorithm for near duplicate image detection. Extensive experiments on three benchmark data sets demonstrate the effectiveness of our deep siamese hash encoding network and load-balanced LSH.

Fast Hash-Based Inter-Block Matching for Screen Content Coding

In the latest High Efficiency Video Coding (HEVC) development, i.e., HEVC screen content coding extensions (HEVC-SCC), a hash-based inter-motion search/block matching scheme is adopted in the reference test model, which brings significant coding gains to code screen content. However, the hash table generation itself may take up to half the encoding time and is thus too complex for practical usage. In this paper, we propose a hierarchical hash design and the corresponding block matching scheme to significantly reduce the complexity of hash-based block matching. The hierarchical structure in the proposed scheme allows large block calculation to use the results of small blocks. Thus, we avoid redundant computation among blocks with different sizes, which greatly reduces complexity without compromising coding efficiency. The experimental results show that compared with the hash-based block matching scheme in the HEVC-SCC test model (SCM)-6.0, the proposed scheme reduces about 77% of hash processing time, which leads to 12% and 16% encoding time savings in random access (RA) and low-delay B coding structures. The proposed scheme has been adopted into the latest SCM. A parallel implementation of the proposed hash table generation on graphics processing unit (GPU) is also presented to show the high parallelism of the proposed scheme, which achieves more than 30 frames/s for 1080p sequences and 60 frames/s for 720p sequences. With the fast hash-based block matching integrated into x265 and the hash table generated on GPU, the encoder can achieve 11.8% and 14.0% coding gains on average for RA and low-delay P coding structures, respectively, for real-time encoding.

ZGLS: a novel flat quorum-based and reliable location management protocol for VANETs

In location management services, a destination advertises its position attributes to a set of vehicles called location servers while, a source obtains these attributes from such location servers to track destination. The location management techniques in VANET have been categorized into flooding-based, flat hashing-based, hierarchical hashing-based and hierarchical quorum-based techniques. In flooding-based location service, destination information is flooded to the entire network which results into high congestion, low throughput and non-scalable network. In flat hashing, a global hash function is applied to compute location servers of each destination which results into higher delay, drop and signaling overhead in large VANETs. In hierarchical hashing, global hash function computes location servers of destination in hierarchical order. It therefore suffers from handover signaling between servers, high load on the top hierarchy and location query delay when source and destination are apart. In hierarchical quorum-based, location servers are identified cluster-wise and therefore it also suffers from the problems similar to hierarchical techniques. To overcome these problems, ZoomOut Geographic Location Service (ZGLS) protocol is proposed which introduces flat quorum-based location management service. In contrast to the aforementioned techniques, the novelty of ZGLS lies in the fact that it has shifted the location server role from hashing-based or clustering-based geographic areas to few 1-hop neighbours, called relatives. The proposed protocol creates a chain of relatives to provide positioning and tracking service. To evaluate signalling overhead, timeliness and the reliability of update and query packets, ZGLS is compared with RLSMP and HRHLS through ns-2 simulations. The results reveal that ZGLS stands out as a better choice for large-scale sparse and dense VANETs.

Hierarchical deep semantic hashing for fast image retrieval

Content-Based large-scale image retrieval has recently attracted considerable attention because of the explosive increase of online images. Inspired by recent advances in convolutional neural networks, we propose a hierarchical deep semantic method for learning similarity function that solves the problems of precision and speed of retrieval in the setting of large-scale environments. The distinctive contribution of our work is a novel approach that can utilize previous knowledge of the semantic hierarchy. When semantic information and a related hierarchy structure are available, significant improvements can be attained. Exploiting hierarchical relationships is the most important thing for large-scale issues. Binary code can be learned from deep neural network for representing the latent concepts that dominate the semantic labels. Different from other supervised methods that require learning an explicit hashing function to map the binary code features from the images, our method learns Hierarchical Deep Semantic Hashing code (HDSH-code) and image representations in an implicit manner, making it suitable for large-scale datasets. An additional contribution is a novel hashing scheme (generated at the same time with semantic information) that is able to reduce the computational cost of retrieval. Comprehensive experiments were conducted on Holidays, Oxford5k/105k, Caltech256 retrieval datasets, our HDSH performs competitively even when the convolutional neural network has been pre-trained for a surrogate unrelated task. We further demonstrates its efficacy and scalability on a large-scale dataset Imagenet with millions of images. With deep hierarchical semantic hashing, we report retrieval times are 0.15ms and 53.92ms on Holidays and Imagenet dataset, respectively.

HHT-Based Security Enhancement Approach with Low Overhead for Coding-Based Reprogramming Protocols in Wireless Sensor Networks

Coding-based reprogramming protocols can effectively and remotely disseminate a new code image to all sensor nodes via wireless channels in wireless sensor networks. However, security service is crucial to these protocols when sensor nodes are deployed in adversarial environments. Existing security schemes can resist Pollution Attack, but the overheads are excessive. In this paper, a security enhancement approach with low overhead based on Hierarchical Hash Tree is proposed to enhance the security of the protocols. Our scheme is composed of two layers of Merkle Tree based on the ideas of hierarchy and aggregation. Then, the security of proposed approach is proven and the overheads of that are analyzed. Furthermore, our scheme is used to implement page authentication of Sreluge protocol, which is a representative reprogramming protocol based on random linear codes. Experimental results show that our scheme can cut authentication overhead by at least 43 % that of Merkle Tree and other overheads have been reduced markedly with the size of code image growing.

Map vs. Unordered Map: An Analysis on Large Datasets

In the current scenario, with the transpiring big data explosion, data sets are often too large to fit completely inside the computers internal memory. In efficient processes, speed is not an option, it is a must. Hence every alternative is explored to further enhance performance, by expanding inplace memory storage that enables more data to be resident in the memory, eliminating operation latency, and even deploying an in-memory database (IMDB) system where all the data can be kept in memory. However, the technique of inmemory data handling is still at an infant stage and not viable in the current scenario. To tackle this problem a hierarchical hashing scheme is discussed where only one component of a big data structure resides in the memory. In this paper two data structures are explored: 1) Map which is implemented as self-balancing binary search trees or more commonly Red Black Trees and 2) Unordered Map which is based on hashing with chaining technique. Serialization and deserialization operations are also performed to free the internal memory and preserve the data structure object for later use. Operations such as read, write are performed, along with documentation of the results and illustrations of visual representations of the two algorithmic data structures.

New Identity-based Key-exposure Free Chameleon Hash from Bilinear Pairings

Chameleon signatures are based on the well established hash-and-sign paradigm, where a chameleon hash function is used to compute the cryptographic message digest. Chameleon signatures simultaneously provide the properties of non-repudiation and non-transferability for the signed message. However, the initial constructions of chameleon signatures suffer from the problem of key exposure of chameleon hashing. Recently, some specific constructions of key-exposure free chameleon hashing are presented in the setting of certificate-based systems, but few works are done in identity-based systems. In this paper, we first analyse the hierarchical identity-based chameleon hash (HIBCH) and key-exposure free identity-based chameleon hash schemes (IBCH) proposed by Bao et al. in 2011 and then propose two improved schemes from bilinear pairings, with the idea of hierarchical key distribution. Moreover, we analyse the security and efficiency of our proposed schemes.

An on-line hot data identification for flash-based storage using sampling mechanism

Efficient hot and cold data identification in computer systems has been a fundamental issue. However, it has been least investigated. In this paper, we propose a novel on-line hot data identification scheme for flash-based storage named HotDataTrap. The main idea is to maintain a working set of potential hot data items in a cache based on a sampling mechanism. This sampling-based scheme enables HotDataTrap to early discard some of the cold items so that it can reduce runtime overheads as well as a waste of memory spaces. Moreover, our two-level hash indexing scheme helps HotDataTrap directly look up a requested item in the cache and save a memory space further by exploiting spatial localities. Both our sampling approach and hierarchical hash indexing scheme empower HotDataTrap to precisely and efficiently identify hot data with a even less memory. Our extensive experiments with various realistic workloads demonstrate that our HotDataTrap outperforms the state-of-the-art scheme by an average of 335% and our two-level hash indexing scheme considerably improves further HotDataTrap performance up to 50.8%.

Hash Subgraph Pairwise Kernel for Protein-Protein Interaction Extraction

Extracting protein-protein interaction (PPI) from biomedical literature is an important task in biomedical text mining (BioTM). In this paper, we propose a hash subgraph pairwise (HSP) kernel-based approach for this task. The key to the novel kernel is to use the hierarchical hash labels to express the structural information of subgraphs in a linear time. We apply the graph kernel to compute dependency graphs representing the sentence structure for protein-protein interaction extraction task, which can efficiently make use of full graph structural information, and particularly capture the contiguous topological and label information ignored before. We evaluate the proposed approach on five publicly available PPI corpora. The experimental results show that our approach significantly outperforms all-path kernel approach on all five corpora and achieves state-of-the-art performance.

Fast Incremental Policy Compilation from Plans in Hybrid Probabilistic Domains

We present the domain-independent HRFF algorithm, which solves goal-oriented HMDPs by incrementally aggregating plans generated by the METRIC-FF planner into a policy defined over discrete and continuous state variables. HRFF takes into account non-monotonic state variables, and complex combinations of many discrete and continuous probability distributions. We introduce new data structures and algorithmic paradigms to deal with continuous state spaces: hybrid hierarchical hash tables, domain determinization based on dynamic domain sampling or on static computation of probability distributions' modes, optimization settings under METRIC-FF based on plan probability and length. We deeply analyze the behavior of HRFF on a probabilistically-interesting structured navigation problem with continuous dead-ends and non-monotonic continuous state variables. We compare with HAO* on the Rover domain and show that HRFF outperforms HAO* by many order of magnitudes in terms of computation time and memory usage. We also experiment challenging and combinatorial HMDP versions of benchmarks from numeric classical planning.

Word shape descriptor-based document image indexing: a new DBH-based approach

In this paper, we propose a novel feature representation for binary patterns by exploiting the object shape information. Initial evaluation of the representation is performed for Bengali and Gujarati script character classification. The extension of the representation for word images is presented subsequently. The proposed feature representation in combination with distance-based hashing is applied for defining novel word image-based document image indexing and retrieval framework. The concept of hierarchical hashing is utilized to reduce the retrieval time complexity. In addition, with the objective of reduction in the size of hashing data structure, the concept of multi-probe hashing is extended for binary mapping functions. The exhaustive experimental evaluation of the proposed framework on a collection of documents belonging to Devanagari, Bengali and English scripts has yielded encouraging results.

VSYNC: Bandwidth-Efficient and Distortion-Tolerant Video File Synchronization

We introduce video-sync (VSYNC), a video file synchronization system that efficiently uses a bidirectional communications link to maintain up-to-date video sources at remote ends to a desired resolution and distortion level. By automatically detecting and transmitting only the differences between video files, VSYNC is able to avoid unnecessary re-transmission of the entire video when there are only minor differences between video copies. A hierarchical hashing scheme is designed to allow synchronization to within some user-defined distortion, white being rate-efficient and computationally tractable. Distributed video coding is used to realize further rate savings when transmitting video updates. VSYNC is bandwidth-efficient and is useful in many scenarios including video backup, video sharing, and video authentication applications. Experimental results show that rate-savings ranging from 2× to 10× can be obtained by VSYNC with about 10% of the frames being edited, compared to re- transmitting the compressed video or using a file synchronization utility such as rsync.