Non-cryptographic Hash Functions Research Articles

Questioning the Criteria for Evaluating Non-cryptographic Hash Functions

Although cryptographic and non-cryptographic hash functions are everywhere, there seems to be a gap in how they are designed. Lots of criteria exist for cryptographic hashes motivated by various security requirements, but on the non-cryptographic side there is a certain amount of folklore that, despite the long history of hash functions, has not been fully explored. While targeting a uniform distribution makes a lot of sense for real-world datasets, it can be a challenge when confronted by a dataset with particular patterns.

An adaptive image steganography algorithm based on the use of non-cryptographic hash functions for data extraction

An adaptive steganography algorithm for data hiding in full-color images based on iterative introduction of minor distortions into blocks of container images and the use of high-speed non-cryptographic hash functions for data extraction is considered. Modification of the minimum number of container elements compared to the length of the hidden message is a distinctive feature of the algorithm. This feature allows the hidden throughput to be increased and the visual and statistical visibility of hidden data to be reduced. The algorithm is compared with modern algorithms of adaptive spatial steganography in terms of assessing the level of distorting changes in stego-containers. In addition, a modified version of the algorithm that implements covert channel multiplexing using a common subset of container elements when embedding various messages into them is considered.

Optimized algorithms and architectures for fast non-cryptographic hash functions in hardware

The design and implementation of eight novel non-cryptographic hash functions is proposed in this paper. Those can suitably be employed in networking and security applications that require fast lookup and/or counting architectures. Our approach is inspired by the design of the existing non-cryptographic hash function Xoodoo-NC, which is constructed through the concatenation of several Xoodoo permutations. In a similar way, this work concatenates several rounds of symmetric-key ciphers to construct non-cryptographic hash functions. The goal is to achieve high performance while maintaining the avalanche properties. Meeting the avalanche criteria is necessary to yield a significant change in the output of hash functions even for the slightest change in the input. The number of rounds required to achieve satisfactory avalanche scores for each cipher is simulated. We implement the corresponding non-cryptographic hash functions on an FPGA and we synthesize them on ASIC to evaluate the occupied resources and the performance. This paper evaluates the performance and area of ASIC implementations optimized for high operating speed. This evaluation is done on the 65 nm TSMC technology. One of the proposed non-cryptographic hash functions, namely PHOTON-NC, outperforms all previously proposed non-cryptographic hash functions in terms of throughput and latency on FPGA, in exchange for an acceptable increase in FPGA resources, while Xoodoo-NC proves to have the highest performance on ASIC.

Hardware-oriented optimization of Bloom filter algorithms and architectures for ultra-high-speed lookups in network applications

This paper optimizes Bloom filter algorithms and hardware architectures for high-speed implementation on FPGAs. A Bloom filter is a data structure that is used to check whether input search data are present in a table of stored data. Bloom filters are extensively used for monitoring network traffic. Improving the speed of Bloom filters can therefore have a significant impact on the speed of many network applications. The most important components determining the speed of Bloom filters are hash functions. While hash functions in Bloom filters do not require strong cryptographic properties, they do need a minimized computational delay. In this work, we evaluate and compare the performance and resource utilization of different Bloom filter algorithms and architectures as well as non-cryptographic hash functions on an FPGA. We also propose a new non-cryptographic hash function, called Xoodoo-NC, derived from the cryptographic permutation Xoodoo. Xoodoo-NC is a reduced-round, reduced-state version of Xoodoo, inheriting Xoodoo’s desired avalanche properties and low logical depth, resulting in an ultra-low-latency non-cryptographic hash function. We evaluate the performance of Bloom filter architectures based on Xoodoo-NC on a Xilinx UltraScale+ FPGA and we compare the performance and resource occupation to existing Bloom filter implementations. We additionally compare our results to memories that use the built-in CAM cores in Xilinx UltraScale+ FPGAs. Our proposed algorithmic and architectural advances lead to Bloom filters that, to the best of our knowledge, outperform all other FPGA-based solutions.

Hunting the pertinency of hash and bloom filter combinations on GPU for fast pattern matching

There has been rapid growth in the field of graphical processing unit (GPU) programming due to the drastic increase in the computing hardware manufacturing. The technology used in these devices is now more affordable and accessible to the general public. With this growth, many serial programming applications that are now being transformed into more efficient parallel programming applications with significant improvement in the performance. The best example for this is parallel implementation of the probabilistic data structure Bloom filter in set membership queries. However, despite of it’s remarkable performance in speed and memory usage, there is a computational overhead in the calculation of hashes in Bloom filter. In this paper, the impact of the choice of hash functions on the qualitative properties of the Bloom filter has been experimentally recorded and the results show that there is a possibility of large performance gap among various hash functions. We have implemented the Bloom filter based pattern matching technique on GPU using compute unified device architecture (CUDA) and benchmark the performance of several cryptographic and non-cryptographic hash functions.

SELECTION OF CONTROLLABLE PARAMETERS OF THE BINARY DATA ARRAY DETECTION ALGORITHM IN THE INTEGRATED MANAGEMENT SYSTEM

The article discusses the selection of parameters for the algorithm for determining binary data arrays included in the control system, developed by the authors using independent substitution methods. Based on the analysis of the algorithms of non-cryptographic hash functions, the hash function based on the linear matching method was selected as the basis for independent substitution methods. Simplified schemes of algorithms developed for creating and comparing identifiers using a set of basic hash functions are given. An array of binary data was selected and based on the appropriate values for the size of the divisible blocks and the number of basic hashfunctions used for independent substitutions. The selection of binary data arrays in information systems integrated into the management system was done for the purpose of intellectual processing of incoming data. The properties of the array of data entering integrated systems are studied. The authors conducted experimental tests in the selected direction and presented the results of similarity assessment measurements for various parameters of the identification algorithm. In addition, the article conductedexperiments on the object of study using the selected mathematical model, based on the analytical conclusions. Initiator elements are studied and analyzed using a set of hash functions. An algorithm for comparison of selected identifiers has been developed. A generation algorithm has been developed to demonstrate and test the proposed solution. Algorithms based on analysis and experiments, and methods for selecting binary data arrays using the ash function have been experimentally tested. Based on the results, the indicators are shown. Based on the results obtained, the analytical conclusions and problem solutions of the research work were recognized

An Enhanced Non-Cryptographic Hash Function

An Enhanced Non-Cryptographic Hash Function

Evolutionary hash functions for specific domains

Hash functions are a key component of many essential applications, ranging from compilers, databases or internet browsers to videogames or network devices. The same reduced set of functions are extensively used and have become “standard de facto” since they provide very efficient results in searches over unsorted sets. However, depending on the characteristics of the data being hashed, the overall performance of these non-cryptographic hash functions can vary dramatically, becoming a very common source of performance loss. Hash functions are difficult to design, they are extremely non-linear and counter-intuitive, and relationships among variables are often intricate and obscure. Surprisingly, very little scientific research is devoted to the design and experimental assessment of these widely used functions. In this work, in addition to performing an up-to-date comparison of state-of-the-art hash functions, we propose the use of evolutionary techniques for designing “ad hoc” non-cryptographic hash functions. Thus, genetic programming will be used to automatically design a tailor-made hash function that can be continuously evolved if needed, so that it is always adapted to real-world dynamic environments. To validate the proposed approach, we have compared several quality metrics for the generated functions and the most widely used non-cryptographic hash functions across eight different scenarios. The results of the evolved hash functions outperformed those of the non-cryptographic hash functions in most of the cases tested.

AUTOMATIC DESIGN OF NONCRYPTOGRAPHIC HASH FUNCTIONS USING GENETIC PROGRAMMING

Noncryptographic hash functions have an immense number of important practical applications owing to their powerful search properties. However, those properties critically depend on good designs: Inappropriately chosen hash functions are a very common source of performance losses. On the other hand, hash functions are difficult to design: They are extremely nonlinear and counterintuitive, and relationships between the variables are often intricate and obscure. In this work, we demonstrate the utility of genetic programming (GP) and avalanche effect to automatically generate noncryptographic hashes that can compete with state‐of‐the‐art hash functions. We describe the design and implementation of our system, called GP‐hash, and its fitness function, based on avalanche properties. Also, we experimentally identify good terminal and function sets and parameters for this task, providing interesting information for future research in this topic. Using GP‐hash, we were able to generate two different families of noncryptographic hashes. These hashes are able to compete with a selection of the most important functions of the hashing literature, most of them widely used in the industry and created by world‐class hashing experts with years of experience.

MemGuard

Memory system reliability is increasingly a concern as memory cell density and capacity continue to grow. The conventional approach is to use redundant memory bits for error detection and correction, with significant storage, cost and power overheads. In this paper, we propose a novel, system-level scheme called MemGuard for memory error detection. With OS-based checkpointing, it is also able to recover program execution from memory errors. The memory error detection of MemGuard is motivated by memory integrity verification using log hashes. It is much stronger than SECDED in error detection, incurs negligible hardware cost and energy overhead and no storage overhead, and is compatible with various memory organizations. It may play the role of ECC memory in consumer-level computers and mobile devices, without the shortcomings of ECC memory. In server computers, it may complement SECDED ECC or Chipkill Correct by providing even stronger error detectio We have comprehensively investigated and evaluated the feasibility and reliability of MemGuard. We show that using an incremental multiset hash function and a non-cryptographic hash function, the performance and energy overheads of Mem- Guard are negligible. We use the mathematical deduction and synthetic simulation to prove that MemGuard is robust and reliable.

Randomness testing of non-cryptographic hash functions for real-time hash table based storage and look-up of URLs

Non-cryptographic hash functions have been investigated to identify their pseudo-random nature when employed in the implementation of hash tables for real-time storage and look-up of uniform resource locators. Statistical studies have been performed on the sequences generated using five widely used non-cryptographic hash functions: (1) CRC, (2) Adler, (3) DJBX33A, (4) FNV, and (5) Murmur. The comparative analysis of tested non-cryptographic hash functions shows that the Adler hash function is not suitable for hash table implementation, whereas, the rest of non-cryptographic hash functions exhibit similar and better randomizing features which make them an attractive choice for hash table implementation.

Performance of the most common non-cryptographic hash functions

Non-cryptographic hash functions NCHFs have an immense number of applications, ranging from compilers and databases to videogames and computer networks. Some of the most important NCHF have been used by major corporations in commercial products. This practical success demonstrates the ability of hashing systems to provide extremely efficient searches over unsorted sets. However, very little research has been devoted to the experimental evaluation of these functions. Therefore, we evaluated the most widely used NCHF using four criteria as follows: collision resistance, distribution of outputs, avalanche effect, and speed. We identified their strengths and weaknesses and found significant flaws in some cases. We also discuss our conclusions regarding general hashing considerations such as selection of the compression map. Our results should assist practitioners and engineers in making more informed choices regarding which function to use for a particular problem. Copyright © 2013 John Wiley & Sons, Ltd.