Parallel construction of minimal perfect hashing functions with neural networks

Abstract

The seeking of minimal perfect hashing functions (MPHF) has a long history and conventional construction methods are sequential algorithms. To parallelize the MPHF construction, a new method using neural networks is proposed in this paper. It constructs a MPHF by training a massive array of neural nets, and the training tasks can be carried out simultaneously. As the total MPHF construction time is proportional to the key set size, the new method can be applied to build MPHFs for large key sets. In one experiment, a MPHF for a dictionary of 24,464 English words is constructed by training an array of 764 multilayered feedforward neural nets. Network training time is reduced by employing an incremental training procedure. Implementation issues concerning persistent object storage and retrieval are also discussed.