On the construction of reduced double array structures by dividing tries

Abstract

Speed and storage capacity are big issues in information retrieval system. For natural language analysis, a double array is one of a data structure of trie, which is a well-known approach to retrieve strings in a dictionary, and helps very fast access in a matrix table with compactness of a list form. In order to realize quite compact structure, this paper presents a compression method by dividing the trie constructed into several pieces (blocks). It enables us to reduce the number of bits representing entries of the double arrays. The trie obtained must trace to the blocks and this causes that retrieval time might be slow because of a state connection. To solve this problem, we propose a new trie construction method to compress and minimize the number of state connections. After applying to a large set of keys, experimental result shows that the storage capacity has been reduced to 50% even though the approach we present has the same retrieval speed.