Parallelism and dictionary based data compression

Abstract

Abstract Because of the size of information involved with the emerging applications in multimedia and the Human Genome Project , parallelism offers the only hope of meeting the challenges of storing such databases and searching through quickly. In this paper, we address dictionary based lossless text compression and give the state-of-the-art in the field of parallelism. Static dictionary compression and sliding window (LZ1) compression have been successfully parallelized by many authors. Dynamic dictionary compression (LZ2) seems hardly parallelizable since some related heuristics are known to be P-complete. In spite of such negative results, the decoding process can be parallelized efficiently for LZ2 compression as well as for static and LZ1. A main issue for implementation purposes in dictionary based compression is to bound the dictionary size [3] , [23] . Differences in terms of parallel complexity are not relevant between compression with bounded and unbounded windows from a theoretical point of view. Much more interesting are the results concerning bounded size dictionary compression with the LZ2 method. When the size of the dictionary is O(logkn ), a bounded size dictionary version (LRU deletion heuristic) of the LZ2 compression algorithm is hard for the class of problems solvable simultaneously in polynomial time and O(log kn) space (that is, SCk). A relaxed variation of this heuristic is the first natural SCk-complete problem (the original heuristic belongs to SCk+1 ). In virtue of these results, it can be argued that there are no practical parallel algorithms for LZ2 compression with LRU deletion heuristic or any other heuristic deleting dictionary elements in a continuous way. For simpler heuristics (SWAP, RESTART, FREEZE), practical parallel algorithms exist.