Time-Sharing Multithreading on Stream-Based Lossless Data Compression

Abstract

To address the performance saturation in communication data path at migrating BigData, the lossless data compression technique is a solution to enhance the bandwidth of the path. However, the recent data compression mechanism needs to treat data streams such as sensor data with very low latency to avoid overhead in the path. This paper focuses on a new loss less data compression mechanism called LCA-DLT that implements a hardware-based fast stream lossless data compression using dictionary-based symbol lookup mechanism. When applying it to a very fast path, the hardware latency increases largely and the clock speed degrades because the dictionary lookup operation becomes bottleneck of the longest delay path in the hardware. This paper proposes a performance improvement technique applying multithreading technique in the dictionary lookup operation. The technique enables a single module of the LCA-DLT to accept multiple data streams by dividing the compression timing in babble stage of the compression/decompression pipeline. According to performance evaluation by a hardware implementation with two threads, although the data compression bandwidth logically becomes half of the original single thread LCA-DLT, the time-sharing multithreading technique reduces required hardware resources and improves the clock frequency.