Dictionary-based Compression Schemes Research Articles

Design of Advanced High Performance Bus Tracer in System on Chip Using Matrix Based Compression for Low Power Applications

This project proposes multilayer advanced high-performance bus architecture for low power applications. The proposed AHB architecture consists of the bus arbiter and the bus tracer (A.R.M.A., 1999. Specification (Rev 2.0) ARM IHI0011A). The bus arbiter, which is self motivated selects the input packet based on the control signals of the incoming packet. So that arbitration leads to a maximum performance. The On-Chip bus is an important system-on-chip infrastructure that connects major hardware components. Monitoring the on-chip bus signals is crucial to the SoC debugging and performance analysis/optimization (Gu, R.T., et al., 2007. A Low Cost Tile-Based 3D Graphics Full Pipeline with Real-Time Performance Monitoring Support for OpenGL ES in Consumer Electronics. 2007 IEEE International Symposium on Consumer Electronics, June; IEEE. pp.1–6). But, such signals are difficult to observe since they are deeply embedded in a SoC and there are often no sufficient I/O pins to access these signals. Therefore, a straightforward approach is to embed a bus tracer in SoC to capture the bus signal trace and store the trace in on-chip storage such as the trace memory which could then be off loaded to outside world for analysis. The bus tracer is capable of capturing the bus trace with different resolutions, all with efficient built in compression mechanisms such as dictionary based compression scheme for address and control signals and differential compression scheme for data. To improve the compression ratio matrix based compression which is lossless compression is used instead of differential compression. This system is designed using Verilog HDL, simulated using Modelsim and synthesized using Xilinx software.

Dictionary Compression in Point Cloud Data Management

Nowadays, massive amounts of point cloud data can be collected thanks to advances in data acquisition and processing technologies such as dense image matching and airborne LiDAR scanning. With the increase in volume and precision, point cloud data offers a useful source of information for natural-resource management, urban planning, self-driving cars, and more. At the same time, on the scale that point cloud data is produced, management challenges are introduced: it is important to achieve efficiency both in terms of querying performance and space requirements. Traditional file-based solutions to point cloud management offer space efficiency, however, they cannot scale to such massive data and provide the declarative power of a DBMS. In this article, we propose a time- and space-efficient solution to storing and managing point cloud data in main memory column-store DBMS. Our solution, Space-Filling Curve Dictionary-Based Compression (SFC-DBC), employs dictionary-based compression in the spatial data management domain and enhances it with indexing capabilities by using space-filling curves. SFC-DBC does so by constructing the space-filling curve over a compressed, artificially introduced dictionary space. Consequently, SFC-DBC significantly optimizes query execution and yet does not require additional storage resources, compared to traditional dictionary-based compression. With respect to space-filling-curve-based approaches, it minimizes storage footprint and increases resilience to skew. As a proof of concept, we develop and evaluate our approach as a research prototype in the context of SAP HANA. SFC-DBC outperforms other dictionary-based compression schemes by up to 61% in terms of space and up to 9.4× in terms of query performance.

Improving Energy Efficiency of Coarse-Grain Reconfigurable Arrays Through Modulo Schedule Compression/Decompression

Modulo-scheduled course-grain reconfigurable array (CGRA) processors excel at exploiting loop-level parallelism at a high performance per watt ratio. The frequent reconfiguration of the array, however, causes between 25% and 45% of the consumed chip energy to be spent on the instruction memory and fetches therefrom. This article presents a hardware/software codesign methodology for such architectures that is able to reduce both the size required to store the modulo-scheduled loops and the energy consumed by the instruction decode logic. The hardware modifications improve the spatial organization of a CGRA’s execution plan by reorganizing the configuration memory into separate partitions based on a statistical analysis of code. A compiler technique optimizes the generated code in the temporal dimension by minimizing the number of signal changes. The optimizations achieve, on average, a reduction in code size of more than 63% and in energy consumed by the instruction decode logic by 70% for a wide variety of application domains. Decompression of the compressed loops can be performed in hardware with no additional latency, rendering the presented method ideal for low-power CGRAs running at high frequencies. The presented technique is orthogonal to dictionary-based compression schemes and can be combined to achieve a further reduction in code size.

Note on the greedy parsing optimality for dictionary-based text compression

Dynamic dictionary-based compression schemes are the most daily used data compression schemes since they appeared in the foundational paper of Ziv and Lempel in 1977, commonly referred to as LZ77. In dynamic setting, LZ77 considers a portion of the previous text as a dictionary and it uses a greedy approach to select, at each step, the longest match between the text and the dictionary. Compression is achieved by replacing matches with encoded dictionary pointers. LZ77 is the base of gzip, zip, rar, 7zip and many others compression software. All these compression schemes use variants of the greedy approach to parse (or factorise) the text into dictionary phrases. Greedy parsing optimality with respect to the number of phrases was proved by Storer et al. (1982) for unbounded LZ77-based dictionaries and by Cohn et al. (1996) for static suffix-closed dictionaries. The optimality of the greedy parsing was never proved for bounded size dictionary which is actually required by all of these practical schemes.In this article, we define the suffix-closed property for dynamic dictionaries, and we show that LZ77-based compression schemes, including the bounded dictionary variants, satisfy this property. Under this condition we prove the optimality of the greedy parsing as a variant of the proof by Cohn et al.

SAR Image Compression Using Multiscale Dictionary Learning and Sparse Representation

In this letter, we focus on a new compression scheme for synthetic aperture radar (SAR) amplitude images. The last decade has seen a growing interest in the study of dictionary learning and sparse representation, which have been proved to perform well on natural image compression. Because of the special techniques of radar imaging, SAR images have some distinct properties when compared with natural images that can affect the design of a compression method. First, we introduce SAR properties, sparse representation, and dictionary learning theories. Second, we propose a novel SAR image compression scheme by using multiscale dictionaries. The experimental results carried out on amplitude SAR images reveal that, when compared with JPEG, JPEG2000, and a single-scale dictionary-based compression scheme, the proposed method is better for preserving the important features of SAR images with a competitive compression performance.

On parsing optimality for dictionary-based text compression—the Zip case

Dictionary-based compression schemes are the most commonly used data compression schemes since they appeared in the foundational paper of Ziv and Lempel in 1977, and generally referred to as LZ77. Their work is the base of Zip, gZip, 7-Zip and many other compression software utilities. Some of these compression schemes use variants of the greedy approach to parse the text into dictionary phrases; others have left the greedy approach to improve the compression ratio. Recently, two bit-optimal parsing algorithms have been presented filling the gap between theory and best practice. We present a survey on the parsing problem for dictionary-based text compression, identifying noticeable results of both a theoretical and practical nature, which have appeared in the last three decades. We follow the historical steps of the Zip scheme showing how the original optimal parsing problem of finding a parse formed by the minimum number of phrases has been replaced by the bit-optimal parsing problem where the goal is to minimise the length in bits of the encoded text.

Grayscale true two-dimensional dictionary-based image compression

Dictionary-based encoding methods are popular forms of data compression. These methods were initially implemented to reduce the one-dimensional correlation in data, since they are designed to compress text. Therefore, they do not take advantage of the fact that adjacent pixels in images are correlated in two dimensions. Previous attempts have been made to adapt dictionary-based compression schemes to consider the two-dimensional nature of images, but mostly for binary images. In this paper, a two-dimensional dictionary-based lossless image compression scheme for grayscale images is introduced. The proposed scheme reduces correlation in image data by finding two-dimensional blocks of pixels that are approximately matched throughout the data and replacing them with short codewords. Test results show that the compression performance of the proposed scheme outperforms and surpasses any other existing dictionary-based lossless compression scheme. The results also show that it slightly outperforms JPEG-2000s compression performance, when it operates in its lossless mode, and it is comparable to JPEG-LS’s and CALIC’s compression performance, where JPEG-2000 and JPEG-LS are the current image compression standards, and CALIC is a Context-based Adaptive Lossless Image Coding scheme.

On-line and off-line heuristics for inferring hierarchies of repetitions in sequences

Hierarchical dictionary-based compression schemes form a grammar for a text by replacing each repeated string with a production rule. While such schemes usually operate on-line, making a replacement as soon as repetition is detected, off-line operation permits greater freedom in choosing the order of replacement. In this paper, we compare the on-line method with three off-line heuristics for selecting the next substring to replace: longest string first, most common string first, and the string that minimizes the size of the grammar locally. Surprisingly, two of the off-line techniques, like the on-line method, run in time linear in the size of the input. We evaluate each technique on artificial and natural sequences. In general, the locally-most-compressive heuristic performs best, followed by most frequent, the on-line technique, and, lagging by some distance, the longest-first technique.

Teletext data change detection and noiseless data compression

The full channel teletext system is a high speed data broadcasting system. Pages of information are broadcast in a cyclic manner. The detection of data change in the information pages is necessary for data analysis, database update and retransmission. Lossless data compression is also necessary to enhance the data throughput in rebroadcasting and to reduce the storage requirement. Performing data change detection and data compression in real time using a software approach in a small machine is impossible for such high speed data. The authors describe the algorithms that are suitable for hardware implementation for both data change detection and noiseless data compression. They also illustrate the use of cyclic redundancy check (CRC) and dictionary based compression scheme using a hashing function to detect changes and compression respectively in the teletext pages.