Dictionary-based Compression Research Articles

Efficient regular expression matching over hybrid dictionary-based compressed data

As a core technique of deep packet inspection, regular expression matching effectively extracts valuable contents from network traffic. Its performance is crucial to network security and big data applications. However, mobile internet brings massive network traffic, and most are compressed for a better user experience, which challenges the performance of matching compressed data. The present works focus on the LZ77 compressed data rather than the new excellent hybrid dictionary-based compression data. Vcdiff, as the representative scheme, combines a static shared dictionary and a self-adaptive dynamic dictionary to ensure better compression density. It also exhibits excellent performance on compression and decompression speed. However, accelerating matching Vcdiff compressed data is only suitable for the string matching scenarios and fails to show an impressive performance. In this paper, we propose ERASER to accelerate regular expression matching over hybrid dictionary-based compressed data, which is also the first approach to accelerate regular expression matching over Vcdiff compressed data. The experiments show that ERASER can double the speed of the existing methods on matching Vcdiff compressed data. It also remarkably improves preprocessing and matching performance, which is better than the state-of-the-art approach over LZ77 compressed data. We also propose a model to explore the correlation between the matching speed and the compression density. The experiments demonstrate that the matching speed of a method will increase with the better compression density of compressed data.

A Novel Low-Power Compression Scheme for Systolic Array-Based Deep Learning Accelerators

The proliferation of deep learning algorithms has catalyzed their utilization to solve a multitude of real-world problems. Algorithms such as deep neural networks (DNNs) are compute- and power-intensive, thereby accentuating the development of hardware platforms like DNN inference accelerators. However, inference execution of large DNNs in resource-constrained environments induces energy bottlenecks in these accelerators. Since large DNNs consist of hundreds of millions of trained parameters, accessing them from the accelerator memory incurs substantial energy. To address this challenge, we propose HardCompress, which, to the best of our knowledge, is the first low-power solution that uses traditional compression strategies pertaining to commercial DNN accelerators in resource-constrained IoT edge devices. The three-step approach involves hardware-based post-quantization trimming of weights, followed by their dictionary-based compression and subsequent decompression by a low-power hardware engine during inference in the accelerator. We evaluate the proposed solution on lightweight networks trained on the MNIST dataset, the compact model trained on the CIFAR-10 dataset, and large DNNs trained on the ImageNet dataset. Performance of HardCompress at different quantization levels has been analyzed. Furthermore, to quantify the effectiveness of the proposed solution, an energy framework that contrasts the DRAM energies of the original and HardCompressed models has been developed. Finally, a fault injection framework which compares the fault resilience of the original model with its HardCompressed counterpart is also proposed. Our results exhibit that HardCompress, without any performance degradation in large DNNs, furnishes a maximum compression of 99.27%, equivalent to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$137\times $ </tex-math></inline-formula> reduction in memory footprint and 0.07 J for 8-bit quantization in the systolic array-based DNN accelerator. Furthermore, our proposed low-power decompression engine incurs an area overhead of only 0.02%; thus, enabling HardCompress’ utilization in resource-constrained environments.

Generalized low-rank nonrigid motion-corrected reconstruction for MR fingerprinting.

Develop a novel low-rank motion-corrected (LRMC) reconstruction for nonrigid motion-corrected MR fingerprinting (MRF). Generalized motion-corrected (MC) reconstructions have been developed for steady-state imaging. Here we extend this framework to enable nonrigid MC for transient imaging applications with varying contrast, such as MRF. This is achieved by integrating low-rank dictionary-based compression into the generalized MC model to reconstruct MC singular images, reducing motion artifacts in the resulting parametric maps. The proposed LRMC reconstruction was applied for cardiac motion correction in 2D myocardial MRF (T1 and T2 ) with extended cardiac acquisition window (~450 ms) and for respiratory MC in free-breathing 3D myocardial and 3D liver MRF. Experiments were performed in phantom and 22 healthy subjects. The proposed approach was compared with reference spin echo (phantom) and with 2D electrocardiogram-triggered/breath-hold MOLLI and T2 gradient-and-spin echo conventional maps (in vivo 2D and 3D myocardial MRF). Phantom results were in general agreement with reference spin-echo measurements, presenting relative errors of approximately 5.4% and 5.5% for T1 and short T2 (<100 ms), respectively. The proposed LRMC MRF reduced residual blurring artifacts with respect to no MC for cardiac or respiratory motion in all cases (2D and 3D myocardial, 3D abdominal). In 2D myocardial MRF, left-ventricle T1 values were 1150 ± 41 ms for LRMC MRF and 1010 ± 56 ms for MOLLI; T2 values were 43.8 ± 2.3 ms for LRMC MRF and 49.5 ± 4.5 ms for T2 gradient and spin echo. Corresponding measurements for 3D myocardial MRF were 1085 ± 30 ms and 1062 ± 29 ms for T1 , and 43.5 ± 1.9 ms and 51.7 ± 1.7 ms for T2 . For 3D liver, LRMC MRF measured liver T1 at 565 ± 44 ms and liver T2 at 35.4 ± 2.4 ms. The proposed LRMC reconstruction enabled generalized (nonrigid) MC for 2D and 3D MRF, both for cardiac and respiratory motion. The proposed approach reduced motion artifacts in the MRF maps with respect to no motion compensation and achieved good agreement with reference measurements.

On the Approximation Ratio of Ordered Parsings

Shannon’s entropy is a clear lower bound for statistical compression. The situation is not so well understood for dictionary-based compression. A plausible lower bound is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {b}$ </tex-math></inline-formula> , the least number of phrases of a general bidirectional parse of a text, where phrases can be copied from anywhere else in the text. Since computing <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {b}$ </tex-math></inline-formula> is NP-complete, a popular gold standard is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {z}$ </tex-math></inline-formula> , the number of phrases in the Lempel-Ziv parse of the text, which is computed in linear time and yields the least number of phrases when those can be copied only from the left. Almost nothing has been known for decades about the approximation ratio of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {z}$ </tex-math></inline-formula> with respect to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {b}$ </tex-math></inline-formula> . In this paper we prove that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$z=O(b\log (n/b))$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> is the text length. We also show that the bound is tight as a function of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> , by exhibiting a text family where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$z = \Omega (b\log n)$ </tex-math></inline-formula> . Our upper bound is obtained by building a run-length context-free grammar based on a locally consistent parsing of the text. Our lower bound is obtained by relating <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {b}$ </tex-math></inline-formula> with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$r$ </tex-math></inline-formula> , the number of equal-letter runs in the Burrows-Wheeler transform of the text. We continue by observing that Lempel-Ziv is just one particular case of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">greedy</i> parses–meaning that it obtains the smallest parse by scanning the text and maximizing the phrase length at each step–, and of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ordered</i> parses–meaning that phrases are larger than their sources under some order. As a new example of ordered greedy parses, we introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lexicographical</i> parses, where phrases can only be copied from lexicographically smaller text locations. We prove that the size <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$v$ </tex-math></inline-formula> of the optimal lexicographical parse is also obtained greedily in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O(n)$ </tex-math></inline-formula> time, that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$v=O(b\log (n/b))$ </tex-math></inline-formula> , and that there exists a text family where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$v = \Omega (b\log n)$ </tex-math></inline-formula> . Interestingly, we also show that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$v = O(r)$ </tex-math></inline-formula> because <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$r$ </tex-math></inline-formula> also induces a lexicographical parse, whereas <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$z = \Omega (r\log n)$ </tex-math></inline-formula> holds on some text families. We obtain some results on parsing complexity and size that hold on some general classes of greedy ordered parses. In our way, we also prove other relevant bounds between compressibility measures, especially with those related to smallest grammars of various types generating (only) the text.

Combination of columnar transposition cipher caesar cipher and lempel ziv welch algorithm in image security and compression

Exchange of information through the medium of the internet allows information can be accessed by unauthorized parties. Techniques for securing information that is commonly used is Cryptography. This research will use a combination of two classic algorithms which are Columnar Transposition Cipher and a Caesar Cipher with a Linear Congruential Generator modification. Information in the form of images often has some of the same pixel value components. For saving the memory then the writers use the technique of compression with Lempel Ziv Welch algorithm, which is a dictionary-based compression algorithm. Stages of the process used in this study are that the pictures will be encrypted using the algorithm of Columnar Transposition Cipher, next step the file that has been encrypted will be encrypted with the Caesar Cipher algorithm using a combination of and Linear Congruential generators are modified. Then the image file that has been encrypted will be compressed using the second algorithm Lempel Ziv Welch. For the implementation of programming language C # systems used. The results of this study showed that the combination of Columnar Transposition Cipher and a Caesar Cipher with a Linear Congruential Generators could keep the confidentiality, integrity, and security of data. Lempel Ziv Welch algorithm produces the value of the average compression ratio (RC) 67.05%, in Compression Ratio (CR) aspect the value is 1.49, and in Space Saving (SS) parameter, the percentage is 32.93%.

Efficient Test Data Compression for SoC through ASRL with Improved Dictionary based Compression Technique

Data compression techniques are explored in this paper, through which system memory size gets reduced in an effective manner. The size of the memory is always a key constraint in the embedded system. Larger memory size increases the bandwidth utilization which raises the cost of hardware and data transmission. It is difficult to transfer large data through the network. Data compression encoding technique is utilized to minimize the data size. The redundant character is reduced or encoding the bits in data is done to reduce the data size. The proposed system focused on lossless compression where the original information of the data is available even though the data size is compressed. The data compression is done through a dictionary-based compression algorithm and Alternating Statistical Run Length code (ASRL). In the existing system of ASRL, the compression ratio is about 65.16% and 67.18% for two benchmark circuits S5378 &amp;S9234. The compression ratio of the test data is increased by combining the ASRL and Improved Dictionary-Based compression Technique. The proposed combined technique provides 80.25%&amp; 82.5% compression ratio for two benchmark circuits S5378 &amp;S9234. This reduces the power dissipation problem in the circuit and thereby the area of the circuit gets reduced.

A dictionary-based text compression technique using quaternary code

Improving encoding and decoding time in compression technique is a great demand to modern users. In bit level compression technique, it requires more time to encode or decode every single bit when a binary code is used. In this research, we develop a dictionary-based compression technique where we use a quaternary tree instead of a binary tree for construction of Huffman codes. Firstly, we explore the properties of quaternary tree structure mathematically for construction of Huffman codes. We study the terminology of new tree structure thoroughly and prove the results. Secondly, after a statistical analysis of English language, we design a variable length dictionary based on quaternary codes. Thirdly, we develop the encoding and decoding algorithms for the proposed technique. We compare the performance of the proposed technique with the existing popular techniques. The proposed technique performs better than the existing techniques with respect to decompression speed while the space requirement increases insignificantly.

Dictionary Compression in Point Cloud Data Management

Nowadays, massive amounts of point cloud data can be collected thanks to advances in data acquisition and processing technologies such as dense image matching and airborne LiDAR scanning. With the increase in volume and precision, point cloud data offers a useful source of information for natural-resource management, urban planning, self-driving cars, and more. At the same time, on the scale that point cloud data is produced, management challenges are introduced: it is important to achieve efficiency both in terms of querying performance and space requirements. Traditional file-based solutions to point cloud management offer space efficiency, however, they cannot scale to such massive data and provide the declarative power of a DBMS. In this article, we propose a time- and space-efficient solution to storing and managing point cloud data in main memory column-store DBMS. Our solution, Space-Filling Curve Dictionary-Based Compression (SFC-DBC), employs dictionary-based compression in the spatial data management domain and enhances it with indexing capabilities by using space-filling curves. SFC-DBC does so by constructing the space-filling curve over a compressed, artificially introduced dictionary space. Consequently, SFC-DBC significantly optimizes query execution and yet does not require additional storage resources, compared to traditional dictionary-based compression. With respect to space-filling-curve-based approaches, it minimizes storage footprint and increases resilience to skew. As a proof of concept, we develop and evaluate our approach as a research prototype in the context of SAP HANA. SFC-DBC outperforms other dictionary-based compression schemes by up to 61% in terms of space and up to 9.4× in terms of query performance.

FCompress: An Algorithm for FASTQ Sequence Data Compression

Background: Biological sequence data have increased at a rapid rate due to the advancements in sequencing technologies and reduction in the cost of sequencing data. The huge increase in these data presents significant research challenges to researchers. In addition to meaningful analysis, data storage is also a challenge, an increase in data production is outpacing the storage capacity. Data compression is used to reduce the size of data and thus reduces storage requirements as well as transmission cost over the internet. Objective: This article presents a novel compression algorithm (FCompress) for Next Generation Sequencing (NGS) data in FASTQ format. Method: The proposed algorithm uses bits manipulation and dictionary-based compression for bases compression. Headers are compressed with reference-based compression, whereas quality scores are compressed with Huffman coding. Results: The proposed algorithm is validated with experimental results on real datasets. The results are compared with both general purpose and specialized compression programs. Conclusion: The proposed algorithm produces better compression ratio in a comparable time to other algorithms.

Improving Energy Efficiency of Coarse-Grain Reconfigurable Arrays Through Modulo Schedule Compression/Decompression

Modulo-scheduled course-grain reconfigurable array (CGRA) processors excel at exploiting loop-level parallelism at a high performance per watt ratio. The frequent reconfiguration of the array, however, causes between 25% and 45% of the consumed chip energy to be spent on the instruction memory and fetches therefrom. This article presents a hardware/software codesign methodology for such architectures that is able to reduce both the size required to store the modulo-scheduled loops and the energy consumed by the instruction decode logic. The hardware modifications improve the spatial organization of a CGRA’s execution plan by reorganizing the configuration memory into separate partitions based on a statistical analysis of code. A compiler technique optimizes the generated code in the temporal dimension by minimizing the number of signal changes. The optimizations achieve, on average, a reduction in code size of more than 63% and in energy consumed by the instruction decode logic by 70% for a wide variety of application domains. Decompression of the compressed loops can be performed in hardware with no additional latency, rendering the presented method ideal for low-power CGRAs running at high frequencies. The presented technique is orthogonal to dictionary-based compression schemes and can be combined to achieve a further reduction in code size.

Variational Decompression of Image Data From DjVu Encoded Files.

The DjVu file format and image compression techniques are widely used in the archival of digital documents. Its key ingredients are the separation of the document into fore- and background layers and a binary switching mask, followed by a lossy, transform-based compression of the former and a dictionary-based compression of the latter. The lossy compression of the layers is based on a wavelet decomposition and bit truncation, which leads, in particular at higher compression rates, to severe compression artifacts in the standard decompression of the layers. The aim of this paper is to break ground for the variational decompression of DjVu files. To this aim, we provide an in-depth analysis and discussion of the compression standard with a particular focus on modeling data constraints for decompression. This allows to carry out DjVu decompression as regularized inversion of the compression procedure. As particular example, we evaluate the performance of such a framework using total variation and total generalized variation regularization. Furthermore, we provide routines for obtaining the necessary data constraints from a compressed DjVu file and for the forward and adjoint transformation operator involved in DjVu compression.

Adaptive Dictionary-based Compression of Protein Sequences

Adaptive Dictionary-based Compression of Protein Sequences

Spatial-temporal traffic flow pattern identification and anomaly detection with dictionary-based compression theory in a large-scale urban network

Traffic flow pattern identification, as well as anomaly detection, is an important component for traffic operations and control. To reveal the characteristics of regional traffic flow patterns in large road networks, this paper employs dictionary-based compression theory to identify the features of both spatial and temporal patterns by analyzing the multi-dimensional traffic-related data. An anomaly index is derived to quantify the network traffic in both spatial and temporal perspectives. Both pattern identifications are conducted in three different geographic levels: detector, intersection, and sub-region. From different geographic levels, this study finds several important features of traffic flow patterns, including the geographic distribution of traffic flow patterns, pattern shifts at different times-of-day, pattern fluctuations over different days, etc. Both spatial and temporal traffic flow patterns defined in this study can jointly characterize pattern changes and provide a good performance measure of traffic operations and management. The proposed method is further implemented in a case study for the impact of a newly constructed subway line. The before-and-after study identifies the major changes of surrounding road traffic near the subway stations. It is found that new metro stations attract more commute traffic in weekdays as well as entertaining traffic during weekends.

An efficient parallelization of longest prefix match and application on data compression

In this article, we describe a new approach to parallelize longest prefix match (LPM) algorithm through bit parallelism, also known as bit-vector approach. This approach makes use of bit-wise computations and leverages bit parallelism. The proposed parallel algorithm will be demonstrated in dictionary-based lossless data compression on general-purpose graphics processing units (GPGPUs). One of the main contributions of this work is redesigning the core part of the data compression algorithm and replacing it with the newly proposed bit-vector LPM solution. Using bit parallelism is a fundamentally new approach for data compression and promising in performance for hybrid CPU-GPU environments. The implementation of the new compression algorithm on GPUs improves the performance of the compression process compared to the previous attempts. Moreover, the bit-vector approach opens new opportunities for improvement and increases the applicability to popular heterogeneous environments.

Efficiency of adaptive fractal image compression with archetype classification and its modifications

The DCT-based JPEG image compression technique is a standard for lossy image compression. But the technique is not suitable at high compression ratios. Again, it is resolution dependent. Alternative compression techniques are introduced to reduce these problems. One such popular technique is fractal-based image compression. In this paper, efficiency of a fractal-based image compression technique is discussed that applies adaptive quadtree partitioning and archetype classification schemes. The adaptive quadtree partitioning scheme is dependent on image pixel context to share some self-similar structure of the image and improves the decoded image quality significantly. The archetype classification scheme tries to base the technique on finding better covering of range by domain that suites better in fractal image compression techniques. Then, modification of the same is also done to enhance the performance by applying dictionary-based loss-less data compression techniques, i.e. OLZW on the fractal compressed image that is basically a collection of affine transforms. The affine transform consists of a number of parameters that are compressed using OLZW technique. After that, the same experiment is carried out by a variant of OLZW, i.e. MOLZW. The experimental results are compared among existing and modified techniques. The modified versions yield not only better image quality but also high encoding speed than their counterparts.

Code Compression for Embedded Systems Using Separated Dictionaries

Engineers must consider performance, power consumption, and cost when designing embedded digital systems; furthermore, memory is a key factor in such systems. Code compression is a technique used in embedded systems to reduce the memory usage. BitMask-based code compression is a modified version of dictionary-based code compression. The basic purpose of BitMask is to record mismatched values and their positions to compress a greater number of instructions; it can be used exclusively or incorporated with the reference instructions to decode the codewords. In this paper, we applied a small separated dictionary, and variable mask numbers were used with the BitMask algorithm to reduce the codeword length of high-frequency instructions. In addition, a novel dictionary selection algorithm was proposed to increase the instruction match rates. The fully separated dictionary method was used to improve the performance of the decompression engine without affecting the compression ratio (CR) (the compressed code size divided by original code size). Based on the experimental results, the proposed method can achieve a 7.5% improvement in the CR with nearly no hardware overhead.

The greedy approach to dictionary-based static text compression on a distributed system

The greedy approach to dictionary-based static text compression can be executed by a finite-state machine. When it is applied in parallel to different blocks of data independently, there is no lack of robustness even on standard large scale distributed systems with input files of arbitrary size. Beyond standard large scale, a negative effect on the compression effectiveness is caused by the very small size of the data blocks. A robust approach for extreme distributed systems is presented in this paper, where this problem is fixed by overlapping adjacent blocks and preprocessing the neighborhoods of the boundaries.

Note on the greedy parsing optimality for dictionary-based text compression

Dynamic dictionary-based compression schemes are the most daily used data compression schemes since they appeared in the foundational paper of Ziv and Lempel in 1977, commonly referred to as LZ77. In dynamic setting, LZ77 considers a portion of the previous text as a dictionary and it uses a greedy approach to select, at each step, the longest match between the text and the dictionary. Compression is achieved by replacing matches with encoded dictionary pointers. LZ77 is the base of gzip, zip, rar, 7zip and many others compression software. All these compression schemes use variants of the greedy approach to parse (or factorise) the text into dictionary phrases. Greedy parsing optimality with respect to the number of phrases was proved by Storer et al. (1982) for unbounded LZ77-based dictionaries and by Cohn et al. (1996) for static suffix-closed dictionaries. The optimality of the greedy parsing was never proved for bounded size dictionary which is actually required by all of these practical schemes.In this article, we define the suffix-closed property for dynamic dictionaries, and we show that LZ77-based compression schemes, including the bounded dictionary variants, satisfy this property. Under this condition we prove the optimality of the greedy parsing as a variant of the proof by Cohn et al.

Research on Compression Storage of Massive Agricultural Data Based on Cloud Environment

With the development of information technology, agriculture data show large amount of data, distributed, heterogeneous characteristics. It is difficult to access and management with the massive data are continuously increasing which affect the large-scale use of agricultural information data. In this paper, the method of compression algorithm is proposed which based on real-time and time space correlation characteristics. All data is divided into several categories by Huffman compression algorithm combines parallel processing cloud platform. Then, the massive agricultural data is compressed and reducing the data storage. Exprienment result show that cloud storage platform with dynamic scalability. Under the same experimental data, the method of this paper has higher compress ratio, and compression consuming less when a larger amount data, compared with the Huffman compress and dictionary-based data compression algorithm.

IBM POWER7+ processor on-chip accelerators for cryptography and active memory expansion

With the heightened focus on computer security, IBM POWER® server workloads are spending an increasing number of cycles performing cryptographic functions. Active memory expansion (AME), a technology to dynamically increase the effective memory capacity of a system by compressing and decompressing memory pages, is also enjoying increasing deployment in POWER server systems. Together, cryptography and AME consume enough central processing unit (CPU) cycles in a typical installation to warrant adding dedicated hardware accelerators on the processor chip to offload the compute-intensive parts of these functions from the processor cores. IBM POWER7+™ is the first POWER server to include on-chip hardware accelerators for symmetric (shared key) and asymmetric (public key) cryptography and memory compression/decompression for AME. A true random number generator (RNG) is also integrated on-chip. This paper describes the hardware accelerator framework, including location relative to the cores and memory, accelerator invocation, data movement, and error handling. A description of each type of accelerator follows, including details of supported algorithms and the corresponding hardware data flows. Algorithms supported include the Advanced Encryption Standard, Secure Hash Algorithm, and Message Digest 5 algorithm as bulk cryptographic functions; asymmetric cryptographic functions in support of RSA and elliptic curve cryptography; and a novel dictionary-based compression algorithm with high throughput supporting AME. A presentation of accelerator performance is included.