Entropy Coding Research Articles

Universal Golomb–Rice Coding Parameter Estimation Using Deep Belief Networks for Hyperspectral Image Compression

For efficient compression of hyperspectral images, we propose a universal Golomb–Rice coding parameter estimation method using deep belief network, which does not rely on any assumption on the distribution of the input data. We formulate the problem of selecting the best coding parameter for a given input sequence as a supervised pattern classification problem. Simulations on the synthesized data and five hyperspectral image datasets show that we can achieve significantly more accurate estimation of the coding parameters, which can translate to slightly higher compression than three state-of-the-art methods. More extensive simulations on additional images from the 2006 AVIRIS datasets show that the proposed method achieved overall compression bitrates comparable with other estimation methods, as well as the sample-adaptive entropy coder employed by the Consultative Committee for Space Data Systems standard for multispectral and hyperspectral data compression. Regarding computational feasibility, we show how to use transferable deep belief networks to speed up training by about five times. We also show that inferring the best coding parameters using a trained deep belief network offers computational advantages over the brute-force search method. As an extension, we propose a novel side-information free codec, where the intersequence correlations can be learned by a differently trained network based on the current sequence to predict reasonably good parameters for coding the next sequence. As another extension, we introduce a variable feature combination architecture, where problem specific heuristics such as the sample means can be incorporated to further improve the estimation accuracy.

Comparison of Effects of Entropy Coding Schemes Cascaded with Set Partitioning in Hierarchical Trees

WT (Wavelet Transform) is considered as landmark for image compression because it represents a signal in terms of functions which are localized both in frequency and time domain. Wavelet sub-band coding exploits the self-similarity of pixels in images and arranges resulting coefficients in different sub-bands. A much simpler and fully embedded codec algorithm SPIHT (Set Partitioning in Hierarchical Trees) is widely used for the compression of wavelet transformed images. It encodes the transformed coefficients depending upon their significance comparative to the given threshold. Statistical analysis reveals that the output bit-stream of SPIHT comprises of long trail of zeroes that can be further compressed, therefore SPIHT is not advocated to be used as sole mean of compression. In this paper, wavelet transformed images have been initially compressed by using SPIHT technique and to attain more compression, the output bit streams of SPIHT are then fed to entropy encoders; Huffman and Arithmetic encoders, for further de-correlation. The comparison of two concatenations has been carried out by evaluating few factors like Bit Saving Capability, PSNR (Peak Signal to Noise Ratio), Compression Ratio and Elapsed Time. The experimental results of these cascading demonstrate that SPIHT combined with Arithmetic coding yields better compression ratio as compared to SPIHT cascaded with Huffman coding. Whereas, SPIHT once combined with Huffman coding is proved to be comparatively efficient.

Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources

We deal with zero-delay source coding of a vector-valued Gauss-Markov source subject to a mean-squared error (MSE) fidelity criterion characterized by the operational zero-delay vector-valued Gaussian rate distortion function (RDF). We address this problem by considering the nonanticipative RDF (NRDF) which is a lower bound to the causal optimal performance theoretically attainable (OPTA) function and operational zero-delay RDF. We recall the realization that corresponds to the optimal "test-channel" of the Gaussian NRDF, when considering a vector Gauss-Markov source subject to a MSE distortion in the finite time horizon. Then, we introduce sufficient conditions to show existence of solution for this problem in the infinite time horizon. For the asymptotic regime, we use the asymptotic characterization of the Gaussian NRDF to provide a new equivalent realization scheme with feedback which is characterized by a resource allocation (reverse-waterfilling) problem across the dimension of the vector source. We leverage the new realization to derive a predictive coding scheme via lattice quantization with subtractive dither and joint memoryless entropy coding. This coding scheme offers an upper bound to the operational zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then for "r" active dimensions of the vector Gauss-Markov source the gap between the obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1 bits/vector. We further show that it is possible when we use vector quantization, and assume infinite dimensional Gauss-Markov sources to make the previous gap to be negligible, i.e., Gaussian NRDF approximates the operational zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian sources of any finite memory under mild conditions. Our theoretical framework is demonstrated with illustrative numerical experiments.

A computationally scalable fast intra coding scheme for HEVC video encoder

The High Efficiency Video Coding (HEVC) adopts 35 intra prediction modes to provide a more precise intra prediction. As a result, HEVC intra coding is much more complex. To provide a flexible solution for battery-powered devices, which include varying power budgets based on battery level, this paper jointly employs statistical distribution of different intra modes, and texture analysis. While the statistical information specifies the most efficient modes for extremely low power budgets, for moderate to high power budgets the best angular modes are predicted by analyzing the residual of the Planar mode, which is demonstrated to represent the dominant edges of the block. Moreover, a scheme is presented that further reduces the complexity by predicting the more important DC and Planar modes using the information provided by the entropy coder. The proposed method imposes almost no computational overhead since it utilizes the existing internal operators of HEVC. Ultimately, an intra prediction scheme with seven predefined levels of complexity is presented that provides coding for various power budgets and devices with various capabilities. The highest level of this scheme provides ~23% total encoding time reduction compared to HM, in all-intra configuration, with only 0.78% penalty on BD-Rate for high end devices. While the lowest level makes encoding possible for extremely low power devices and low battery plans with ~52% time reduction (almost the entire share of intra coding) at the cost of 9.6% BD-Rate, which is shown to be nearly minimal for this level of processing power.

A Low Area Fully Pipelined Implementation of JPEG on FPGA

This paper presents a low-area and high-throughput design and implementation of JPEG encoder on FPGA. The design consists of three main components: (1) 2-D DCT module, based on the row-column decomposition technique, (2) Quantization in zigzag ordering, based on look-up tables, and (3) Entropy coder, transforming the quantized DCT coefficients into JPEG words. All components are fully pipelined and optimized for FPGA resource utilization. The proposed implementation of JPEG encoder is able to encode 143 and 71 SDTV frames per second with 720x480 gray scale and color pixels per frame, respectively, on Xilinx Spartan 6 FPGA. Moreover, the proposed architecture is capable of encoding at least 53 and 26 HD Ready TV frames per second with 1280x720 gray scale and color pixels per frame, respectively, on this FPGA chip. Thus, the proposed JPEG encoder architecture is well-suited to various image and video compressing applications where performance and area are significantly important.

Gaussian Approximation-Based Lossless Compression of Smart Meter Readings

Automation metering services, load forecasting, and energy feedback are among the great benefits of smart meters. These meters are usually connected using Narrowband power line communication to transmit the collected waveform readings. The huge volume of these streams, the limited-bandwidth, energy, and required storage space pose a unique management challenge. Compression of these streams has a significant opportunity to solve these issues. Therefore, this paper proposes a new lossless smart meter readings compression algorithm. The uniqueness is in representing smart meter streams using few parameters. This is effectively achieved using Gaussian approximation based on dynamic-nonlinear learning technique. The margin space between the approximated and the actual readings is measured. The significance is that the compression will be only for margin space limited points rather than the entire stream of readings. The margin space values are then encoded using burrow-wheeler transform followed by move-to-front and run-length to eliminate the redundancy. Entropy encoding is finally applied. Both mathematical and empirical experiments have been thoroughly conducted to prove the significant enhancement of the entropy (i.e., almost reduced by half) and the resultant compression ratio (i.e., 3.8:1) which is higher than any known lossless algorithm in this domain.

Algorithm optimization and hardware implementation for Merge mode in HEVC

Merge mode is a new tool for improving inter-frame coding efficiency in high-efficiency video coding. This tool can save the bitrate for the motion vector by sharing this vector with neighboring blocks. Merge is a process that selects a candidate motion vector by calculating the cost of rate-distortion. However, this process requires a large number of complex computations and memory access, thereby resulting in the low efficiency of hardware implementation. This paper proposes a new Merge candidate decision scheme that determines the most favorable Merge candidate from a full list of candidates by comparing the sum of absolute transformed difference with the weighted header bit instead of performing a complex calculation for sum of squared difference and entropy coding process in HM16.7. The simulation results show that the performance of the proposed algorithm is close to that of HM16.7 and increases the BD-rate only by 0.22–1.21%. The multilevel pipelines architecture is also exploited in the hardware design. The weighted header bit operation is performed by using the look-up table, which reduces both the complexity and encoding clock cycle. The designed system is implemented with a register transfer level code. The synthesis results from the Design Compiler show that compared with other architecture, the proposed architecture offers great advantages in resource utilization and can process 1920 × 1080 at 353 frame/s for P-slices with a clock frequency of 1057 MHz and logic gate count of 285.2 K.

VLSI Implementation of an Efficient Lossless EEG Compression Design for Wireless Body Area Network

Data transmission of electroencephalography (EEG) signals over Wireless Body Area Network (WBAN) is currently a widely used system that comes together with challenges in terms of efficiency and effectivity. In this study, an effective Very-Large-Scale Integration (VLSI) circuit design of lossless EEG compression circuit is proposed to increase both efficiency and effectivity of EEG signal transmission over WBAN. The proposed design was realized based on a novel lossless compression algorithm which consists of an adaptive fuzzy predictor, a voting-based scheme and a tri-stage entropy encoder. The tri-stage entropy encoder is composed of a two-stage Huffman and Golomb-Rice encoders with static coding table using basic comparator and multiplexer components. A pipelining technique was incorporated to enhance the performance of the proposed design. The proposed design was fabricated using a 0.18 μm CMOS technology containing 8405 gates with 2.58 mW simulated power consumption under an operating condition of 100 MHz clock speed. The CHB-MIT Scalp EEG Database was used to test the performance of the proposed technique in terms of compression rate which yielded an average value of 2.35 for 23 channels. Compared with previously proposed hardware-oriented lossless EEG compression designs, this work provided a 14.6% increase in compression rate with a 37.3% reduction in hardware cost while maintaining a low system complexity.

Properties and Design of Variable-to-Variable Length Codes

For the entropy coding of independent and identically distributed (i.i.d.) binary sources, variable-to-variable length (V2V) codes are an interesting alternative to arithmetic coding. Such a V2V code translates variable length words of the source into variable length code words by employing two prefix-free codes. In this article, several properties of V2V codes are studied, and new concepts are developed. In particular, it is shown that the redundancy of a V2V code cannot be zero for a binary i.i.d. source {X} with 0 < p X (1) < 0.5. Furthermore, the concept of prime and composite V2V codes is proposed, and it is shown why composite V2V codes can be disregarded in the search for particular classes of minimum redundancy codes. Moreover, a canonical representation for V2V codes is proposed, which identifies V2V codes that have the same average code length function. It is shown how these concepts can be employed to greatly reduce the complexity of a search for minimum redundancy (size-limited) V2V codes.

Bilateral adaptive quantization in HEVC

In this paper, we proposed an adaptive quantization algorithm for High Efficiency Video Coding (HEVC) to boost the encoding performance. The transform coefficients in a Transform Unit (TU) inherit the energy concentration property. However, they are equally quantized before entropy coding. With equal quantization technique, the energy distribution of transform coefficients and the scanning pattern following the quantization stage is not properly considered. In order to quantize the coefficients adaptively, we proposed an improved algorithm to quantize the coefficients. For each coefficient, both the magnitude and its ordinal number scanned in entropy coding process are taken into account. The quantization parameter of each coefficient in a TU is adaptively calculated by the bilateral factors accordingly. We tested our method on the latest HM16.0. An average performance of -0.27% on BD-Rate and 7.07% computing time saving are achieved in the case of the commonly used Low Delay P configuration, which demonstrated the effectiveness of the proposed algorithm.

A two-stage rice-based lossless compression method for telemetry data

The article presents results of a study of a proposed lossless compression method for telemetry data. A two-stage scheme of lossless compression, consisting of decorrelation and entropy coding, is presented. In experiments, a bitwise XOR operation is implemented as a decorrelator and rice coding method as entropy coder. A comparison is performed between two compression methods: one is based only on rice coding method, while the other is based on the proposed method. The results are based on estimates of the gain in variance and entropy of the output signal from the decorrelator. In experiments, parameters of telemetry information of automatic control systems are studied, such as temperature, pressure and positioning data. Streams of telemetry frames, with different frame structures: one and two level of commutation, in IRIG-106 standard format, are formed and tested. Based on experimental results, conclusions are done for lossless telemetry data compression.

S-EMG Signal Compression in One-Dimensional and Two-Dimensional Approaches.

This paper presents algorithms designed for one-dimensional (1-D) and 2-D surface electromyographic (S-EMG) signal compression. The 1-D approach is a wavelet transform based encoder applied to isometric and dynamic S-EMG signals. An adaptive estimation of the spectral shape is used to carry out dynamic bit allocation for vector quantization of transformed coefficients. Thus, an entropy coding is applied to minimize redundancy in quantized coefficient vector and to pack the data. In the 2-D approach algorithm, the isometric or dynamic S-EMG signal is properly segmented and arranged to build a 2-D representation. The high efficient video codec is used to encode the signal, using 16-bit-depth precision, all possible coding/prediction unit sizes, and all intra-coding modes. The encoders are evaluated with objective metrics, and a real signal data bank is used. Furthermore, performance comparisons are also shown in this paper, where the proposed methods have outperformed other efficient encoders reported in the literature.

Improving Efficiency of Entropy Coding Method in Modern Video Compression Standards

Modern video coding standards have high coding efficiency, but the encoding performance has to be improved to meet the growing multimedia applications. The paper deals with the entropy encoding methods and algorithms in video coding standard H.264/AVC and H.265/HEVC. Context-based Adaptive Variable Length Coding (CAVLC) for the H.264/AVC standard was originally designed for lossy video coding, and as such does not yield adequate performance for lossless video coding. Context-Adaptive Binary Arithmetic Coding (CABAC) is a method of entropy coding first introduced in H.264/AVC and now used in the standard H.265/HEVC. While it provides high coding efficiency, the data dependencies in H.264/AVC CABAC make it challenging to parallelize and thus, limit its throughput. Accordingly, during the standardization of entropy coding for HEVC, both coding efficiency and throughput were considered. Based on an analysis of their advantages and disadvantages, a method called the entropy coding algorithm using the enumerative coding of the hierarchical approach is proposed. The proposed algorithm consists of the Context-Adaptive Binary Arithmetic Coding algorithm and the enumerative coding algorithm with a hierarchical approach. The proposed algorithm is tested in the Visual C ++ development environment on various test video sequences. The results of the experiments showed a greater efficiency of coding of multimedia data (the proposed one reduces on average up to 15% of the storage volume compared to the traditional CABAC method), while the method requires a longer coding time (approximately twice). The proposed method can be recommended for use in telecommunication systems for storage, transmission and processing of multimedia data, where a high degree of compression is required first.

Energy efficient surveillance system using WVSN with reweighted sampling in modified fast Haar wavelet transform domain

Wireless visual sensor network (WVSN) consists of a large number of nodes that are capable of acquiring, compressing and transmitting images. Surveillance becomes a vital application area of WVSN as they can be deployed in various environments to monitor and collect information. The lifetime of the nodes in the network depends on the energy consumption. Hence in this paper, block compressed sensing (BCS) based image transmission technique that utilizes Energy based Reweighted sampling (ERWS) in Modified Fast Haar Wavelet Transform (MFHWT) domain is proposed to reduce energy consumption considerably. Sparse binary random matrix is used to acquire CS measurements in MFHWT domain. In addition, the proposed technique also maintains the image quality. The developed algorithm is applied and tested for a car parking lot monitoring system. It is evident from the simulation results that the proposed method achieves better PSNR values even for fewer measurements. Experimental analysis is performed using Atmega 128 processor of Mica mote in WinAVR by Atmel. The proposed method has approximately 85.5% lesser energy consumption than other Compressed Sensing (CS) methods. Lossless Entropy Coding is applied to the ERWS measurements and considerable reduction in number of transmitted bits is also achieved. The algorithm has also been tested in WINGZ mote in real time.

Edge-Based and Prediction-Based Transformations for Lossless Image Compression

Pixelated images are used to transmit data between computing devices that have cameras and screens. Significant compression of pixelated images has been achieved by an “edge-based transformation and entropy coding” (ETEC) algorithm recently proposed by the authors of this paper. The study of ETEC is extended in this paper with a comprehensive performance evaluation. Furthermore, a novel algorithm termed “prediction-based transformation and entropy coding” (PTEC) is proposed in this paper for pixelated images. In the first stage of the PTEC method, the image is divided hierarchically to predict the current pixel using neighboring pixels. In the second stage, the prediction errors are used to form two matrices, where one matrix contains the absolute error value and the other contains the polarity of the prediction error. Finally, entropy coding is applied to the generated matrices. This paper also compares the novel ETEC and PTEC schemes with the existing lossless compression techniques: “joint photographic experts group lossless” (JPEG-LS), “set partitioning in hierarchical trees” (SPIHT) and “differential pulse code modulation” (DPCM). Our results show that, for pixelated images, the new ETEC and PTEC algorithms provide better compression than other schemes. Results also show that PTEC has a lower compression ratio but better computation time than ETEC. Furthermore, when both compression ratio and computation time are taken into consideration, PTEC is more suitable than ETEC for compressing pixelated as well as non-pixelated images.

Performance optimization of dual stage algorithm for lossless data compression and decompression

In this paper, an optimized dual stage architecture is proposed which is the combination of Lempel-Ziv-Welch (LZW) Algorithm at the first phase and Arithmetic Coding being the later part of Architecture. LZW Algorithm is a lossless compression algorithm and code here for each character is available in the dictionary which reduces 5-bits per cycle as compared to ASCII. In arithmetic coding the numbers are represented by an interval of real numbers from zero to one according to their probabilities. It is an entropy coding and is lossless in nature. The text information is allowed to pass through the proposed architecture and it gets compressed to the higher rate.

Fractal Image Compression Based on High Entropy Values Technique

There are many attempts tried to improve the encoding stage of FIC because it consumed time. These attempts worked by reducing size of the search pool for pair range-domain matching but most of them led to get a bad quality, or a lower compression ratio of reconstructed image. This paper aims to present a method to improve performance of the full search algorithm by combining FIC (lossy compression) and another lossless technique (in this case entropy coding is used). The entropy technique will reduce size of the domain pool (i. e., number of domain blocks) based on the entropy value of each range block and domain block and then comparing the results of full search algorithm and proposed algorithm based on entropy technique to see each of which give best results (such as reduced the encoding time with acceptable values in both compression quali-ty parameters which are C. R (Compression Ratio) and PSNR (Image Quality). The experimental results of the proposed algorithm proven that using the proposed entropy technique reduces the encoding time while keeping compression rates and reconstruction image quality good as soon as possible.

Comparison of Entropy Coding mechanism on IEEE1857.2 Lossless Audio Compression Standard

<p>This paper has two objectives. First, we aim to review and analyze the performance of the IEEE 1857.2 standard, focusing is on the Golomb-Rice and Arithmetic entropy algorithms as well as the effect of the pre-processing block on these entropy blocks. The pre-processing block normalizes the error residue of the Linear Predictive encoder, which then is passed to Entropy block, where the selector chooses the entropy encoder to use. The second objective is to present results from experimenting different existing algorithms available to benchmark it’. The results are discussed, and comparisons are made to identify the effect on compression ratio and encoding speed of the lossless encoder. As well as this, comparison is made to analyze effects of enabling and disabling the pre-processing output to the Entropy Coding block. We concluded that pre-processing block works well to flatten the output at lower predictor order for all the sound types, but works best at improving the residual output for music sound type.</p>

Dynamic Power Reduction in Scalable Neural Recording Interface Using Spatiotemporal Correlation and Temporal Sparsity of Neural Signals

We report a scalable neural recording interface with embedded lossless compression to reduce dynamic power consumption ( $\text{P}_{D}$ ) for data transmission in high-density neural recording systems. We investigated the characteristics of neural signals and implemented effective lossless compression for local field potential (LFP) and extracellular action potential (EAP or spike) in separate signal paths. For LFP, spatial–temporal (spatiotemporal) correlation of the LFP signals is exploited in a $\Delta $ -modulated $\Delta \Sigma $ analog-to-digital converter ( $\Delta -\Delta \Sigma $ ADC) and a dedicated digital difference circuit. Then, statistical redundancy is further eliminated through entropy encoding without information loss. For spikes, only essential parts of waveforms in the spikes are extracted from the raw data by using spike detectors and reconfigurable analog memories. The prototype chip was fabricated using 180-nm CMOS processes, incorporating 128 channels into a modular architecture that is easily scalable and expandable for high-density neural recordings. The fabricated chip achieved the data rate reduction for the LFPs and spikes by a factor of 5.35 and 10.54, respectively, from the proposed compression scheme. Consequently, $P_{D}$ was reduced by 89%, when compared to the uncompressed case. We also achieved the state-of-the-art recording performance of 3.37 $\mu \text{W}$ per channel, 5.18 $\mu V_{\mathrm {rms}}$ noise, and 3.41 ${\text {NEF}}^{2}V_{\mathrm {DD}}$ .

Directional Transforms for Video Coding Based on Lifting on Graphs

In this paper, we describe and optimize a general scheme based on lifting transforms on graphs for video coding. A graph is constructed to represent the video signal. Each pixel becomes a node in the graph and links between nodes represent similarity between them. Therefore, spatial neighbors and temporal motion-related pixels can be linked, while nonsimilar pixels (e.g., pixels across an edge) may not be. Then, a lifting-based transform, in which filtering operations are performed using linked nodes, is applied to this graph, leading to a 3D (spatio-temporal) directional transform, which can be viewed as an extension of wavelet transforms for video. The design of the proposed scheme requires four main steps: 1) graph construction; 2) graph splitting; 3) filter design; and 4) extension of the transform to different levels of decomposition. We focus on the optimization of these steps in order to obtain an effective transform for video coding. Furthermore, based on this scheme, we propose a coefficient reordering method and an entropy coder leading to a complete video encoder that achieves better coding performance than a motion-compensated temporal filtering wavelet-based encoder, and a simple encoder derived from H.264/AVC that makes use of similar tools as our proposed encoder (reference software JM15.1 configured to use one reference frame, no subpixel motion estimation, and $16 \times 16$ inter and $4 \times 4$ intra modes).