Entropy Coding Research Articles

Low bit rate underwater video image compression and coding method based on wavelet decomposition

In view of the limited bandwidth of underwater video image transmission, a low bit rate underwater video compression coding method is proposed. Based on the preprocessing process of wavelet transform and coefficient down-sampling, the visual redundancy of underwater image is removed and the computational coefficients and coding bits are reduced. At the same time, combined with multi-level wavelet decomposition, inter frame motion compensation, entropy coding and other methods, according to the characteristics of different types of frame image data, reduce the number of calculations and improve the coding efficiency. The experimental results show that the reconstructed image quality can meet the visual requirements, and the average compression ratio of underwater video can meet the requirements of underwater acoustic channel transmission rate.

Multisymbol Architecture of the Entropy Coder for H.265/HEVC Video Encoders

In video compression, throughputs of entropy encoders based on arithmetic coding are limited. This article presents the architecture of the entropy coder able to process in each clock cycle much more binary symbols than previous works. The architecture takes advantage of the multisymbol implementation of the binary arithmetic coder (BAC) developed earlier. To balance high throughputs of the BAC, fast implementations of binarization, context modeling, and probability model (PM) update are developed. The main improvement in the symbol rate stems from the decomposition of the processing path into many parallel ones. Critical paths associated with state transitions are shortened since each path updates the PM only for one context selected in each clock cycle. The negative impact on the symbol rate is compensated by the context-based symbol reordering. Although paths have variable bin/symbol rates, the applied buffering strategy improves the continuity of two data streams directed to the BAC, separately for context-coded and bypass-mode symbols. The entropy coder synthesized for the 90-nm TSMC technology consumes 273-k gates and operates at 570 MHz. It achieves the average symbol rate of 13.08 bins per clock cycle and the throughput of 7455 Mbins/s for high-quality H.265/HEVC compression.

Large-Alphabet Semi-Static Entropy Coding Via Asymmetric Numeral Systems

An entropy coder takes as input a sequence of symbol identifiers over some specified alphabet and represents that sequence as a bitstring using as few bits as possible, typically assuming that the elements of the sequence are independent of each other. Previous entropy coding methods include the well-known Huffman and arithmetic approaches. Here we examine the newer asymmetric numeral systems (ANS) technique for entropy coding and develop mechanisms that allow it to be efficiently used when the size of the source alphabet is large—thousands or millions of symbols. In particular, we examine different ways in which probability distributions over large alphabets can be approximated and in doing so infer techniques that allow the ANS mechanism to be extended to support large-alphabet entropy coding. As well as providing a full description of ANS, we also present detailed experiments using several different types of input, including data streams arising as typical output from the modeling stages of text compression software, and compare theproposed ANS variants with Huffman and arithmetic coding baselines, measuring both compression effectiveness and also encoding and decoding throughput. We demonstrate that in applications in which semi-static compression is appropriate, ANS-based coders can provide an excellent balance between compression effectiveness and speed, even when the alphabet is large.

LUISA: Decoupling the Frequency Model From the Context Model in Prediction-Based Compression

AbstractPrediction-based compression methods, like prediction by partial matching, achieve a remarkable compression ratio, especially for texts written in natural language. However, they are not efficient in terms of speed. Part of the problem concerns the usage of dynamic entropy encoding, which is considerably slower than the static alternatives. In this paper, we propose a prediction-based compression method that decouples the context model from the frequency model. The separation allows static entropy encoding to be used without a significant overhead in the meta-data embedded in the compressed data. The result is a reasonably efficient algorithm that is particularly suited for small textual files, as the experiments show. We also show it is relatively easy to built strategies designed to handle specific cases, like the compression of files whose symbols are only locally frequent.

A Block-Based Arithmetic Entropy Encoding Scheme for Medical Images

The digitization of human body, especially for treatment of diseases can generate a large volume of data. This generated medical data has a large resolution and bit depth. In the field of medical diagnosis, lossless compression techniques are widely adopted for the efficient archiving and transmission of medical images. This article presents an efficient coding solution based on a predictive coding technique. The proposed technique consists of Resolution Independent Gradient Edge Predictor16 (RIGED16) and Block Based Arithmetic Encoding (BAAE). The objective of this technique is to find universal threshold values for prediction and provide an optimum block size for encoding. The validity of the proposed technique is tested on some real images as well as standard images. The simulation results of the proposed technique are compared with some well-known and existing compression techniques. It is revealed that proposed technique gives a higher coding efficiency rate compared to other techniques.

Designing a GPU-parallel algorithm for raw SAR data compression: A focus on parallel performance estimation

When a Synthetic Aperture Radar (SAR) acquires raw data using a satellite or airborne platform, it must be transferred to the ground for further processing. For example, SAR raw data need a so-called ’focusing’ signal processing to render it into a visible image. Such processing is time and computing consuming, and it is commonly carried out in computing centres. Since the data transfer rate is a typical limitation when communicating with the ground station, compression is necessary to reduce transmission time. So far, this procedure has been implemented in application-specific hardware, but recent adoption of avionic computational GPUs opened to new high-performance onboard perspectives. Due to the limited availability of avionic GPUs, we focused on parallel performance estimation starting from measures relative to a similar off-the-shelf solution. In this paper, we present a GPU algorithm for raw SAR data compression, which uses 1-dimensional DCT transforms, followed by quantisation and entropy coding. We evaluate results using ENVISAT (Environmental Satellite) ASAR Image Mode level 0 data by measuring compression rates, statistical parameters, and distortion on decompressed and then focused images. Moreover, by evaluating the Algorithmic Overhead induced by the parallelisation strategy, we predict the best thread-block configuration for possible adoption of such a GPU algorithm on one of the most available avionic hardware.

Ensemble Learning-Based Rate-Distortion Optimization for End-to-End Image Compression

End-to-end image compression using trained deep networks as encoding/decoding models has been developed substantially in the recent years. Previous work is limited in using a single encoding/decoding model, whereas we explore the usage of multiple encoding/decoding models as an ensemble. We propose several methods to obtain multiple models. First, we adopt the boosting strategy to train multiple networks with diversity as an ensemble. Second, we train an ensemble of multiple probability distribution models to reduce the distribution gap for efficient entropy coding. Third, we present a geometric transform-based self-ensemble method. The multiple models can be regarded as the multiple coding modes, similar to those in non-deep video coding schemes. We further adopt block-level model/mode selection at the encoder side to pursue rate-distortion optimization, where we use hierarchical block partitioning to improve the adaptation ability. Compared with single-model end-to-end compression, our proposed method improves the compression efficiency significantly, leading to 21% BD-rate reduction on the Kodak dataset, without increasing the decoding complexity. On the other hand, when keeping the same compression efficiency, our method can use much simplified decoding models, where the floating-point operations are reduced by 70%.

Superpixel based compression of hyperspectral image with modified dictionary and sparse representation

ABSTRACT Sparse representation provides an efficient way for the compression of hyperspectral images in the literature. In this work, an improved version of the Spectral-Spatial Adaptive Sparse Representation (SSASR), Modified SSASR (MSSASR), is proposed for hyperspectral image compression. In the first step of the proposed method, superpixel maps are generated for adaptive spatio-spectral representation. Then, the best possible dictionary is computed for the representation of the data. Afterwards, sparse coefficients are determined depending on the dictionary by Simultaneous Orthogonal Matching Pursuit (SOMP) method. In the final step, the dictionary and sparse coefficients are encoded by quantization and entropy encoding. This paper has the following novelties: modified dictionary learning step, new ordering scheme and Differential Pulse Code Modulation (DPCM) usage. Owing to modified dictionary learning, the sparse coefficients can be represented more compact than traditional SSASR. By using of new ordering scheme, it is not needed to send the superpixel map as side information. Moreover, DCPM usage lowers the magnitudes of sparse coefficients. Thanks to these modifications, the proposed method achieves an important improvement on compression performance. In the experimental results, the proposed method is compared with PCA+JPEG2000, DWT+JPEG2000, 3D-SPECK, 3D-TARP and SSASR methods on Indian Pines, Washington DC Mall, Jasper Ridge and Moffett Field scenes. The evaluation is carried out not only using distance and similarity metrics, namely, signal-to-noise ratio, mean spectral angle and mean spectral correlation metrics but also computation times. Additionally, reconstruction quality in anomaly regions is also used for comparison. Experimental results show that the proposed method outperforms the other compression methods in terms of quality metrics and anomaly preserving performance.

Distributed video coding based on classification of frequency bands with block texture conditioned key frame encoder for wireless capsule endoscopy

Wireless capsule endoscopy (WCE) has provided remarkable improvement in diagnosing gastrointestinal disorders by scanning the entire digestive tract. The system still need refinement, to upgrade the quality of images, frame rate and battery life. The principal component of the system that can address these issues is low complexity video compressor. Motivated by low computational complexity requirements of WCE video encoding, this paper presents a distributed video coding framework based on frequency bands classification. The lower frequency bands are used to generate good quality side information (SI) as they exhibit high temporal correlation. This reduces the complexity of hash generation at the encoder, thus eliminating the latency in SI creation. Apart from this, SI creation involves only a simple block search and doesn’t depend on Wyner–Ziv (WZ) bands. Also different approach for distributed coding of sub-sampled chroma components of WZ frame is proposed. Low complexity JPEG based key frame encoding is proposed that take advantage of WCE image textural properties to reduce the complexity of encoding smooth blocks by 81% at the quantization and encoding stage. Other novel features include use of discrete Tchebichef transform (DTT), Golomb–Rice code for entropy coding. Performance evaluation shows that the proposed method achieves 60% improvement in compression over Motion JPEG with low computational complexity.

Video Codec Using Flexible Block Partitioning and Advanced Prediction, Transform and Loop Filtering Technologies

This paper describes a joint response to the Call for Proposals by Samsung, Huawei, GoPro, and HiSilicon on Video Compression with Capability beyond HEVC/H.265, jointly issued by ITU-T SG16 Q.6 (VCEG) and ISO/IEC JTC1/SC29/WG11 (MPEG). In the proposed codec, the coding framework supports hierarchical splitting with binary and ternary trees and flexible coding order representations. Additionally, novel compression tools on inter/intra prediction, in-loop filtering, and entropy coding have been proposed. The proposed compression scheme provides significantly higher compression capability than the state-of-the-art HEVC/H.265 standard for SDR (Standard Dynamic Range) category while maintaining complexity acceptable for emerging applications. When all the proposed algorithmic tools are used, the proposed video codec achieves approximately 40% bit-saving for the SDR cetegory on average compared to HEVC/H.265 anchor.

Unsupervised Idealization of Nano-Electronic Sensors Recordings with Concept Drifts: An Information Theory Approach for Non-Stationary Single-Molecule Data Analysis

Unsupervised Idealization of Nano-Electronic Sensors Recordings with Concept Drifts: An Information Theory Approach for Non-Stationary Single-Molecule Data Analysis

Toverline{T} and EE, with implications for (A)dS subregion encodings

We initiate a study of subregion dualities, entropy, and redundant encoding of bulk points in holographic theories deformed by Toverline{T} and its generalizations. This includes both cut off versions of Anti de Sitter spacetime, as well as the generalization to bulk de Sitter spacetime, for which we introduce two additional examples capturing different patches of the bulk and incorporating the second branch of the square root dressed energy formula. We provide new calculations of entanglement entropy (EE) for more general divisions of the system than the symmetric ones previously available. We find precise agreement between the gravity side and deformed-CFT side results to all orders in the deformation parameter at large central charge. An analysis of the fate of strong subadditivity for relatively boosted regions indicates nonlocality reminiscent of string theory. We introduce the structure of operator algebras in these systems. The causal and entanglement wedges generalize to appropriate deformed theories but exhibit qualitatively new behaviors, e.g. the causal wedge may exceed the entanglement wedge. This leads to subtleties which we express in terms of the Hamiltonian and modular Hamiltonian evolution. Finally, we exhibit redundant encoding of bulk points, including the cosmological case.

Efficient lossless compression scheme for multi-channel ECG signal processing

Electrocardiogram (ECG) represents the recording of the heart’s electrical activity and is used to diagnose heart disease nowadays. The diagnosis requires huge time consumption for acquiring enough multi-channel data. The storage and transmission of 12 lead ECG data results in massive cost. In this work, we propose a multi-channel ECG lossless compression which uses the adaptive linear prediction for intra and inter channel decorrelation. We also use the adaptive Golomb-Rice codec for entropy coding. The proposed technique for adaptive linear prediction and Golomb-Rice codec is based on the performance of passed samples. Thus, the coefficient of linear prediction and Golomb-Rice codec will make self-adjustments during the process. We evaluate the proposed algorithm with MIT-BIH Arrhythmia database for single-channel compression, and Physikalisch-Technische Bundesanstalt database (PTB) for multi-channel compression. The overall compression scheme is also implemented in embedded system with an ARM Cortex-M4 processor for real-time demonstration.

Medical Image Compression Techniques with Wavelet Discrete Transformation and Entropy Encoding

Medical imaging is a presentment of human organ parts. Medical imaging is saved on a film; therefore, it needs a big saving quota. Compressing is a process to remove redundancy from a piece of information without reducing its quality. This study recommended compressed medical image with DWT (Discrete Wavelet Transform) with adaptive threshold added and entropy copying with the Run Length Encoding (RLE) coding. This study is comparing several parameters, such as compressed ratio and compressed image file size, and PSNR (Peak Signal to Noise Ratio) for analyzing the quality of reconstructive image. The study showed that the comparison of rate, compressed ratio, and PSNR tracing of Haar and Daubechies doesn’t have a significant difference. Comparison of rate, compressed ratio, and PSNR tracing on the hard and soft threshold is the rate of the sold threshold is lower than the hard threshold. The optimal outcome of this study is to use a soft threshold.

Fractal Image Compression Using Canonical Huffman Coding

Image Compression techniques have become a very important subject with the rapid growth of multimedia application. The main motivations behind the image compression are for the efficient and lossless transmission as well as for storage of digital data. Image Compression techniques are of two types; Lossless and Lossy compression techniques. Lossy compression techniques are applied for the natural images as minor loss of the data are acceptable. Entropy encoding is the lossless compression scheme that is independent with particular features of the media as it has its own unique codes and symbols. Huffman coding is an entropy coding approach for efficient transmission of data. This paper highlights the fractal image compression method based on the fractal features and searching and finding the best replacement blocks for the original image. Canonical Huffman coding which provides good fractal compression than arithmetic coding is used in this paper. The result obtained depicts that Canonical Huffman coding based fractal compression technique increases the speed of the compression and has better PNSR as well as better compression ratio than standard Huffman coding.

High efficiency coding for surveillance videos based on selective based fast intra coding

HEVC, the successor of popular H.264 video compression standard targets the new era of ultra high-resolution videos with higher frame rates. Lossless coding mode of high-efficiency video coding (HEVC) relies on efficient intra prediction methods for achieving higher compression. In this paper, we propose a method called selective based intra prediction (SBIP) which is a combination of Block based intra prediction (BP) and Boundary region based SAP (BRSAP) for surveillance videos. First, we calculate the threshold complexity for the block and based on the condition the CU early termination is done. Then, we calculate a prediction for each block by using the BP and BRSAP method separately. After calculating both the prediction based on some condition the final prediction of the block is performed which selects the best prediction for each pixel to reduce the residual energy thereby increasing the coding efficiency. At last, the transform, quantization, and entropy coding are performed to compress the input video signal. The experimental results are conducted for surveillance videos test sequences for 30 frames from each test sequence. The results are evaluated based on encoding time and peak signal to noise ratio (PSNR).

Low-Complexity Rate-Distortion Optimization of Sampling Rate and Bit-Depth for Compressed Sensing of Images.

Compressed sensing (CS) offers a framework for image acquisition, which has excellent potential in image sampling and compression applications due to the sub-Nyquist sampling rate and low complexity. In engineering practices, the resulting CS samples are quantized by finite bits for transmission. In circumstances where the bit budget for image transmission is constrained, knowing how to choose the sampling rate and the number of bits per measurement (bit-depth) is essential for the quality of CS reconstruction. In this paper, we first present a bit-rate model that considers the compression performance of CS, quantification, and entropy coder. The bit-rate model reveals the relationship between bit rate, sampling rate, and bit-depth. Then, we propose a relative peak signal-to-noise ratio (PSNR) model for evaluating distortion, which reveals the relationship between relative PSNR, sampling rate, and bit-depth. Finally, the optimal sampling rate and bit-depth are determined based on the rate-distortion (RD) criteria with the bit-rate model and the relative PSNR model. The experimental results show that the actual bit rate obtained by the optimized sampling rate and bit-depth is very close to the target bit rate. Compared with the traditional CS coding method with a fixed sampling rate, the proposed method provides better rate-distortion performance, and the additional calculation amount amounts to less than 1%.

SHyLoC 2.0: A Versatile Hardware Solution for On-Board Data and Hyperspectral Image Compression on Future Space Missions

In this paper, we present the design, implementation and results of a set of IP cores that perform on-board hyperspectral image compression according to the CCSDS 123.0-B-1 lossless standard, specifically designed to be suited for on-board systems and for any kind of hyperspectral sensor. As entropy coder, the sample-adaptive entropy coder defined in the 123.0-B-1 standard or the low-complexity block-adaptive encoder defined by the CCSDS 121.0-B-2 lossless standard could be used. Both IPs, 123.0-B-1 and 121.0-B-2, are part of SHyLoC 2.0, and can be used together for compression of hyperspectral images, being also possible the compression of any kind of data using only the 121-IP. SHyLoC 2.0 improves and extends the capabilities of SHyLoC 1.0, currently available at the ESA IP Cores library, increasing its compression efficiency and throughput, without compromising the resources footprint. Moreover, it incorporates new features, such as the unit-delay predictor option defined by the CCSDS 121.0-B-2 standard, and burst capabilities in the external memory interface of the CCSDS 123-IP, among others. Dedicated architectures have been designed for all the possible input image sample arrangements, in order to maximise throughput and reduce the hardware resources utilization. The design is technology-agnostic, enabling the mapping of the VHDL code in different FPGAs or ASICs. Results are presented for a representative group of well-known space-qualified FPGAs, including the new NanoXplore BRAVE family. A maximum throughput of 150 MSamples/s is obtained for Xilinx Virtex XQR5VFX130 when the SHyLoC 2.0 CCSDS-123 IP is configured in Band-Interleaved by Pixel (BIP) order, using only the 4% of LUTs and less than the 1% of internal memory.

Learning Sparse Low-Precision Neural Networks With Learnable Regularization

We consider learning deep neural networks (DNNs) that consist of low-precision weights and activations for efficient inference of fixed-point operations. In training low-precision networks, gradient descent in the backward pass is performed with high-precision weights while quantized low-precision weights and activations are used in the forward pass to calculate the loss function for training. Thus, the gradient descent becomes suboptimal, and accuracy loss follows. In order to reduce the mismatch in the forward and backward passes, we utilize mean squared quantization error (MSQE) regularization. In particular, we propose using a learnable regularization coefficient with the MSQE regularizer to reinforce the convergence of high-precision weights to their quantized values. We also investigate how partial L2 regularization can be employed for weight pruning in a similar manner. Finally, combining weight pruning, quantization, and entropy coding, we establish a low-precision DNN compression pipeline. In our experiments, the proposed method yields low-precision MobileNet and ShuffleNet models on ImageNet classification with the state-of-the-art compression ratios of 7.13 and 6.79, respectively. Moreover, we examine our method for image super resolution networks to produce 8-bit low-precision models at negligible performance loss.

Precise Estimation of Probabilities in CABAC Using the Cauchy Optimization Method

The algorithm of entropy coding that is now widely used in video compression is the Context-based Adaptive Binary Arithmetic Coding (CABAC). This paper presents a modified, improved version of CABAC, called the CABAC+. The author's idea for the improvement is to use in CABAC a more accurate estimation of probabilities of data symbols. The basis of the proposal is to use the Cauchy optimization method, in order to minimize the number of bits that are produced by the entropy encoder. The application of the CABAC improvement in the High Efficiency Video Coding (HEVC) technology increased the compression efficiency of entropy coding by 0.6% to 1.2%, depending on the parameters of the method and scenario of experiments. The use of the proposed solution increases the decoding time of a video, but the method virtually does not change the complexity of a video encoder.