Rate-distortion Tradeoff Research Articles

Efficient Computation of the Quantum Rate-Distortion Function

The quantum rate-distortion function plays a fundamental role in quantum information theory, however there is currently no practical algorithm which can efficiently compute this function to high accuracy for moderate channel dimensions. In this paper, we show how symmetry reduction can significantly simplify common instances of the entanglement-assisted quantum rate-distortion problems. This allows us to better understand the properties of the quantum channels which obtain the optimal rate-distortion trade-off, while also allowing for more efficient computation of the quantum rate-distortion function regardless of the numerical algorithm being used. Additionally, we propose an inexact variant of the mirror descent algorithm to compute the quantum rate-distortion function with provable sublinear convergence rates. We show how this mirror descent algorithm is related to Blahut-Arimoto and expectation-maximization methods previously used to solve similar problems in information theory. Using these techniques, we present the first numerical experiments to compute a multi-qubit quantum rate-distortion function, and show that our proposed algorithm solves faster and to higher accuracy when compared to existing methods.

Offline and Online Optical Flow Enhancement for Deep Video Compression

Video compression relies heavily on exploiting the temporal redundancy between video frames, which is usually achieved by estimating and using the motion information. The motion information is represented as optical flows in most of the existing deep video compression networks. Indeed, these networks often adopt pre-trained optical flow estimation networks for motion estimation. The optical flows, however, may be less suitable for video compression due to the following two factors. First, the optical flow estimation networks were trained to perform inter-frame prediction as accurately as possible, but the optical flows themselves may cost too many bits to encode. Second, the optical flow estimation networks were trained on synthetic data, and may not generalize well enough to real-world videos. We address the twofold limitations by enhancing the optical flows in two stages: offline and online. In the offline stage, we fine-tune a trained optical flow estimation network with the motion information provided by a traditional (non-deep) video compression scheme, e.g. H.266/VVC, as we believe the motion information of H.266/VVC achieves a better rate-distortion trade-off. In the online stage, we further optimize the latent features of the optical flows with a gradient descent-based algorithm for the video to be compressed, so as to enhance the adaptivity of the optical flows. We conduct experiments on two state-of-the-art deep video compression schemes, DCVC and DCVC-DC. Experimental results demonstrate that the proposed offline and online enhancement together achieves on average 13.4% bitrate saving for DCVC and 4.1% bitrate saving for DCVC-DC on the tested videos, without increasing the model or computational complexity of the decoder side.

On the Robustness of Neural-Enhanced Video Streaming against Adversarial Attacks

The explosive growth of video traffic on today's Internet promotes the rise of Neural-enhanced Video Streaming (NeVS), which effectively improves the rate-distortion trade-off by employing a cheap neural super-resolution model for quality enhancement on the receiver side. Missing by existing work, we reveal that the NeVS pipeline may suffer from a practical threat, where the crucial codec component (i.e., encoder for compression and decoder for restoration) can trigger adversarial attacks in a man-in-the-middle manner to significantly destroy video recovery performance and finally incurs the malfunction of downstream video perception tasks. In this paper, we are the first attempt to inspect the vulnerability of NeVS and discover a novel adversarial attack, called codec hijacking, where the injected invisible perturbation conspires with the malicious encoding matrix by reorganizing the spatial-temporal bit allocation within the bitstream size budget. Such a zero-day vulnerability makes our attack hard to defend because there is no visual distortion on the recovered videos until the attack happens. More seriously, this attack can be extended to diverse enhancement models, thus exposing a wide range of video perception tasks under threat. Evaluation based on state-of-the-art video codec benchmark illustrates that our attack significantly degrades the recovery performance of NeVS over previous attack methods. The damaged video quality finally leads to obvious malfunction of downstream tasks with over 75% success rate. We hope to arouse public attention on codec hijacking and its defence.

A Universal Optimization Framework for Learning-based Image Codec

Recently, machine learning-based image compression has attracted increasing interests and is approaching the state-of-the-art compression ratio. But unlike traditional codec, it lacks a universal optimization method to seek efficient representation for different images. In this paper, we develop a plug-and-play optimization framework for seeking higher compression ratio, which can be flexibly applied to existing and potential future compression networks. To make the latent representation more efficient, we propose a novel latent optimization algorithm to adaptively remove the redundancy for each image. Additionally, inspired by the potential of side information for traditional codecs, we introduce side information into our framework, and integrate side information optimization with latent optimization to further enhance the compression ratio. In particular, with the joint side information and latent optimization, we can achieve fine rate control using only single model instead of training different models for different rate-distortion trade-offs, which significantly reduces the training and storage cost to support multiple bit rates. Experimental results demonstrate that our proposed framework can remarkably boost the machine learning-based compression ratio, achieving more than 10% additional bit rate saving on three different representative network structures. With the proposed optimization framework, we can achieve 7.6% bit rate saving against the latest traditional coding standard VVC on Kodak dataset, yielding the state-of-the-art compression ratio.

CBANet: Toward Complexity and Bitrate Adaptive Deep Image Compression Using a Single Network.

In this work, we propose a new deep image compression framework called Complexity and Bitrate Adaptive Network (CBANet) that aims to learn one single network to support variable bitrate coding under various computational complexity levels. In contrast to the existing state-of-the-art learning-based image compression frameworks that only consider the rate-distortion trade-off without introducing any constraint related to the computational complexity, our CBANet considers the complex rate-distortion-complexity trade-off when learning a single network to support multiple computational complexity levels and variable bitrates. Since it is a non-trivial task to solve such a rate-distortion-complexity related optimization problem, we propose a two-step approach to decouple this complex optimization task into a complexity-distortion optimization sub-task and a rate-distortion optimization sub-task, and additionally propose a new network design strategy by introducing a Complexity Adaptive Module (CAM) and a Bitrate Adaptive Module (BAM) to respectively achieve the complexity-distortion and rate-distortion trade-offs. As a general approach, our network design strategy can be readily incorporated into different deep image compression methods to achieve complexity and bitrate adaptive image compression by using a single network. Comprehensive experiments on two benchmark datasets demonstrate the effectiveness of our CBANet for deep image compression. Code is released at https://github.com/JinyangGuo/CBANet-release.

MmWave Networking and Edge Computing for Scalable 360° Video Multi-User Virtual Reality.

We investigate a novel multi-user mobile Virtual Reality (VR) arcade system for streaming scalable 8K 360° video with low interactive latency, while providing high remote scene immersion fidelity and application reliability. This is achieved through the integration of embedded multi-layer 360° tiling, edge computing, and wireless multi-connectivity that comprises sub-6 GHz and mmWave (millimeter wave) links. The sub-6 GHz band is used for broadcast of the base layer of the entire 360° panorama to all users, while the directed mmWave links are used for high-rate transmission of VR-enhancement layers that are specific to the viewports of the individual users. The viewport-specific enhancements can comprise compressed and raw 360° tiles, decoded first at the edge server. We aim to maximize the smallest immersion fidelity for the delivered 360 content across all VR users, given rate, latency and computing constraints. We characterize analytically the rate-distortion trade-offs across the spatiotemporal 360° panorama and the computing power required to decompress 360° tiles. The proposed solution consists of geometric programming algorithms and an intermediate step of graph-theoretic VR user to mmWave access point assignment. The results reveal a significant improvement (8 - 10 dB) in delivered VR user immersion fidelity and spatial resolution (8K vs. 4K) compared to a state-of-the-art method based on sub-6 GHz transmission only. We also show that an increasing number of raw 360° tiles are sent, as the mmWave network link data rate or the edge server/user computing power increase. Finally, we demonstrate that in order to hypothetically deliver the same immersion fidelity, the reference method would incur a much higher (2.5-4.5x) system latency.

An Information-Theoretic Approach to Collaborative Integrated Sensing and Communication for Two-Transmitter Systems

This paper considers information-theoretic models for integrated sensing and communication (ISAC) over multi-access channels (MAC) and device-to-device (D2D) communication. The models are general and include as special cases scenarios with and without perfect or imperfect state-information at the MAC receiver as well as causal state-information at the D2D terminals. For both setups, we propose collaborative sensing ISAC schemes where terminals not only convey data to the other terminals but also state-information that they extract from their previous observations. This state-information can be exploited at the other terminals to improve their sensing performances. Indeed, as we show through examples, our schemes improve over previous non-collaborative schemes in terms of their achievable rate-distortion tradeoffs. For D2D we propose two schemes, one where compression of state information is separated from channel coding and one where it is integrated via a hybrid coding approach.

Optimized video compression with residual split attention and swin-block artifact contraction

Research in video compression has seen significant advancement in the last several years. However, the existing deep learning-based algorithms continue to be plagued by erroneous motion compression and ineffective motion compensation architectures, resulting in compression errors with a lower rate–distortion trade-off. To overcome these challenges, we present an end-to-end purely deep learning-based video compression method through a set of primary operations (e.g., motion estimation, motion compression, motion compensation, residual compression, and artifact contraction) differently. A deep residual attention split (DRAS) block is introduced for motion compression networks to pay more attention to certain image regions to create more effective features for the decoder while boosting the rate–distortion optimization (RDO) efficiency. A channel residual block (CRB) is proposed in motion compensation to yield a more accurate predicted frame, potentially improving the residual frame. To mitigate the compression errors, an artifact contraction module (ACM) by residual swin convolution UNet block is included in this model to improve the reconstruction quality. To improve the final frame, a buffer is added to fine-tune the previous reference frames. These modules combine with a loss function by assessing the trade-off and enhancing the decoded video quality. A comprehensive ablation study demonstrates the effectiveness of the proposed blocks and modules for video compression. Experimental results show the competitive performance of the proposed method on four benchmark datasets.

Learned Compression Framework With Pyramidal Features and Quality Enhancement for SAR Images

Current image compression algorithms based on transforms can achieve ideal performance for natural images, but do not do well with synthetic aperture radar (SAR) images. We propose a learned compression framework with pyramidal features and quality enhancement to fully exploit the redundancy among image pixels and to improve the compression bitrate and reconstruction quality. Based on the variational autoencoder (VAE) architecture, pyramidal decomposition is performed at the first autoencoder to extract both global and coarse feature maps. The latent distribution is modeled by the second hyperprior autoencoder with a single-Gaussian model for more accurate and flexible entropy estimation. Universal quantization is applied to consolidate the entropy estimation accuracy of the hyperprior network. To further improve reconstruction quality, a residual dense network (RDN) is adopted to fully capture local and global features. Experimental results demonstrate that the proposed framework provides a better rate-distortion tradeoff than standard codecs such as JPEG, JPEG2000, and learning-based methods on both the Sandia and ICEYE datasets.

Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach

This paper investigates task-oriented communication for edge inference, where a low-end edge device transmits the extracted feature vector of a local data sample to a powerful edge server for processing. It is critical to encode the data into an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">informative</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">compact</i> representation for low-latency inference given the limited bandwidth. We propose a learning-based communication scheme that jointly optimizes feature extraction, source coding, and channel coding in a task-oriented manner, i.e., targeting the downstream inference task rather than data reconstruction. Specifically, we leverage an information bottleneck (IB) framework to formalize a rate-distortion tradeoff between the informativeness of the encoded feature and the inference performance. As the IB optimization is computationally prohibitive for the high-dimensional data, we adopt a variational approximation, namely the variational information bottleneck (VIB), to build a tractable upper bound. To reduce the communication overhead, we leverage a sparsity-inducing distribution as the variational prior for the VIB framework to sparsify the encoded feature vector. Furthermore, considering dynamic channel conditions in practical communication systems, we propose a variable-length feature encoding scheme based on dynamic neural networks to adaptively adjust the activated dimensions of the encoded feature to different channel conditions. Extensive experiments evidence that the proposed task-oriented communication system achieves a better rate-distortion tradeoff than baseline methods and significantly reduces the feature transmission latency in dynamic channel conditions.

Learning-Based Scalable Image Compression With Latent-Feature Reuse and Prediction

Recently, learning-based image compression model has attracted much attention due to its impressive performance and ease of optimization, compared with traditional DCT and wavelet-based image compression standards. Most learning-based image compression models are trained to minimize joint rate-distortion (RD) loss on one single RD trade-off point. However, in many multimedia applications, due to communication constraints, or display adaptation needs for different spatial formats, bit rates or power, it is necessary to provide a variety of image versions for different client devices. To fulfill this requirement, typical end-to-end image compression methods have to compress an image into several bit streams independently by a number of pre-trained networks, which are resource-consuming because of redundancy among these streams. To address this problem, inspired by traditional scalable video coding framework, we propose a learning-based end-to-end quality and spatial scalable image compression (QSSIC) model in multi-layer structure, in which each layer could generate one bitstream corresponding to a specified resolution and image fidelity. This scalability is achieved by exploring the potential of feature-domain representation prediction and reuse. To be specific, firstly, bitstreams of previous layers are used to predict the current layer representations which contains the enhancement information, and then only prediction residuals need to be coded in enhancement layers. Secondly, previous bitstreams are reused in image reconstruction in higher layers to provide basic information. The proposed model could be optimized in an end-to-end manner. Extensive experiments show that our method outperforms state-of-art deep neural networks (DNN)-based auto-encoders in simulcast scenarios. In addition, our method has a better performance than the traditional scalable image compression method scalable extension of H.264/AVC (SVC) and is comparable to scalable extension of H.265/HEVC (SHVC).

Taylor rate-distortion trade-off with Laplace correction-assisted coding in HEVC encoder

Taylor rate-distortion trade-off with Laplace correction-assisted coding in HEVC encoder

Taylor rate-distortion trade-off with Laplace correction-assisted coding in HEVC encoder

Taylor rate-distortion trade-off with Laplace correction-assisted coding in HEVC encoder

Improved Coefficient Recovery and Its Application for Rewritable Data Embedding.

JPEG is the most commonly utilized image coding standard for storage and transmission purposes. It achieves a good rate–distortion trade-off, and it has been adopted by many, if not all, handheld devices. However, often information loss occurs due to transmission error or damage to the storage device. To address this problem, various coefficient recovery methods have been proposed in the past, including a divide-and-conquer approach to speed up the recovery process. However, the segmentation technique considered in the existing method operates with the assumption of a bi-modal distribution for the pixel values, but most images do not satisfy this condition. Therefore, in this work, an adaptive method was employed to perform more accurate segmentation, so that the real potential of the previous coefficient recovery methods can be unleashed. In addition, an improved rewritable adaptive data embedding method is also proposed that exploits the recoverability of coefficients. Discrete cosine transformation (DCT) patches and blocks for data hiding are judiciously selected based on the predetermined precision to control the embedding capacity and image distortion. Our results suggest that the adaptive coefficient recovery method is able to improve on the conventional method up to 27% in terms of CPU time, and it also achieved better image quality with most considered images. Furthermore, the proposed rewritable data embedding method is able to embed 20,146 bits into an image of dimensions .

Optimal block search mechanism using deep recurrent neural network for enabling the code-efficiency in HEVC

Purpose High-efficiency video coding (HEVC) is the latest video coding standard that has better coding efficiency than the H.264/advanced video coding (AVC) standard. The purpose of this paper is to design and develop an effective block search mechanism for the video compression-HEVC standard such that the developed compression standard is applied for the communication applications. Design/methodology/approach In the proposed method, an rate-distortion (RD) trade-off, named regressive RD trade-off is used based on the conditional autoregressive value at risk (CaViar) model. The motion estimation (ME) is based on the new block search mechanism, which is developed with the modification in the Ordered Tree-based Hex-Octagon (OrTHO)-search algorithm along with the chronological Salp swarm algorithm (SSA) based on deep recurrent neural network (deepRNN) for optimally deciding the shape of search, search length of the tree and dimension. The chronological SSA is developed by integrating the chronological concept in SSA, which is used for training the deep RNN for ME. Findings The competing methods used for the comparative analysis of the proposed OrTHO-search based RD + chronological-salp swarm algorithm (RD + C-SSA) based deep RNN are support vector machine (SVM), fast encoding framework, wavefront-based high parallel (WHP) and OrTHO-search based RD method. The proposed video compression method obtained a maximum peak signal-to-noise ratio (PSNR) of 42.9180 dB and a maximum structural similarity index measure (SSIM) of 0.9827. Originality/value In this research, an effective block search mechanism was developed with the modification in the OrTHO-search algorithm along with the chronological SSA based on deepRNN for the video compression-HEVC standard.

Regressive rate-distortion trade-off with weighted entropy coding for HEVC encoding

High-Efficiency Video Coding (HEVC) is the standard employed for the subsequent compressing of the video without degrading the quality of the image. HEVC renders effective performance compared to the existing compression standards as the encoding time is minimal. With the concern for quality and video compression, this paper proposes a modified version for HEVC encoding using the Ordered Tree-based Hex-Octagon based block Search and Rate-Distortion trade-off (OrTHO-Search-based RD) for motion estimation and weighted Context-Adaptive Binary Arithmetic Coding (CABAC). The proposed OrTHO-Search-based RD-dependent HEVC renders a good-quality video after compression with half the compression standard when compared with the other existing compression standards. In the motion estimation block, the OrTHO-Search is employed with a new RD trade-off in such a way that the Conditional Autoregressive Value at Risk concept modifies the existing RD trade-off measure. The bit rates of the proposed method are reduced with effective coding. The experimentation and analysis of the methods are performed using four videos from CIPR SIF Sequences and one video from Xiph.org Video Test Media datasets and the analysis based on Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), reveals that the proposed method acquired the maximal PSNR of 45.1132 dB, maximal SSIM of 0. 9918, and minimal computational time of 0.2135 min.

Edge-aware coding tree unit hierarchical partitioning for quality scalable compression of depth maps

The scalable extension of the high efficiency video coding standard (SHVC) combines the large compression efficiency and high visual quality of HEVC, with the possibility of encoding different versions of the same video in a single bitstream. However, this comes at the cost of high computational complexity. In this context, many research works aim to reduce this complexity for texture images. We aim at achieving the same objective, but for depth maps whose characteristics make them different from conventional texture images. Depth maps are indeed characterized by areas of smoothly varying grey levels separated by sharp discontinuities at object boundaries. Preserving these discontinuities is crucial to enable high quality of synthesized views at the receiver side. In this paper, we propose a fast depth maps encoding scheme for quality scalable HEVC while exploiting depth maps characteristics, SHVC Coding Unit (CU) depth information and the correlation between the Base Layer (BL) and the Enhancement Layers (ELs) of SHVC. If a CU corresponds to a depth smooth region, we maintain the same best coding depth of its co-located in the BL. If a CU is a sharp region, the best coding depth is computed in the same way as in the original SHVC. Experiments are conducted and satisfying results are obtained as the proposed method improves the SHVC coding speed without a significant impact on the synthesized views Rate-Distortion tradeoff

Reduced-Complexity End-to-End Variational Autoencoder for on Board Satellite Image Compression

Recently, convolutional neural networks have been successfully applied to lossy image compression. End-to-end optimized autoencoders, possibly variational, are able to dramatically outperform traditional transform coding schemes in terms of rate-distortion trade-off; however, this is at the cost of a higher computational complexity. An intensive training step on huge databases allows autoencoders to learn jointly the image representation and its probability distribution, possibly using a non-parametric density model or a hyperprior auxiliary autoencoder to eliminate the need for prior knowledge. However, in the context of on board satellite compression, time and memory complexities are submitted to strong constraints. The aim of this paper is to design a complexity-reduced variational autoencoder in order to meet these constraints while maintaining the performance. Apart from a network dimension reduction that systematically targets each parameter of the analysis and synthesis transforms, we propose a simplified entropy model that preserves the adaptability to the input image. Indeed, a statistical analysis performed on satellite images shows that the Laplacian distribution fits most features of their representation. A complex non parametric distribution fitting or a cumbersome hyperprior auxiliary autoencoder can thus be replaced by a simple parametric estimation. The proposed complexity-reduced autoencoder outperforms the Consultative Committee for Space Data Systems standard (CCSDS 122.0-B) while maintaining a competitive performance, in terms of rate-distortion trade-off, in comparison with the state-of-the-art learned image compression schemes.

Patch-Based Holographic Image Sensing

Holographic representations of data enable distributed storage with progressive refinement when the stored packets of data are made available in any arbitrary order. In this paper, we propose and test patch-based transform coding holographic sensing of image data. Our proposal is optimized for progressive recovery under random order of retrieval of the stored data. The coding of the image patches relies on the design of distributed projections ensuring best image recovery, in terms of the $\ell_2$ norm, at each retrieval stage. The performance depends only on the number of data packets that has been retrieved thus far. Several possible options to enhance the quality of the recovery while changing the size and number of data packets are discussed and tested. This leads us to examine several interesting bit-allocation and rate-distortion trade offs, highlighted for a set of natural images with ensemble estimated statistical properties.

Taylor rate-distortion trade-off and adaptive block search for HEVC encoding

The advancement in high-efficiency video coding (HEVC) is adapted for defining the subsequent generation compression model to offer efficient compression without affecting the image quality. The HEVC offers improved performance than the existing compression models. Accordingly, this work develops an approach for video compression by proposing weighted entropy coding and adaptive block search-based rate-distortion (R-D) trade-off, named Taylor R-D trade-off. The adaptive block search R-D trade-off, by integrating the hexagon-based tree search algorithm (HBTSA), along with the Taylor R-D trade-off for initiating the block search process of motion estimation in video coding, which selects the optimal block. Initially, the frames are extorted from the input video. Then, the video frames are divided into macroblocks to perform the adaptive block search. Further, the suitable blocks are selected and given to the encoding process by weighted context-adaptive binary arithmetic coding (CABAC) that employs a weighted entropy function to persist the video quality after the compression. The results analysis shows that the proposed HBTSA method has improved PSNR and SSIM values of 42.717dB, and 0.991, respectively, using football, coast guard, garden, and tennis videos.