Rate-distortion Research Articles

Stable successive Neural Image Compression via coherent demodulation-based transformation

Neural Image Compression (NIC) has made significant strides in recent years. However, the existing NIC methods demonstrate instability issues during iterative re-compression cycles, which can degrade image quality with each cycle. This paper introduces a novel framework aimed at enhancing the stability of NIC methods. We first conducted a theoretical analysis and identified that the instability in current NIC methods stems from a lack of idempotency in transformations. Drawing from the domain of signal processing, we then examined the principles of idempotency in coherent demodulation techniques. This examination led to the identification of three foundational principles that inform the design of stable transformations: the cosine function, parameter sharing, and low-pass filtering. Leveraging these insights, we propose the innovative Coherent Demodulation-based Transformation (CDT), which is designed to address the stability challenges in NIC by incorporating these principles into its architecture. The experimental results suggest that CDT not only significantly improve the re-compression stability but also preserves the codec’s rate–distortion performance. Furthermore, it can be broadly applied in current NIC structures. The effectiveness of the module endorses the viability of designing transformation networks based on Coherent Demodulation principles, playing a crucial role in enhancing stability of NIC. The code will be available at https://github.com/baoyu2020/Stable_SuccessiveNIC.

High efficiency deep image compression via channel-wise scale adaptive latent representation learning

Recent learning based neural image compression methods have achieved impressive rate–distortion (RD) performance via the sophisticated context entropy model, which performs well in capturing the spatial correlations of latent features. However, due to the dependency on the adjacent or distant decoded features, existing methods require an inefficient serial processing structure, which significantly limits its practicability. Instead of pursuing computationally expensive entropy estimation, we propose to reduce the spatial redundancy via the channel-wise scale adaptive latent representation learning, whose entropy coding is spatially context-free and parallelizable. Specifically, the proposed encoder adaptively determines the scale of the latent features via a learnable binary mask, which is optimized with the RD cost. In this way, lower-scale latent representation will be allocated to the channels with higher spatial redundancy, which consumes fewer bits and vice versa. The downscaled latent features could be well recovered with a lightweight inter-channel upconversion module in the decoder. To compensate for the entropy estimation performance degradation, we further develop an inter-scale hyperprior entropy model, which supports the high efficiency parallel encoding/decoding within each scale of the latent features. Extensive experiments are conducted to illustrate the efficacy of the proposed method. Our method achieves bitrate savings of 18.23%, 19.36%, and 27.04% over HEVC Intra, along with decoding speeds that are 46 times, 48 times, and 51 times faster than the baseline method on the Kodak, Tecnick, and CLIC datasets, respectively.

Leveraging occupancy map to accelerate video-based point cloud compression

Video-based Point Cloud Compression enables point cloud streaming over the internet by converting dynamic 3D point clouds to 2D geometry and attribute videos, which are then compressed using 2D video codecs like H.266/VVC. However, the complex encoding process of H.266/VVC, such as the quadtree with nested multi-type tree (QTMT) partition, greatly hinders the practical application of V-PCC. To address this issue, we propose a fast CU partition method dedicated to V-PCC to accelerate the coding process. Specifically, we classify coding units (CUs) of projected images into three categories based on the occupancy map of a point cloud: unoccupied, partially occupied, and fully occupied. Subsequently, we employ either statistic-based rules or machine-learning models to manage the partition of each category. For unoccupied CUs, we terminate the partition directly; for partially occupied CUs with explicit directions, we selectively skip certain partition candidates; for the remaining CUs (partially occupied CUs with complex directions and fully occupied CUs), we train an edge-driven LightGBM model to predict the partition probability of each partition candidate automatically. Only partitions with high probabilities are retained for further Rate–Distortion (R–D) decisions. Comprehensive experiments demonstrate the superior performance of our proposed method: under the V-PCC common test conditions, our method reduces encoding time by 52% and 44% in geometry and attribute, respectively, while incurring only 0.68% (0.66%) BD-Rate loss in D1 (D2) measurements and 0.79% (luma) BD-Rate loss in attribute, significantly surpassing state-of-the-art works.

Quantum communications for image transmission over error‐prone channels

AbstractThe introduction of quantum communications, enabled by advancements in quantum computing, is expected to play a significant role in the field of communications. Inherent properties of quantum objects, such as superposition and entanglement, have the potential to provide novel solutions to overcome the challenges encountered by classical communication systems in bandwidth‐intensive applications such as media transmission. This research explores the performance of a quantum communication system in image transmission using quantum superposition and investigates its performance using a simple quantum channel model. With an increase in channel noise, there are significant gains in the rate distortion performance of images transmitted over the quantum channel compared to an ideal classical channel. This novel attempt at constructing a quantum communication‐based image transmission system indicates the potential of the approach to be applied to satisfy the ever‐increasing demands of high‐quality media transmission applications.

CFNet: An infrared and visible image compression fusion network

Image fusion aims to acquire a more complete image representation within a limited physical space to more effectively support practical vision applications. Although the currently popular infrared and visible image fusion algorithms take practical applications into consideration. However, they did not fully consider the redundancy and transmission efficiency of image data. To address this limitation, this paper proposes a compression fusion network for infrared and visible images based on joint CNN and Transformer, termed CFNet. First of all, the idea of variational autoencoder image compression is introduced into the image fusion framework, achieving data compression while maintaining image fusion quality and reducing redundancy. Moreover, a joint CNN and Transformer network structure is proposed, which comprehensively considers the local information extracted by CNN and the global long-distance dependencies emphasized by Transformer. Finally, multi-channel loss based on region of interest is used to guide network training. Not only can color visible and infrared images be fused directly but more bits can be allocated to the foreground region of interest, resulting in a superior compression ratio. Extensive qualitative and quantitative analyses affirm that the proposed compression fusion algorithm achieves state-of-the-art performance. In particular, rate–distortion performance experiments demonstrate the great advantages of the proposed algorithm for data storage and transmission. The source code is available at https://github.com/Xiaoxing0503/CFNet.

Quantum soft-covering lemma with applications to rate-distortion coding, resolvability and identification via quantum channels

We propose a quantum soft-covering problem for a given general quantum channel and one of its output states, which consists in finding the minimum rank of an input state needed to approximate the given channel output. We then prove a one-shot quantum covering lemma in terms of smooth min-entropies by leveraging decoupling techniques from quantum Shannon theory. This covering result is shown to be equivalent to a coding theorem for rate distortion under a posterior (reverse) channel distortion criterion by two of the present authors. Both one-shot results directly yield corollaries about the i.i.d. asymptotics, in terms of the coherent information of the channel. The power of our quantum covering lemma is demonstrated by two additional applications: first, we formulate a quantum channel resolvability problem, and provide one-shot as well as asymptotic upper and lower bounds. Second, we provide new upper bounds on the unrestricted and simultaneous identification capacities of quantum channels, in particular separating for the first time the simultaneous identification capacity from the unrestricted one, proving a long-standing conjecture of the last author.

Fast Depth Map Coding Algorithm for 3D-HEVC Based on Gradient Boosting Machine

Three-Dimensional High-Efficiency Video Coding (3D-HEVC) has been extensively researched due to its efficient compression and deep image representation, but encoding complexity continues to pose a difficulty. This is mainly attributed to redundancy in the coding unit (CU) recursive partitioning process and rate–distortion (RD) cost calculation, resulting in a complex encoding process. Therefore, enhancing encoding efficiency and reducing redundant computations are key objectives for optimizing 3D-HEVC. This paper introduces a fast-encoding method for 3D-HEVC, comprising an adaptive CU partitioning algorithm and a rapid rate–distortion-optimization (RDO) algorithm. Based on the ALV features extracted from each coding unit, a Gradient Boosting Machine (GBM) model is constructed to obtain the corresponding CU thresholds. These thresholds are compared with the ALV to further decide whether to continue dividing the coding unit. The RDO algorithm is used to optimize the RD cost calculation process, selecting the optimal prediction mode as much as possible. The simulation results show that this method saves 52.49% of complexity while ensuring good video quality.

Rate-Distortion Theory by and for Energy-Based Models

Rate-Distortion Theory by and for Energy-Based Models

JND-based multi-module cooperative perceptual optimization for HEVC

Since the just noticeable distortion (JND) reflects the tolerance limit of human visual system (HVS) to coding distortion directly, JND-based perceptual video coding (PVC) plays an increasingly significant role in video compression with the developing explosion of video data. In this paper, we focus on the coupling effect among coding modules and provide a JND-based multi-module cooperative perceptual optimization (JMCPO) scheme for HEVC. The main contribution of the proposed JMCPO scheme includes the following three aspects. (1) Based on quantization distortion estimation, an adaptive perceptual quantization scheme is proposed using the binary search approach on the premise of that the quantization distortion is infinitely close to the estimated JND threshold. (2) The coupling effect among coding modules is first analyzed and a novel perceptual residual filtering scheme is presented based on statistic analysis of coupling strength. (3) The JMCPO scheme is finally developed through collaborative optimization of residual filtering, quantization and rate–distortion optimization. Experimental results show that the proposed JMCPO scheme saves more bitrate with better subjective and objective qualities.

On the rate–distortion–perception–semantics tradeoff in low-rate regime for lossy compression

This paper formulates a multi-letter information theoretical optimization problem to characterize the tradeoff between rate, distortion, perception, and semantics. It turns out that our formulation is related to existing works, such as the rate–distortion theory, utility-privacy tradeoff, and rate–distortion–perception tradeoff. In this paper, we solve this problem in the low-rate regime by applying a local geometric approach, which allows us to express the optimal solutions analytically. By using such an approach, we obtain the analytical solution of the multi-letter optimization problem, which demonstrates the optimal rate–distortion–perception–semantics tradeoff, and characterizes the gain from increasing the number of letters as well as the limiting performance. Moreover, our result also validates the single-letter optimality of the scenarios above, which reveals the information theoretical insights of system and algorithm designs.

Towards real-time practical image compression with lightweight attention

The rate–distortion (RD) performance of learning-based image compression (LIC) has already surpassed that of traditional Versatile Video Coding (VVC) intra-coding. However, the gain of the compression efficiency is at the cost of high computational complexity, which is prohibitively expensive and becomes a bottleneck for practical applications. To this end, we proposed an efficient LIC method with lightweight designs for real-time practical applications, which achieves a better trade-off between compression performance and computational complexity. Specifically, residual connected lightweight attention units (RLAUs) are stacked for feature extraction, which achieves effective global spatial information embedding while keeping relatively low complexity. Meanwhile, a trainable channel-gained adaptive module (CGAM) is introduced into the nonlinear transform network and multi-stage context model. This module re-distributes the importance of different channels, further improving compression efficiency. The experimental results demonstrate that the proposed method achieves better compression efficiency than VVC intra-coding. Furthermore, compared to other state-of-the-art LIC approaches, the proposed method significantly reduces the coding time while maintaining comparable coding efficiency. The source codes are available at https://github.com/llsurreal919/LightweightLIC.

Introduction to mean dimension

The purpose of this survey is to explain basic ideas of mean dimension theory. Mean dimension represents a number of parameters per unit time for describing a dynamical system. It was introduced by Gromov in 1999. His motivation is to propose a new approach to geometric analysis over noncompact manifolds. Rather independently of this original motivation, Lindenstrauss and Weiss found applications of mean dimension to classical problems in topological dynamics. In this survey we start from the viewpoint of Lindenstrauss–Weiss and first review the applications to topological dynamics. Next we explain the relation to geometric analysis, in particular holomorphic curve theory. Mean dimension is connected to rate distortion theory, which is a branch of information theory studying the trade-off between distortion and compression rate in lossy data compression. In the last three sections we review a variational principle between mean dimension and rate distortion function and then discuss its possible application to holomorphic curve theory.

Rate-Adaptable Multitask-Oriented Semantic Communication: An Extended Rate–Distortion Theory-Based Scheme

Rate-Adaptable Multitask-Oriented Semantic Communication: An Extended Rate–Distortion Theory-Based Scheme

Adaptive HEVC video steganograhpy based on PU partition modes

High EfficiencyVideoCoding (HEVC) −based steganography has gained attention as a prominent research focus. Especially, block structure based HEVC video steganography has received increasing attention due to commendable performance. However, current block structure- based steganography algorithms confront with challenges such as reduced coding efficiency and limited capacity. To avoid these problems, an adaptive video steganography algorithm based on Prediction Unit (PU) partition mode in I-frames is proposed. This is done through the analysis of the block division process and the visual distortion resulting from the modification of the PU partition mode in HEVC. The PU block structure is utilized as steganographic covers, and the Rate Distortion Optimization (RDO) technique is introduced to establish an adaptive distortion function for Syndrome-trellis code (STC). Further comparison is performed between the proposed method and the state-of-the-art steganography algorithms, confirming its advantages in embedding capacity, compression efficiency, visual quality, and resistance to video steganalysis.

Transform coefficient distribution modeling based frame-level adaptive rate control for VVC

Rate control is a key issue in the research of video coding optimization. In this work, we propose a statistical modeling based frame-level distortion estimation model and a distortion dependency based rate control optimization algorithm for Versatile Video Coding (VVC). VVC has incorporated into more advanced coding tools, resulting in the distribution of transform coefficient more sharper. In response to this phenomenon, the TCD of VVC is modeled by the probability density function (PDF) of Cauchy distribution and generalized Gaussian distribution (GGD) individually. Then, a sliding window-based frame-level distortion model is proposed, which is used for derivation of inter-frame distortion dependency factor. According to the derived distortion dependency factor, a frame-level rate control optimization algorithm is presented to update the coding parameters in practical coding process. Experiments have shown that the proposed method could effectively improve rate–distortion performance, achieving an average rate distortion performance improvement of 0.72% in sequences blow 4K resolution, and 1.47% in 4K sequences in low delay P frame (LDP) configuration.

Bayesian Reinforcement Learning With Limited Cognitive Load.

All biological and artificial agents must act given limits on their ability to acquire and process information. As such, a general theory of adaptive behavior should be able to account for the complex interactions between an agent's learning history, decisions, and capacity constraints. Recent work in computer science has begun to clarify the principles that shape these dynamics by bridging ideas from reinforcement learning, Bayesian decision-making, and rate-distortion theory. This body of work provides an account of capacity-limited Bayesian reinforcement learning, a unifying normative framework for modeling the effect of processing constraints on learning and action selection. Here, we provide an accessible review of recent algorithms and theoretical results in this setting, paying special attention to how these ideas can be applied to studying questions in the cognitive and behavioral sciences.

A PRECISE RESPIRATORY AND HEART RATE DETECTION METHOD FOR MILLIMETER-WAVE RADAR

Millimeter wave radar as a type of noncontact sensor can more conveniently and insensibly obtain breathing and heartbeat signals. However, due to the weak radar echo signal and the influence of noise, there may be phase ambiguity and separation distortion of breathing and heartbeat signals during the processing, which affects the accuracy of detection. In this paper, an enhanced extended differential and cross multiplication (EDACM) algorithm is presented to obtain chest position signals without phase ambiguity. A specific order finite impulse response (FIR) filter with a Blackman window is applied to extract breath signals without waveform distortion from phase signals, and the heartbeat signal without rate distortion is extracted from the phase signal by means of the designed Chebyshev II filter. The measured results show that the proposed method achieves 96.78% accuracy for heart rate detection and 94.44% accuracy for respiratory rate detection, and improves the accuracy of heart rate detection by 2.22% and respiratory rate detection by 4.06% compared to the band-pass filter method under the same parameter conditions.

Efficient Computation of the Quantum Rate-Distortion Function

The quantum rate-distortion function plays a fundamental role in quantum information theory, however there is currently no practical algorithm which can efficiently compute this function to high accuracy for moderate channel dimensions. In this paper, we show how symmetry reduction can significantly simplify common instances of the entanglement-assisted quantum rate-distortion problems. This allows us to better understand the properties of the quantum channels which obtain the optimal rate-distortion trade-off, while also allowing for more efficient computation of the quantum rate-distortion function regardless of the numerical algorithm being used. Additionally, we propose an inexact variant of the mirror descent algorithm to compute the quantum rate-distortion function with provable sublinear convergence rates. We show how this mirror descent algorithm is related to Blahut-Arimoto and expectation-maximization methods previously used to solve similar problems in information theory. Using these techniques, we present the first numerical experiments to compute a multi-qubit quantum rate-distortion function, and show that our proposed algorithm solves faster and to higher accuracy when compared to existing methods.

Structural Properties of the Wyner-Ziv Rate Distortion Function: Applications for Multivariate Gaussian Sources.

The main focus of this paper is the derivation of the structural properties of the test channels of Wyner's operational information rate distortion function (RDF), R¯(ΔX), for arbitrary abstract sources and, subsequently, the derivation of additional properties for a tuple of multivariate correlated, jointly independent, and identically distributed Gaussian random variables, {Xt,Yt}t=1∞, Xt:Ω→Rnx, Yt:Ω→Rny, with average mean-square error at the decoder and the side information, {Yt}t=1∞, available only at the decoder. For the tuple of multivariate correlated Gaussian sources, we construct optimal test channel realizations which achieve the informational RDF, R¯(ΔX)=▵infM(ΔX)I(X;Z|Y), where M(ΔX) is the set of auxiliary RVs Z such that PZ|X,Y=PZ|X, X^=f(Y,Z), and E{||X-X^||2}≤ΔX. We show the following fundamental structural properties: (1) Optimal test channel realizations that achieve the RDF and satisfy conditional independence, PX|X^,Y,Z=PX|X^,Y=PX|X^,EX|X^,Y,Z=EX|X^=X^. (2) Similarly, for the conditional RDF, RX|Y(ΔX), when the side information is available to both the encoder and the decoder, we show the equality R¯(ΔX)=RX|Y(ΔX). (3) We derive the water-filling solution for RX|Y(ΔX).

Information theoretic clustering for coarse-grained modeling of non-equilibrium gas dynamics

We present a new framework towards the objective of learning coarse-grained models based on the maximum entropy principle. We show that existing methods for assigning clusters using the maximum entropy approach are heuristic or sub-optimal. We propose a machine learning framework informed by rate-distortion theory to learn optimal cluster assignments, aiming to improve the effectiveness of the coarse-graining process. Our approach involves transforming the discrete optimization problem into a probabilistic and continuous form. Our inverse modeling approach involves the development of a fully differentiable, adjoint-driven dynamics solver for use with gradient-based optimization techniques. The entire framework is end-to-end differentiable, facilitating backward pass gradient computation to flow through the ODE solve and probabilistic coarse-graining to train a classifier. We demonstrate application in the evolution of particle quantum states in non-equilibrium conditions. The high dimensionality of these equations poses a significant challenge for efficient and accurate computations, even for seemingly simple chemical systems. Training is performed using a loss function informed by rate-distortion theory to cluster the rovibrational states of some molecule, effectively forming a reduced order model of the master equations. We also introduce variable transformations along with several ways of improving the tractability of the training process. These allow the framework to process the extremely high-dimensional discrete optimization problem successfully. Our method is general, and to demonstrate the effectiveness in a controlled setting, we apply it to a one-dimensional Gaussian source quantization problem for which an analytical solution is known, followed by the problem of isothermal relaxation of a ▪ system with O(104) degrees of freedom.