Rate-distortion Research Articles

Information-Theoretic Approaches to Differential Privacy

This tutorial studies relations between differential privacy and various information-theoretic measures by using several selective articles. In particular, we present how these connections can provide new interpretations for the privacy guarantee in systems that deploy differential privacy in an information-theoretic framework. Accordingly, the tutorial delivers an extensive summary on the existing literature that makes use of information-theoretic measures and tools such as mutual information, min-entropy, Kullback-Leibler divergence, and rate-distortion function for quantification and characterization of differential privacy in various settings.

Neural Network Based Rate Control for Versatile Video Coding

In this work, we propose a neural network based rate control algorithm for Versatile Video Coding (VVC). The proposed method relies on the modeling of the Rate-Quantization (R-Q) and Distortion-Quantization (D-Q) relationships in a data driven manner based upon the characteristics of prediction residuals. In particular, a pre-analysis framework is adopted, in an effort to obtain the prediction residuals which govern the Rate-Distortion (R-D) behaviors. By inferring from the prediction residuals with deep neural networks, the Coding Tree Unit (CTU) level R-Q and D-Q model parameters are derived, which could efficiently guide the optimal bit allocation. Subsequently, the coding parameters, including Quantization Parameter (QP) and λ, at both frame and CTU levels, are obtained according to allocated bit-rates. We implement the proposed rate control algorithm on VVC Test Model (VTM-13.0). Experimental results exhibit that the proposed rate control algorithm achieves 0.77% BD-Rate savings under Low Delay B (LDB) configurations when compared to the default rate control algorithm used in VTM-13.0. For Random Access (RA) configurations, 1.77% BD-Rate savings can be observed. Furthermore, with better bit-rate estimation, more stable buffer status can be observed, further demonstrating the advantages of the proposed rate control method.

Quality-Constrained Encoding Optimization for Omnidirectional Video Streaming

Omnidirectional video streaming is usually implemented based on the representations of tiles, where the tiles are obtained by splitting the video frame into several rectangular areas and each tile is converted into multiple representations with different resolutions and encoded at different bitrates. One key issue in omnidirectional video streaming is how to choose the optimal representations for each tile at the server to save the overall transmission bitrate to all users while offering them satisfactory quality. This is different from the adaptive bitrate-based method that optimizes the downloading procedure of individual users, where the given video representations are stored on the server. In this work, we focus on optimization for the encoding of omnidirectional video streaming by using the optimal combination of tile representations. To achieve our goal, we formulate the selection of the representations into an optimization problem in which the transmission bitrate of all the representations is minimized with a quality constraint. By using this constraint, we can improve the average quality of omnidirectional videos for users. More specifically, we first construct the tile-level rate-distortion (R-D) model and determine the available tile bandwidth based on the previous viewers’ statistics. Then, we formulate the representation selection problem based on the obtained R-D model and tile bandwidth. Finally, we solve this problem to obtain the optimal combination of tile representations so that we can transmit the omnidirectional video to users with satisfactory quality but low bitrate. The experimental results demonstrate the effectiveness of our proposed approach when it is applied to omnidirectional video streaming.

The Information Bottleneck's Ordinary Differential Equation: First-Order Root Tracking for the Information Bottleneck.

The Information Bottleneck (IB) is a method of lossy compression of relevant information. Its rate-distortion (RD) curve describes the fundamental tradeoff between input compression and the preservation of relevant information embedded in the input. However, it conceals the underlying dynamics of optimal input encodings. We argue that these typically follow a piecewise smooth trajectory when input information is being compressed, as recently shown in RD. These smooth dynamics are interrupted when an optimal encoding changes qualitatively, at a bifurcation. By leveraging the IB's intimate relations with RD, we provide substantial insights into its solution structure, highlighting caveats in its finite-dimensional treatments. Sub-optimal solutions are seen to collide or exchange optimality at its bifurcations. Despite the acceptance of the IB and its applications, there are surprisingly few techniques to solve it numerically, even for finite problems whose distribution is known. We derive anew the IB's first-order Ordinary Differential Equation, which describes the dynamics underlying its optimal tradeoff curve. To exploit these dynamics, we not only detect IB bifurcations but also identify their type in order to handle them accordingly. Rather than approaching the IB's optimal tradeoff curve from sub-optimal directions, the latter allows us to follow a solution's trajectory along the optimal curve under mild assumptions. We thereby translate an understanding of IB bifurcations into a surprisingly accurate numerical algorithm.

Information-theoretic principles in incremental language production

I apply a recently emerging perspective on the complexity of action selection, the rate-distortion theory of control, to provide a computational-level model of errors and difficulties in human language production, which is grounded in information theory and control theory. Language production is cast as the sequential selection of actions to achieve a communicative goal subject to a capacity constraint on cognitive control. In a series of calculations, simulations, corpus analyses, and comparisons to experimental data, I show that the model directly predicts some of the major known qualitative and quantitative phenomena in language production, including semantic interference and predictability effects in word choice; accessibility-based ("easy-first") production preferences in word order alternations; and the existence and distribution of disfluencies including filled pauses, corrections, and false starts. I connect the rate-distortion view to existing models of human language production, to probabilistic models of semantics and pragmatics, and to proposals for controlled language generation in the machine learning and reinforcement learning literature.

Reducing Video Coding Complexity Based on CNN-CBAM in HEVC

High-efficiency video coding (HEVC) outperforms H.264 in coding efficiency. However, the rate–distortion optimization (RDO) process in coding tree unit (CTU) partitioning requires an exhaustive exploration of all possible quad-tree partitions, resulting in high encoding complexity. To simplify this process, this paper proposed a convolution neural network (CNN) based optimization algorithm combined with a hybrid attention mechanism module. Firstly, we designed a CNN compatible with the current coding unit (CU) size to accurately predict the CU partitions. In addition, we also designed a convolution block to enhance the information interaction between CU blocks. Then, we introduced the convolution block attention module (CBAM) into CNN, called CNN-CBAM. This module concentrates on important regions in the image and attends to the target object correctly. Finally, we integrated the CNN-CBAM into the HEVC coding framework for CU partition prediction in advance. The proposed network was trained, validated, and tested using a large scale dataset covering various scenes and objects, which provides extensive samples for intra-frame CU partition prediction in HEVC. The experimental findings demonstrate that our scheme can reduce the coding time by up to 64.05% on average compared to a traditional HM16.5 encoder, with only 0.09 dB degradation in BD-PSNR and a 1.94% increase in BD-BR.

A CTU-Level Screen Content Rate Control for Low-Delay Versatile Video Coding

In this paper, a rate control scheme for screen content video coding is proposed for the Versatile Video Coding (VVC) standard. In view of the critical challenges arising from the spatial and temporal unnaturalness of screen content sequences, the proposed method relies on the specifically designed pre-analysis such that the content information regarding the scene complexity can be obtained. As such, the estimated residual complexity is then incorporated into the proposed complexity-aware rate models and distortion models, leading to the optimal bit allocations for each frame and coding tree unit (CTU). In particular, the optimization problem can be analytically solved with the proposed models, and the coding parameters such as Lagrangian multiplier λ and quantization parameter of each frame and CTU could be delicately derived according to the allocated bits through the proposed analytical models. Extensive experiments have been conducted to evaluate the effectiveness of the proposed method. Compared to the default hierarchical λ-domain rate control and other screen content rate control algorithms, the proposed method could achieve obvious RD performance gain, and the bit-rate accuracy could be improved.

A new paradigm for high-capacity reversible data hiding with pixel repetition and adaptive embedding

Reversible data hiding (RDH) offers a useful attribute of retaining the visual semantics with security properties, for the effective management and protection of digital images with their metadata. This paper introduces a Reversible Data Hiding (RDH) scheme with Pixel Repetition (PR) to selectively embed additional secret data. bits utilizing the image redundancy created with PR. A generalized framework of PR is first developed and the rate–distortion criterion is extended for adaptive embedding. An input image is defined to be scaled-up by replacing each original pixel (called seed) with a block of shape s×s containing the seed, where s is the scaling factor. For better utilizing this PR-based image redundancy, we then propose an RDH scheme with two-phase embedding for s=2. The first phase of the scheme embeds 4-bit data in each 2 × 2 image block using a mapping function, which is designed to modify the repeated pixels, keeping the seed intact in each block. Being invertible, the mapping function also allows another phase of embedding in the preserved seeds using an adaptive selection of their least significant bits. Compared to the prominent PR-based and other related schemes, an impressive rate–distortion performance of the proposed scheme is recorded with a greater potential for the applications that require a high and varying embedding rate with better image quality.

Inflation Targeting, Exchange Rate Distortion and Output Stabilisation A Case of Zimbabwe Disinflation

Inflation targeting (IT) is practised in many countries and the real effects of strict inflation targeting were investigated on several occasions and less on other variants of IT. The study investigates the effects of inflation targeting lite on output. In particular, the compatibility of inflation targeting lite with managed exchange rate regime on output stabilization. The inflation targeting regime, the operating environment and exchange rate regimes were reviewed for Zimbabwe. An ARDL model was adopted as the empirical strategy to investigate the compatibility problem and the real effects of IT lite. Modelling output as output gap, inflation targeting lite and the exchange rate regimes were found to be inconsistent with output stabilization. IT lite and exchange rate premium increased the output gap, however the interaction between IT lite and exchange rate premium had negative effects on output. These results hold for both the short run and long run. However, considering the official rate and parallel rate separately, parallel rate has a negative effect but its interaction with IT has positive effects. The study recommends that developing countries should consider the exchange rate systems before the adoption of IT as a monetary policy framework. A floating exchange rate seemed to be compatible with IT lite.

An image compression and encryption scheme for similarity retrieval

With the development of cloud computing, people usually outsource encrypted images for saving storage and protecting privacy. However, traditional image encryption methods not only hinder the availability of images such as similarity retrieval, but also degrade the compression performance. To address this issue, we propose a retrievable image compression and encryption method(RICE). RICE takes into account the contradiction of image compression, availability and security, then propose a cascaded information bottleneck model, which includes the compression information bottleneck and the security and availability information bottleneck. The former is converted into a rate distortion problem and its optimal solution is sought by a convolutional neural network(CNN)-based compression network which includes channel space attention module and discrete wavelet transform(DWT) module. To solve the later, we propose a feature partition method to find a retrieval subset that balances the contradiction between security and availability, and design a DNA-based deterministic encryption method for this subset to support ciphertext retrieval. The ciphertext of the retrieved subset is sent to the proposed similarity search fully connected network(SimFcNet) to improve the retrieval accuracy. The remaining subset is encrypted by Non-deterministic encryption to further improve security. In general, the method RICE we proposed supports similarity retrievable in compressed domain ciphertext, and can achieve excellent performance. Experimental results show that our method is 36.56% higher than JPEG2000 at compression ratio of 60:1 in MS-SSIM, the accuracy of ciphertext retrieval can reach 0.828, and the security of ciphertext is close to that of traditional encryption methods.

Block-Adaptive Point Cloud Attribute Coding With Region-Aware Optimized Transform

Block-based compression scheme shows remarkable success in image and video coding. However, existing tree-type block partition methods usually divide point clouds into clusters with few or disjoint points due to the irregular sampling, which is adverse for the subsequent transform to exploit the local region correlation. Moreover, the widely-used optimal transform, e.g., Discrete Cosine Transform (DCT), is deduced with the assumption of the specified probability model. Thus these transforms cannot adequately conform to diverse statistical characteristics of point cloud attributes. To address the above two problems, we propose a block-adaptive codec with the optimized transform for point cloud attribute compression, including Progressive Clustering (PC) for block partition and Region-Aware Signal Modeling (RASM) for transform. PC is designed via the split-and-merge strategy. In the splitting phase, over-segmented clusters are obtained by iteratively searching the nearest neighbors to maintain the spatial continuity of points in one cluster. In the merging step, blocks are determined by merging over-segmented clusters under the guidance of minimizing texture complexity, which adjusts block sizes to adapt to texture variation. RASM adaptively captures the color correlations to respond to diverse statistical characteristics by optimizing the overall Rate-Distortion (RD) cost. Then, the optimal transform bases are obtained via the eigen-decomposition of the color correlation representation with respect to the Gaussian Markov Random Field, where input colors are obtained considering the linear relationship between geometry and color variations. The linear coefficients also need to be encoded to combine geometry to reconstruct the transform bases at the decoder in the same way as the encoder. Particularly, we accelerate the RASM by constraining the independent relationship between signals in the local region, which hardly influences RD performance. Extensive experiments not only indicate the effectiveness of PC and RASM but also show the significant RD gains achieved by our approach compared with several state-of-the-art platforms.

Zorro: Valid, Sparse, and Stable Explanations in Graph Neural Networks

With the ever-increasing popularity and applications of graph neural networks, several proposals have been made to explain and understand the decisions of a graph neural network. Explanations for graph neural networks differ in principle from other input settings. It is important to attribute the decision to input features and other related instances connected by the graph structure. We find that the previous explanation generation approaches that maximize the mutual information between the label distribution produced by the model and the explanation to be restrictive. Specifically, existing approaches do not enforce explanations to be valid, sparse, or robust to input perturbations. In this paper, we lay down some of the fundamental principles that an explanation method for graph neural networks should follow and introduce a metric <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RDT-Fidelity</i> as a measure of the explanation's effectiveness. We propose a novel approach Zorro based on the principles from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rate-distortion theory</i> that uses a simple combinatorial procedure to optimize for RDT-Fidelity. Extensive experiments on real and synthetic datasets reveal that Zorro produces sparser, stable, and more faithful explanations than existing graph neural network explanation approaches.

Machine Learning Based Efficient QT-MTT Partitioning Scheme for VVC Intra Encoders

The next-generation Versatile Video Coding (VVC) standard introduces a new Multi-Type Tree (MTT) block partitioning structure that supports Binary-Tree (BT) and Ternary-Tree (TT) splits in both vertical and horizontal directions. This new approach leads to five possible splits at each block depth. It thereby improves the coding efficiency of VVC over that of the preceding High Efficiency Video Coding (HEVC) standard, which only supports Quad-Tree (QT) partitioning with a single split per block depth. However, MTT also has brought a considerable impact on encoder computational complexity. This paper proposes a two-stage learning-based technique to tackle the complexity overhead of MTT in VVC intra encoders. In our scheme, the input block is first processed by a Convolutional Neural Network (CNN) to predict its spatial features through a vector of probabilities describing the partition at each 4×4 edge. Subsequently, a Decision Tree (DT) model leverages this vector of spatial features to predict the most likely splits at each block. Finally, based on this prediction, only the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> most likely splits are processed by the Rate-Distortion (RD) process of the encoder. In order to train our CNN and DT models on a wide range of image contents, we also propose a public VVC frame partitioning dataset based on existing image dataset encoded with the VVC reference software encoder. Our solution relying on the top-3 configuration reaches 47.4% complexity reduction for a negligible bitrate increase of 0.79%. A top-2 configuration enables a higher complexity reduction of 70.4% for 2.49% bitrate loss. These results emphasize a better trade-off between VTM intra-coding efficiency and complexity reduction compared to the state-of-the-art solutions. The source code of the proposed method and the training dataset are made publicly available at GitHub.

Asymmetric Learned Image Compression With Multi-Scale Residual Block, Importance Scaling, and Post-Quantization Filtering

Recently, deep learning-based image compression has made significant progresses, and has achieved better rate-distortion (R-D) performance than the latest traditional method, H.266/VVC, in both MS-SSIM metric and the more challenging PSNR metric. However, a major problem is that the complexities of many leading learned schemes are too high. In this paper, we propose an efficient and effective image coding framework, which achieves similar R-D performance with lower complexity than the state of the art. First, we develop an improved multi-scale residual block (MSRB) that can expand the receptive field and capture global information more efficiently, which further reduces the spatial correlation of the latent representations. Second, an importance scaling network is introduced to directly scale the latents to achieve content-adaptive bit allocation without sending side information, which is more flexible than previous importance map methods. Third, we apply a post-quantization filter (PQF) to reduce the quantization error, motivated by the Sample Adaptive Offset (SAO) filter in video coding. Moreover, our experiments show that the performance of the system is less sensitive to the complexity of the decoder. Therefore, we design an asymmetric paradigm, in which the encoder employs three stages of MSRBs to improve the learning capacity, whereas the decoder only uses one stage of MSRB, which reduces the decoder complexity and still yields satisfactory performance. Experimental results show that compared to the state-of-the-art method, the encoding and decoding time of the proposed method are about 17 times faster, and the R-D performance is only reduced by about 1% on both Kodak and Tecnick-40 datasets, which is still better than H.266/VVC(4:4:4) and other leading learning-based methods. Our source code is publicly available at https://github.com/fengyurenpingsheng.

Signaling games in multiple dimensions: Geometric properties of equilibrium solutions

Signaling game problems investigate communication scenarios where encoder(s) and decoder(s) have misaligned objectives due to the fact that they either employ different cost functions or have inconsistent priors. This problem has been studied in the literature for scalar sources under various setups. In this paper, we consider multi-dimensional sources under quadratic criteria in the presence of a bias leading to a mismatch in the criteria, where we show that the generalization from the scalar setup is more than technical. We show that the Nash equilibrium solutions lead to structural richness due to the subtle geometric analysis the problem entails, with consequences in both system design, the presence of linear Nash equilibria, and an information theoretic problem formulation. We first provide a set of geometric conditions that must be satisfied in equilibrium considering any multi-dimensional source. Then, we consider independent and identically distributed sources and characterize necessary and sufficient conditions under which an informative linear Nash equilibrium exists. These conditions involve the bias vector that leads to misaligned costs. Depending on certain conditions related to the bias vector, the existence of linear Nash equilibria requires sources with a Gaussian or a symmetric density. Moreover, in the case of Gaussian sources, our results have a rate–distortion theoretic implication that achievable rates and distortions in the considered game theoretic setup can be obtained from its team theoretic counterpart.

Rate-distortion theory of neural coding and its implications for working memory.

Rate-distortion theory provides a powerful framework for understanding the nature of human memory by formalizing the relationship between information rate (the average number of bits per stimulus transmitted across the memory channel) and distortion (the cost of memory errors). Here we show how this abstract computational-level framework can be realized by a model of neural population coding. The model reproduces key regularities of visual working memory, including some that were not previously explained by population coding models. We verify a novel prediction of the model by reanalyzing recordings of monkey prefrontal neurons during an oculomotor delayed response task.

Effects of Different Full-Reference Quality Assessment Metrics in End-to-End Deep Video Coding

Visual quality assessment is often used as a key performance indicator (KPI) to evaluate the performance of electronic devices. There exists a significant association between visual quality assessment and electronic devices. In this paper, we bring attention to alternative choices of perceptual loss function for end-to-end deep video coding (E2E-DVC), which can be used to reduce the amount of data generated by electronic sensors and other sources. Thus, we analyze the effects of different full-reference quality assessment (FR-QA) metrics on E2E-DVC. First, we select five optimization-suitable FR-QA metrics as perceptual objectives, which are differentiable and thus support back propagation, and use them to optimize an E2E-DVC model. Second, we analyze the rate–distortion (R-D) behaviors of an E2E-DVC model under different loss function optimizations. Third, we carry out subjective human perceptual tests on the reconstructed videos to show the performance of different FR-QA optimizations on subjective visual quality. This study reveals the effects of the competing FR-QA metrics on E2E-DVC and provides a guide for further future study on E2E-DVC in terms of perceptual loss function design.

D-Semifaithful Codes That are Universal Over Both Memoryless Sources and Distortion Measures

We prove the existence of codebooks for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> -semifaithful lossy compression that are simultaneously universal with respect to both the class of finite-alphabet memoryless sources and the class of all bounded additive rational distortion measures. By applying independent random selection of the codewords according to a mixture of all memoryless sources, we achieve redundancy rates that are within <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (log <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n/n</i> ) close to the empirical rate-distortion function of every given source vector with respect to every bounded, rational distortion measure.

The Role of Fidelity in Goal-Oriented Semantic Communication: A Rate Distortion Approach

We study a variant of a robust description source coding framework, which is a relevant model for goal-oriented semantic information transmission, via its corresponding characterization. Considering two individual single-letter separable distortion constraints and input and output data acting as the intrinsic and extrinsic message, respectively, we first derive a lower bound on the optimal rates of the problem, as well as necessary and sufficient conditions for this bound to be tight. Subsequently, we prove a general result that provides in parametric form the optimal solution of the characterization of this problem. Capitalizing on these results, we examine the structure of the solution for one case study of general binary alphabets under Hamming distortions and solve in closed form a special case. We also solve another general binary alphabet case where a Hamming and an erasure distortion coexist, as a means to highlight the importance of selecting the type of the distortion constraint in goal-oriented semantic communication. Furthermore, we develop a goal-oriented Blahut-Arimoto (BA) algorithm, which can be used for the computation of any finite alphabet intrinsic or extrinsic message under individual distortion criteria. Finally, we revisit the problem for multidimensional independent and identically distributed (i.i.d.) jointly Gaussian processes with individual mean-square error (MSE) distortion constraints, providing new insights that have previously been overlooked. This work reveals the cardinal role of context-dependent fidelity criteria in goal-oriented semantic communication.

Sustaining Fairness via Incremental Learning

Machine learning systems are often deployed for making critical decisions like credit lending, hiring, etc. While making decisions, such systems often encode the user's demographic information (like gender, age) in their intermediate representations. This can lead to decisions that are biased towards specific demographics. Prior work has focused on debiasing intermediate representations to ensure fair decisions. However, these approaches fail to remain fair with changes in the task or demographic distribution. To ensure fairness in the wild, it is important for a system to adapt to such changes as it accesses new data in an incremental fashion. In this work, we propose to address this issue by introducing the problem of learning fair representations in an incremental learning setting. To this end, we present Fairness-aware Incremental Representation Learning (FaIRL), a representation learning system that can sustain fairness while incrementally learning new tasks. FaIRL is able to achieve fairness and learn new tasks by controlling the rate-distortion function of the learned representations. Our empirical evaluations show that FaIRL is able to make fair decisions while achieving high performance on the target task, outperforming several baselines.