Distortion Constraints Research Articles

Bounds on the Sum-Rate of MIMO Causal Source Coding Systems with Memory under Spatio-Temporal Distortion Constraints.

In this paper, we derive lower and upper bounds on the OPTA of a two-user multi-input multi-output (MIMO) causal encoding and causal decoding problem. Each user’s source model is described by a multidimensional Markov source driven by additive noise process subject to three classes of spatio-temporal distortion constraints. To characterize the lower bounds, we use state augmentation techniques and a data processing theorem, which recovers a variant of rate distortion function as an information measure known in the literature as nonanticipatory -entropy, sequential or nonanticipative RDF. We derive lower bound characterizations for a system driven by an Gaussian noise process, which we solve using the SDP algorithm for all three classes of distortion constraints. We obtain closed form solutions when the system’s noise is possibly non-Gaussian for both users and when only one of the users is described by a source model driven by a Gaussian noise process. To obtain the upper bounds, we use the best linear forward test channel realization that corresponds to the optimal test channel realization when the system is driven by a Gaussian noise process and apply a sequential causal DPCM-based scheme with a feedback loop followed by a scaled ECDQ scheme that leads to upper bounds with certain performance guarantees. Then, we use the linear forward test channel as a benchmark to obtain upper bounds on the OPTA, when the system is driven by an additive non-Gaussian noise process. We support our framework with various simulation studies.

Coupled Aeropropulsive Optimization of a Three-Dimensional Boundary-Layer Ingestion Propulsor Considering Inlet Distortion

Boundary-layer ingestion (BLI) promises increased aircraft efficiency, but excessive inlet distortion must be avoided to prevent fans that are too heavy or structurally infeasible. We propose a new approach to study the effect of distortion on BLI propulsors via an aeropropulsive design optimization with a constraint on inlet distortion using highly efficient gradient-based methods with analytic derivatives. The fully coupled aeropropulsive model includes a three-dimensional Reynolds-averaged Navier–Stokes aerodynamic analysis of the flow and a thermodynamic cycle model of the propulsor fan. We minimize the shaft power required at cruise for the aft BLI thruster by varying the propulsor size and the shape of the propulsor nacelle and aft fuselage. Optimizations are performed with and without the inlet distortion constraint for different aft-propulsor sizes to study the impact of the constraint on overall BLI performance. The results show that imposing a distortion constraint increased the required propulsor shaft power by up to 1.2% relative to the unconstrained case, and that smaller propulsors suffer lower performance degradation than larger ones. The results underline the importance of using a fully coupled aeropropulsive model to predict BLI performance and demonstrate the power of aeropropulsive optimization in the design of BLI configurations.

A Theory for Optimizing Insecure Internetwork Connections

While internetwork connections among private and public networks are inevitable and essential for enhancing the data rate and range of communication among networks, they introduce new vulnerabilities and risks of security breach to the interconnected networks. In this article, a novel information-theoretic model is proposed to analyze the tradeoff between network security and data-rate enhancement considering homogeneous and heterogeneous internetwork connections between two networks. Various patterns of internetwork connections among gateway nodes are studied. The implications of internetwork connections on data manipulation due to security breach by attackers are investigated by modeling the internetwork connections as a noisy communication channel. Specifically, the fundamental limit of the maximum achievable capacity afforded by internetwork connections is characterized, while considering the impact of security vulnerabilities that the internetwork connections introduce. For a nominal data-rate below the fundamental limit, the effect of security breach can be diminished to an arbitrarily low level by channel coding. Notably, any nominal data-rate above the fundamental limit is not achievable; hence, a packet-loss distortion is introduced to keep the data-rate below the fundamental limit. Moreover, the fundamental limit is used to find the optimal number of internetwork connections between two networks under prescribed data-rate and packet-loss distortion constraints.

Generalized Gaussian Multiterminal Source Coding: The Symmetric Case

Consider a generalized multiterminal source coding system, where $\binom{\ell }{ m}$ encoders, each observing a distinct size- $m$ subset of $\ell $ ( $\ell \geq 2$ ) zero-mean unit-variance exchangeable Gaussian sources with correlation coefficient $\rho $ , compress their observations in such a way that a joint decoder can reconstruct the sources within a prescribed mean squared error distortion based on the compressed data. The optimal rate-distortion performance of this system was previously known only for the two extreme cases $m=\ell $ (the centralized case) and $m=1$ (the distributed case), and except when $\rho =0$ , the centralized system can achieve strictly lower compression rates than the distributed system under all non-trivial distortion constraints. Somewhat surprisingly, it is established in the present paper that the optimal rate-distortion performance of the afore-described generalized multiterminal source coding system with $m\geq 2$ coincides with that of the centralized system for all distortions when $\rho \leq 0$ and for distortions below an explicit positive threshold (depending on $m$ ) when $\rho > 0$ . Moreover, when $\rho > 0$ , the minimum achievable rate of generalized multiterminal source coding subject to an arbitrary positive distortion constraint $d$ is shown to be within a finite gap (depending on $m$ and $d$ ) from its centralized counterpart in the large $\ell $ limit except for possibly the critical distortion $d=1-\rho $ .

CMB spectral distortions constraints on primordial black holes, cosmic strings and long lived unstable particles revisited

We calculate the spectral distortions from Hawking evaporation of primordial black holes before the epoch of recombination, taking into account emission of all standard model particles, including quark and gluons, and evolving the resulting particle cascades in the expanding Universe. We show that the constraints on the abundance of primordial black holes are stronger by more than an order of magnitude compared to the previous calculations which take only the primary photon emission into account. We also show that the shapes of the spectral distortions is different from the y or i-type distortions and are sensitive to the mass of the primordial black holes. We also extend previous constraints on the decay of long lived unstable particles before recombination to additional decay channels. We show that for dark matter mass ≲ 1 GeV, the spectral distortion shape is a function of the dark matter mass as well as the decay channel to standard model particles. We also provide new spectral distortion constraints on superconducting cosmic string decay. We explicitly show that consideration of emitted photon spectrum from string decay is not only important for the future experiments but also for already available COBE-FIRAS data.

Summarization of information systems based on rough set theory

In the rough set theory proposed by Pawlak, the concept of reduct is very important. The reduct is the minimum attribute set that preserves the partition of the universe. A great deal of research in the past has attempted to reduce the representation of the original table. The advantage of using a reduced representation table is that it can summarize the original table so that it retains the original knowledge without distortion. However, using reduct to summarize tables may encounter the problem of the table still being too large, so users will be overwhelmed by too much information. To solve this problem, this article considers how to further reduce the size of the table without causing too much distortion to the original knowledge. Therefore, we set an upper limit for information distortion, which represents the maximum degree of information distortion we allow. Under this upper limit of distortion, we seek to find the summary table with the highest compression. This paper proposes two algorithms. The first is to find all summary tables that satisfy the maximum distortion constraint, while the second is to further select the summary table with the greatest degree of compression from these tables.

Zero-Delay Multiple Descriptions of Stationary Scalar Gauss-Markov Sources

In this paper, we introduce the zero-delay multiple-description problem, where an encoder constructs two descriptions and the decoders receive a subset of these descriptions. The encoder and decoders are causal and operate under the restriction of zero delay, which implies that at each time instance, the encoder must generate codewords that can be decoded by the decoders using only the current and past codewords. For the case of discrete-time stationary scalar Gauss—Markov sources and quadratic distortion constraints, we present information-theoretic lower bounds on the average sum-rate in terms of the directed and mutual information rate between the source and the decoder reproductions. Furthermore, we show that the optimum test channel is in this case Gaussian, and it can be realized by a feedback coding scheme that utilizes prediction and correlated Gaussian noises. Operational achievable results are considered in the high-rate scenario using a simple differential pulse code modulation scheme with staggered quantizers. Using this scheme, we achieve operational rates within // of the theoretical lower bounds for varying description rates.

Tunable Measures for Information Leakage and Applications to Privacy-Utility Tradeoffs

We introduce a tunable measure for information leakage called maximal alpha-leakage. This measure quantifies the maximal gain of an adversary in inferring any (potentially random) function of a dataset from a release of the data. The inferential capability of the adversary is, in turn, quantified by a class of adversarial loss functions that we introduce as $\alpha$-loss, $\alpha\in[1,\infty]$. The choice of $\alpha$ determines the specific adversarial action and ranges from refining a belief (about any function of the data) for $\alpha=1$ to guessing the most likely value for $\alpha=\infty$ while refining the $\alpha^{th}$ moment of the belief for $\alpha$ in between. Maximal alpha-leakage then quantifies the adversarial gain under $\alpha$-loss over all possible functions of the data. In particular, for the extremal values of $\alpha=1$ and $\alpha=\infty$, maximal alpha-leakage simplifies to mutual information and maximal leakage, respectively. For $\alpha\in(1,\infty)$ this measure is shown to be the Arimoto channel capacity of order $\alpha$. We show that maximal alpha-leakage satisfies data processing inequalities and a sub-additivity property thereby allowing for a weak composition result. Building upon these properties, we use maximal alpha-leakage as the privacy measure and study the problem of data publishing with privacy guarantees, wherein the utility of the released data is ensured via a hard distortion constraint. Unlike average distortion, hard distortion provides a deterministic guarantee of fidelity. We show that under a hard distortion constraint, for $\alpha>1$ the optimal mechanism is independent of $\alpha$, and therefore, the resulting optimal tradeoff is the same for all values of $\alpha>1$. Finally, the tunability of maximal alpha-leakage as a privacy measure is also illustrated for binary data with average Hamming distortion as the utility measure.

Texture-Distortion-Constrained Joint Source-Channel Coding of Multi-View Video Plus Depth-Based 3D Video

A novel joint source and channel coding scheme tailored to 3D video is proposed in this paper to minimize the end-to-end view synthesis distortion within a given total bit rate for both texture and depth as well as a maximum tolerable distortion constraint for texture. First, we formulate a joint texture and depth coding mode selection strategy for error-resilient source coding of multi-view video plus depth-based 3D video through using the Lagrange multiplier method. Then, by considering the effect of residual errors after channel coding, we evolve to a more general formulation that jointly optimizes error-resilient source coding and channel coding in an integrated manner for unequal error protection between texture and depth, for which a theoretic solution using a proposed dual-trellis is derived. Finally, we extend the general formulation by including the texture distortion constraint. We show how to optimize the view synthesis quality while simultaneously catering to the texture quality constraint. Experimental results demonstrate the proposed algorithm has much better performance than existing related work.

Markov Information Bottleneck to Improve Information Flow in Stochastic Neural Networks

While rate distortion theory compresses data under a distortion constraint, information bottleneck (IB) generalizes rate distortion theory to learning problems by replacing a distortion constraint with a constraint of relevant information. In this work, we further extend IB to multiple Markov bottlenecks (i.e., latent variables that form a Markov chain), namely Markov information bottleneck (MIB), which particularly fits better in the context of stochastic neural networks (SNNs) than the original IB. We show that Markov bottlenecks cannot simultaneously achieve their information optimality in a non-collapse MIB, and thus devise an optimality compromise. With MIB, we take the novel perspective that each layer of an SNN is a bottleneck whose learning goal is to encode relevant information in a compressed form from the data. The inference from a hidden layer to the output layer is then interpreted as a variational approximation to the layer’s decoding of relevant information in the MIB. As a consequence of this perspective, the maximum likelihood estimate (MLE) principle in the context of SNNs becomes a special case of the variational MIB. We show that, compared to MLE, the variational MIB can encourage better information flow in SNNs in both principle and practice, and empirically improve performance in classification, adversarial robustness, and multi-modal learning in MNIST.

The Time-Invariant Multidimensional Gaussian Sequential Rate-Distortion Problem Revisited

We revisit the sequential rate-distortion (SRD) tradeoff problem for vector-valued Gauss–Markov sources with mean-squared error distortion constraints. Our study is partly motivated by the question recently raised in the paper “Rate-cost tradeoffs in control” (in Proc. 54th Annu. Allerton Conf. Commun., Control, Comput. , 2016, pp. 1157–1164) regarding the correctness of its solution algorithm known in the literature. We show via a counterexample that the dynamic reverse water-filling algorithm suggested by (15) of the paper “Stochastic linear control over a communication channel” ( IEEE Trans. Autom. Control , vol. 49, pp. 1549–1561, 2004) is not applicable to this problem, and consequently, the closed-form expression of the asymptotic SRD function derived in (17) of the paper “Stochastic linear control over a communication channel” ( IEEE Trans. Autom. Control , vol. 49, pp. 1549–1561, 2004) is not correct in general. Nevertheless, we show that the multidimensional Gaussian SRD function is semidefinite representable, and thus, it is readily computable.

New CMB spectral distortion constraints on decaying dark matter with full evolution of electromagnetic cascades before recombination

Current constraints on energy injection in the form of energetic particles before the epoch of recombination using CMB spectral distortions assume that all energy goes into $y$ and $\ensuremath{\mu}$-type distortions. We revisit these constraints with exact calculations of the spectral distortions by evolving the electromagnetic cascades. The actual spectral distortion differs in shape and amplitude from the $y$-type distortion and depends on the energy and nature of injected particles. The constraints on the energy injection processes such as dark matter decay can be relaxed by as much as a factor of 5.

Information and Energy Transmission With Experimentally Sampled Harvesting Functions

This paper considers the problem of simultaneous information and energy transmission, where the energy harvesting function is only known experimentally at sample points, e.g., due to nonlinearities and parameter uncertainties in harvesting circuits. We investigate the performance loss due to this partial knowledge of the harvesting function in terms of transmitted energy and information. In particular, we assume that the harvesting function is a subclass of the Sobolev space and consider two cases, where the experimental samples are either taken noiselessly or in the presence of noise. Using constructive function approximation and regression methods for noiseless and noisy samples, respectively, we show that the worst loss in energy transmission vanishes asymptotically as the number of samples increase. Similarly, the loss in information rate vanishes in the interior of the energy domain; however, it does not always vanish at maximal energy. We further show that the same principle applies in multicast settings, such as medium access in the Wi-Fi protocol. We also consider the end-to-end source–channel communication problem under source distortion constraint and channel energy requirement, where both distortion and harvesting functions are known only at samples.

The partial side information problem with additional reconstructions

We consider a quadratic Gaussian distributed lossy source coding setup with an additional constraint of identical reconstructions between the encoder and the decoder. The setup consists of two correlated Gaussian sources, wherein one of them has to be reconstructed to be within some distortion constraint and match with a corresponding reconstruction at the encoder, while the other source acts as coded side information. We study the trade-off between the rates of two encoders for a given distortion constraint on the reconstruction. An explicit characterization of this trade-off is the main result of the paper. We also give close inner and outer bounds for the discrete memoryless version of the problem.

Robust Distributed Compression of Symmetrically Correlated Gaussian Sources

Consider a lossy compression system with $\ell $ -distributed encoders and a centralized decoder. Each encoder compresses its observed source and forwards the compressed data to the decoder for joint reconstruction of the target signals under the mean-squared-error distortion constraint. It is assumed that the observed sources can be expressed as the sum of the target signals and the corruptive noises, which are generated independently from two symmetric multivariate Gaussian distributions. Depending on the parameters of such distributions, the rate-distortion limit of this system is characterized either completely or at least for sufficiently low distortions. The results are further extended to the robust distributed compression setting, where the outputs of a subset of encoders may also be used to produce a non-trivial reconstruction of the corresponding target signals. In particular, we obtain in the high-resolution regime a precise characterization of the minimum achievable reconstruction distortion based on the outputs of $k+1$ or more encoders when every $k$ out of all $\ell $ encoders are operated collectively in the same mode that is greedy in the sense of minimizing the distortion incurred by the reconstruction of the corresponding $k$ target signals with respect to the average rate of these $k$ encoders.

Asymptotic Rate-Distortion Analysis of Symmetric Remote Gaussian Source Coding: Centralized Encoding vs. Distributed Encoding.

Consider a symmetric multivariate Gaussian source with ℓ components, which are corrupted by independent and identically distributed Gaussian noises; these noisy components are compressed at a certain rate, and the compressed version is leveraged to reconstruct the source subject to a mean squared error distortion constraint. The rate-distortion analysis is performed for two scenarios: centralized encoding (where the noisy source components are jointly compressed) and distributed encoding (where the noisy source components are separately compressed). It is shown, among other things, that the gap between the rate-distortion functions associated with these two scenarios admits a simple characterization in the large ℓ limit.

Distortion-Constraint-Based Group Sparse Channel Estimation Under <inline-formula> <tex-math notation="LaTeX">$\alpha$ </tex-math> </inline-formula>-Stable Noise

The ever-increasing requirements of wireless communications have inspired the search for a better method to tackle the problem of group sparse channel estimation in practical applications. Sparsity with group structure is encountered in numerous applications, but efforts to devise group sparse adaptive methods remain scarce, especially under impulse noise with symmetric alpha stable (SαS) statistics. In this paper, we propose an improved adaptive algorithm using the distortion constraints based group sparse recursive least square (DC-GRLS) to exploit channel group sparsity and obtain robust performance under the background of α stable noise. We introduce distortion constraints combined with the mixed norms (l p,q norm), to obtain the relative balance between correctiveness and conservativeness. The MATLAB simulation results reveal that the improved algorithm can improve robustness under α stable noise when compared with the l p,q group algorithms and it can effectively predict the channel impulse response for a group sparse structure.

Privacy Against a Hypothesis Testing Adversary

Privacy against an adversary (AD) that tries to detect the underlying privacy-sensitive data distribution is studied. The original data sequence is assumed to come from one of the two known distributions, and the privacy leakage is measured by the probability of error of the binary hypothesis test carried out by the AD. A management unit (MU) is allowed to manipulate the original data sequence in an online fashion while satisfying an average distortion constraint. The goal of the MU is to maximize the minimal type II probability of error subject to a constraint on the type I probability of error assuming an adversarial Neyman–Pearson test, or to maximize the minimal error probability assuming an adversarial Bayesian test. The asymptotic exponents of the maximum minimal type II probability of error and the maximum minimal error probability are shown to be characterized by a Kullback–Leibler divergence rate and a Chernoff information rate, respectively. Privacy performances of particular management policies, the memoryless hypothesis-aware policy and the hypothesis-unaware policy with memory, are compared. The proposed formulation can also model adversarial example generation with minimal data manipulation to fool classifiers. At last, the results are applied to a smart meter privacy problem, where the user’s energy consumption is manipulated by adaptively using a renewable energy source in order to hide user’s activity from the energy provider.

Estimation of Data Hiding Capacities in Spatio-Chromatic Fourier Transform (SCFT) Representations of Color Images

Despite the abundance of color images, a vast majority of digital image watermarking algorithms are specifically designed for gray-scale images in terms of perceptual modeling, watermark embedding and data hiding capacity. This paper tries to bridge this gap and provides information theoretic estimates of data hiding capacities for standard color images using an improved version of the CIEL*a*b* color space. These capacity estimates are derived based on appropriate color models of the spatio-chromatic Fourier transform (SCFT) of the host image, distortion constraints imposed on the watermarking game players (watermark embedder and attacker), and side information available to both players. This paper considers full-frame and block-based SCFT statistical representations for color images and derive data hiding bounds for these image sources assuming invariance to affine and geometric attacks. Finally, the estimated data hiding capacities are used to assess the performance of a color-based SCFT image watermarking algorithm. The performance results highlight the importance of combining the estimated hiding capacities and perceptual model in designing a high capacity, robust and perceptual color image watermarking system.

CrowdDBS: A Crowdsourced Brightness Scaling Optimization for Display Energy Reduction in Mobile Video

Mobile display has become one of the most power-hungry components in mobile video viewing. Currently, mobile devices can reduce the display energy by performing dynamic brightness scaling (DBS) under the distortion constraint of video signals. We observe that there is a pitfall preventing current practice from systematic display energy reduction. In particular, existing objective DBS schemes lack direct connection to the subjective human perception on DBS-enabled videos, which is the key to achieving human-centered energy-experience optimization. To overcome this pitfall, we present CrowdDBS, a crowdsourced display energy reduction framework for mobile video viewing. CrowdDBS is empowered by a set of crowdsourcing studies that uncover the relationship between human perception and DBS frequency, magnitude, and temporal consistency, respectively. Motivated by the insights obtained from these studies, CrowdDBS employs a suit of designs and a DBS optimization framework to optimize the energy-experience tradeoff in mobile video viewing. Comprehensive experimental results and user evaluations under a variety of practical settings show that CrowdDBS can achieve 37 percent device energy reduction on average while guaranteeing satisfactory user experience in mobile video viewing.