Source Coding Problem Research Articles

Scalable Joint Source and Channel Coding of Meshes

This paper proposes a new approach for joint source and channel coding (JSCC) of meshes, simultaneously providing scalability and optimized resilience against transmission errors. An unequal error protection approach is followed, to cope with the different error-sensitivity levels characterizing the various resolution and quality layers produced by the input scalable source codec. The number of layers and the protection levels to be employed for each layer are determined by solving a joint source and channel coding problem. In this context, a novel fast algorithm for solving the optimization problem is conceived, enabling a real-time implementation of the JSCC rate-allocation. An instantiation of the proposed JSCC approach is demonstrated for MeshGrid, which is a scalable 3-D object representation method, part of MPEG-4 AFX. In this context, the L-inflnite distortion metric is employed, which is to our knowledge a unique feature in mesh coding. Numerical results show the superiority of the L-inflnite norm over the classical L-2 norm in a JSCC setting. One concludes that the proposed joint source and channel coding approach offers resilience against transmission errors, provides graceful degradation, enables a fast real-time implementation, and preserves all the scalability features and animation capabilities of the employed scalable mesh codec.

On Successive Refinement With Causal Side Information at the Decoders

Consider a process, {Xi Yi,Zi}infin i=1 producing independent copies of a triplet of jointly distributed random variables (RVs). The part of the process - the source - is observed at the encoder, and is supposed to be reproduced at two decoders, decoder 1 and decoder 2, where the {Zi} and the {Yi} parts of the process are observed, respectively, in a causal manner. The communication between the encoder and the decoders is carried out across two memoryless channels in two successive communication stages. In the first stage, the compressed transmission is available to both decoders, but only decoder 1 reconstructs the source (according to the received data stream and its causal side information (SI){Zi}). In the second stage, the second decoder reconstructs the source according to {Yi} and the transmissions of the encoder at both stages. It is desired to find necessary and sufficient conditions such that the distortions incurred (in each stage) will not exceed given thresholds. First, a single-letter characterization of achievable rates is derived for a pure source-coding problem with successive refinement and causal SI at the decoders. Then, for a joint source-channel coding setting, a separation theorem is proved, asserting that in the limit of long blocks, no optimality is lost by first applying lossy successive-refinement source coding, regardless of the channels, and then applying good channel codes to each one of the resulting bit streams, regardless of the source. Next, conditions for a source to be successively refinable in two different senses are established, and finally, it is shown that the binary-symmetric source is successively refinable.

On functional duality in multiuser source and channel coding problems with one-sided collaboration

In this paper, we address duality in a variety of multiuser source and channel coding problems under different scenarios of one-sided inter-terminal collaboration at either the transmitter or at the receiver. First we consider duality between broadcast channel coding and distributed source coding problems. We also consider a new source coding problem in this paper, which we refer to as distributed reconstruction source coding. This problem is closely related to the multiple description source coding problem. We then consider duality between this problem and the multiple access channel coding problem. Our notion of duality in this paper is in a functional sense, where the optimal encoder mapping for a multiuser source coding problem becomes identical to the optimal decoder mapping for the dual multiuser channel coding problem, and vice versa. For ease of illustration we give the formulation only for systems which involve either two encoders or two decoders. Our formulation can be easily extended to the multiuser (more than two noncollaborating terminals) case. We present the precise mathematical conditions under which these encoder-decoder mappings are swappable in the two dual multiuser communication problems, identifying the key roles played by the source distortion and channel cost measures respectively in the multiuser source and channel coding problems in capturing this duality.

Two-way successively refined joint source-channel coding

Consider a source, {X/sub i/,Y/sub i/}/sub i=1//sup /spl infin//, producing independent copies of a pair of jointly distributed random variables (RVs). The {X/sub i/} part of the process is observed at some location, say A, and is supposed to be reproduced at a different location, say B, where the {Y/sub i/} part of the process is observed. Similarly, {Y/sub i/} should be reproduced at location A. The communication between the two locations is carried out across two memoryless channels in K iterative bi-directional rounds. In each round, the source components are reconstructed at the other locations based on the information exchanged in all previous rounds and the source component known at that location, and it is desired to find the amount of information that should be exchanged between the two locations in each round, so that the distortions incurred (in each round) will not exceed given thresholds. Our setting extends the results of Steinberg and Merhav as well as Kaspi, combining the notion of successive refinement with this of two-way interactive communication. We first derive a single-letter characterization of achievable rates for a pure source-coding problem with successive refinement. Then, for a joint source-channel coding setting, we prove a separation theorem, asserting that in the limit of long blocks, no optimality is lost by first applying lossy (two-way) successive-refinement source coding, regardless of the channels, and then applying good channel codes to each one of the resulting bitstreams, regardless of the source.

On dualities in multiterminal coding problems

It has been shown recently that under certain conditions there exist dualities between different multiterminal (MT) source and channel coding problems. Following these results, we study lossless MT source coding and deterministic MT channel coding problems and point out different dualities between them. In particular, we show that there exists a functional between a Slepian-Wolf (SW) coding problem and a deterministic broadcast channel (DBC) coding problem and between a lossless multiple-description (MD) coding problem and a deterministic multiple-access channel (DMAC) coding problem. In analogy to the established between DBC and DMAC coding problems, we further propose a similar between SW and lossless MD coding problems; in this way, we form a closed duality loop of four MT coding problems, which imposes the existence of a single common rate point in the achievable rate regions of all four dual problems. We also consider in zero-error MT coding and shed light on practical code design with an example. Finally, extension to the case with only one lossless/deterministic component in the source/channel coding problem is provided.

On the Error Exponent of Trellis Source Coding

In this paper, we develop a single-letter lower bound on the error exponent for the problem of trellis source coding. We demonstrate that for the case of a binary source with the Hamming distortion measure, and for rates close to the rate-distortion curve, this bound is superior to Marton's block-coding exponent, for the same computational complexity.

The Multiple-Access Channel With Partial State Information at the Encoders

This work studies the multiple-access channel (MAC) controlled by random parameters, with full side information at the decoder, and partial, rate limited, side information (SI) at the encoders. A single-letter characterization of the capacity region is derived, for the special case where the SI is degraded. Here degraded SI refers to the case where the SI available at one of the encoders is a subset of the SI available at the other encoder. Inner and outer bounds are derived on the capacity region of that channel for the general case where there are no restrictions on the structure of the SI at the two encoders. The techniques employed for coding the rate-limited SI, and the achievable regions so obtained, are closely related to the problems of hierarchical source coding, and multiple descriptions.

Efficient fault-diagnosis algorithms for all-optical WDM networks with probabilistic link failures

This paper investigates the fault-diagnosis problem for all-optical wavelength-division-multiplexing (WDM) networks. A family of failure-localization algorithms that exploit the unique properties of all-optical networks is proposed. Optical probe signals are sequentially sent along a set of designed lightpaths, and the network state is inferred from the result of this set of end-to-end measurements. The design objective is to minimize the diagnosis effort (e.g., the average number of probes) to locate failures. By establishing a mathematical equivalence between the fault-diagnosis problem and the source-coding problem in information theory, we obtain a tight lower bound for the minimum average number of probes per edge (of the network modeled as a graph) as H/sub b/(p), the entropy of the individual edges. Using the rich set of results from coding theory to solve the fault-diagnosis problem, it is shown that the "2/sup m/-splitting" probing scheme is optimum for the special case of single failure over a linear network. A class of near-optimum run-length probing schemes that have low computation complexity is then developed. Analytical and numerical results suggest that the average number of probes per edge for the run-length probing scheme is uniformly bounded above by (1+/spl epsiv/)H/sub b/(p) and converges to the entropy lower bound as the failure probability decreases. From an information-theoretic perspective, it is shown that the run-length probing scheme outperforms the greedy probing scheme of the same computational complexity. The investigation reveals a guideline for efficient fault-diagnosis schemes: Each probe should provide approximately 1 bit of information, and the total number of probes required is approximately equal to the entropy of the state of the network. This result provides an insightful guideline to reduce the overhead cost of fault management for all-optical networks and can further the understanding of the relationship between information entropy and network management. Several practical issues are also addressed in the implementation of run-length probing schemes over all-optical WDM networks.

Source Coding Algorithms Using the Randomness of a Past Sequence

We propose source coding algorithms that use the randomness of a past sequence. The proposed algorithms solve the problems of multi-terminal source coding, rate-distortion source coding, and source coding with partial side information at the decoder. We analyze the encoding rate and the decoding error rate in terms of almost-sure convergence.

Rate–Distortion Approach to Databases: Storage and Content-Based Retrieval

This paper investigates the relationship between rate-distortion theory and efficient content-based data retrieval from high-dimensional databases. We consider database design as the encoding of a data object sequence, and retrieval from the database as the decoding of the sequence using side information (i.e., the query) available only at the decoder. We show that, in this setting, the optimal asymptotic tradeoff between the search time R/sub s/ (bits per data object read from the storage device) and the expected search accuracy D/sub s/ (relevance of the retrieved data set) is given by the Wyner-Ziv solution with a side-information-dependent distortion measure. Moreover, the data indexing and retrieval problem is, in general, inseparable from the data compression problem. Data items selected by the search procedure, which can be stored in the disk with a limited total rate of R/sub r/ /spl ges/ R/sub s/, need to be presented at a prescribed expected reconstruction quality D/sub r/. This is, hence, a problem of scalable source coding or refinement, albeit with differing layer distortion measures to quantify search and reconstruction quality, respectively. We derive a single-letter characterization of all achievable quadruples {R/sub s/,R/sub r/,D/sub s/,D/sub r/}, and prove conditions for successive refinability without rate loss. Finally, we show that the special case D/sub s/=D/sub r/=0 is nontrivial and of practical interest in this context, as it can impose acceptable search and reconstruction qualities for each individual data item and for the entire query space with high probability, in contradistinction with standard average distortion requirements. The region of achievable {R/sub s/,R/sub r/} is obtained by adapting Rimoldi's characterization to a new regular scalable coding problem.

Distortion bounds for vector quantizers with finite codebook size

Upper and lower bounds are presented for the distortion of the optimal N-point vector quantizer applied to k-dimensional signals. Under certain smoothness conditions on the source distribution, the bounds are shown to hold for each and every value of N, the codebook size. These results extend bounds derived in the high-resolution limit, which assume that the number of code vectors is arbitrarily large. Two approaches to the upper bound are presented. The first, constructive construction, achieves the correct asymptotic rate of convergence as well as the correct dependence on the source density, although leading to an inferior value for the constant. The second construction, based on a random coding argument, is shown to additionally achieve a value of the constant which is much closer to the best known result derived within the asymptotic theory. Lower bound results derived in the correspondence are again shown to possess the correct asymptotic form and yield a constant which is almost indistinguishable from the best value achieved in the asymptotic regime. Finally, application of the results to the problem of source coding yields upper bounds on the distortion rate function for a wide class of processes.

Distortion-rate bounds for fixed- and variable-rate multiresolution source codes

The source coding theorem for stationary sources describes the optimal performance theoretically achievable by fixed- and variable-rate block quantizers. The source coding theorem may be generalized by considering the problem of multiresolution or successive refinement source coding, which is the topic of this work. Given a distortion vector (D/sub 1/,...,D/sub L/), this work describes the family of achievable rate vectors (R/sub 1/,...,R/sub L/) for describing a stationary source at L resolutions, where the description at the first resolution is given at rate R/sub 1/ and achieves an expected distortion no greater than D/sub 1/, the description at the second resolution includes both the first description and a refining description of rate R/sub 2/ and achieves expected distortion no greater than D/sub 2/, and so on. The work includes performance bounds for both fixed- and variable-rate source codes on discrete-time stationary ergodic sources and discrete-time stationary nonergodic sources for any integer number of resolutions L/spl ges/1. For L=1, the source coding theorems for stationary sources result. For L>1, the results extend previous theorems for discrete-alphabet memoryless sources.

A method to remove variations in source codes

Variations in source codes of computer programs cause difficult problems in source code handling systems, such as program understanding systems. This paper proposes a new method to remove these variations. First, a representation method of programs is discussed. The proposed representation is free from three kinds of variations and it can be easily rewritten. Next, a prototype variation removal system is evaluated. The system utilizes the representation and removes 13 kinds of variations by program rewriting. It has removed 20–83% variations in the source codes implemented by undergraduates and postgraduates.

Fast and efficient coding of information sources

The author considers the problem of source coding and investigates the cases of known and unknown statistics. The efficiency of the compression codes can be estimated by three characteristics: 1) the redundancy (r), defined as the maximal difference between the average codeword length and Shannon entropy in case the letters are generated by a Bernoulli source; 2) the size (in bits) of the encoder and the encoder programs (S) when implemented on a computer; and 3) the average time required for encoding and decoding of a single letter (T). He investigates S and T as a function of r when r/spl rarr/0. All known methods may be divided into two classes. The Ziv-Lempel codes and their variants fall under the first class, and the arithmetic code with the Lynch-Davisson code fall under the second one. The codes from the first class need exponential memory size S=0(exp(1/r)) for redundancy r when r/spl rarr/O. The methods from the second class have a small memory size but a low encoding speed: S=0(1/r/sup const/), T=0(log/sup const/(1/r) log log (1/r)). In this paper, the author presents a code that combines the merits of both classes; the memory size is small and the speed is high: S=0(1/r/sup const/),T=0(log/sup const/(1/r) log log (1/r)).< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An inductive inference procedure to minimize prediction error

Considering inductive inference and deductive inference as not individual processes but a serial process of information processing, the serial inference procedure is studied for two purposes: the compression of observed facts and the prediction of new well-formed formulas. A serial inference process scheme that uses the correspondence to source coding and prediction problems is proposed. The optimal inference procedures for the two purposes are shown in the proposed scheme.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Analysis and further results on adaptive entropy-coded quantization

Buffer underflow and overflow problems associated with entropy coding are completely eliminated by effectively imposing reflecting walls at the buffer endpoints. Synchronous operation of the AECQ (adaptive entropy-coded quantizer) encoder and decoder is examined in detail, and it is shown that synchronous operation is easily achieved without side information. A method is developed to explicitly solve for the buffer-state probability distribution and the resulting average distortion when memoryless buffer-state feedback is used as well as when the source is stationary and memoryless. This method is then used as a tool in the design of low-distortion AECQ systems, with particular attention given to developing source scale-invariant distortion performance. It is shown that the introduction of reflecting buffer walls in a properly designed AECQ system results in a very small rate-distortion performance penalty and that the resulting AECQ system can be an extremely simple and effective solution to the stationary memoryless source-coding problem for a wide range of source types. Operation with nonstationary sources is also examined. >

Reliable transmission of two correlated sources over an asymmetric multiple-access channel (Corresp.)

Necessary and sufficient conditions are derived for the transmission of two arbitrarily correlated sources over a discrete memoryless asymmetric multiple-access channel. It is shown that in this situation the classical separation principle of Shannon (the factorization of the joint source-channel transmission problem into separate source and channel coding problems) applies. This asymmetric case is the first non-trivial situation of a multiple-access channel with arbitrarily correlated sources in which the sufficient conditions found for the reliable transmission of the sources over the channel turn out to be necessary as well. Furthermore, it is demonstrated that these necessary and sufficient conditions continue to hold if feedback is available to one or both of the encoders.

On a Source-Coding Problem with Two Channels and Three Receivers

This paper treats the problem of communicating a memoryless unit-variance Gaussian source to three receivers. Two channels are available, each with a separate receiver. A third receiver has the outputs of both channels available. We obtain an expression for the simultaneously achievable distortions (mean-squared error). This problem applies to the following situation: Assume that high-quality reproduction of a source is desired at a single receiver which is connected to the source over a pair of links operating in parallel. Further assume that the links are unreliable in that either may fail, and that the source encoder is unaware of the failures. One can then ask how “robust” a system designed for this situation can be. That is, what are the limits on the fidelity achievable when both links are functioning if graceful degradation is required during the failure of either link? An inverse relation between performance in the two modes is obtained in the sense that, as performance in the presence of both links approaches its theoretical optimum, average distortion during failures becomes large. Conversely, if near-ideal performance during link failures is desired, then the distortion achieved when both links operate is far from its optimum value.

Recent Progress in Communication Theory in Japan

This paper reviews recent progress in communication theory in Japan, mainly in the past three years. The topics to be discussed include 1) waveform transmission in band-limited channels and active transmission lines, 2) source-coding problems such as optimal quantizing characteristics, delta modulation, and linear transformation, 3) transmission codes, where mainly multilevel codes have been discussed, 4) new information transmission and modulation schemes, and 5) theories of traffic and switchingsystem design.