Autoregressive Sources Research Articles

Bayesian classification with multivariate autoregressive sources that might have different orders

The main objective of this paper is to develop an exact Bayesian technique that can be used to assign a multivariate time series realization to one of several autoregressive sources, with unknown coefficients and precision, that might have different orders. The foundation of the proposed technique is to develop the posterior mass function of a classification vector, in an easy form, using the conditional likelihood function. A multivariate time series realization is assigned to the multivariate autoregressive source with the largest posterior probability. A simulation study, with uniform prior mass function, is carried out to demonstrate the performance of the proposed technique and to test its adequacy in handling the multivariate classification problems. The analysis of the numerical results supports the adequacy of the proposed technique in solving the classification problems with multivariate autoregressive sources.

A bound on the rate of a system for encoding an unknown Gaussian autoregressive source

To obtain the rate distortion function of an autoregressive (AR) source when the source parameters are fixed but unknown, it is common simply to insert the estimated parameters into the known-spectrum rate distortion expressions. This approach, although asymptotically correct, ignores any error in estimating the parameters, ignores distortion incurred by the need to encode the parameters for transmission to the receiver, and does not include a component for the side information. We derive a lower bound to the rate distortion performance of a relatively general system for encoding an unknown Gaussian AR source with respect to the mean-squared error (MSE) fidelity criterion. Asymptotically in the estimator frame length n and in the encoder blocklength N, the lower bound equals the usual Gaussian rate distortion function for small distortions.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Maximum-likelihood localization of narrow-band autoregressive sources via the EM algorithm

The authors derive an expectation-maximization algorithm for the maximum-likelihood estimation of the directions of arrival of multiple narrowband autoregressive (AR) signals embedded in noise. The proposed algorithm simultaneously estimates the location parameters, the AR coefficients, and the signals. The additional structural information that the sources are of the AR type enables one to resolve the case in which the number of sources is equal to or possibly larger than the number of sensors. The authors provide examples demonstrating that the proposed algorithm can resolve two spectrally and spatially closely spaced sources by using a two-sensor array. >

Localization by narrow-band autoregressive sources by passive sensor arrays

The authors present a method for estimating the location of narrowband autoregressive (AR) sources with unknown spectral characteristics. Unique localization of multiple sources by antenna arrays requires that the number of sensors be greater than the number of sources. When additional information is given this is no longer true. The authors assume that the signals structure is of the AR type. They present a localization procedure which consists of two stages: in the first one the coefficients of the AR signals are estimated, and in the second the directions of arrival are estimated. Simulation results demonstrate the performance of the algorithm under different scenarios.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Mismatched DPCM encoding of autoregressive processes

A method for computing the mean squared error distortion of differential pulse code modulation (DPCM) applied to Gaussian autoregressive sources is developed. This extends previous work wherein the code predictor was matched to the source. A two-dimensional version of the projection method for the computation of the stationary distribution of the joint source-state process is developed from which distortion can be readily evaluated. An iterative algorithm is used to optimize the quantizer for a given source and predictor. The results show that the matched predictor is very nearly optimal but not exactly so, and that DPCM is fairly robust to mismatch of the prediction coefficient to the correlation coefficient of a first-order autoregressive source.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

On the continuity of the stationary state distribution of DPCM

Continuity and singularity properties of the stationary state distribution of differential pulse code modulation (DPCM) are explored. Two-level DPCM (i.e. delta modulation) operating on a first-order autoregressive source is considered, and it is shown that, when the magnitude of the DPCM prediction coefficient is between zero and one-half, the stationary state distribution is singularly continuous; i.e. it is not discrete but concentrates on an uncountable set with a Lebesgue measure of zero. Consequently, it cannot be represented with a probability density function. For prediction coefficients with magnitude greater than or equal to one-half, the distribution is pure, i.e. either absolutely continuous and representable with a density function, or singular. This problem is compared to the well-known and still substantially unsolved problem of symmetric Bernoulli convolutions.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Convergence of the projection method for an autoregressive process and a matched DPCM code

The key step in the analysis of differential pulse code modulation (DPCM) is to find the steady-state probability distribution of a random process in terms of which the code distortion can be evaluated. For the case of an autoregressive source and a matched DPCM code, a well-known approximation technique has been used for the evaluation of the steady-state distribution of the prediction error process. However, the validity of this approximation method has not been justified before. A framework in which this approximation technique can be viewed as the projection method for the solution of integral equations is established. Sufficient conditions under which the approximation method can be rigorously justified are obtained.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The Design of Predictive Trellis Waveform Coders Using the Generalized Lloyd Algorithm

Trellis source codes consist of a finite-state machine decoder and a trellis search algorithm, such as the Viterbi algorithm, as the encoder. The encoder experiments with a local copy of the decoder and determines the best channel path map in the sense that it will yield the smallest average distortion between the source sequence and the reproduction sequence given the codebook. In this paper we present a coding system and a design algorithm for predictive trellis coding. Results obtained via simulation are compared for trellis and predictive trellis codes designed for first-order autoregressive sources with Gaussian and Laplacian innovations and for sampled speech. On a random source which models speech, simulation results of the predictive and nonpredictive trellis codes designed by the generalized Lloyd algorithm and those obtained by other researchers are compared. Issues related to computational complexity, the effects of initial codebook selection, training sequence segmentation, search length, channel errors, and algorithm convergence are addressed.

A transform tree code for stationary Gaussian sources

A new tree code is introduced for discrete-time stationary Gaussian sources with hounded, integrable power spectra and the squared-error distortion measure. The codewords in the tree are reconstructions of Karhunen-Loeve transforms of the source words. The branching factor and the number of code letters per branch may vary with level in the tree. A theorem that guarantees the existence of an optimal code for any code rate using such a tree is proved. The proof uses the random coding argument in conjunction with a theorem on survival of a branching process with random environment. A suboptimal but computationally affordable realization of the theorem's coding technique was used for encoding simulations for six autoregressive sources at rates of 1.0, 0.50, 0.25 , and 0.10 bits per source symbol. The average distortion results were generally within 1 dB of the distortion-rate bound but varied widely depending on the source and rate. The results were compared with those for transform quantization simulations for the same sources and rates. The tree code always performed better but only by an average of 0.44 dB all sources and rates. Longer source blocks and more intensive search would certainly improve the performance of the tree codes, but at the expense of extra computation and storage.

Rate-distortion performance of DPCM schemes for autoregressive sources

An analysis of the rate-distortion performance of differential pulse code modulation (DPCM) schemes operating on discrete-time auto-regressive processes is presented. The approach uses an iterative algorithm for the design of the predictive quantizer subject to an entropy constraint on the output sequence. At each stage the iterative algorithm optimizes the quantizer structure, given the probability distribution of the prediction error, while simultaneously updating the distribution of the resulting prediction error. Different orthogonal expansions specifically matched to the source are used to express the prediction error density. A complete description of the algorithm, including convergence and uniqueness properties, is given. Results are presented for rate-distortion performance of the optimum DPCM scheme for first-order Gauss-Markov and Laplace-Markov sources, including comparisons with the corresponding rate-distortion bounds. Furthermore, asymptotic formulas indicating the high-rate performance of these schemes are developed for both first-order Gaussian and Laplacian autoregressive sources.

Fourier Encoding of Closed Planar Boundaries

A circular Gaussian autoregressive (CGAR) source is used as a model for closed planar curves. A class of suboptimal encoding schemes is considered which separately quantize the Fourier coefficients of the boundary. Application of rate-distortion theoretic techniques leads to parametric equations describing the optimal encoding bound. Interpretation of these equations establishes a sampling criterion and a computationally efficient transform encoding scheme for the suboptimal class. Several variants of this transform encoding scheme are suggested and compared to the encoding bound.

Interpolative DPCM

Recent results have established that DPCM is quite inefficient when encoding tightly correlated autoregressive sources at low rates. This work presents a slightly more complex code generator based on interpolation for rate 1 encoding of such sources and evaluates its performance via simulation for Gaussian and Laplacian distributed general stationary autoregressive sources. Improvements in the range of 3 dB over the LMS predictive quantizer for speech-like correlated sources at rate 1 have been realized with no tree searching. Another 1 dB is gained with a moderate tree searching. The code generator is less sensitive to the quantizer step size and a locally mismatched source than DPCM, two desirable features for encod, ing quasi-stationary sources.

A trellis code construction and coding theorem for stationary Gaussian sources (Corresp.)

For stationary discrete-time Gaussian sources and the squared-error distortion measure, a trellis source code is constructed. The encoder consists of a Karhunen-Loeve transform on the source output followed by a search on a trellis structured code, where the decoder is a time-variant nonlinear filter. The corresponding code theorem is proved using the random coding argument. The proof technique follows that of Viterbi and Omura, who proved the trellis coding theorem for memoryless sources. The resultant coding scheme is implementable and applicable at any nonzero rate to a stationary Gaussian source with a bounded and continuous power spectrum. Therefore. for stationary sources, it is more general than Berger's tree coding scheme, which is restricted to autoregressive Gaussian sources in a region of high rate (low distortion).

Low-rate tree coding of autoregressive sources (Corresp.)

The performance of differential pulse-code modulation and random codes is evaluated experimentally for a range of autoregressive sources, including Gaussian and Laplacian sources of orders <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1, 2</tex> , and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</tex> encoded at rates <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1, 2</tex> , and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">7</tex> . Correlations are typical of those encountered in speech and images. Results show that the gain from a fixed moderate degree of code searching increases as the rate decreases, and increases consistently with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1 /D_{o}</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{o}</tex> is the Berger-Gray critical distortion. Laplacian and Gaussian sources behave similarly. Random codes perform better than DPCM codes for lightly correlated Laplacian sources but are otherwise worse.

Adaptive decision-feedback predictive coding of digital sources

A 1-bit adaptive decision-feedback predictive coding (ADFPC) system is derived on the assumption of ‘good’ decisions. The predictor in the ADFPC is made adaptive by projecting the gradient, at each iteration, along the co-ordinate which exhibits the maximum absolute gradient. This gradient-seeking method requires that the stepsize only needs to be optimum with respect to a single coefficient (co-ordinate) during each iteration. A formula for calculating the stepsize at each iteration is obtained via a minimisation of the squared prediction error. Simulation results obtained in the encoding of a 1st-order autoregressive source and a monochrome image (with the analogue sources digitised by a 6-bit quantiser prior to encoding) reveal that the ADFPC is robust and exhibits good performance.

Combinatorial properties of good codes for binary autoregressive sources (Corresp.)

Let M be a binary autoregressive source to be encoded within a specified Hamming distortion \delta . A binary n -tuple is called \sigma -central if it is at distance \leq n(\delta + \sigma) from at least 2^{nH(\delta - \sigma)} typical sequences produced by the source M . It is first shown that, in the region where the Shannon rate-distortion bound is achieved, there exist good codes consisting only of \sigma -central words. Next, the characterization problem is studied; the basic conjecture is that a central sequence is well-characterized by its level, which is the Hamming weight of an image sequence. The problem is solved for the memoryless source. In general, if N(k,r) is defined to be the mean number of typical n -tuples at distance \leq r = n \delta from the n -tuples of level k=n \xi , then it is shown that n^{-l} \log N(k,r) becomes arbitrarily close to H(\delta) for an explicitly determined unique value of \xi .

A Fake Process Approach to Data Compression

The problem of designing a good decoder for a timeinvariant tree-coding data compression system is equivalent to that of finding a good low rate "fake process" for the original source, where the fake is produced by a time-invariant nonlinear filtering of an independent, identically distributed sequence of uniformly distributed discrete random variables and "goodness" is measured by the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\bar{\rho}</tex> -distance between the fake and the original source. Several simple ad hoc techniques for obtaining such fake processes are introduced and shown by simulation to provide an improvement of typically 1-2 dB over optimum quantization, delta modulation, and predictive quantization for one-bit per symbol compression of Gaussian memoryless, autoregressive, and moving average sources. In addition, the fake process viewpoint provides a new intuitive explanation of why delta modulation and predictive quantization work as well as they do on Gaussian autoregressive sources.

Intraframe Sequential Picture Coding

This paper generalizes time-discrete autoregressive source coding results of rate-distortion theory to two dimensions. A 2-D discrete autoregressive source is defined and shown to produce a 2-D widesense Markovian field. The rate distortion function of the source is then obtained under assumption of Gaussian field statistics and a squared error fidelity criterion. A procedure for generating an ensemble of 2-D codewords whose statistics satisfy the variational equations for R(D) is given. These 2-D codewords are, by space-time mappings, 1-D tree codes, and it is noted that a tree coding theorem of Jelinek, Berger, Davis and Hellman applies. The problem of instrumenting nearly optimum 2-D sequential encoding is discussed briefly. The paper stresses potential application to image coder design.

Tree encoding of speech

Recently developed methods of tree source coding with a fidelity criterion are applied to speech coding. We first demonstrate that tree codes are inherent in A-D speech convertors of the waveform following type and point to ordinary and adaptive delta modulation and differential pulse code modulation (DPCM) as examples. Insights of coding theory improve these trees at low rates; we offer two new code classes, one obtained by smoothing the DPCM tree and one using the rate-distortion theory of autoregressive sources. Using these codes, we study the performance of a simple synchronous tree searching algorithm called the M -algorithm that maintains a small fixed number of paths in contention. 1 and 2 bit/sample code trees, used to encode actual speech at 8, 10, and 16 kbits/s, yield improved dynamic range and channel error resistance, and 4-8 dB improvement in mean-square error (mse) over ordinary single-path searched DPCM. These improvements in excess of analytical estimates suggest that tree coding methods perform better with real-life sources than previously thought.

Two-dimensional Bayesian estimate of images

A dynamic model for pictorial data that can be represented by a random field of an exponential autocorrelation function is developed. A partial difference equation describes the dynamic model and is used to realize a two-dimensional recursive filter that gives a Bayesian-estimate of the pictorial data from a noisy observation of the data. It is assumed that the noise is additive, white, and uncorrelated with the signal. Practical application of the estimation technique is illustrated by applying the results to enhance several pictures. A comparison of this technique and its one-dimensional counterpart (Kalman filter) is made, and generalization of the estimation technique to other autoregressive sources is considered.