Autoregressive Sources Research Articles

Dynamic deconvolution and identification of independent autoregressive sources

We consider a multi‐variate system , where the unobserved components are independent AR(1) processes and the number of sources is greater than the number of observed outputs. We show that the mixing matrix , the AR(1) coefficients and distributions of can be identified (up to scale factors of ), which solves the dynamic deconvolution problem. The proof is constructive and allows us to introduce simple consistent estimators of all unknown scalar and functional parameters of the model. The approach is illustrated by an estimation and identification of the dynamics of unobserved short‐ and long‐run components in a time series. Applications to causal models with structural innovations are also discussed, such as the identification in error‐in‐variables models and causal mediation models.

Tracking Unstable Autoregressive Sources Over Discrete Memoryless Channels

We consider the problem of tracking, in realtime, an unstable autoregressive (AR) source over a discrete memoryless channel (DMC). We present computable achievable bounds on the optimal tracking error for general DMCs, and we particularize these bounds to the binary erasure, packet erasure, and binary symmetric channels. The achievable bounds in this paper are proved using a partially separate source quantization and channel coding architecture. We do not use complete or strict separation in usual Shannon sense: 1) the quantiser’s resolution is optimized against the error-correction capabilities of the channel code and the channel code is optimized against an AR Hamming distortion function matched to the source (a weighted Hamming distortion function that provides unequal error protection to different parts of the AR source). The achievability results for general DMCs are proved by combining the AR Hamming distortion function with new realtime (streaming) versions of the random coding union and dependence testing bounds . When applied to erasure channels, these general bounds combine with simple converses to demonstrate that the channel’s cutoff rate plays an important role in realtime tracking.

A Low-Complexity Analog Linear Coding Scheme for Transmitting Asymptotically WSS AR Sources

In this letter, we give a low-complexity analog linear coding scheme for transmitting finite-length data blocks of asymptotically wide sense stationary (AWSS) autoregressive (AR) sources through additive white Gaussian noise (AWGN) channels.

Efficient blind source extraction of noisy mixture utilising a class of parallel linear predictor filters

This study presents a novel blind source extraction of a noisy mixture using a class of parallel linear predictor filters. Analysis of a noisy mixture equation is carried out to address new autoregressive source signal model based on the covariance matrix of the whitened data. A method of interchanging the rules of filter inputs is proposed such that this matrix becomes the filter input while the estimated source signals are considered as the parallel filter coefficients. As the matrix has unity norm and unity eigenvalues, the filter becomes independent on the mixture signal norm and eigenvalues variations, thus solving drastically the ambiguity due to the dependency of the filter on the mixture power levels if the mixture is considered as the filter input. Furthermore, the unity eigenvalues of the matrix result in a very fast convergence in two iterations. Simulation results show that the model is capable of extracting the unknown source signals and removing noise when the input signal-to-noise ratio is varied from −20 to 80 dB.

Rate-Distortion Function Upper Bounds for Gaussian Vectors and Their Applications in Coding AR Sources.

In this paper, we give upper bounds for the rate-distortion function (RDF) of any Gaussian vector, and we propose coding strategies to achieve such bounds. We use these strategies to reduce the computational complexity of coding Gaussian asymptotically wide sense stationary (AWSS) autoregressive (AR) sources. Furthermore, we also give sufficient conditions for AR processes to be AWSS.

Energy-Efficient Resource Allocation for MIMO-OFDM Systems Serving Random Sources With Statistical QoS Requirement

This paper optimizes resource allocation that maximizes the energy efficiency (EE) of wireless systems with statistical quality of service (QoS) requirement, where a delay bound and its violation probability need to be guaranteed. To avoid wasting energy when serving random sources over wireless channels, we convert the QoS exponent, a key parameter to characterize statistical QoS guarantee under the framework of effective bandwidth and effective capacity, into multi-state QoS exponents dependent on the queue length. To illustrate how to optimize resource allocation, we consider multi-input-multi-output orthogonal frequency division multiplexing (MIMO-OFDM) systems. A general method to optimize the queue length based bandwidth and power allocation (QRA) policy is proposed, which maximizes the EE under the statistical QoS constraint. A closed-form optimal QRA policy is derived for massive MIMO-OFDM system with infinite antennas serving the first order autoregressive source. The EE limit obtained from infinite delay bound and the achieved EEs of different policies under finite delay bounds are analyzed. Simulation and numerical results show that the EE achieved by the QRA policy approaches the EE limit when the delay bound is large, and is much higher than those achieved by existing policies considering statistical QoS provision when the delay bound is stringent.

Separation of Dependent Autoregressive Sources Using Joint Matrix Diagonalization

This letter proposes a novel technique for the blind separation of autoregressive (AR) sources. The latter relies on the joint diagonalization (JD) of appropriate AR matrix coefficients of the observed signals and can be applied to the separation of statistically dependent sources. The developed algorithm is referred to as ‘DARSS-JD’ (for Dependent AR Source Separation using JD). Through the simulation experiments, DARSS-JD is shown to overcome existing second order separation methods with a relatively moderate computational cost.

Single-channel separation using underdetermined blind autoregressive model and least absolute deviation

Abstract A novel “artificial stereo” mixture is proposed to resemble a synthetic stereo signal for solving the signal-channel blind source separation (SCBSS) problem. The proposed SCBSS framework takes the advantages of the following desirable properties: one microphone; no training phase; no parameter turning; independent of initialization and a priori data of the sources. The artificial stereo mixture is formulated by weighting and time-shifting the single-channel observed mixture. Separability analysis of the proposed mixture model has also been elicited to examine that the artificial stereo mixture is separable. For the separation process, mixing coefficients of sources are estimated where the source signals are modeled by the autoregressive process. Subsequently, a binary time–frequency mask can then be constructed by evaluating the least absolute deviation cost function. Finally, experimental testing on autoregressive sources has shown that the proposed framework yields superior separation performance and is computationally very fast compared with existing SCBSS methods.

On Construction and Simulation of Autoregressive Sources With Near-Laplace Marginals

In this paper, we focus upon the problem of modeling and simulation of stationary non-Gaussian time series. In particular, we consider a first order autoregressive process whose marginal distribution is close to the Laplace density. This model allows us to simulate correlated non-Gaussian signals typically appearing in speech analysis, compression, and noise synthesis. The Monte Carlo rejection method is applied to develop efficient algorithms for simulation of the proposed autoregressive process. We also extend our theory and algorithms to the related issue of constructing a correlated bivariate time-series model with near-Laplace margins. A theoretical analysis of the average complexity of the proposed simulation algorithms is included.

On Predictive Coding for Erasure Channels Using a Kalman Framework

We present a new design method for robust low-delay coding of auto-regressive (AR) sources for transmission across erasure channels. The method is based on Linear Predictive Coding (LPC) with Kalman estimation at the decoder. The method designs the encoder and decoder offline through an iterative algorithm based on minimization of the trace of the decoder state error covariance. The design method applies to stationary AR sources of any order. Simulation results show considerable performance gains, when the transmitted quantized prediction errors are subject to loss, in terms of Signal-to-Noise Ratio (SNR) compared to the same coding framework optimized for no loss. We furthermore investigate the impact on decoding performance when channel losses are correlated. We find that the method still provides substantial improvements in this case despite being designed for i.i.d. losses.

A Note on Rate-Distortion Functions for Nonstationary Gaussian Autoregressive Processes

Source coding theorems and Shannon rate-distortion functions were studied for the discrete-time Wiener process by Berger and generalized to nonstationary Gaussian autoregressive processes by Gray and by Hashimoto and Arimoto. Hashimoto and Arimoto provided an example apparently contradicting the methods used in Gray, implied that Gray's rate-distortion evaluation was not correct in the nonstationary case, and derived a new formula that agreed with previous results for the stationary case and held in the nonstationary case. In this correspondence it is shown that the rate-distortion formulas of Gray and Hashimoto and Arimoto are in fact consistent and that the example of Hashimoto and Arimoto does not form a counterexample to the methods or results of the earlier paper. Their results do provide an alternative, but equivalent, formula for the rate-distortion function in the nonstationary case and they provide a concrete example that the classic Kolmogorov formula differs from the autoregressive formula when the autoregressive source is not stationary. Some observations are offered on the equality of the asymptotic distributions of the eigenvalues of the sequence of inverse autocorrelation matrices of possibly nonstationary autoregressive processes and of their Toeplitz approximations.

Non-combinatorial estimation of independent autoregressive sources

Identification of mixed independent subspaces is thought to suffer from combinatorial explosion of two kinds: the minimization of mutual information between the estimated subspaces and the search for the optimal number and dimensions of the subspaces. Here we show that independent autoregressive process analysis, under certain conditions, can avoid this problem using a two-phase estimation process. We illustrate the solution by computer demonstration.

Dynamic rate shaping of compressed digital video

We discuss new theoretical and experimental results on the dynamic rate shaping (DRS) approach for transcoding compressed video bitstreams (MPEG-1, MPEG-2, MPEG-4, H.261, as well as JPEG). We analyze the behavior of DRS assuming a first order autoregressive source. We propose a set of low complexity algorithms for both constrained and unconstrained DRS and substantiate the almost-optimal experimental performance of the memoryless algorithm by assuming a first order autoregressive source. By deriving the statistical and rate-distortion characteristics of different components of the interframe rate shaping problem, we offer an explanation as to why the set of optimal breakpoint values for any frame is somewhat invariant to the accumulated motion compensated shaping error from past frames. We also present an extensive experimental study on the various DRS algorithms (causally optimal, memoryless, and rate-based) both in their constrained and generalized forms. The study proves the computational viability of the DRS approach to transcoding and identifies a range of rate shaping ratios for which it is better than requantization, both complexity-wise as well as in performance. This result is significant in that it opens up the way to construct much simpler memoryless algorithms that give minimal penalty in achieved quality, not just for this but possibly other types of algorithms. This is also the very first use of matrix perturbation theory for tracking the spectral behavior of the autocorrelation matrix of the source signal and the motion residual it yields.

Mean Buffer Contents in Discrete-Time Single-Server Queues with Heterogeneous Sources

We consider discrete-time single-server queues fed by independent, heterogeneous sources with geometrically distributed idle periods. While being active, each source generates some cells depending on the state of the underlying Markov chain. We first derive a general and explicit formula for the mean buffer contents in steady state when the underlying Markov chain of each source has finite states. Next we show the applicability of the general formula to queues fed by independent sources with infinite-state underlying Markov chains and discrete phase-type active periods. We then provide explicit formulas for the mean buffer contents in queues with Markovian autoregressive sources and greedy sources. Further we study two limiting cases in general settings, one is that the lengths of active periods of each source are governed by an infinite-state absorbing Markov chain, and the other is the model obtained by the limit such that the number of sources goes to infinity under an appropriate normalizing condition. As you will see, the latter limit leads to a queue with (generalized) M/G/? input sources. We provide sufficient conditions under which the general formula is applicable to these limiting cases.

Separation of an Instantaneous Mixture of Gaussian Autoregressive Sources by the Exact Maximum Likelihood Approach

This paper deals with the problem of blind separation of an instantaneous mixture of Gaussian autoregressive sources, without additive noise, by the exact maximum likelihood approach. The maximization of the likelihood function is divided, using relaxation, into two suboptimization problems, solved by relaxation methods as well. The first one consists of the estimation of the separating matrix when the autoregressive structure of the sources is fixed. The second one aims at estimating this structure when the separating matrix is fixed. We show that the first problem is equivalent to the determinant maximization of the separating matrix under nonlinear constraints. We prove the existence and the consistency of the maximum likelihood estimator. We also give the expression of Fisher's information matrix. Then, we study, by computer simulations, the performance of our estimator and show the improvement of its achievements w.r.t. both quasimaximum likelihood and second-order blind identification (SOBI) estimators.

Theoretical foundations of transform coding

Discusses various aspects of transform coding, including: source coding, constrained source coding, the standard theoretical model for transform coding, entropy codes, Huffman codes, quantizers, uniform quantization, bit allocation, optimal transforms, transforms visualization, partition cell shapes, autoregressive sources, transform optimization, synthesis transform optimization, orthogonality and independence, and departures form the standard model.

Bayesian Classification of Multivariate Autoregressive Sources with Unknown Order. By: Samir M. Shaarawy and Ahmed L. Daif and Mona A. EL Shafy

Bayesian Classification of Multivariate Autoregressive Sources with Unknown Order. By: Samir M. Shaarawy and Ahmed L. Daif and Mona A. EL Shafy

Hyperspectral image compression using entropy-constrained predictive trellis coded quantization

A training-sequence-based entropy-constrained predictive trellis coded quantization (ECPTCQ) scheme is presented for encoding autoregressive sources. For encoding a first-order Gauss-Markov source, the mean squared error (MSE) performance of an eight-state ECPTCQ system exceeds that of entropy-constrained differential pulse code modulation (ECDPCM) by up to 1.0 dB. In addition, a hyperspectral image compression system is developed, which utilizes ECPTCQ. A hyperspectral image sequence compressed at 0.125 b/pixel/band retains an average peak signal-to-noise ratio (PSNR) of greater than 43 dB over the spectral bands.

Bias correction in effective bandwidth estimation

Abstract Call admission in ATM networks involves a trade-off between ensuring an adequate quality of service to users and exploiting the scale efficiencies of statistical multiplexing. Achieving a good trade-off requires some knowledge of the source traffic. Its effective bandwidth has been proposed as a measure that captures characteristics which are relevant to quality of service provisioning. The effective bandwidth of a source is not known a priori, but needs to be estimated from an observation of its output. We show that direct estimators that have been proposed for this purpose are biased when the source traffic is autocorrelated. By explicitly computing the bias for auto-regressive and Markov sources, we devise a bias correction scheme that does not require knowledge of the model parameters. This is achieved by exploiting a scaling property of the bias that is insensitive to model parameters, and that has the same form for both auto-regressive and Markov sources. This leads us to conjecture that the scaling property may be valid in greater generality and can be used to obtain unbiased effective bandwidth estimates for real traffic. Use of our bias correction technique enables us to obtain accurate estimates of effective bandwidths using relatively short block lengths. The latter is important both because the variance of the estimator increases with the block length, and because real traffic may well be non-stationary, requiring that estimates be obtained from short data records.

Information-theoretic performance of quadrature mirror filters

Most existing quadrature mirror filters (QMFs) closely match the derived closed-form expression for an efficient class of QMFs. We use the closed-form expressions to derive the relationship between information-theoretic loss and the frequency selectivity of the QMF, by calculating first-order entropy as well as rate-distortion theoretic performance of a two band QMF system. We find that practical QMFs do not suffer a significant information-theoretic loss with first-order autoregressive Gaussian sources. With second-order autoregressive sources we find that practical QMFs suffer a notable information-theoretic loss when the bandwidth of the source is extremely narrow, but incur a small loss when the bandwidth is wider. We suggest that our results broadly apply to higher order autoregressive sources as well.