Noncausal Estimation Research Articles

Resolvent-based optimal estimation of transitional and turbulent flows

Abstract

Generalized Savitzky–Golay filters for identification of nonstationary systems

The problem of identification of nonstationary systems using noncausal estimation schemes is considered and a new class of identification algorithms, combining the basis functions approach with local estimation technique, is described. Unlike the classical basis function estimation schemes, the proposed local basis function estimators are not used to obtain interval approximations of the parameter trajectory, but provide a sequence of point estimates corresponding to consecutive instants of time. Based on the results of theoretical analysis conducted for nonstationary finite impulse response systems the paper proposes two mechanisms for adaptive selection of the number of basis functions and the size of the local analysis window.

Universities and Intergenerational Social Mobility in Brazil: Examining Patterns by Race and Gender

This paper analyzes social mobility as realized by students of a high-quality public flagship university in Brazil, the Federal University of Pernambuco (UFPE), and compares with mobility at US institutions, applying the methodology of Chetty et al. (2017). Intergenerational income mobility is analyzed using the family income of students matriculating to UFPE in 2005–2006 and individual earnings 12–13 years later. Upward mobility is defined as the percentage of students who attain the highest quintile of individual earnings among those who matriculated from the lowest income families. We find that mobility rates are higher at UFPE than at comparator US universities as calculated by Chetty et al. (2017). While these are non-causal estimates, they nonetheless suggest that public universities can play a key role in facilitating upward social mobility in Brazil. Disaggregating by gender, we find higher mobility rates for men than for women in both UFPE and US comparator institutions. Using UFPE admissions data, we are able to explore the role of both ability and major choice on mobility gaps by gender and race. For both women and Afro-Brazilians, the proxy for ability (college entry exam) does not explain the gap in reaching the top earnings quintile compared to white males. However, the choice of major is found to be an important factor in limiting mobility for these demographic groups.

On Noncausal Identification of Nonstationary Multivariate Autoregressive Processes

The problem of identification of nonstationary multivariate autoregressive processes using noncausal local estimation schemes is considered and a new approach to joint selection of the model order and the estimation bandwidth is proposed. The new selection rule, based on evaluation of pseudoprediction errors, is compared with the previously proposed one, based on the modified Akaike's final prediction error criterion.

New results on estimation bandwidth adaptation

The problem of identification of a nonstationary autoregressive signal using non-causal estimation schemes is considered. Noncausal estimators can be used in applications that are not time-critical, i.e., do not require real-time processing. A new adaptive estimation bandwidth selection rule based on evaluation of pseudoprediction errors is proposed, allowing one to adjust tracking characteristics of noncausal estimators to unknown and/or time-varying degree of signal nonstationary. The new rule is compared with the previously proposed one, based on the generalized Akaike’s final prediction error criterion.

Optimal Enforcement of Causality in Non-Parametric Transfer Function Estimation

Traditionally, non-parametric impulse and frequency response functions are estimated by taking the ratio of power spectral density estimates. However, this approach may often lead to non-causal estimates. In this letter, we derive a closed form expression for the impulse response estimator by smoothed empirical transfer function estimate, which allows optimal enforcement of causality on non-parametric estimators based on spectral analysis. The new method is shown to be asymptotically unbiased and of minimum covariance in a positive semidefinite sense among a broad class of linear estimators. Numerical simulations illustrate the performance of the new estimator.

Explicit Causal Recursive Estimators for Continuous-Time Bivariate Markov Chains

A bivariate Markov chain comprises a pair of random processes which are jointly Markov. In this paper, both processes are assumed to be continuous-time with finite state space. One of the two processes is observable, while the other is an underlying process which affects the statistical properties of the observable process. Neither the observable, nor the underlying process , is required to be a Markov chain. Examples of bivariate Markov chains include the Markov modulated Markov process (MMMP), the Markov modulated Poisson process (MMPP), and the batch Markovian arrival process (BMAP). We develop explicit causal recursions for estimating the number of jumps from one state to another, and the total sojourn time in each state, of a general bivariate Markov chain. Explicit causal recursions of these statistics were previously developed for the MMMP and the MMPP using the transformation of measure approach. We argue that this approach cannot be extended to a general bivariate Markov chain. Instead, we modify the approach developed by Ryden for noncausal estimation of the same statistics of an MMPP, and use the state augmentation approach of Zeitouni and Dembo and a matrix recursion from Stiller and Radons, to derive the causal recursions. The new recursions do not require any numerical integration or sampling scheme of the continuous-time bivariate Markov chain.

Estimation With a Helper Who Knows the Interference

We consider the problem of estimating a signal corrupted by independent interference with the assistance of a cost-constrained helper who knows the interference causally or noncausally. When the interference is known causally, we characterize the minimum distortion incurred in estimating the desired signal. In the noncausal case, we present a general achievable scheme for discrete memoryless systems and novel lower bounds on the distortion for the binary and Gaussian settings. Our Gaussian setting coincides with that of assisted interference suppression introduced by Grover and Sahai. Our lower bound for this setting is based on the relation recently established by Verdu between divergence and minimum mean squared error. We illustrate with a few examples that this lower bound can improve on those previously developed. Our bounds also allow us to characterize the optimal distortion in several interesting regimes. Moreover, we show that causal and noncausal estimation are not equivalent for this problem. Finally, we consider the case where the desired signal is also available at the helper. We develop new lower bounds for this setting that improve on those previously developed, and characterize the optimal distortion up to a constant multiplicative factor for some regimes of interest.

Likelihood Ratios and Inference for Poisson Channels

In recent years, infinite-dimensional methods have been introduced for Gaussian channel estimation. The aim of this paper is to study the application of similar methods to Poisson channels. In particular, we compute the noncausal conditional mean estimator of a Poisson channel using the likelihood ratio and the discrete Malliavin gradient. This algorithm is suitable for numerical implementation via the Monte-Carlo scheme. As an application, we provide a new proof of a very deep and remarkable formula in Information Theory obtained recently in the literature and relating the derivatives of the input-output mutual information of a general Poisson channel and the conditional mean estimator of the input regardless the distribution of the latter. The use of the aforementioned stochastic analysis techniques allows us to extend these results to more general channels such as mixed Gaussian-Poisson channels.

Causal binary mask estimation for speech enhancement using sparsity constraints

While most single-channel noise reduction algorithms fail to improve speech intelligibility, the ideal binary mask (IBM) has demonstrated substantial intelligibility improvements for both normal- and impaired-hearing listeners. However, this approach exploits oracle knowledge of the target and interferer signals to preserve only the time-frequency regions that are target-dominated. Single-channel noise suppression algorithms trying to approximate the IBM using locally estimated signal-to-noise ratios without oracle knowledge have had limited success. Thought of in another way, the IBM exploits the disjoint placement of the target and interferer in time and frequency to create a time-frequency signal representation that is more sparse (i.e., has fewer non-zeros). In recent work (in preparation for ICASSP 2013), we have introduced a novel time-frequency masking algorithm based on a sparse approximation algorithm from the signal processing literature. However, the algorithm employs a non-causal estimator. The present work introduces an improved de-noising algorithm that uses more realistic frame-based (causal) computations to estimate a binary mask.

The Relationship Between Causal and Noncausal Mismatched Estimation in Continuous-Time AWGN Channels

A continuous-time finite-power process with distribution P is observed through an AWGN channel, at a given signal-to-noise ratio (SNR), and is estimated by an estimator that would have minimized the mean-square error if the process had distribution Q. We show that the causal filtering mean-square error (MSE) achieved at SNR level snr is equal to the average value of the noncausal (smoothing) MSE achieved with a channel whose SNR is chosen uniformly distributed between 0 and snr. Emerging as the bridge for equating these two quantities are mutual information and relative entropy. Our result generalizes that of Guo, Shamai, and Verdu (2005) from the nonmismatched case, where P = Q, to general P and Q. Among our intermediate results is an extension of Duncan's theorem, that relates mutual information and causal MMSE, to the case of mismatched estimation. Some further extensions and implications are discussed. Key to our findings is the recent result of Verdu on mismatched estimation and relative entropy.

Fixed-Interval Optimal Smoothing for Linear Impulsive Systems

In this paper we study the fixed-interval optimal smoothing problems for a class of linear continuous-time systems with impulsive effects. We adopt a maximum likelihood (ML) approach and derive a maximum likelihood noncausal estimator on the fixed time interval. We derive an extension of the Fraser's algorithm for the linear discrete-time smoother to the impulsive systems utilizing the filtering theory of the impulsive systems.

Optimal and suboptimal smoothing algorithms for identification of time-varying systems with randomly drifting parameters

Noncausal estimation algorithms, which involve smoothing, can be used for off-line identification of nonstationary systems. Since smoothing is based on both past and future data, it offers increased accuracy compared to causal (tracking) estimation schemes, incorporating past data only. It is shown that efficient smoothing variants of the popular exponentially weighted least squares and Kalman filter-based parameter trackers can be obtained by means of backward-time filtering of the estimates yielded by both algorithms. When system parameters drift according to the random walk model and the adaptation gain is sufficiently small, the properly tuned two-stage Kalman filtering/smoothing algorithm, derived in the paper, achieves the Cramér–Rao type lower smoothing bound, i.e. it is the optimal noncausal estimation scheme. Under the same circumstances performance of the modified exponentially weighted least-squares algorithm is often only slightly inferior to that of the Kalman filter-based smoother.

Optimal and Suboptimal Smoothing Algorithms for Identification of Time-varying Systems with Randomly Drifting Parameters

Noncausal estimation algorithms, which involve smoothing, can be used for off-line identification of nonstationary systems. Since smoothing is based on both past and future data, it offers increased accuracy compared to causal (tracking) estimation schemes, incorporating past data only. It is shown that efficient smoothing variants of the popular exponentially weighted least squares and Kalman filter based parameter trackers can be obtained by means of backward-time filtering of the estimates yielded by both algorithms. When system parameters drift according to the random walk model, the properly tuned two-stage Kalman filtering/smoothing algorithm, derived in the paper, achieves the Cramér-Rao type lower smoothing bound, i.e. it is the optimal noncausal estimation scheme. Under the same circumstances performance of the modified exponentially weighted least squares algorithm is often only slightly inferior to that of the Kalman filter based smoother.

On the lower smoothing bound in identification of time-varying systems

In certain applications of nonstationary system identification the model-based decisions can be postponed, i.e. executed with a delay. This allows one to incorporate in the identification process not only the currently available information, but also a number of “future” data points. The resulting estimation schemes, which involve smoothing, are not causal. Assuming that the infinite observation history is available, the paper establishes the lower steady-state estimation bound for any noncausal estimator applied to a linear system with randomly drifting coefficients (under Gaussian assumptions). This lower bound complements the currently available one, which is restricted to causal estimators.

Relaxed statistical model for speech enhancement and a priori SNR estimation

In this paper, we propose a statistical model for speech enhancement that takes into account the time-correlation between successive speech spectral components. It retains the simplicity associated with the Gaussian statistical model, and enables the extension of existing algorithms to noncausal estimation. The sequence of speech spectral variances is a random process, which is generally correlated with the sequence of speech spectral magnitudes. Causal and noncausal estimators for the a priori SNR are derived in agreement with the model assumptions and the estimation of the speech spectral components. We show that a special case of the causal estimator degenerates to a decision-directed estimator with a time-varying frequency-dependent weighting factor. Experimental results demonstrate the improved performance of the proposed algorithms.

On Mutual Information, Likelihood Ratios, and Estimation Error for the Additive Gaussian Channel

This paper considers the model of an arbitrary distributed signal x observed through an added independent white Gaussian noise w, y=x+w. New relations between the minimal mean square error of the non-causal estimator and the likelihood ratio between y and \omega are derived. This is followed by an extended version of a recently derived relation between the mutual information I(x;y) and the minimal mean square error. These results are applied to derive infinite dimensional versions of the Fisher information and the de Bruijn identity. The derivation of the results is based on the Malliavin calculus.

Speech enhancement using super-Gaussian speech models and noncausal a priori SNR estimation

A priori signal-to-noise ratio (SNR) estimation is of major consequence in speech enhancement applications. Recently, we introduced a noncausal recursive estimator for the a priori SNR based on a Gaussian speech model, and showed its advantage compared to using the decision-directed estimator. In particular, noncausal estimation facilitates a distinction between speech onsets and noise irregularities. In this paper, we extend our noncausal estimation approach to Gamma and Laplacian speech models. We show that the performance of noncausal estimation, when applied to the problem of speech enhancement, is better under a Laplacian model than under Gaussian or Gamma models. Furthermore, the choice of the specific speech model has a smaller effect on the enhanced speech signal when using the noncausal a priori SNR estimator than when using the decision-directed method.

Speech Enhancement Using a Noncausal A Priori SNR Estimator

We propose a noncausal estimator for the a priori signal-to-noise ratio (SNR), and a corresponding noncausal speech enhancement algorithm. In contrast to the decision-directed estimator of Ephraim and Malah (1984), the noncausal estimator is capable of discriminating between speech onsets and noise irregularities. Onsets of speech are better preserved, while a further reduction of musical noise is achieved. Experimental results show that the noncausal estimator yields a higher improvement in the segmental SNR, lower log-spectral distortion, and better Perceptual Evaluation of Speech Quality scores (PESQ, ITU-T P.862).

Minimax estimation of unknown deterministic signals in colored noise

The estimation of a deterministic signal corrupted by random noise is considered. The strategy is to find a linear noncausal estimator which minimizes the maximum mean square error over an a priori set of signals. This signal set is specified in terms of frequency/energy constraints via the discrete Fourier transform. Exact filter expressions are given for the case of additive white noise. For the case of additive colored noise possessing a continuous power spectral density, a suboptimal filter is derived whose asymptotic performance is optimal. Asymptotic expressions for the minimax estimator error are developed for both cases. The minimax filter is applied to random data and is shown to solve asymptotically a certain worst-case Wiener filter problem. >