Mean Square Error Sense Research Articles

Asymptotic Properties of Generalized Cross Validation Estimators for Regularized System Identification

In this paper, we study the asymptotic properties of the generalized cross validation (GCV) hyperparameter estimator and establish its connection with the Stein’s unbiased risk estimators (SURE) as well as the mean squared error (MSE). It is shown that as the number of data goes to infinity, the GCV has the same asymptotic property as the SURE does and both of them converge to the best hyperparameter in the MSE sense. We illustrate the efficacy of the result by Monte Carlo simulations.

Mean‐square error constrained approach to robust stochastic iterative learning control

A Kalman filtering-based robust iterative learning control algorithm is proposed in this study for linear stochastic systems with uncertain dynamics and unknown noise statistics. Firstly, a learning gain matrix is designed for the nominal case by minimising the trace of the mean-square matrix of the input tracking error. Theoretical results show that the proposed algorithm guarantees not only the asymptotic but also monotonic convergence of the input tracking error in the mean-square error sense, especially when random noises are Gaussian distributed the proposed algorithm is further proved to be asymptotically efficient. In addition, a new mean-square error constrained approach is presented in designing the robust learning gain matrix, taking into account model uncertainties. A sufficient condition is provided such that the mean-square matrix of the input tracking error is constrained within a predesigned upper bound which can monotonically converge to zero. Finally, numerical examples considering both structured and unstructured model uncertainties are included to illustrate the effectiveness of the proposed algorithms.

ROAST: Rapid Orthogonal Approximate Slepian Transform

In this paper, we provide a Rapid Orthogonal Approximate Slepian Transform (ROAST) for the discrete vector that one obtains when collecting a finite set of uniform samples from a baseband analog signal. The ROAST offers an orthogonal projection which is an approximation to the orthogonal projection onto the leading discrete prolate spheroidal sequence (DPSS) vectors (also known as Slepian basis vectors). As such, the ROAST is guaranteed to accurately and compactly represent not only oversampled bandlimited signals but also the leading DPSS vectors themselves. Moreover, the subspace angle between the ROAST subspace and the corresponding DPSS subspace can be made arbitrarily small. The complexity of computing the representation of a signal using the ROAST is comparable to the FFT, which is much less than the complexity of using the DPSS basis vectors. We also give non-asymptotic results to guarantee that the proposed basis not only provides a very high degree of approximation accuracy in a mean squared error sense for bandlimited sample vectors, but also that it can provide high-quality approximations of all sampled sinusoids within the band of interest.

State estimation for networked systems with Markovian communication constraints and multiple packet dropouts

In this paper, the state estimation problem for discrete-time networked systems with communication constraints and random packet dropouts is considered. The communication constraint is that, at each sampling instant, there is at most one of the various transmission nodes in the networked systems is allowed to access a shared communication channel, and then the received data are transmitted to a remote estimator to perform the estimation task. The channel accessing process of those transmission nodes is determined by a finite-state discrete-time Markov chain, and random packet dropouts in remote data transmission are modeled by a Bernoulli distributed white sequence. Using Bayes’ rule and some results developed in this study, two state estimation algorithms are proposed in the sense of minimum mean-square error. The first algorithm is optimal, which can exactly compute the minimum mean-square error estimate of system state. The second algorithm is a suboptimal algorithm obtained under a lot of Gaussian hypotheses. The proposed suboptimal algorithm is recursive and has time-independent complexity. Computer simulations are carried out to illustrate the performance of the proposed algorithms.

Optimal fusion estimation for stochastic systems with cross-correlated sensor noises

This paper is concerned with the optimal fusion of sensors with cross-correlated sensor noises. By taking linear transformations to the measurements and the related parameters, new measurement models are established, where the sensor noises are decoupled. The centralized fusion with raw data, the centralized fusion with transformed data, and a distributed fusion estimation algorithm are introduced, which are shown to be equivalent to each other in estimation precision, and therefore are globally optimal in the sense of linear minimum mean square error (LMMSE). It is shown that the centralized fusion with transformed data needs lower communication requirements compared to the centralized fusion using raw data directly, and the distributed fusion algorithm has the best flexibility and robustnessand proper communication requirements and computation complexity among the three algorithms (less communication and computation complexity compared to the existed distributed Kalman filtering fusion algorithms). An example is shown to illustrate the effectiveness of the proposed algorithms.

Channel Training for MIMO Relay Systems in the Presence of Interference

In this paper, we propose an optimal channel training method in the sense of linear minimum mean square error for multiple-input multiple-output amplify-and-forward relay systems in the presence of interference. By using a relay processing matrix, the received interference is cancelled or mitigated. To make the problem tractable, we first derive the optimal structure of the training signal and the relay processing matrix. Then, we propose an optimal channel training method. In order to reduce the complexity, we also propose an asymptotically optimal channel training method. Simulation results demonstrate that the two proposed methods outperform the conventional training signal design methods in terms of both the mean square error and in the bit error rate.

Poisson Wiener filtering with non-local weighted parameter estimation using stochastic distances

The Wiener filter is a classical approach for signal denoising and it is known to be the optimum filter in the sense of the linear minimum mean square error (LMMSE). In general, knowledge of some statistical parameters of the underlying signal and noise are required to compute the Wiener filter. These parameters are usually estimated from the corrupted image using a local neighborhood, thus assuming that the signal and noise are locally stationary. However, images corrupted by signal-dependent noise, such as Poisson noise, are not locally stationary. In this paper, we are proposing a novel Wiener filtering approach for Poisson corrupted images considering a non-local weighted parameter estimation. In the proposed method, named Poisson Non-Local Wiener filter (PNL-Wiener), filtering parameters are estimated from the degraded image considering a non-local neighborhood, where the weights of the estimation function are computed based on the stochastic distances between image patches. Experimental results show that the proposed method is competitive to other state-of-the-art denoising methods designed specifically for Poisson corrupted images, yet providing better preservation of edges and fine details in the images.

High-Efficiency Image Coding via Near-Optimal Filtering

Wiener filtering, which has been widely used in the field of image restoration, is statistically optimal in the sense of mean square error. The adaptive loop filter in video coding inherits the design of Wiener filters, and has been proved to achieve significant improvement on compression performance by reducing coding artifacts and providing high-quality references for subsequent frames. To further improve the compression performance via filtering technique, we explore the factors that may hinder the potential performance of Wiener-based filters, and propose a near-optimal filter learning scheme for high-efficiency image coding. Based on the analyses, we observe that the foremost factor affecting the performance of Wiener-based filters is the divergence of statistical characteristics of training samples, instead of the filter taps or shapes. In view of this, we propose an iterative training method to derive the near-optimal Wiener filter parameters by simultaneously labeling sample categories at the pixel level. These parameters are compressed and transmitted to the decoder side to improve the quality of decoded images by reducing the coding artifacts. Experimental results show that the proposed scheme achieves significant bitrate savings compared with high-efficiency video coding in high-bitrate intra coding scenario.

State estimation for discrete-time Markov jump linear systems with time-correlated and mode-dependent measurement noise

The state estimation problem for discrete-time Markov jump linear systems corrupted by time-correlated and mode-dependent measurement noise is considered where the time-correlated and mode-dependent measurement noise is described via a discrete-time stochastic system with Markov parameter and Kronecker delta function. By defining the measurement noise in this manner, both time-correlation and periodic step change caused by the change of system environment or structure can be embodied in the measurement noise. A novel “distributed measurement differencing method” is applied to the problem of state estimation under consideration so that two algorithms are obtained using some results presented in this paper. The first algorithm is optimal in the sense of minimum mean-square error, which can exactly compute the minimum mean-square error estimate of system state. The second algorithm is suboptimal and the suboptimality of the algorithm is caused by using some Gaussian hypotheses. The two proposed algorithms are recursive and the proposed suboptimal algorithm has a time-independent complexity. The performance of the proposed suboptimal algorithm is illustrated using computer simulations.

Hermite Polynomial Uncorrelated Conversion Filter for Bearings-Only Tracking

Passive bearings-only target tracking is a challenging estimation problem due to high nonlinearity and low observability. In this paper, a novel Hermite polynomial uncorrelated conversion filter is proposed within the linear minimum mean square error framework to improve the target tracking performance. The uncorrelated conversion is a nonlinear transformation of the original measurement that is uncorrelated with it. It can be regarded as a pseudomeasurement to augment the measurement space so that additional information can be exploited to update the state estimate in a nonlinear way. Due to the orthogonality property of the Hermite polynomials in the probability domain, each uncorrelated conversion is constructed as a Hermite polynomial, which is not only uncorrelated with the original measurement but also uncorrelated with other Hermite polynomial uncorrelated conversions automatically. This method enables a systematic way to generate multiple uncorrelated conversions. Moreover, it is shown that, under the linear measurement model, the Hermite polynomial uncorrelated conversion filter is equivalent to the linear minimum mean square error estimator. The simulation results illustrate the superiority of the Hermite polynomial uncorrelated conversion filter over the conventional nonlinear Gaussian filters in the sense of the root mean square error and consistency.

Maximum-Likelihood Approach With Bayesian Refinement for Multichannel-Wiener Postfiltering

Multichannel signal enhancement is an important task, e.g., in modern speech communication. It can be approached in a mean-square error sense by multichannel Wiener filtering, which is often implemented as a cascade of spatial beamforming and spectral enhancement postfiltering. In this paper, we propose a novel Bayesian perspective on the problem of postfilter adaptation. Specifically, we present a two-step procedure for postfilter estimation. We first derive maximum-likelihood (ML) optimal estimators of the target signal and noise power spectral densities (PSD) employing the high-dimensional receiver data in conjunction with blindly estimated channels. When constructing a Wiener postfilter for noisy speech based on the respective speech and noise PSD estimates, this ML-based postfilter is capable of performing good speech enhancement, while leaving musical noise. Hence, we extend to a Bayesian estimation approach in a second step. By statistically modeling the ML-based postfilter estimation and further proposing a prior for the ideal Wiener postfilter, we derive a Bayesian refinement of the ML-based postfilter. This refinement then solely depends on the ML-based postfilter and is independent of the underlying high-dimensional data. Our Bayesian refinement is therefore implemented as a lightweight table look-up. Both proposed postfilters perform well in terms of segmental signal-to-noise ratio and perceptual evaluation of speech quality in various acoustical setups with nonstationary noise. The Bayes-refined postfilter offers consistent improvement over the ML-based postfilter and a single-channel speech enhancement baseline.

Critical Issues on Kalman Filter with Colored and Correlated System Noises

AbstractThe Kalman filtering (KF) is optimal under the assumption that both process and observation noises are independent white Gaussian noise. However, this assumption is not always satisfied in real‐world navigation campaigns. In this paper, two types of KF methods are investigated, i.e. augmented KF (AKF) and the second moment information based KF (SMIKF) with colored system noises, including process and observation noises. As a popular noise‐whitening method, the principle of AKF is briefly reviewed for dealing with the colored system noises. The SMIKF method is developed for the colored and correlated system noises, which directly compensates for the covariance through stochastic model in the sense of minimum mean square error. To accurately implement the SMIKF, a refined SMIKF is further derived regarding the continuous‐time dynamic model rather than the discrete one. The computational burdens of the proposed SMIKF along with representative methods are analyzed and compared. The simulation results demonstrate the performances of proposed methods.

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

ABSTRACTIn the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.

Team-optimal distributed MMSE estimation in general and tree networks

We construct team-optimal estimation algorithms over distributed networks for state estimation in the finite-horizon mean-square error (MSE) sense. Here, we have a distributed collection of agents with processing and cooperation capabilities. These agents observe noisy samples of a desired state through a linear model and seek to learn this state by interacting with each other. Although this problem has attracted significant attention and been studied extensively in fields including machine learning and signal processing, all the well-known strategies do not achieve team-optimal learning performance in the finite-horizon MSE sense. To this end, we formulate the finite-horizon distributed minimum MSE (MMSE) when there is no restriction on the size of the disclosed information, i.e., oracle performance, over an arbitrary network topology. Subsequently, we show that exchange of local estimates is sufficient to achieve the oracle performance only over certain network topologies. By inspecting these network structures, we propose recursive algorithms achieving the oracle performance through the disclosure of local estimates. For practical implementations we also provide approaches to reduce the complexity of the algorithms through the time-windowing of the observations. Finally, in the numerical examples, we demonstrate the superior performance of the introduced algorithms in the finite-horizon MSE sense due to optimal estimation.

A Dual-Path 4-Phase Nonuniform Wideband Receiver With Digital MMSE Harmonic Rejection Equalizer

This paper presents a dual-path nonuniform blocker tolerant wideband receiver that employs digital harmonic rejection to suppress local oscillator (LO) harmonic interferers. The proposed receiver performs harmonic rejection of any LO harmonic including seventh and ninth using only four uniformly spaced clocks each with 25% duty cycle. An adaptive minimum mean-squared error (MMSE) harmonic rejection equalizer is developed that minimizes the desired signal distortion in the mean-squared error sense in the presence of harmonic interferers and the correlated noise between the two paths. The chip prototype is fabricated in 130 nm CMOS process and achieved harmonic rejection ratio (HRR) >75 dB for LO harmonic interferers up to the measured 11th harmonic. The 3.1 dB noise figure (NF) at 1 GHz frequency, 14.5 dBm out-of-band third-order intermodulation intercept point (OBIIP3) at 80 MHz offset from 1 GHz and higher than 10 dB return loss in the frequency range of 100–1450 MHz is measured for the proposed receiver.

Time-Varying Weighted Optimal Empirical Mode Decomposition Using Multiple Sets of Basis Functions

Empirical mode decomposition (EMD) is a favorite tool for analyzing nonlinear and non-stationary signals. It decomposes any signal into a finite set of oscillation modes consisting of intrinsic mode functions and a residual function. Superimposing all these modes reconstructs the signal without any information loss. In addition to satisfying the perfect reconstruction property, however, there is no implication about the reconstruction optimality of the EMD. The lack of optimality restricts the signal recovery capability of the EMD in the presence of disturbances. Only a few attempts are made to meet this deficiency. In this paper, we propose a new algorithm named as time-varying weighted EMD. By this algorithm, original signal is reconstructed in the minimum mean-square error sense through the EMD followed by time-varying weightings of the oscillation modes. Determining the time-varying weights for the oscillation modes constitutes the backbone of the algorithm. Aiming to determine the time-varying weights of the oscillation modes; we use multiple sets of basis functions. The effectiveness of the proposed algorithm is demonstrated by computer simulations involving real biomedical signals. Simulation results show that the proposed algorithm exhibits better performance than that of its existing counterparts in terms of lower mean-square error and higher signal-to-error ratio.

Distributed estimation for stochastic hybrid systems with event‐triggered sensor schedule

This study deals with the state estimation problem for stochastic hybrid systems with event-triggered sensor schedule. By employing the basic interacting multiple model (IMM) approach, a novel event-triggered state estimator has been proposed for stochastic hybrid systems based on a closed-loop schedule rule. To compute the mode probabilities in the IMM estimator, a set of sigma points are generated to produce the pseudo-measurements when the sensor measurements are unavailable. Then, the event-triggered state estimator is extended to develop a distributed estimator in the sense of linear minimum mean square error, where the fused estimates are used to implement the re-initialisation in the interacting stage of the IMM estimator. The performance of proposed estimators is illustrated through the Monte Carlo simulations involving tracking a manoeuvring target in the two-dimensional experiment. Simulation results show that the performance of the proposed estimator is compared with that of the existing filter with a reduced computational burden.

Computational intelligence methodology for the analysis of RC circuit modelled with nonlinear differential order system

In this study, we solve nonlinear initial value problems arising in circuit analysis by applying bio-inspired computational intelligence technique using feed-forward artificial neural networks (ANNs) optimized with genetic algorithms (GAs), sequential quadratic programming (SQP), and their combined scheme. The system of resister–capacitor (RC) circuit having nonlinear capacitance is mathematically modelled with unsupervised ANNs by defining an energy function in mean-square error (MSE) sense. The objectives are to minimize the MSE for which the parameters of the networks are estimated initially with GA-based global search and in steady state with SQP algorithm for efficient local search. We consider a set of scenarios to evaluate the performance of the proposed scheme for different resistance and capacitance values along with current variations in the nonlinear RC circuit system. The results are compared with well-established fully explicit Runge–Kutta numerical solver in order to verify the accuracy of the applied bio-inspired heuristics. To prove the worth of the scheme, a comprehensive statistical analysis is provided for the performance metrics based on root MSE, mean absolute error, Theil’s inequality coefficient, Nash–Sutcliffe efficiency, variance account for, and the coefficient of determination (R 2).

Minimum Variance Distortionless Response Estimators for Linear Discrete State-Space Models

For linear discrete state-space models, under certain conditions, the linear least-mean-squares filter estimate has a convenient recursive predictor/corrector format, aka the Kalman filter. The purpose of this paper is to show that the linear minimum variance distortionless response (MVDR) filter shares exactly the same recursion, except for the initialization which is based on a weighted least-squares estimator. If the MVDR filter is suboptimal in mean-squared error sense, it is an infinite impulse response distortionless filter (a deconvolver) which does not depend on the prior knowledge (first- and second-order statistics) on the initial state. In other words, the MVDR filter can be pre-computed and its behaviour can be assessed in advance independently of the prior knowledge on the initial state.

Optimal signal reconstruction based on time-varying weighted empirical mode decomposition

Empirical mode decomposition (EMD) is a tool developed for analyzing nonlinear and non-stationary signals. It is capable of splitting any signal into a set of oscillation modes known as intrinsic mode functions and a residual function. Although the EMD satisfies the perfect signal reconstruction property by superimposing all the oscillation modes, it is not based on any optimality criterion. The lack of optimality limits the signal recovery performance of the EMD in the presence of disturbances such as noise and interference. In this paper, we propose a new algorithm, termed, time-varying weighted EMD, which gives the best estimate of a given signal in the minimum mean-square error sense. The main idea of the proposed algorithm is to reconstruct the original signal through the EMD followed by time-varying weightings of the oscillation modes. Simulations including two real-life signals are performed to show the superiority of the proposed algorithm.