Noise-corrupted Observations Research Articles

Hybrid stochastic epidemic SIR models with hidden states

This paper focuses on realistic hybrid SIR models that take into account stochasticity. The proposed systems are applicable to most incidence rates that are used in the literature including the bilinear incidence rate, the Beddington–DeAngelis incidence rate, and a Holling type II functional response. Given that many diseases can lead to asymptomatic infections, this paper looks at a system of stochastic differential equations that also includes a class of hidden state individuals, for which the infection status is unknown. Assuming that the direct observation of the percentage of hidden state individuals being infected, α(t), is not given and only a noise-corrupted observation process is available. Using nonlinear filtering techniques in conjunction with an invasion type analysis, this paper shows that the long-term behavior of the disease is governed by a threshold λ∈R that depends on the model parameters. It turns out that if λ<0 the number I(t) of infected individuals converges to zero exponentially fast (extinction). However, if λ>0, the infection is endemic and the system is persistent. We showcase our theorems by applying them in some illuminating examples.

Bayesian operator inference for data-driven reduced-order modeling

This work proposes a Bayesian inference method for the reduced-order modeling of time-dependent systems. Informed by the structure of the governing equations, the task of learning a reduced-order model from data is posed as a Bayesian inverse problem with Gaussian prior and likelihood. The resulting posterior distribution characterizes the operators defining the reduced-order model, hence the predictions subsequently issued by the reduced-order model are endowed with uncertainty. The statistical moments of these predictions are estimated via a Monte Carlo sampling of the posterior distribution. Since the reduced models are fast to solve, this sampling is computationally efficient. Furthermore, the proposed Bayesian framework provides a statistical interpretation of the regularization term that is present in the deterministic operator inference problem, and the empirical Bayes approach of maximum marginal likelihood suggests a selection algorithm for the regularization hyperparameters. The proposed method is demonstrated on two examples: the compressible Euler equations with noise-corrupted observations, and a single-injector combustion process.

LPV Model Order Selection from Noise-corrupted Output and Scheduling Signal Measurements

In parametric identification of Linear Parameter-Varying (LPV) systems, it is important to achieve a low variance of the model estimate by limiting the number of parameters to be identified. This is the well known “model order selection” problem, which consists of selecting the number of input and output delays and the basis functions characterizing the dependence of the LPV model parameters on the scheduling signal. Ignoring the effect of noise on the observations of the scheduling signals may lead to a bias in the final estimate and, as a consequence, also to an incorrect selection of the model order. In this paper, we introduce a “bias-corrected cost function” for the identification of LPV systems from noise-corrupted observations of the output and scheduling variable. The introduced cost function provides a bias-free parameter estimation along with model order selection. The proposed identification approach has two main advantages: (i) the problem of model order selection can be handled by adding a LASSO-like penalty term to the bias-corrected cost function; (ii) it provides a bias-free cost as a criterion to tune some hyper-parameters influencing the final parameter estimate.

A Partition-Based Random Search for Stochastic Constrained Optimization via Simulation

We consider the global optimization problem over finite solution space with a deterministic objective function and stochastic constraints, where noise-corrupted observations of the constraint measures are evaluated via simulation. This problem is challenging in that the solution space often lacks rich structure that can be utilized in identifying the optimal solution, and the feasibility of a solution cannot be known for certain, due to the noisy measurements of the constraints. To tackle these two issues, we adopt a partitioning scheme to explore the solution space and develop a feasibility detection procedure to detect the feasibility of the sampled solutions. A new random search method, called partition-based random search with multi-constraint feasibility detection (PRS-MFD), is proposed. It is shown that PRS-MFD converges to the set of global optima with probability one. The significantly higher efficiency of it is demonstrated by numerical experiments.

A Distributed Navigation Strategy for Mobile Sensor Networks With the Probabilistic Wireless Links

Mobile sensor networks have been widely used to predict the spatio-temporal physical phenomena for various scientific and engineering applications. To accommodate the realistic models of mobile sensor networks, we incorporated probabilistic wireless communication links based on packet reception ratio (PRR) with distributed navigation. We then derived models of mobile sensor networks that predict Gaussian random fields from noise-corrupted observations under probabilistic wireless communication links. For the given model with probabilistic wireless communication links, we derived the prediction error variances for further sampling locations. Moreover, we designed a distributed navigation that minimizes the network cost function formulated in terms of the derived prediction error variances. Further, we have shown that the solution of distributed navigation with the probabilistic wireless communication links for mobile sensor networks are uniformly ultimately bounded with respect to that of the distributed one with the R-disk communication model. According to Monte Carlo simulation results, agent trajectories under distributed navigation with the probabilistic wireless communication links are similar to those with the R-disk communication model, which confirming the theoretical analysis.

Detecting periodicities with Gaussian processes

We consider the problem of detecting and quantifying the periodic component of a function given noise-corrupted observations of a limited number of input/output tuples. Our approach is based on Gaussian process regression, which provides a flexible non-parametric framework for modelling periodic data. We introduce a novel decomposition of the covariance function as the sum of periodic and aperiodic kernels. This decomposition allows for the creation of sub-models which capture the periodic nature of the signal and its complement. To quantify the periodicity of the signal, we derive a periodicity ratio which reflects the uncertainty in the fitted sub-models. Although the method can be applied to many kernels, we give a special emphasis to the Matérn family, from the expression of the reproducing kernel Hilbert space inner product to the implementation of the associated periodic kernels in a Gaussian process toolkit. The proposed method is illustrated by considering the detection of periodically expressed genes in thearabidopsisgenome.

Bounded Conditional Mean Imputation with Observation Uncertainties and Acoustic Model Adaptation

Automatic speech recognition systems use noise compensation and acoustic model adaptation to increase robustness towards speaker and environmental variation. The current work focuses on noise compensation with bounded conditional mean imputation (BCMI). BCMI approaches are missing-data methods which operate on the assumption that noise-corrupted observations can be divided into reliable and unreliable components. BCMI methods substitute the unreliable components with a clean speech posterior distribution. The posterior means can be used as clean speech estimates and the posterior variances can be introduced in acoustic model likelihood calculation as observation uncertainties. In addition, we propose in the current work that similar uncertainties are introduced in acoustic model adaptation. Evaluation with speech data recorded in diverse public and car environments indicates that the proposed uncertainties improve adaptation performance. When uncertainties were used in acoustic model likelihood calculation and adaptation, the proposed imputation and adaptation system introduced 15%-84% relative error reductions to an uncompensated baseline system performance.

Simulation-based optimization over discrete sets with noisy constraints

This article considers a constrained optimization problem over a discrete set where noise-corrupted observations of the objective and constraints are available. The problem is challenging because the feasibility of a solution cannot be known for certain, due to the noisy measurements of the constraints. To tackle this issue, a method is proposed that converts constrained optimization into the unconstrained optimization problem of finding a saddle point of the Lagrangian. The method applies stochastic approximation to the Lagrangian in search of the saddle point. The proposed method is shown to converge, under suitable conditions, to the optimal solution almost surely as the number of iterations grows. The effectiveness of the proposed method is demonstrated numerically in three settings: (i) inventory control in a periodic review system; (ii) staffing in a call center; and (iii) staffing in an emergency room.

Identification of autoregressive moving average systems based on noise compensation in the correlation domain

This study presents a new scheme for the identification of minimum-phase autoregressive moving average (ARMA) systems from noise-corrupted observations. From the autocorrelation function (ACF) of the observed data, exploiting the characteristics of the zero lag, the authors develop a set of equations containing the lower lags of the ACF and solve the corresponding quadratic eigenvalue problem in order to estimate simultaneously the AR parameters and the observation noise variance. In the proposed identification technique, both the white noise and the periodic impulse-train excitations are considered for the purpose of practical applications. In order to estimate the MA parameters, first, a residual signal is obtained by filtering the noisy observations via the estimated AR parameters. A noise-subtraction algorithm is proposed utilising the estimated noise variance and the AR parameters to reduce the effect of noise from the ACF of the residual signal. The MA parameters are then estimated by using the spectral factorisation corresponding to the noise-compensated ACF of the residual signal. In order to demonstrate the effectiveness of the proposed method, extensive experimentations are performed considering synthetic ARMA systems of different orders in the presence of noise and results are compared with those of some of the existing methods. Computer simulations demonstrate a superior identification results in terms of estimation accuracy and consistency even under heavy noisy conditions. Simulation results are also provided for the identification of a human vocal-tract system using natural speech signals showing a superior performance of the proposed technique in terms of the power spectral density of the synthesised speech signal.

Noisy Constrained Maximum-Likelihood Linear Regression for Noise-Robust Speech Recognition

Adaptive training is a widely used technique for building speech recognition systems on nonhomogeneous training data. Recently, there has been interest in applying these approaches for situations where there is significant levels of background noise in the training data. Various schemes for adaptive training are based on noise-, or speaker-, specific transforms of features to yield estimates of the clean speech. However, when there are high levels of background noise, these clean speech estimates may be poor resulting in degradations in performance. In this paper, a new approach for adaptive training on noise-corrupted training data is presented. It extends a popular form of linear transform for model-based adaptation and adaptive training, constrained MLLR (CMLLR), to reflect additional uncertainty from noise-corrupted observations. This new form of adaptation transform is called noisy CMLLR (NCMLLR). NCMLLR uses a modified version of generative model between clean speech and noisy observation, similar to factor analysis (FA). However, in contrast to FA here the generative model describes an adaptation transform, rather than a covariance matrix structure. The use of NCMLLR for adaptive training using an expectation-maximization approach is described. Discriminative adaptive training with NCMLLR is also described based on the minimum phone error criterion. Experimental results comparing NCMLLR with standard adaptive training schemes are given on a noise-corrupted version of Resource Management, the ARPA 1994 CSRNAB Spoke 10 task, and in-car recorded data.

Stochastic optimization algorithms for barrier dividend strategies

This work focuses on finding optimal barrier policy for an insurance risk model when the dividends are paid to the share holders according to a barrier strategy. A new approach based on stochastic optimization methods is developed. Compared with the existing results in the literature, more general surplus processes are considered. Precise models of the surplus need not be known; only noise-corrupted observations of the dividends are used. Using barrier-type strategies, a class of stochastic optimization algorithms are developed. Convergence of the algorithm is analyzed; rate of convergence is also provided. Numerical results are reported to demonstrate the performance of the algorithm.

Recursive Identification of EIV ARMA Processes

Easily computable recursive algorithms are proposed for estimating coefficients of A(z), C(z), and the covariance matrix Rw of wk for the multivariate ARMA process A(z)yk = C(z)wk on the basis of the noise-corrupted observations ηk Δ yk + εk. the estimates converge to the true ones under reasonable conditions. An illustrative example is provided, and the simulation results are shown to be consistent with the theoretical analysis.

Language-measure-theoretic optimal control of probabilistic finite-state systems

Supervisory control theory for discrete event systems, introduced by Ramadge and Wonham, is based on a non-probabilistic formal language framework. However, models for physical processes inherently involve modelling errors and noise-corrupted observations, implying that any practical finite-state approximation would require consideration of event occurrence probabilities. Building on the concept of signed real measure of regular languages, this paper formulates a comprehensive theory for optimal control of finite-state probabilistic processes. It is shown that the resulting discrete-event supervisor is optimal in the sense of elementwise maximizing the renormalized langauge measure vector for the controlled plant behaviour and is efficiently computable. The theoretical results are validated through several examples including the simulation of an engineering problem.

Fast filtering of noisy autoregressive signals

Autoregressive (AR) models are used in a wide variety of applications concerning the recovery of signals from noise-corrupted observations. In all real contexts of this kind also an additive broadband observation noise is present and the filtering of the observations is usually performed by means of standard Kalman filtering that requires a state space realization of the AR model to describe the observed process and the solution, at every step, of the Riccati equation. This paper proposes a faster filtering algorithm suitable for stationary processes and based on the decomposition of Toeplitz matrices described in [J. Rissanen, Algorithms for triangular decomposition of block Hankel and Toeplitz matrices with application to factoring positive matrix polynomials, Math. Comput. 27 (January 1973) 147–154] that operates directly on AR models. The computational complexity of the proposed algorithm increases only linearly with the order of the process.

Energy constrained frequency-domain normalized LMS algorithm for blind channel identification

This paper deals with the blind adaptive identification of single-input multi-output (SIMO) finite impulse response acoustic channels from noise-corrupted observations. The normalized multichannel frequency-domain least-mean-squares (NMCFLMS) algorithm [1] is known to be a very effective and efficient technique for identification of such channels when noise effects can be ignored. It, however, misconverges in presence of noise [2]. In this paper, we present an analysis of noise effects on the NMCFLMS algorithm and propose a novel technique for ameliorating such misconvergence characteristics of the NMCFLMS algorithm for blind channel identification (BCI) with noise by attaching a spectral constraint in the adaptation rule. Experimental results demonstrate that the robustness of the NMCFLMS algorithm for BCI can be significantly improved using such a constraint.

Universal Minimax Discrete Denoising Under Channel Uncertainty

The goal of a denoising algorithm is to recover a signal from its noise-corrupted observations. Perfect recovery is seldom possible and performance is measured under a given single-letter fidelity criterion. For discrete signals corrupted by a known discrete memoryless channel (DMC), the Discrete Universal DEnoiser (DUDE) was recently shown to perform this task asymptotically optimally, without knowledge of the statistical properties of the source. In the present work, we address the scenario where, in addition to the lack of knowledge of the source statistics, there is also uncertainty in the channel characteristics. We propose a family of discrete denoisers and establish their asymptotic optimality under a minimax performance criterion which we argue is appropriate for this setting. As we show elsewhere, the proposed schemes can also be implemented computationally efficiently

Algorithms for discrete denoising under channel uncertainty

The goal of a denoising algorithm is to reconstruct a signal from its noise-corrupted observations. Perfect reconstruction is seldom possible and performance is measured under a given fidelity criterion. In a recent work, the authors addressed the problem of denoising unknown discrete signals corrupted by a discrete memoryless channel when the channel, rather than being completely known, is only known to lie in some uncertainty set of possible channels. A sequence of denoisers was derived for this case and shown to be asymptotically optimal with respect to a worst-case criterion argued most relevant to this setting. In the present work we address the implementation and complexity of this denoiser for channels parametrized by a scalar, establishing its practicality. We show that for symmetric channels, the problem can be mapped into a convex optimization problem, which can be solved efficiently. We also present empirical results suggesting the potential of these schemes to do well in practice. A key component of our schemes is an estimator of the subset of channels in the uncertainty set that are feasible in the sense of being able to give rise to the noise-corrupted signal statistics for some channel input distribution. We establish the efficiency of this estimator, both algorithmically and experimentally. We also present a modification of the recently developed discrete universal denoiser (DUDE) that assumes a channel based on the said estimator, and show that, in practice, the resulting scheme performs well. For concreteness, we focus on the binary alphabet case and binary symmetric channels, but also discuss the extensions of the algorithms to general finite alphabets and to general channels parameterized by a scalar.

Denoising by Sparse Approximation: Error Bounds Based on Rate-Distortion Theory

If a signal x is known to have a sparse representation with respect to a frame, it can be estimated from a noise-corrupted observation y by finding the best sparse approximation to y. Removing noise in this manner depends on the frame efficiently representing the signal while it inefficiently represents the noise. The mean-squared error (MSE) of this denoising scheme and the probability that the estimate has the same sparsity pattern as the original signal are analyzed. First an MSE bound that depends on a new bound on approximating a Gaussian signal as a linear combination of elements of an overcomplete dictionary is given. Further analyses are for dictionaries generated randomly according to a spherically-symmetric distribution and signals expressible with single dictionary elements. Easily-computed approximations for the probability of selecting the correct dictionary element and the MSE are given. Asymptotic expressions reveal a critical input signal-to-noise ratio for signal recovery.

On the optimality of symbol-by-symbol filtering and denoising

We consider the problem of optimally recovering a finite-alphabet discrete-time stochastic process {X/sub t/} from its noise-corrupted observation process {Z/sub t/}. In general, the optimal estimate of X/sub t/ will depend on all the components of {Z/sub t/} on which it can be based. We characterize nontrivial situations (i.e., beyond the case where (X/sub t/,Z/sub t/) are independent) for which optimum performance is attained using operations (a.k.a. decoding), meaning that the optimum estimate of X/sub t/ depends solely on Z/sub t/. For the case where {X/sub t/} is a stationary binary Markov process corrupted by a memoryless channel, we characterize the necessary and sufficient condition for optimality of symbol-by-symbol operations, both for the filtering problem (where the estimate of X/sub t/ is allowed to depend only on {Z/sub t'/}/sub t'/spl les/t/) and the denoising problem (where the estimate of X/sub t/ is allowed dependence on the entire noisy process). It is then illustrated how our approach, which consists of characterizing the support of the conditional distribution of the noise-free symbol given the observations, can be used for characterizing the entropy rate of the binary Markov process corrupted by the binary-symmetric channel (BSC) in various asymptotic regimes. For general noise-free processes (not necessarily Markov), general noise processes (not necessarily memoryless), and general index sets (random fields) we obtain an easily verifiable sufficient condition for the optimality of symbol-by-symbol operations and illustrate its use in a few special cases. For example, for binary processes corrupted by a BSC, we establish, under mild conditions, the existence of a /spl delta//sup */>0 such that the say-what-you-see scheme is optimal provided the channel crossover probability is less than /spl delta//sup */. Finally, we show how for the case of a memoryless channel the large deviations (LD) performance of a symbol-by-symbol filter is easy to obtain, thus characterizing the LD behavior of the optimal schemes when these are singlet decoders (and constituting the only known cases where such explicit characterization is available).

Spectrum Estimation by Noise-Compensated Data Extrapolation

High-resolution spectrum estimation techniques have been extensively studied in recent publications. Knowledge of the noise variance is vital for spectrum estimation from noise-corrupted observations. This paper presents the use of noise compensation and data extrapolation for spectrum estimation. We assume that the observed data sequence can be represented by a set of autoregressive parameters. A recently proposed iterative algorithm is then used for noise variance estimation while autoregressive parameters are used for data extrapolation. We also present analytical results to show the exponential decay characteristics of the extrapolated samples and the frequency domain smoothing effect of data extrapolation. Some statistical results are also derived. The proposed noise-compensated data extrapolation approach is applied to both the autoregressive and FFT-based spectrum estimation methods. Finally, simulation results show the superiority of the method in terms of bias reduction and resolution improvement for sinusoids buried in noise.