Additive Noise Process Research Articles

Stochastic resonance in noisy threshold neurons

Stochastic resonance occurs when noise improves how a nonlinear system performs. This paper presents two general stochastic-resonance theorems for threshold neurons that process noisy Bernoulli input sequences. The performance measure is Shannon mutual information. The theorems show that small amounts of independent additive noise can increase the mutual information of threshold neurons if the neurons detect subthreshold signals. The first theorem shows that this stochastic-resonance effect holds for all finite-variance noise probability density functions that obey a simple mean constraint that the user can control. A corollary shows that this stochastic-resonance effect occurs for the important family of (right-sided) gamma noise. The second theorem shows that this effect holds for all infinite-variance noise types in the broad family of stable distributions. Stable bell curves can model extremely impulsive noise environments. So the second theorem shows that this stochastic-resonance effect is robust against violent fluctuations in the additive noise process.

Asymptotic properties of least-squares estimates of Hammerstein-Wiener models

This paper investigates the asymptotic properties of least squares estimates of Hammerstein-Wiener model structures, and in doing so establishes consistency and asymptotic normality under fairly mild conditions on the additive noise process, the inputs and the static non-linearities. In relation to the asymptotic distributional results, a consistent procedure for the estimation of the asymptotic variance of the parameter estimates is provided. A key theme of this paper is to demonstrate how recent results from the econometrics literature may be employed in an engineering setting. In this respect the Hammerstein-Wiener model structure serves as a demonstration example. A simulation study complements the theoretical findings.

Noise robustness of nonlinear filters for image recognition.

We analyze the performance of the Fourier plane nonlinear filters in terms of signal-to-noise ratio (SNR). We obtain a range of nonlinearities for which SNR is robust to the variations in input-noise bandwidth. This is shown both by analytical estimates of the SNR for nonlinear filters and by experimental simulations. Specifically, we analyze the SNR when Fourier plane nonlinearity is applied to the input signal. Using the Karhunen-Loève series expansion of the noise process, we obtain precise analytic expressions of the SNR for Fourier plane nonlinear filters in the presence of various types of additive-noise processes. We find a range of nonlinearities that need to be applied that keep the output SNR of the filter stable relative to changes in the noise bandwidth.

NON-CHAOTIC RESPONSE OF NON-LINEAR OSCILLATORS UNDER COMBINED DETERMINISTIC AND WEAK STOCHASTIC EXCITATIONS

Non-linear oscillators under harmonic and/or weak stochastic excitations are considered in this paper. Under harmonic excitations alone, an analytical technique based on a set of exponential transformations followed by harmonic balancing is proposed to solve for a variety of one-periodic orbits. The stability boundaries for such orbits in the associated parameter space are constructed using the Floquet theory. Under a combination of harmonic and weak stochastic excitations, a stochastic perturbation approach around the deterministic orbit is adopted to obtain response statistics in terms of the evolving moment functions. In the present study, the stochastic perturbation is assumed to be an additive white noise process and equations for the evolving moments are derived using Ito differential rule. A fifth order cumulant neglect closure is used to close the infinite hierarchy of moment equations. Limited numerical results are presented to illustrate the implementation of the proposed scheme. The method is found to be quite versatile and admits ready extensions to Md.o.f. systems under combined harmonic and white or non-white, multiplicative or additive random excitations.

A systematic approach to the design and analysis of optimum DPSK receivers for generalized diversity communications over Rayleigh fading channels

A generalized diversity channel is introduced that models a variety of wireless communication systems that use time, frequency, multipath, and/or antenna diversity with various interbranch correlations between signaling waveforms and the fading and additive noise processes. In the context of this general model, a systematic approach to the design and analysis of optimum noncoherent differential phase-shift keying (DPSK) receivers is introduced. In particular, it is shown how the minimum error probability (MEP) and the generalized likelihood ratio tests (GLRT) can be applied to obtain optimal noncoherent combining rules. A comparative error-rate analysis of the GLRT and MEP detectors and an ad hoc equal-gain combiner is provided for binary signaling, and the suitability of the three schemes is determined as a function of fading characteristics. The asymptotic bit-error-rate analysis is undertaken for the MEP detector for slow and fast fading channels. An estimator-detector decomposition of the noncoherent MEP rule is obtained which allows an insightful comparative study of the fundamental limits of binary phase-shift keying and DPSK modulation-detection methods for both slow and fast fading. The results of this paper are also applicable to postdecorrelative receivers in multiuser channels.

Simulation using the state-space representation of noisy dynamic systems to determine effective integrated process control designs

There has been considerable interest in investigating control policies that can be applied to dynamic systems to take advantage of engineering process control (EPC) and statistical process control (SPC). We design two simulation models using SIMULINK to represent control of noisy dynamic systems given by a state-space representation. We use the models on systems with dynamic behavior that have been degraded by an additive ARMA noise process. Our findings show that the combination of EPC and SPC is a more effective policy on a complex dynamic system than on a simple dynamic system.

Stochastic sensitivity analysis and Langevin simulation for neural network learning

A comprehensive theoretical framework is proposed for the learning of a class of gradient-type neural networks with an additive Gaussian white noise process. The study is based on stochastic sensitivity analysis techniques, and formal expressions are obtained for stochastic learning laws in terms of functional derivative sensitivity coefficients. The present method, based on Langevin simulation techniques, uses only the internal states of the network and ubiquitous noise to compute the learning information inherent in the stochastic correlation between noise signals and the performance functional. In particular, the method does not require the solution of adjoint equations of the back-propagation type. Thus, the present algorithm has the potential for efficiently learning network weights with significantly fewer computations. Application to an unfolded multi-layered network is described, and the results are compared with those obtained by using a back-propagation method.

Neural network learning based on stochastic sensitivity analysis

A comprehensive theoretical framework is proposed for the learning of a class of gradient-type neural networks with an additive Gaussian white noise process. The study is based on stochastic sensitivity analysis techniques, and formal expressions are obtained for the stochastic learning laws in terms of the functional derivative sensitivity coefficients. The present method efficiently processes the learning information inherent in the stochastic correlation between the signal and corresponding noise processes without the need for actually computing equations of the back-propagation type. New stochastic implementations of the Hebbian and competitive learning laws are derived to elucidate this theoretical development.

Minimum variance time-frequency distribution kernels for signals in additive noise

We derive the minimum variance time-frequency distribution kernel for signals in additive circular complex Gaussian white noise processes. It is shown that the kernel that minimizes the average variance and simultaneously satisfies the time-frequency constraints for the noise-only ease remains optimum in the presence of frequency-modulated (FM) signals.

Feedback does not increase the capacity of discrete channels with additive noise

We consider discrete-time finite alphabet channels with additive random noise. We show that output feedback does not increase the capacity of such channels. This result holds in the most general case; i.e., for arbitrary additive noise processes.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Cramer-Rao bounds for discrete-time nonlinear filtering problems

In this note, a Cramer-Rao bound for the mean squared error that can be achieved with nonlinear observations of a nonlinear pth order autoregressive (AR) process where both the process and observation noise covariances can be state dependent is presented. The major limitation is that the AR process must be driven by an additive white Gaussian noise process that has a full-rank covariance. A numerical example demonstrating the tightness of the bound for a particular problem is included.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Gaussian multiaccess channels with ISI: capacity region and multiuser water-filling

The capacity region of a two-user Gaussian multiaccess channel with intersymbol interference (ISI) in which the inputs pass through respective linear systems and are superimposed before being corrupted by an additive Gaussian noise process is discussed. A geometrical method for obtaining the optimal input power spectral densities and the capacity region is presented. This method can be viewed as a nontrivial generalization of the single-user water-filling argument. It is shown that, as in the traditional memoryless multiaccess channel, frequency-division multiaccess (FDMA) with optimally selected frequency bands for each user achieves the total capacity of the multiuser Gaussian multiaccess channel with ISI. However, the capacity region of the two-user channel with memory is, in general, not a pentagon unless the channel transfer functions for both users are identical.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Upper bounds on the capacity of the digital magnetic recording channel

Upper bounds are derived for the channel capacity of a model for a digital magnetic disk storage device. The model is similar to that of C.A. French and J.K. Wolf (1988) in that it contains a linear filter to represent the Lorentzian step response and two additive Gaussian noise processes to represent the electronic and media noises. It differs in that both the lineal bit density and the track density are parameters that may be varied, as opposed to just track density. Using typical media parameters, it is estimated that, for this model, the channel capacity (in bits per unit area) may be increased up to a factor of approximately 20 by appropriately increasing the lineal and track densities. >

Sensitivity to initial conditions in stochastic systems.

The time evolution of the mean deviation of initially close trajectories in a stochastic dynamical system is investigated. It is shown both for additive and linearly coupled multiplicative noise that the mean deviation loses its dependence on initial conditions for long times. For shorter times a power law is found for certain types of additive noise processes, in sharp contrast to the exponential separation of initially nearby trajectories in deterministic chaotic systems. Exponential time evolution is obtained for linearly coupled multiplicative noise after an initial transient during which more complex regimes, including a superexponential stage, can take place.

Comparison of location models for stochastic processes

Let $$\mathfrak{E}$$ (Q) be the statistical experiment based on the observation of an unknown function in the presence of an additive noise process with distributionQ. The (possible) loss of information whenQ is replaced by some other noise distributionP is measured by the deficiency of $$\mathfrak{E}$$ (P) relative to $$\mathfrak{E}$$ (Q). This deficiency and its relation to the variational distance ofP andQ are studied mainly for Gaussian noise processes. Gaussian diffusion processes and special set-indexed processes are treated in detail.

Previsualized image vector quantization with optimized pre- and postprocessors

The optimal previsualized image vector quantization method for compressing digital images to a bit rate of 0.75 bpp or below with moderately low to very low subjective distortion is presented. The encoding method incorporates a visual model as part of the distortion measure. By modeling the quantization noise as an additive signal-dependent noise process, an optimum pre- and postprocessing system, which minimizes the mean-squared error measured inside the visual model, is derived. The analysis of the system performance and a coordinate descent design algorithm are discussed. A set of experiments was conducted using the optimum system, and the results were compared to those obtained by other methods. The study shows that the images quantized by the method presented exhibit much less sawtooth, blocking, and contouring effects and higher subjective quality. Images of surprising quality have been produced by this method at a bit rate of about 0.1 bpp with a compression ratio of 80:1 relative to a normal 8 bpp original.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Random coding for additive Gaussian channels with feedback

A random coding strategy for discrete-time additive Gaussian noise channels with feedback is analyzed. It has long been known that feedback may increase the capacity of such channels as long as the additive noise process is not white. Here, it is proved that a strictly positive gain is always achieved and that, as the signal power goes to zero, the ratio of feedback capacity to capacity without feedback may be strictly greater than one if the noise spectrum has a null. This is not the case when the noise spectrum is bounded away from zero. It is also demonstrated that random coding, where the codewords are chosen from an ensemble of stationary Gaussian sequences, does not achieve capacity. >

A critique of several failure detection approaches for navigation systems

Additional references relating to the work of J.E. White and J.L. Speyer (ibid., vol.AC-32, p.593-603, July 1987) are reported and commented on. White and Speyer discussed predecessor studies that dealt with failure detection in navigation applications described by stochastic time-invariant linear system with additive Gaussian white process and measurement noises being present, and sought to use Kalman filters tuned in this application context to detect a priori specified failures. It is argued that, since the predecessor work dealt exclusively with time-invariant deterministic systems devoid of noise terms, exclusive use of observers suffice for failure detection in this more benign context. >

SPECTRAL ESTIMATION AND DECONVOLUTION FOR A LINEAR TIME SERIES MODEL

Abstract. A vector linear time series model is observed as the sum of a convolution of an unknown signal and an additive noise process. The main objective is the estimation or deconvolution of the signal when the spectra of the signal and noise processes are unknown. We prove the strong consistency of a class of nonparametric spectral estimators derived by maximizing a particular Gaussian likelihood function. We also study the mean square convergence of the finite‐sample deconvolution estimators as a function of the sample length T, the filter length M and the spectral bandwidth BT=LT/T.

Analysis of the digital data transition tracking loop in correlated Gaussian noise

The behaviour of a, practical self-bit synchronizer, called the digital data transition tracking loop, bas been studied when the input additive noise process (assumed to the zero-mean Gaussian) is correlated. Both linear and non-linear analyses of the above synchronizer in the correlated case have been carried out and the results are compared with those for the case of uncorrelated noise.