Prior Statistical Information Research Articles

Subjectivity in Application of the Principle of Maximum Entropy

Complete prior statistical information is currently required in the majority of statistical evaluations of complex models. The principle of maximum entropy is often utilized in this context to fill in the missing pieces of available information and is normally claimed to be fair and objective. A rarely discussed aspect is that it relies upon testable information, which is never known but estimated, i.e. results from processing of raw data. The subjective choice of this processing strongly affects the result. Less conventional posterior completion of information is equally accurate but is computationally superior to prior, as much less information enters the analysis. Our recently proposed methods of lean deterministic sampling are examples of very few approaches that actively promote the use of minimal incomplete prior information. The inherited subjective character of maximum entropy distributions and the often critical implications of prior and posterior completion of information are here discussed and illustrated, from a novel perspective of consistency, rationality, computational efficiency and realism.

One-Bit Measurements With Adaptive Thresholds

We introduce a new method for adaptive one-bit quantization of linear measurements and propose an algorithm for the recovery of signals based on generalized approximate message passing (GAMP). Our method exploits the prior statistical information on the signal for estimating the minimum-mean-squared error solution from one-bit measurements. Our approach allows the one-bit quantizer to use thresholds on the real line. Given the previous measurements, each new threshold is selected so as to partition the consistent region along its centroid computed by GAMP. We demonstrate that the proposed adaptive-quantization scheme with GAMP reconstruction greatly improves the performance of signal and image recovery from one-bit measurements.

Color Constancy with Spatio-Spectral Statistics

We introduce an efficient maximum likelihood approach for one part of the color constancy problem: removing from an image the color cast caused by the spectral distribution of the dominating scene illuminant. We do this by developing a statistical model for the spatial distribution of colors in white balanced images (i.e., those that have no color cast), and then using this model to infer illumination parameters as those being most likely under our model. The key observation is that by applying spatial band-pass filters to color images one unveils color distributions that are unimodal, symmetric, and well represented by a simple parametric form. Once these distributions are fit to training data, they enable efficient maximum likelihood estimation of the dominant illuminant in a new image, and they can be combined with statistical prior information about the illuminant in a very natural manner. Experimental evaluation on standard data sets suggests that the approach performs well.

Minimum Divergence Approaches for Robust Classification of Ground Moving Targets

This work addresses the problem of automatic target recognition (ATR) using micro-Doppler information obtained by a low-resolution ground surveillance radar. Gaussian mixture models (GMMs) are used to represent the prior statistical information of threatening ground moving targets such as walking personnel and tracked or wheeled vehicles. A minimum divergence (MD) classification approach with a variety of distance measures is proposed. The proposed MD classification approach is robust with respect to modeling errors and can be efficiently used in low signal-to-noise (SNR) and training data deficient scenarios. The MD classifier is implemented using a variety of computationally efficient approximations of distance measures between GMMs. Performance of the MD classifier was analyzed using collected radar measurements and the influence of different distance measures and their approximations on classification performance is assessed. The proposed MD classifier outperforms the maximum likelihood (ML) classifier in low-SNR and training data deficient scenarios while providing a computationally efficient implementation.

Practical Estimation of Rapidly Varying Channels for OFDM Systems

We propose a novel pilot-aided algorithm for estimation of rapidly varying wireless channels in OFDM systems. Our approach is specifically designed for channels varying on the scale of a single OFDM symbol duration, which occur, for example, in mobile WiMAX, WAVE, and DVB-T. From the pilot information, we recover information about the channel taps in the framework of the Basis Expansion Model (BEM). We derive explicit formulas for the BEM coefficients in terms of the receive signal. Algebraically, the algorithm is FFT-based, and can be easily implemented in hardware. For a system with L channels taps, our method uses O(L log L) operations and O(L) memory per OFDM symbol. This complexity is the best possible up to the order of magnitude. Previously published methods require O(L2) operations and O(L2) memory. Numerical simulations illustrate performance gains achieved by our estimator at sufficiently high Doppler frequencies. Our approach does not assume any prior statistical information.

Compressive Estimation of Doubly Selective Channels in Multicarrier Systems: Leakage Effects and Sparsity-Enhancing Processing

We consider the application of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">compressed sensing</i> (CS) to the estimation of doubly selective channels within pulse-shaping multicarrier systems (which include orthogonal frequency-division multiplexing (OFDM) systems as a special case). By exploiting sparsity in the delay-Doppler domain, CS-based channel estimation allows for an increase in spectral efficiency through a reduction of the number of pilot symbols. For combating leakage effects that limit the delay-Doppler sparsity, we propose a sparsity-enhancing basis expansion and a method for optimizing the basis with or without prior statistical information about the channel. We also present an alternative CS-based channel estimator for (potentially) strongly time-frequency dispersive channels, which is capable of estimating the ¿off-diagonal¿ channel coefficients characterizing intersymbol and intercarrier interference (ISI/ICI). For this estimator, we propose a basis construction combining Fourier (exponential) and prolate spheroidal sequences. Simulation results assess the performance gains achieved by the proposed sparsity-enhancing processing techniques and by explicit estimation of ISI/ICI channel coefficients.

Inter-Carrier Interference Estimation in OFDM Systems With Unknown Noise Distributions

There are a number of approaches to estimating carrier frequency offset (CFO) that causes inter-carrier interference (ICI) in orthogonal frequency division multiplexing (OFDM) systems: self-cancelation method, the extended Kalman filter (EKF), particle filter, H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">infin</sub> filter (HF), etc. In particular, the HF is of interest because prior statistical noise information is not necessarily required in its application. Cost reference particle filter (CRPF), newly developed in the particle filtering framework, has the same feature as HF; it also does not require the prior noise information of the state and the measurement equation. In this letter, we compare and analyze the performances of two similar methods. The simulation results show that CRPF outperforms HF, particularly when the bit energy to noise ratio of the measurement is low. Therefore, CRPF is very effective and robust, especially when the noise statistics are unknown with a low bit energy to noise ratio.

Note Onset Detection via Nonnegative Factorization of Magnitude Spectrum

A novel approach for onset detection of musical notes from audio signals is presented. In contrast to most commonly used conventional approaches, the proposed method features new detection functions constructed from the linear temporal bases that are obtained from the decomposition of musical spectra using nonnegative matrix factorization (NMF). Three forms of detection function, namely, first-order difference function, psychoacoustically motivated relative difference function, and constant-balanced relative difference function, are considered. As the approach works directly on input data, no prior knowledge or statistical information is therefore required. Practical issues, including the choice of the factorization rank and detection robustness to instruments, are also examined experimentally. Due to the scalability issue with the generated nonnegative matrix, the proposed method is only applied to relatively short, single instrument (or voice) recordings. Numerical examples are provided to show the good performance of the proposed method, including comparisons between the three detection functions.

A field-based procedure for determining number of waste sorts for solid waste characterization

Current methodologies for estimating the number of waste sorts require prior information on statistics (mean and standard deviation) for specific waste categories. The methods often require an iterative procedure or computer software to estimate the number of sorts required and the required statistics may change both spatially and temporally. To overcome the need for prior statistical information and avoid the need for iterative effort, a real-time methodology is developed to determine the required number of waste sorts for a waste category while sampling in the field for solid waste characterization. The information on required numbers of waste sorts is field-based and evolves during the sampling by utilizing real-time data to characterize the means and standard deviations. The proposed approach does not require prior information. The application of proposed methodology is demonstrated using the data from the Greater Vancouver Regional District (GVRD) for the primary categories of paper, glass, and secondary category glass-beverage.

A new MMSE channel estimation algorithm for OFDM systems

In this report, we propose a new MMSE channel estimation algorithm for OFDM systems. It is well-known that time domain maximum likelihood estimators (MSEs) show highly accurate impulse response estimation by using the time domain long preamble of the OFDM frame. On the other hand, the impulse response estimation based on the minimum mean square error (MMSE) criterion achieves superior channel estimation in low SNR conditions; however, it requires prior statistical information such as delay profiles of channels. In addition, the computational complexity becomes large because of the inverse matrix calculation. In order to overcome these issues, we propose to estimate the power delay profile of the channel by using the MLE, and to suppress the increase of the computational complexity for the matrix inverse calculation by applying the conjugate gradient method without degradation of the PER performance.

Blind channel estimation for redundant precoded OFDM systems

Considering that channel estimation can play a crucial role in coherent detection of the information symbols in each data block, a blind channel estimation approach is proposed for redundant precoded orthogonal frequency-division multiplexing (OFDM) systems. A redundant linear frequency-domain precoder is applied to each pair of blocks before they enter the OFDM system. Because of the introduced structure, the frequency-selective channel can be identified at the receiver based on autocorrelation operations, singular value decomposition (SVD), and by resolving the scalar ambiguity. The proposed channel estimation method has low computational complexity and requires no prior statistical information on channel or noise. And the proposed blind method has high spectral efficiency owing to exploiting no training sequence. Computer simulations confirm that this proposed blind channel estimation method can identify the frequency-selective channels perfectly and obtain a good performance.

Nonparametric shape priors for active contour-based image segmentation

When segmenting images of low quality or with missing data, statistical prior information about the shapes of the objects to be segmented can significantly aid the segmentation process. However, defining probability densities in the space of shapes is an open and challenging problem. In this paper, we propose a nonparametric shape prior model for image segmentation problems. In particular, given example training shapes, we estimate the underlying shape distribution by extending a Parzen density estimator to the space of shapes. Such density estimates are expressed in terms of distances between shapes, and we consider the L 2 distance between signed distance functions for shape density estimation, in addition to a distance measure based on the template metric. In particular, we consider the case in which the space of shapes is interpreted as a manifold embedded in a Hilbert space. We then incorporate the learned shape prior distribution into a maximum a posteriori (MAP) estimation framework for segmentation. This results in an optimization problem, which we solve using active contours. We demonstrate the effectiveness of the resulting algorithm in segmenting images that involve low-quality data and occlusions. The proposed framework is especially powerful in handling “multimodal” shape densities.

Improved Performance of Bayesian Solutions for Inverse Electrocardiography Using Multiple Information Sources

The usual goal in inverse electrocardiography (ECG) is to reconstruct cardiac electrical sources from body surface potentials and a mathematical model that relates the sources to the measurements. Due to attenuation and smoothing that occurs in the thorax, the inverse ECG problem is ill-posed and imposition of a priori constraints is needed to combat this ill-posedness. When the problem is posed in terms of reconstructing heart surface potentials, solutions have not yet achieved clinical utility; limitations include the limited availability of good a priori information about the solution and the lack of a "good" error metric. We describe an approach that combines body surface measurements and standard forward models with two additional information sources: statistical prior information about epicardial potential distributions and sparse simultaneous measurements of epicardial potentials made with multielectrode coronary venous catheters. We employ a Bayesian methodology which offers a general way to incorporate these information sources and additionally provides statistical performance analysis tools. In a simulation study, we first compare solutions using one or more of these information sources. Then, we study the effects of varying the number of sparse epicardial potential measurements on reconstruction accuracy. To evaluate accuracy, we used the Bayesian error covariance as well as traditional error metrics such as relative error. Our results show that including even sparsely sampled information from coronary venous catheters can substantially improve the reconstruction of epicardial potential distributions and that a Bayesian framework provides a feasible approach to using this information. Moreover, computing the Bayesian error standard deviations offers a means to indicate confidence in the results even in the absence of validation data.

Joint gravity and magnetic inversion in 3D using Monte Carlo methods

We jointly invert gravity and magnetic data following a Monte Carlo method that provides estimation for a 3D model of the structure and physical properties of the medium. In particular, the model layer geometry and the density and magnetic susceptibility fields within layers are estimated, and their uncertainties are described with posterior probabilities. This method combines the gravity and magnetic data with prior information of the mass density and magnetic susceptibility statistics, and statistical constraints on the model interface positions. The resulting model realizations jointly comply with the observations and the prior statistical information.

Distributed target classification and tracking in sensor networks

The highly distributed infrastructure provided by sensor networks supports fundamentally new ways of designing surveillance systems. In this paper, we discuss sensor networks for target classification and tracking. Our formulation is anchored on location-aware data routing to conserve system resources, such as energy and bandwidth. Distributed classification algorithms exploit signals from multiple nodes in several modalities and rely on prior statistical information about target classes. Associating data to tracks becomes simpler in a distributed environment, at the cost of global consistency. It may be possible to filter clutter from the system by embedding higher level reasoning in the distributed system. Results and insights from a recent field test at 29 Palms Marine Training Center are provided to highlight challenges in sensor networks.

Wavelet-Based Regularization of Dynamic Data Reconciliation

Dynamic data reconciliation can supply more accurate data for dynamic optimization, dynamic fault diagnosis, and control by means of incorporating process information in some mathematical model. It will be an ill-posed inverse problem if the sensitive input variables are unmeasured; here, the sensitive input variable is defined as the variable that, if it is unmeasured, can only be estimated through the differentiation of other measured variables. In such a case, existing methods cannot obtain correct and usable data effectively. To address the problem, based on the principle of regularization, the wavelets are adopted to construct regular operators. And, a new approach is proposed to determine the optimal scale level corresponding to the optimal approximate operator in which the prior statistical information of the signal is utilized. The algorithm can deal with the estimation of unknown sensitive input variable effectively. The results show that more accurate estimation of the sensitive input variable can be obtained by using the proposed method as compared with the one obtained by using existing collocation methods based on polynomials.

Non-Markovian Optimal Prediction

Optimal prediction methods compensate for a lack of resolution in the numerical solution of complex problems through the use of prior statistical information. We know from previous work that in the presence of strong underresolution a good approximation needs a non-Markovian memory, determined by an equation for the orthogonal, i.e., unresolved, dynamics. We present a simple approximation of the orthogonal dynamics, which involves an ansatz and a Monte-Carlo evaluation of autocorrelations. The analysis provides a new understanding of the fluctuation-dissipation formulas of statistical physics. An example is given.

Optimal Prediction for Hamiltonian Partial Differential Equations

Optimal prediction methods compensate for a lack of resolution in the numerical solution of time-dependent differential equations through the use of prior statistical information. We present a new derivation of the basic methodology, show that field-theoretical perturbation theory provides a useful device for dealing with quasi-linear problems, and provide a nonlinear example that illuminates the difference between a pseudo-spectral method and an optimal prediction method with Fourier kernels. Along the way, we explain the differences and similarities between optimal prediction, the representer method in data assimilation, and duality methods for finding weak solutions. We also discuss the conditions under which a simple implementation of the optimal prediction method can be expected to perform well.

Optimal prediction and the Mori-Zwanzig representation of irreversible processes.

Optimal prediction methods compensate for a lack of resolution in the numerical solution of complex problems through the use of prior statistical information. We point out a relation between optimal prediction and the statistical mechanics of irreversible processes, and use a version of the Mori-Zwanzig formalism to produce a higher-order optimal prediction method.

Unresolved computation and optimal predictions

We present methods for predicting the solution of time-dependent partial differential equations when that solution is so complex that it cannot be properly resolved numerically, but when prior statistical information can be found. The sparse numerical data are viewed as constraints on the solution, and the gist of our proposal is a set of methods for advancing the constraints in time so that regression methods can be used to reconstruct the mean future. For linear equations we offer general recipes for advancing the constraints; the methods are generalized to certain classes of nonlinear problems, and the conditions under which strongly nonlinear problems and partial statistical information can be handled are briefly discussed. Our methods are related to certain data acquisition schemes in oceanography and meteorology. © John Wiley & Sons, Inc.