Parametric Probability Density Research Articles

Sequential Markov Chain Monte Carlo (MCMC) model discrimination

AbstractIn this paper a new approach to model discrimination is presented that takes advantage of Markov Chain Monte Carlo (MCMC) methods. It combines an experimental criterion first proposed by Roth (Roth, Design of Experiments for Discrimination Among rival Models, PhD, Thesis, Princeton University, New Jersey, USA, 1965) with an adaptation of a model selection method described by Chib and Jeliazkov [Chib and Jeliazkov, Stat. Neerl. 59, 30–44 (2005)], which uses an Acceptance–Rejection Metropolis–Hastings algorithm to evaluate the model marginal likelihood thus enabling the calculation of model posterior probabilities. It does so without requiring any linearisation of nonlinear models. In designing model discrimination experiments using the Roth criterion, MCMC methods are again used to find the mean of the predicted values by integrating over the entire parameter probability density function. The method is illustrated using the well‐known chemical reaction kinetics example first discussed by Box and Hill [Box and Hill, Technometrics 9, 57–71 (1967)]. The results indicate that the method is very successful in identifying the correct model. Higher error levels and more complex kinetics require on average more model discrimination experiments. © 2012 Canadian Society for Chemical Engineering

Baseline Probabilities for the Seasonal Prediction of Meteorological Drought

AbstractThe inherent persistence characteristics of various drought indicators are quantified to extract predictive information that can improve drought early warning. Predictive skill is evaluated as a function of the seasonal cycle for regions within North America. The study serves to establish a set of baseline probabilities for drought across multiple indicators amenable to direct comparison with drought indicator forecast probabilities obtained when incorporating dynamical climate model forecasts. The emphasis is on the standardized precipitation index (SPI), but the method can easily be applied to any other meteorological drought indicator, and some additional examples are provided. Monte Carlo resampling of observational data generates two sets of synthetic time series of monthly precipitation that include, and exclude, the annual cycle while removing serial correlation. For the case of no seasonality, the autocorrelation (AC) of the SPI (and seasonal precipitation percentiles, moving monthly averages of precipitation) decays linearly with increasing lag. It is shown that seasonality in the variance of accumulated precipitation serves to enhance or diminish the persistence characteristics (AC) of the SPI and related drought indicators, and the seasonal cycle can thereby provide an appreciable source of drought predictability at regional scales. The AC is used to obtain a parametric probability density function of the future state of the SPI that is based solely on its inherent persistence characteristics. In addition, a method is presented for determining the optimal persistence of the SPI for the case of no serial correlation in precipitation (again, the baseline case). The optimized, baseline probabilities are being incorporated into Internet-based tools for the display of current and forecast drought conditions in near–real time.

혈관 내 초음파 영상에서 내강 경계면 자동 분할

혈관 내 초음파 영상(IVUS: Intravascular ultrasoundimages)에서 내강(Lumen) 경계 영역을 검출하는 것은 환자의 심혈관 상태를 파악하는데 중요한 정보를 제공하며, 이를 통해 심혈관계 질환을 예측하고 진단할 수 있다. 따라서 정확하게 내강 경계를 분할하는 것은 매우 중요한 단계이다. 본 논문에서는 비모수적 확률 밀도 함수와 스무딩 함수를 사용하여 자동으로 내강 영역을 분할하는 기법을 제안한다. 각각의 혈관내 초음파 영상들을 극좌표 이미지로 변환 후 웨이블릿 변환을 적용하여 초기 관심 점들을 검출한다. 초기 관심점들 중에서 잡음과 칼슘에 의해 발생된 튀는 점들을 제거하기 위해 비모수적 밀도 함수와 스무딩 함수를 이용하여 튀는 점들을 제거하고 경계면에 해당하는 중요 관심 점만을 남긴다. 마지막으로, 다항곡선 접합(Polynomial curve fitting) 함수를 정의하고 다항식과 실제 내강 경계선에 접합된 관심 점을 이용하여 자연스러운 내강 경계면을 추정한다. 본 논문에서 제안한 방법을 다양한 초음파 영상에 대해 실험한 결과, 기존에 제안된 방법 보다 정확하게 경계면을 검출함을 알 수 있었다. Accurately segmenting lumen border in intravascular ultrasound images (IVUS) is very important to study vascular wall architecture for diagnosis of the cardiovascular diseases. After each of IVUS image is transformed to a polar coordinated image, initial points are detected using wavelet transform. Then, lumen border is initialized as the set of important points using non parametric probability density function and smoothing function by removing outlier initial points occurred by noises and artifacts. Finally, polynomial curve fitting is applied to obtain real lumen border using filtered important points. The evaluation of proposed method was performed with related method and the proposed method produced accurate lumen contour detection when compared to another method in most types of IVUS images.

Partially Adaptive Estimation of Interval Censored Regression Models

Several valuable data sources, including the census and National Longitudinal Survey of Youth, include data measured using interval responses. Many empirical studies attempt estimation by assuming the data correspond to the interval midpoints and then use OLS or maximum likelihood assuming normality. Stata performs maximum likelihood estimates (MLE) under the assumption of normality, allowing for intra-group variation. In the presence of heteroskedasticity or distributional misspecification, these estimates are inconsistent. In this paper we focus on an estimation procedure that helps prevent distributional misspecification for interval censored data. We explore the application of partially adaptive estimation, which builds on the MLE framework with families of flexible parametric probability density functions which include the normal as a limiting case. These methods are used to estimate determinants associated with household expenditures based on US Census data. Monte Carlo Simulations are performed to compare the relative efficiency of the different methods of estimation. We find that the flexible nature of our proposed partially adaptive estimation technique significantly reduces estimator bias and improves efficiency in the presence of distributional misspecification.

Interpreting LHC SUSY searches in the phenomenological MSSM

Abstract We interpret within the phenomenological MSSM (pMSSM) the results of SUSY searches published by the CMS collaboration based on the first ~1 fb−1 of data taken during the 2011 LHC run at 7 TeV. The pMSSM is a 19-dimensional parametrization of the MSSM that captures most of its phenomenological features. It encompasses, and goes beyond, a broad range of more constrained SUSY models. Performing a global Bayesian analysis, we obtain posterior probability densities of parameters, masses and derived observables. In contrast to constraints derived for particular SUSY breaking schemes, such as the CMSSM, our results provide more generic conclusions on how the current data constrain the MSSM.

A flood pulse driven fish population model for the Okavango Delta, Botswana

The Okavango Delta is a large, remote, and ecologically significant wetland located in Botswana that receives a strong annual flood pulse. Although the hydrology in flood pulsed systems is often theorized to drive fish population dynamics, in the Okavango Delta there are no monitoring or modeling studies that quantify this complex ecological relationship. The objective of this work was to produce and analyze a mechanistic fish population model of the Okavango Delta that is driven by the annual flood pulse in order to corroborate the theory that Delta fish populations are driven by the flood signal. The model tracked age cohorts over time with density dependant recruitment, mortality, and vulnerability components. Global sensitivity analysis identified the parameters that were the most important in determining the model outcome. Monte Carlo filtering truncated prior parameter probability density functions and refined model uncertainty. One of the unique outcomes of this research was the identification of polishing parameters, i.e. model parameters that are essential in obtaining optimal model performance by matching output variability, though they are not important in changing the magnitude of model results. The flood coefficient (a scaling factor that describes how recruitment changes with the magnitude of the flood) was shown to be a polishing parameter, providing quantitative evidence that floods are a driver of fish population dynamics in the Delta. This linkage between the flood pulse and fish population dynamics provides quantitative information that is necessary for making informed decisions regarding the management of hydrologic and ecological resources in the Okavango Delta.

Evaluation of C-band SAR polarimetric parameters for discrimination of first-year sea ice types

In this study, the classification potential of polarimetric parameters derived after Cloude–Pottier decomposition, Touzi decomposition, Freeman–Durden decomposition, normalized radar cross section measurements, phase differences, and statistical synthetic aperture radar correlation measures is evaluated by relating them to three pre-identified sea ice types and wind-roughened open water. A combined approach that constitutes a visual inspection of estimated probability densities of the polarimetric parameters and quantitative analysis using supervised classifications (k means and maximum likelihood) is adopted. Polarimetric parameters are iteratively combined in pairs and triplets to test for their ice type discrimination potential. Sensitivity of polarimetric parameters to radar incidence angle is also examined. Our results demonstrated strong but variable sensitivity of polarimetric parameters to different ice types, which was dependent on radar incidence angle. Results of parameter evaluation demonstrated that no single parameter discriminates significantly (>60%) between all the ice types considered in the study. Combining two low correlated parameters increased the classification accuracy by 10%–22%. Combining the third polarimetric parameter did not necessarily improve the classification results. However, the best classification results were achieved using a combination of three parameters.

Stochastic inversion for electromagnetic geophysics: Practical challenges and improving convergence efficiency

Traditional deterministic geophysical inversion algorithms are not designed to provide a robust evaluation of uncertainty that reflects the limitations of the geophysical technique. Stochastic inversions, which do provide a sampling-based measure of uncertainty, are computationally expensive and not straightforward to implement for nonexperts (nonstatisticians). Our results include stochastic inversion for magnetotelluric and controlled source electromagnetic data. Two Markov Chain sampling algorithms (Metropolis-Hastings and Slice Sampler) can significantly decrease the computational expense compared to using either sampler alone. The statistics of the stochastic inversion allow for (1) variances that better reveal the measurement sensitivities of the two different electromagnetic techniques than traditional techniques and (2) models defined by the median and modes of parameter probability density functions, which produce amplitude and phase data that are consistent with the observed data. In general, parameter error estimates from the covariance matrix significantly underestimate the true parameter error, whereas the parameter variance derived from Markov chains accurately encompass the error.

Geoacoustic inversion from ambient noise data using a trans-dimensional Bayesian approach

Geoacoustic inversion of seabed parameters from ambient noise in shallow water is a promising technique, with potential advantages over active survey methods such as low environmental impact, easier deployment procedures, and less restrictive hardware requirements. Bayesian inversion provides a framework to estimate the posterior probability density (PPD) of geoacoustic parameters. Parameter and uncertainty estimates can be obtained from PPD moments, such as the maximum a posteriori model, means, correlations, and marginal distributions. A fundamental step in the inversion is the selection of a model parametrization (i.e., number of seabed layers) consistent with the data information content. Recent developments in Bayesian inversion of seismic and active-source acoustic data have considered a trans-dimensional approach to model selection, where the number of model parameters is treated as unknown. Different models are sampled according to their support by the data, accounting for parametrization uncertainty in the geoacoustic parameter uncertainty estimates. This work applies a trans-dimensional reversible-jump Markov chain Monte Carlo algorithm to ambient noise reflection coefficient data. The approach is demonstrated with data collected on the Malta Plateau using a vertical line array. [Work supported by ONR.]

A robust probabilistic Braille recognition system

As a structured document, Braille is the most common means of reading and study for visually handicapped people. The need for converting Braille documents into a computer-readable format has motivated research into the implementation of Braille recognition systems. The main theme of this research is to propose robust probabilistic approaches to different steps of Braille Recognition. The method is meant to be very general in terms of being independent of those parameters of the Braille document such as skewness, scale, and spacing of the page, lines, and characters. For a given Braille document, a statistical method is proposed for estimating the scaling, spacing, and skewness parameters, whereby the detected dots of the Braille document are modeled using a parameterized probability density function. Skewness, scaling, and line spacing are estimated as a solution of a maximum-likelihood (ML) problem, which is solved using expectation maximization. Based on those parameters, each line of the Braille document is extracted, and each of three rows of individual lines is separated based on the vertical projection of the Braille dots. Finally, a scale-independent automatic document gridding procedure is proposed for dot localization and character detection based on a hidden Markov model.

TYPE Ia SUPERNOVA LIGHT CURVE INFERENCE: HIERARCHICAL MODELS IN THE OPTICAL AND NEAR-INFRARED

We have constructed a comprehensive statistical model for Type Ia supernova (SN Ia) light curves spanning optical through near-infrared (NIR) data. A hierarchical framework coherently models multiple random and uncertain effects, including intrinsic supernova (SN) light curve covariances, dust extinction and reddening, and distances. An improved BayeSN Markov Chain Monte Carlo code computes probabilistic inferences for the hierarchical model by sampling the global probability density of parameters describing individual SNe and the population. We have applied this hierarchical model to optical and NIR data of 127 SNe Ia from PAIRITEL, CfA3, Carnegie Supernova Project, and the literature. We find an apparent population correlation between the host galaxy extinction AV and the ratio of total-to-selective dust absorption RV. For SNe with low dust extinction, AV ≲ 0.4, we find RV ≈ 2.5–2.9, while at high extinctions, AV ≳ 1, low values of RV < 2 are favored. The NIR luminosities are excellent standard candles and are less sensitive to dust extinction. They exhibit low correlation with optical peak luminosities, and thus provide independent information on distances. The combination of NIR and optical data constrains the dust extinction and improves the predictive precision of individual SN Ia distances by about 60%. Using cross-validation, we estimate an rms distance modulus prediction error of 0.11 mag for SNe with optical and NIR data versus 0.15 mag for SNe with optical data alone. Continued study of SNe Ia in the NIR is important for improving their utility as precise and accurate cosmological distance indicators.

Inverse problems in cancellous bone: estimation of the ultrasonic properties of fast and slow waves using Bayesian probability theory.

Quantitative ultrasonic characterization of cancellous bone can be complicated by artifacts introduced by analyzing acquired data consisting of two propagating waves (a fast wave and a slow wave) as if only one wave were present. Recovering the ultrasonic properties of overlapping fast and slow waves could therefore lead to enhancement of bone quality assessment. The current study uses Bayesian probability theory to estimate phase velocity and normalized broadband ultrasonic attenuation (nBUA) parameters in a model of fast and slow wave propagation. Calculations are carried out using Markov chain Monte Carlo with simulated annealing to approximate the marginal posterior probability densities for parameters in the model. The technique is applied to simulated data, to data acquired on two phantoms capable of generating two waves in acquired signals, and to data acquired on a human femur condyle specimen. The models are in good agreement with both the simulated and experimental data, and the values of the estimated ultrasonic parameters fall within expected ranges.

Bayesian inversion of microtremor array dispersion data in southwestern British Columbia

This paper applies Bayesian inversion, with evaluation of data errors and model parametrization, to produce the most-probable shear-wave velocity profile together with quantitative uncertainty estimates from microtremor array dispersion data. Generally, the most important property for characterizing earthquake site response is the shear-wave velocity (VS) profile. The microtremor array method determines phase velocity dispersion of Rayleigh surface waves from multi-instrument recordings of urban noise. Inversion of dispersion curves for VS structure is a non-unique and non-linear problem such that meaningful evaluation of confidence intervals is required. Quantitative uncertainty estimation requires not only a non-linear inversion approach that samples models proportional to their probability, but also rigorous estimation of the data error statistics and an appropriate model parametrization. This paper applies a Bayesian formulation that represents the solution of the inverse problem in terms of the posterior probability density (PPD) of the geophysical model parameters. Markov-chain Monte Carlo methods are used with an efficient implementation of Metropolis-Hastings sampling to provide an unbiased sample from the PPD to compute parameter uncertainties and inter-relationships. Nonparametric estimation of a data error covariance matrix from residual analysis is applied with rigorous a posteriori statistical tests to validate the covariance estimate and the assumption of a Gaussian error distribution. The most appropriate model parametrization is determined using the Bayesian information criterion, which provides the simplest model consistent with the resolving power of the data. Parametrizations considered vary in the number of layers, and include layers with uniform, linear and power-law gradients. Parameter uncertainties are found to be underestimated when data error correlations are neglected and when compressional-wave velocity and/or density (nuisance) parameters are fixed in the inversion. Bayesian inversion of microtremor array data is applied at two sites in British Columbia, the area of highest seismic risk in Canada, to study the ability to recover an accurate VS profile in relatively deep and shallow geological settings on the Fraser River delta in Greater Vancouver and in Victoria, respectively. A well-resolved VS profile to at least 110 m depth is determined at the Fraser River delta site for a power-law gradient parametrization. At the Victoria site, a layer with low VS and a weak linear gradient is indicated to 15–18 m depth, above much higher velocity material. Invasive VS measurements from seismic cone penetration testing and/or surface-to-downhole methods are used to assess the reliability of the Bayesian microtremor inversion results, with excellent agreement obtained at both sites.

ADVANCES IN CROSS-ENTROPY METHODS

Abstract

Improving land surface models with FLUXNET data

Abstract. There is a growing consensus that land surface models (LSMs) that simulate terrestrial biosphere exchanges of matter and energy must be better constrained with data to quantify and address their uncertainties. FLUXNET, an international network of sites that measure the land surface exchanges of carbon, water and energy using the eddy covariance technique, is a prime source of data for model improvement. Here we outline a multi-stage process for "fusing" (i.e. linking) LSMs with FLUXNET data to generate better models with quantifiable uncertainty. First, we describe FLUXNET data availability, and its random and systematic biases. We then introduce methods for assessing LSM model runs against FLUXNET observations in temporal and spatial domains. These assessments are a prelude to more formal model-data fusion (MDF). MDF links model to data, based on error weightings. In theory, MDF produces optimal analyses of the modelled system, but there are practical problems. We first discuss how to set model errors and initial conditions. In both cases incorrect assumptions will affect the outcome of the MDF. We then review the problem of equifinality, whereby multiple combinations of parameters can produce similar model output. Fusing multiple independent and orthogonal data provides a means to limit equifinality. We then show how parameter probability density functions (PDFs) from MDF can be used to interpret model validity, and to propagate errors into model outputs. Posterior parameter distributions are a useful way to assess the success of MDF, combined with a determination of whether model residuals are Gaussian. If the MDF scheme provides evidence for temporal variation in parameters, then that is indicative of a critical missing dynamic process. A comparison of parameter PDFs generated with the same model from multiple FLUXNET sites can provide insights into the concept and validity of plant functional types (PFT) – we would expect similar parameter estimates among sites sharing a single PFT. We conclude by identifying five major model-data fusion challenges for the FLUXNET and LSM communities: (1) to determine appropriate use of current data and to explore the information gained in using longer time series; (2) to avoid confounding effects of missing process representation on parameter estimation; (3) to assimilate more data types, including those from earth observation; (4) to fully quantify uncertainties arising from data bias, model structure, and initial conditions problems; and (5) to carefully test current model concepts (e.g. PFTs) and guide development of new concepts.

Cassini Radar Data: Estimation of Titan's Lake Features by Means of a Bayesian Inversion Algorithm

The analysis derived from the Cassini SAR imagery reflects the complex Titan's surface morphology with a wide range of backscattering coefficients and peculiar features such as periodic structures and lakelike features, which were observed on July 22, 2006, when polar areas were first imaged, and are considered good candidates to be filled with liquid hydrocarbons. In this paper, the modeling description of lakes is addressed by means of a double-layer model which considers an upper liquid-hydrocarbon layer and a lower layer compatible with the radar response of the neighboring areas. This model is introduced into a Bayesian framework for the purpose of inferring the likely ranges of some parameters and, in particular, of the optical thickness of the hypothesized liquid-hydrocarbon layer and of the wind speed. The main idea is to use the information contained in the parameter probability density function, which describes how probability is distributed among the different values of parameters according to the various scenarios considered. The analysis carried out on lakes and surrounding areas on flybys T16 and T19 determines optical thickness values from 0.2 to 6. For T25 flyby, the inferred values of optical thickness indicate that a limit value of optical thickness may be 9. Considering that, beyond these values, the signal from the bottom layer is completely attenuated, information on the wind speed on the upper layer can be inferred. The found mean values of wind speed are around 0.2-0.3 m/s according to different hypotheses on the upper layer dielectric constant.

Damage identification in vibrating rotor-shaft systems by efficient sampling approach

In the paper a stochastic method for fault detection and identification in the shafts of rotating machines is proposed. This approach is based on the Monte Carlo simulations of rotor-shaft lateral–torsional–longitudinal vibrations mutually coupled by transverse cracks of various possible and randomly selected depths and locations on the shaft. For this purpose the structural hybrid model of a real faulty object is applied. This model is characterized by a high practical reliability and great computational efficiency, so important for many hundred thousand single numerical simulations necessary for a creation of the databases applied for inverse problem solution finally leading to crack identification. These databases are created with an arbitrary assumed probability densities of crack parameters which ensures appropriate spread of the dynamic responses of the considered faulty mechanical system. A sufficiently large database determined for the investigated object enable us to estimate almost immediately, i.e. within less than 1 s, the crack depth and axial position with identification errors not exceeding 9% and 5%, respectively. Thus, the proposed method seems to be a very convenient diagnostic tool for engineering applications in the industry.

Parametric Reduced-Order Models for Probabilistic Analysis of Unsteady Aerodynamic Applications

DOI: 10.2514/1.35850 We address the problem of propagating input uncertainties through a computational fluid dynamics model. Methods such as Monte Carlo simulation can require many thousands (or more) of computational fluid dynamics solves, rendering them prohibitively expensive for practical applications. This expense can be overcome with reduced-order models that preserve the essential flow dynamics. The specific contributions of this paper are as follows: first, to derive a linearized computational fluid dynamics model that permits the effects of geometry variations to be represented with an explicit affine function; second, to propose an adaptive sampling method to derive a reduced basis that is effective over the joint probability density of the geometry input parameters. The method is applied to derive efficient reduced models for probabilistic analysis of a two-dimensional problem governedbythelinearized Eulerequations.Reduced-order modelsthatachieve 3-orders-of-magnitude reduction in the number of states are shown to accurately reproduce computational fluid dynamics Monte Carlo simulation results at a fraction of the computational cost.

Uncertainty estimation in simultaneous Bayesian tracking and environmental inversion

This paper develops a Bayesian approach for two related inverse problems: tracking an acoustic source when ocean environmental parameters are unknown, and determining environmental parameters using acoustic data from an unknown (moving) source. The formulation considers source and environmental parameters as unknown random variables constrained by noisy acoustic data and by prior information on parameter values (e.g., physical limits for environmental properties) and on inter-parameter relationships (limits on radial and vertical source speed). The goal is not simply to estimate parameter values, but to rigorously determine parameter uncertainty distributions, thereby quantifying the information content of the data/prior to resolve source and environmental parameters. Results are presented as marginal posterior probability densities (PPDs) for environmental parameters and joint marginal PPDs for source ranges and depths. Given the numerically intensive inversion, an efficient Markov-chain Monte Carlo importance-sampling approach is developed which combines Metropolis and heat-bath Gibbs' sampling, employs efficient proposal distributions based on a linearized PPD approximation, and considers nonunity sampling temperatures to ensure a complete parameter search. The approach is illustrated with two simulated examples representing tracking a quiet submerged source and geoacoustic inversion using noise from an unknown ship of opportunity. In both cases, source, seabed, and water-column parameters are unknown.

Statistical conversion of speech parameter trajectory for mapping between features of different modalities

A state-of-the-art speech parameter conversion technique and its application to a mapping between features of different modalities are reviewed. Many statistical approaches to the parameter conversion have been studied particularly for voice conversion in speech synthesis research. A typical method conducts the parameter conversion frame by frame based on the minimum mean square error using a Gaussian mixture model of the joint probability density of input and output parameters [Y. Stylianou et al., IEEE Trans. SAP, 6(2), 131-142 (1998)]. Although this method is reasonably effective, the deterioration of the conversion accuracy is caused by essential problems of the frame-based conversion process. Recently a conversion method based on the maximum likelihood estimation of a parameter trajectory has been proposed [T. Toda et al., IEEE Trans. ASLP, 15(8), 2222-2235 (2007)]. This method realizes the appropriate converted parameter sequence by (1) using not only static but also dynamic feature statistics and (2) considering a global variance feature of the converted parameters. It has been reported that this method is effective in several applications such as a spectral determination from articulatory movements, an acoustic-to-articulatory inversion mapping, and a conversion of body-transmitted speech into air-transmitted speech.