Model Parameter Estimation Problem Research Articles

Using the Markov chain Monte Carlo method to estimate model order

Markov chain Monte Carlo (MCMC) methods are widely used in the solution of parameter estimation problems arising in acoustics and other applications. The use of MCMC to estimate the parameters of a single model is well established. However, in many applications, there is not a single model for the data but rather a number of competing models. A common method of dealing with multiple models is to use MCMC to compute the posterior probability and estimate the parameter values of each model in turn. However, for problems with many models, it is more efficient to combine the parameter spaces of all models into a single space and use MCMC to perform across-model sampling of the joint space. Although the development of an MCMC algorithm of this sort is sufficiently difficult so as to be unprofitable for the non-specialist, the acoustician wishing to solve their multi model parameter estimation problem using MCMC can still do so using an existing algorithm. This presentation gives an overview and brief tutorial of MCMC for parameter estimation and then discusses and gives an example of using the open source computer program BayeSys [Skilling, 2004] to determine the model order of a simple atomic model.

Objective, Bayesian Analysis of Fluorescence Correlation Spectroscopy Data

Fluctuations in fluorescence intensity provide quantitative insight into a range of molecular processes including diffusion, active transport, and binding kinetics in living cells and tissues. Classical fluorescence correlation spectroscopy (FCS) measures fluctuations in fluorescence intensity in a small detection volume to infer molecular transport properties from governing continuum diffusion-convection equations. Essential to the interpretation of FCS data is the use of an underlying mathematical model that governs the molecular process under study. Autocorrelations in fluctuations in fluorescence intensity may then be fit to the analytical solution of the model, and corresponding molecular properties may be inferred. While the choice of physical process may be unambiguous for special cases of free molecular diffusion or convective transport in solution, the correct choice of model becomes considerably less unambiguous in the application of FCS to complex biological processes in living cells or extracellular matrices. For this reason, an objective and unbiased approach to model selection and parameter estimation is of interest. The method of Bayesian inference provides such a framework. At the level of parameter estimation, Bayesian inference is similar to Maximum Likelihood Estimation when flat priors are chosen for initial parameter estimates. At the level of model selection, however, Bayesian inference assigns explicit model probabilities that are proportional to their marginal likelihoods by considering the full range of parameter values and their posterior probability distributions rather than only using point-wise, Maximum Likelihood estimates, thereby appropriately penalizing model complexity and preventing over-fitting of experimental data. Here we illustrate application of Bayesian inference to the problem of model selection and parameter estimation of molecular transport properties from FCS data, comparing it to traditional model selection approaches.

A MEASURE-THEORETIC COMPUTATIONAL METHOD FOR INVERSE SENSITIVITY PROBLEMS I: METHOD AND ANALYSIS.

We consider the inverse sensitivity analysis problem of quantifying the uncertainty of inputs to a deterministic map given specified uncertainty in a linear functional of the output of the map. This is a version of the model calibration or parameter estimation problem for a deterministic map. We assume that the uncertainty in the quantity of interest is represented by a random variable with a given distribution, and we use the law of total probability to express the inverse problem for the corresponding probability measure on the input space. Assuming that the map from the input space to the quantity of interest is smooth, we solve the generally ill-posed inverse problem by using the implicit function theorem to derive a method for approximating the set-valued inverse that provides an approximate quotient space representation of the input space. We then derive an efficient computational approach to compute a measure theoretic approximation of the probability measure on the input space imparted by the approximate set-valued inverse that solves the inverse problem.

Joint Model Selection and Parameter Estimation by Population Monte Carlo Simulation

In this paper, we study the problem of joint model selection and parameter estimation under the Bayesian framework. We propose to use the Population Monte Carlo (PMC) methodology in carrying out Bayesian computations. The PMC methodology has recently been proposed as an efficient sampling technique and an alternative to Markov Chain Monte Carlo (MCMC) sampling. Its flexibility in constructing transition kernels allows for joint sampling of parameter spaces that belong to different models. The proposed method is able to estimate the desired a posteriori distributions accurately. In comparison to the Reversible Jump MCMC (RJMCMC) algorithm, which is popular in solving the same problem, the PMC algorithm does not require burn-in period, it produces approximately uncorrelated samples, and it can be implemented in a parallel fashion. We demonstrate our approach on two examples: sinusoids in white Gaussian noise and direction of arrival (DOA) estimation in colored Gaussian noise, where in both cases the number of signals in the data is a priori unknown. Both simulations show the effectiveness of our proposed algorithm.

On the Relation Between Sparse Reconstruction and Parameter Estimation With Model Order Selection

We examine the relationship between sparse linear reconstruction and the classic problem of continuous parametric modeling. In sparse reconstruction, one wishes to recover a sparse amplitude vector from a measurement that is described as a linear combination of a small number of discrete additive components. Recent results in the compressive sensing literature have provided fast sparse reconstruction algorithms with guaranteed performance bounds for problems with certain structure. In this paper, we show an explicit connection between sparse reconstruction and parameter/order estimation and demonstrate how sparse reconstruction may be used to solve model order selection and parameter estimation problems. The structural assumption used in compressive sensing to guarantee reconstruction performance-the Restricted Isometry Property-is not satisfied in the general parameter estimation context. Nonetheless, we develop a method for selecting sparsity parameters such that sparse reconstruction mimics classic order selection criteria such as Akaike information criterion (AIC) and Bayesian information criterion (BIC). We compare the performance of the sparse reconstruction approach with traditional model order selection/parameter estimation techniques for a sinusoids-in-noise example. We find that the two methods have comparable performance in most cases, and that sparse linear modeling performs better than traditional model-based parameter/order estimation for closely spaced sinusoids with low signal-to-noise ratio.

Approximating stochastic biochemical processes with Wasserstein pseudometrics

Modelling stochastic processes inside the cell is difficult due to the size and complexity of the processes being investigated. As a result, new approaches are needed to address the problems of model reduction, parameter estimation, model comparison and model invalidation. Here, the authors propose addressing these problems by using Wasserstein pseudometrics to quantify the differences between processes. The method the authors propose is applicable to any bounded continuous-time stochastic process and pseudometrics between processes are defined only in terms of the available outputs. Algorithms for approximating Wasserstein pseudometrics are developed from experimental or simulation data and demonstrate how to optimise parameter values to minimise the pseudometrics. The approach is illustrated with studies of a stochastic toggle switch and of stochastic gene expression in E. coli.

Comparison of different algorithms for simultaneous estimation of multiple parameters in kinetic metabolic models.

Computational models in systems biology are usually characterized by a lack of reliable parameter values. This is especially true for kinetic metabolic models. Experimental data can be used to estimate these missing parameters. Different optimization techniques have been explored to solve this challenging task but none has proved to be superior to the other. In this paper we review the problem of parameter estimation in kinetic models. We focus on the suitability of four commonly used optimization techniques of parameter estimation in biochemical pathways and make a comparison between those methods. The suitability of each technique is evaluated based on the ability of converging to a solution within a reasonable amount of time. As most local optimization methods fail to arrive at a satisfactory solution we only considered the global optimization techniques. A case study of the upper part of Glycolysis consisting 15 parameters is taken as the benchmark model for evaluating these methods.

Optimal designs for both model discrimination and parameter estimation

The KL-optimality criterion has been recently proposed to discriminate between any two statistical models. However, designs which are optimal for model discrimination may be inadequate for parameter estimation. In this paper, the DKL-optimality criterion is proposed which is useful for the dual problem of model discrimination and parameter estimation. An equivalence theorem and a stopping rule for the corresponding iterative algorithms are provided. A pharmacokinetics application and a bioassay example are given to show the good properties of a DKL-optimum design.

FS–TARMA Models for Non–Stationary Vibration Analysis: An overview and comparison

The problem of non–stationary vibration modelling and analysis via Functional Series Time-dependent Autoregressive Moving Average (FS-TARMA) models is addressed. An overview of the methods for the FS-TARMA model parameter estimation and structure selection problems is presented, with their relative merits highlighted. An experimental comparison and assessment study based on the output–only identification of the time–varying dynamics of a retractable arm structure is performed.

A Bayesian framework to identify principal intravoxel diffusion profiles based on diffusion-weighted MR imaging

In this paper we introduce a new method to characterize the intravoxel anisotropy based on diffusion-weighted imaging (DWI). The proposed solution, under a fully Bayesian formalism, deals with the problem of joint Bayesian Model selection and parameter estimation to reconstruct the principal diffusion profiles or primary fiber orientations in a voxel. We develop an efficient stochastic algorithm based on the reversible jump Markov chain Monte Carlo (RJMCMC) method in order to perform the Bayesian computation. RJMCMC is a good choice for this problem because of its ability to jump between models of different dimensionality. This methodology provides posterior estimates of the parameters of interest (fiber orientation, diffusivities etc) unconditional of the model assumed. It also gives an empirical posterior distribution of the number of primary nerve fiber orientations given the DWI data. Different probability maps can be assessed using this methodology: 1) the intravoxel fiber orientation map (or orientational distribution function) that gives the probability of finding a fiber in a particular spatial orientation; 2) a three-dimensional map of the probability of finding a particular number of fibers in each voxel; 3) a three-dimensional MaxPro (maximum probability) map that provides the most probable number of fibers for each voxel.In order to study the performance and reliability of the presented approach, we tested it on synthetic data; an ex-vivo phantom of intersecting capillaries; and DWI data from a human subject.

Time-domain Bayesian detection and estimation of noisy damped sinusoidal signals applied to NMR spectroscopy

The problem of model detection and parameter estimation for noisy signals arises in different areas of science and engineering including audio processing, seismology, electrical engineering, and NMR spectroscopy. We have adopted the Bayesian modeling framework to jointly detect and estimate signal resonances. This considers a model of the time-domain complex free induction decay (FID) signal as a sum of exponentially damped sinusoidal components. The number of model components and component parameters are considered unknown random variables to be estimated. A Reversible Jump Markov Chain Monte Carlo technique is used to draw samples from the joint posterior distribution on the subspaces of different dimensions. The proposed algorithm has been tested on synthetic data, the 1H NMR FID of a standard of l-glutamic acid and a blood plasma sample. The detection and estimation performance is compared with Akaike information criterion (AIC), minimum description length (MDL) and the matrix pencil method. The results show the Bayesian algorithm superior in performance especially in difficult cases of detecting low-amplitude and strongly overlapping resonances in noisy signals.

Study on joint Bayesian model selection and parameter estimation method of GTD model

The Bayesian method is applied to the joint model selection and parameter estimation problem of the GTD model. An algorithm based on RJ-MCMC is designed. This algorithm not only improves the model order selection and parameter estimation accuracy by exploiting the priori information of the GTD model, but also solves the mixed parameter estimation problem of the GTD model properly. Its performance is tested using numerical simulations and data generated by electromagnetic code. It is shown that it gives good model order selection and parameter estimation results, especially for low SNR, closely-spaced components and short data situations.

MULTIPLE STRATEGIES FOR PARAMETER ESTIMATION VIA A HYBRID METHOD: A COMPARATIVE STUDY

In this paper, two different strategies are considered for application of a previously proposed hybrid method designed to parameter estimation. This method combines the feedforward neural networks ability to produce initial parameter estimates close to the true values with the fast convergence of the Levenberg-Marquardt method using such estimates. The first strategy is of general applicability, while the second one is intended for models having a structure defined by various blocks in series. The neuromuscular blockade model parameter estimation problem is taken as the case study for comparing the performances of the two strategies.

On the interpretation of a continuous-time model identification method in terms of regularization

This paper presents an interpretation of a continuous-time model identification method in terms of regularization. We show that, in the case of linear filter method, data filtering corresponds to a regularized derivative estimation. We also give the explicit form of the minimized criterion. In addition, we propose a new structure based on the use of a true regularization filter whose performances are compared with the GPMF method. Finally, we propose a new formulation of continuous-time model identification as a joint output signal and model parameter estimation problem.

Criterion-robust optimal designs for model discrimination and parameter estimation in Fourier regression models

Consider the problem of discriminating between two competitive Fourier regression on the circle [− π, π] and estimating parameters in the models. To find designs which are efficient for both model discrimination and parameter estimation, Zen and Tsai (some criterion-robust optimal designs for the dual problem of model discrimination and parameter estimation, Indian J. Statist. 64, 322–338) proposed a multiple-objective optimality criterion for polynomial regression models. In this work, taking the same M γ -criterion, which puts weight γ (0⩽γ⩽1) for model discrimination and 1− γ for parameter estimation, and using the techniques of projection design, the corresponding M γ -optimal design for Fourier regression models is explicitly derived in terms of canonical moments. The behavior of the M γ -optimal designs is investigated under different weighted selection criterion. And the extreme value of the minimum M γ -efficiency of any M γ′ -optimal design is obtained at γ′=γ ∗ , which results in the M γ ∗ -optimal design to be served as a criterion-robust optimal design for the problem.

Nonlinear parameter identification of models for masonry

This paper investigates the parameter estimation problem for brick masonry models. An identification procedure is proposed in which the uncertainties of known parameters and/or errors of measurements are its elements of distinction. The minimization process of the discrepancies between experimental data and theoretical measurements takes place by means of a first order iterative method. The identification procedure is applied to two different problems: the calibration of an interface model for brick–mortar joint in its functional form through monotonic experimental tests; to evaluate the unknown parameters of a continuum model for brick masonry walls in its non-holonomic form by means of in-plane cyclic shear–compression test of masonry panels. The general framework of the non-linear estimate methodology, the parameter identification problems and the numerical results are presented.

Simultaneous parameter estimation and model structure determination in FTIR spectroscopy by global MINLP optimization

We address the problem of simultaneous model structure determination and parameter estimation in infrared spectroscopy. For given measurements of concentrations ( C) and absorbances ( A), we seek to find the constant of analogy (Θ) in reverse Beer's law ( C=Θ A). Two approaches are described and compared in this paper. Both utilize Akaike's information criterion (AIC) to obtain an estimate of the constant. The first method is frequently used in practice and requires the iterative solution of mixed-integer convex quadratic optimization problems. The second method is a novel one that requires the solution of a single mixed-integer nonconvex nonlinear program for which we develop a global optimization algorithm. Computational results demonstrate that the latter approach provides better solutions for all of the eleven problems solved in this paper. Our computational experiments also reveal the importance of bounding the errors and number of model parameters when minimizing AIC.

Bayesian analysis of generalized frequency-modulated signals

General frequency-modulated (GFM) signals can be used to characterize many vibrations in dynamic environments, with applications to engine monitoring and sonar. Most work into parameter estimation of such signals assumes knowledge of the number of carrier frequencies. In this paper, we make no such assumption and use Bayesian techniques to address jointly the problem of model selection and parameter estimation. Following the work of Andrieu and Doucet (see ibid., vol.47, p.2667-76, 1999), who addressed the problem for nonmodulated sinusoids, a posterior distribution for the parameters and model order is obtained. This distribution is too complicated for analytical extraction of moments and to sample from directly; therefore, we use a reversible jump Markov chain Monte Carlo (RJMCMC) algorithm to draw samples from the distribution. Some simulated examples are presented to illustrate the algorithm's performance.

Radioisotope tracer study in a sludge hygienization research irradiator (SHRI)

A radioisotope tracer study has been carried out in a batch type sludge hygienization research irradiator with flow from top to bottom, the objective being to measure flow rate, circulation and mixing times and to investigate the hydrodynamic behaviour of the irradiator for identifying the cause(s) of malfunction. A stimulus–response technique with NH 82 4Br as a tracer was used to measure the above parameters. Experiments were carried out at three different flow rates, i.e 1.0, 0.64 and 0.33 m 3/min. Three combined models based on a set of differential equations are proposed and used to simulate the measured tracer concentration curves. The obtained parameters were used to estimate dead volume and analyse hydrodynamic behaviour of the irradiator. The nonlinear regression problem of model parameter estimation was solved using the Marquardt–Levenberg method. The measured flow rate was found to be in good agreement with the values shown by the flow meter. The circulation times were found to be half of the mixing times. A simple approach for estimation of dose based on a known vertical dose-rate profile inside the irradiator is presented. About one-fourth of the volume of the irradiator was found to be dead at lower flow rates and this decreased with increase in flow rate. At higher flow rates, a semi stagnant volume was found with slow exchange of flow between the active and dead volumes.

Joint Bayesian model selection and estimation of noisy sinusoids via reversible jump MCMC

In this paper, the problem of joint Bayesian model selection and parameter estimation for sinusoids in white Gaussian noise is addressed. An original Bayesian model is proposed that allows us to define a posterior distribution on the parameter space. All Bayesian inference is then based on this distribution. Unfortunately, a direct evaluation of this distribution and of its features, including posterior model probabilities, requires evaluation of some complicated high-dimensional integrals. We develop an efficient stochastic algorithm based on reversible jump Markov chain Monte Carlo methods to perform the Bayesian computation. A convergence result for this algorithm is established. In simulation, it appears that the performance of detection based on posterior model probabilities outperforms conventional detection schemes.