Framework For Parameter Estimation Research Articles

Convergence of posteriors for structurally non-identified problems using results from the theory of inverse problems

Abstract. We consider convergence of the posterior distribution in a Bayesian parameter estimation framework in the large sample size limit for structurally non-identified problems. These belong to the class of ill-posed problems, and the large sample theory is not applicable here. In particular, the influence of the prior distribution does not vanish in the large sample size limit. We review recent results in this area and present ideas inspired from the theory of ill-posed inverse problems that can be used towards a more general concept of posterior convergence for non-identified problems.

SEMANTIC MOTION CONCEPT RETRIEVAL IN NON-STATIC BACKGROUND UTILIZING SPATIAL-TEMPORAL VISUAL INFORMATION

Motion concepts mean those concepts containing motion information such as racing car and dancing. In order to achieve high retrieval accuracy comparing with those static concepts such as car or person in semantic retrieval tasks, the temporal information has to be considered. Additionally, if a video sequence is captured by an amateur using a hand-held camera containing significant camera motion, the complexities of the uncontrolled backgrounds would aggravate the difficulty of motion concept retrieval. Therefore, the retrieval of semantic concepts containing motion in non-static background is regarded as one of the most challenging tasks in multimedia semantic analysis and video retrieval. To address such a challenge, this paper proposes a motion concept retrieval framework including a motion region detection model and a concept retrieval model that integrates the spatial and temporal information in video sequences. The motion region detection model uses a new integral density method (adopted from the idea of integral images) to quickly identify the motion regions in an unsupervised way. Specially, key information locations on video frames are first obtained as maxima and minima of the result of Difference of Gaussian (DoG) function. Then a motion map of adjacent frames is generated from the diversity of the outcomes from the Simultaneous Partition and Class Parameter Estimation (SPCPE) framework. The usage of the motion map is to filter key information locations into key motion locations (KMLs) that imply the regions containing motion. The motion map can also indicate the motion direction which guides the proposed "integral density" approach to locate the motion regions quickly and accurately. Based on the motion region detection model, moving object-level information is extracted for semantic retrieval. In the proposed conceptual retrieval model, temporally semantic consistency among the consecutive shots is analyzed and presented into a conditional probability model, which is then used to re-rank the similarity scores to improve the final retrieval results. The results of our proposed novel motion concept retrieval framework are not only illustrated visually demonstrating its robustness in non-static background, but also verified by the promising experimental results demonstrating that the concept retrieval performance can be improved by integrating the spatial and temporal visual information.

Estimating Parameters in Physical Models through Bayesian Inversion: A Complete Example

All mathematical models of real-world phenomena contain parameters that need to be estimated from measurements, either for realistic predictions or simply to understand the characteristics of the model. Bayesian statistics provides a framework for parameter estimation in which uncertainties about models and measurements are translated into uncertainties in estimates of parameters. This paper provides a simple, step-by-step example---starting from a physical experiment and going through all of the mathematics---to explain the use of Bayesian techniques for estimating the coefficients of gravity and air friction in the equations describing a falling body. In the experiment we dropped an object from a known height and recorded the free fall using a video camera. The video recording was analyzed frame by frame to obtain the distance the body had fallen as a function of time, including measures of uncertainty in our data that we describe as probability densities. We explain the decisions behind the various choices of probability distributions and relate them to observed phenomena. Our measured data are then combined with a mathematical model of a falling body to obtain probability densities on the space of parameters we seek to estimate. We interpret these results and discuss sources of errors in our estimation procedure.

Inference for Interest Rate Models Using Milstein’s Approximation

A class of martingale estimating functions based on the first two moments of the observed process provides a convenient framework for estimating the parameters of diffusion processes [1]. In the Bayesian set up, combined estimating functions had been studied for diffusion processes in [2] with filtering applications. However, when the conditional mean and the conditional variance are functions of parameters of interest in a diffusion process model, the basic martingales generating components of quadratic estimating functions are such that one is an absolute continuous function with respect to the other [3, p. 94]. Hence, the combined martingale estimating functions cannot be constructed for continuous-time diffusion processes. In this paper, a general framework for parameter estimation of discretely observed interest rate models is developed by using the Milstein approximation and closed form expressions for the information gain are also obtained. The method is used to study the estimates of the parameters for an extended version of the CoxIngersoll-Ross interest rate model.

Modeling 3‐D spatio‐temporal biogeochemical processes with a forest of 1‐D statistical emulators

This paper focuses on the spatio‐temporal dynamical processes in lower trophic level marine ecosystems, where various sources of uncertainty make statistical modeling difficult. Such dynamical processes exhibit nonlinearity in time and potential nonstationarity in space. Planktonic organisms are microscopic, making it difficult to measure their abundance and resulting in limited data. Further, deterministic, component‐based ecosystem models contain a large number of parameters, some of which can be difficult to estimate. We consider a Bayesian hierarchical framework for parameter estimation that uses an approximation to the dynamical models for computational feasibility. Specifically, we develop a computationally inexpensive first‐order statistical emulator for a one‐dimensional NPZD model with iron limitation. Then, we introduce a novel approach to the modeling of three‐dimensional lower trophic level marine ecosystem processes, linking the one‐dimensional emulators via a two‐dimensional spatial field on the parameters. This methodology is used to estimate important biological parameters on the coastal Gulf of Alaska, leading to a reduction in Bayesian credible interval width compared with a nonspatial model. Copyright © 2012 John Wiley & Sons, Ltd.

Detection of FLT3 Internal Tandem Duplication in Targeted, Short-Read-Length, Next-Generation Sequencing Data

A recurrent somatic mutation frequently found in cytogenetically normal acute myeloid leukemia (AML) is internal tandem duplication (ITD) in the fms-related tyrosine kinase 3 gene (FLT3). This mutation is generally detected in the clinical laboratory by PCR and electrophoresis-based product sizing. As the number of clinically relevant somatic mutations in AML increases, it becomes increasingly attractive to incorporate FLT3 ITD testing into multiplex assays for many somatic mutations simultaneously, using next-generation sequencing (NGS). However, the performance of most NGS analysis tools for identifying medium-size insertions such as FLT3 ITD mutations is largely unknown. We used a multigene, targeted NGS assay to obtain deep sequence coverage (>1000-fold) of FLT3 and 26 other genes from 22 FLT3 ITD-positive and 29 ITD-negative specimens to examine the performance of several commonly used NGS analysis tools for identifying FLT3 ITD mutations. ITD mutations were present in hybridization-capture sequencing data, and Pindel was the only tool out of the seven tested that reliably detected these insertions. Pindel had 100% sensitivity (95% CI = 83% to 100%) and 100% specificity (95% CI = 88% to 100%) in our samples; Pindel provided accurate ITD insertion sizes and was able to detect ITD alleles present at estimated frequencies as low as 1%. These data demonstrate that FLT3 ITDs can be reliably detected in panel-based, next-generation sequencing assays.

Margin Based Learning: A Framework for Acoustic Model Parameter Estimation

Statistical learning theory has been introduced in the field of machine learning since last three decades. In speech recognition application, SLT combines generalization function and empirical risk in single margin based objective function for optimization. This paper incorporated separation (misclassification) measures conforming to conventional discriminative training criterion in loss function definition of margin based method to derive the mathematical framework for acoustic model parameter estimation and discuss some important issues related to hinge loss function of the derived model to enhance the performance of speech recognition system.

Markov chain Monte Carlo approach to parameter estimation in the FitzHugh-Nagumo model

Excitability is observed in a variety of natural systems, such as neuronal dynamics, cardiovascular tissues, or climate dynamics. The stochastic FitzHugh-Nagumo model is a prominent example representing an excitable system. To validate the practical use of a model, the first step is to estimate model parameters from experimental data. This is not an easy task because of the inherent nonlinearity necessary to produce the excitable dynamics, and because the two coordinates of the model are moving on different time scales. Here we propose a Bayesian framework for parameter estimation, which can handle multidimensional nonlinear diffusions with large time scale separation. The estimation method is illustrated on simulated data.

Multi-criteria parameter estimation for the Unified Land Model

Abstract. We describe a parameter estimation framework for the Unified Land Model (ULM) that utilizes multiple independent data sets over the continental United States. These include a satellite-based evapotranspiration (ET) product based on MODerate resolution Imaging Spectroradiometer (MODIS) and Geostationary Operational Environmental Satellites (GOES) imagery, an atmospheric-water balance based ET estimate that utilizes North American Regional Reanalysis (NARR) atmospheric fields, terrestrial water storage content (TWSC) data from the Gravity Recovery and Climate Experiment (GRACE), and streamflow (Q) primarily from the United States Geological Survey (USGS) stream gauges. The study domain includes 10 large-scale (≥105 km2) river basins and 250 smaller-scale (<104 km2) tributary basins. ULM, which is essentially a merger of the Noah Land Surface Model and Sacramento Soil Moisture Accounting Model, is the basis for these experiments. Calibrations were made using each of the data sets individually, in addition to combinations of multiple criteria, with multi-criteria skill scores computed for all cases. At large scales, calibration to Q resulted in the best overall performance, whereas certain combinations of ET and TWSC calibrations lead to large errors in other criteria. At small scales, about one-third of the basins had their highest Q performance from multi-criteria calibrations (to Q and ET) suggesting that traditional calibration to Q may benefit by supplementing observed Q with remote sensing estimates of ET. Model streamflow errors using optimized parameters were mostly due to over (under) estimation of low (high) flows. Overall, uncertainties in remote-sensing data proved to be a limiting factor in the utility of multi-criteria parameter estimation.

A Poisson common factor model for projecting mortality and life expectancy jointly for females and males

We examine the application of a Poisson common factor model for the projection of mortality jointly for females and males. The model structure is an extension of the classical Lee–Carter method in which there is a common factor for the aggregate population, while a number of additional sex-specific factors can also be incorporated. The Poisson distribution is a natural choice for modelling the number of deaths, and its use provides a formal statistical framework for model selection, parameter estimation, and data analysis. Our results for Australian data show that this model leads to projected life expectancy values similar to those produced by the separate projection of mortality for females and males, but possesses the additional advantage of ensuring that the projected male-to-female ratio for death rates at each age converges to a constant. Moreover, the randomness of the corresponding residuals indicates that the model fit is satisfactory.

A novel cost function to estimate parameters of oscillatory biochemical systems

Oscillatory pathways are among the most important classes of biochemical systems with examples ranging from circadian rhythms and cell cycle maintenance. Mathematical modeling of these highly interconnected biochemical networks is needed to meet numerous objectives such as investigating, predicting and controlling the dynamics of these systems. Identifying the kinetic rate parameters is essential for fully modeling these and other biological processes. These kinetic parameters, however, are not usually available from measurements and most of them have to be estimated by parameter fitting techniques. One of the issues with estimating kinetic parameters in oscillatory systems is the irregularities in the least square (LS) cost function surface used to estimate these parameters, which is caused by the periodicity of the measurements. These irregularities result in numerous local minima, which limit the performance of even some of the most robust global optimization algorithms. We proposed a parameter estimation framework to address these issues that integrates temporal information with periodic information embedded in the measurements used to estimate these parameters. This periodic information is used to build a proposed cost function with better surface properties leading to fewer local minima and better performance of global optimization algorithms. We verified for three oscillatory biochemical systems that our proposed cost function results in an increased ability to estimate accurate kinetic parameters as compared to the traditional LS cost function. We combine this cost function with an improved noise removal approach that leverages periodic characteristics embedded in the measurements to effectively reduce noise. The results provide strong evidence on the efficacy of this noise removal approach over the previous commonly used wavelet hard-thresholding noise removal methods. This proposed optimization framework results in more accurate kinetic parameters that will eventually lead to biochemical models that are more precise, predictable, and controllable.

A generalized reference state extends the applicability of approximative kinetics in metabolic engineering

AbstractBACKGROUND: The use of approximative kinetics for modeling enzyme kinetics and for metabolic engineering purposes has become increasingly attractive. Despite a list of advantages, the proposed use of a steady state for reference limits the use of such kinetic formats.RESULTS: Here, to avoid these limitations, an alternative formulation is proposed for the linlog approximative kinetics introducing ‘a characteristic set’ for normalization. The necessary equations for the parameter (elasticities and the reaction rate of the characteristic set) estimation framework using data obtained from various types of perturbation experiments are given. The use of the new formulation is exemplified with three illustrative examples: parameter estimation in steady state and in dynamic experiments; and prediction of the effect of a mutation in enzyme kinetics properties. The implications of the new formulation are discussed.CONCLUSIONS: By introducing a so‐called characteristic set, we show that the approximative quality of this kinetics is improved and this formulation can be used to analyze and model data from a broader set of experimental scenarios. Copyright © 2012 Society of Chemical Industry

Visualization and Curve-Parameter Estimation Strategies for Efficient Exploration of Phenotype Microarray Kinetics

BackgroundThe Phenotype MicroArray (OmniLog® PM) system is able to simultaneously capture a large number of phenotypes by recording an organism's respiration over time on distinct substrates. This technique targets the object of natural selection itself, the phenotype, whereas previously addressed ‘-omics’ techniques merely study components that finally contribute to it. The recording of respiration over time, however, adds a longitudinal dimension to the data. To optimally exploit this information, it must be extracted from the shapes of the recorded curves and displayed in analogy to conventional growth curves.MethodologyThe free software environment R was explored for both visualizing and fitting of PM respiration curves. Approaches using either a model fit (and commonly applied growth models) or a smoothing spline were evaluated. Their reliability in inferring curve parameters and confidence intervals was compared to the native OmniLog® PM analysis software. We consider the post-processing of the estimated parameters, the optimal classification of curve shapes and the detection of significant differences between them, as well as practically relevant questions such as detecting the impact of cultivation times and the minimum required number of experimental repeats.ConclusionsWe provide a comprehensive framework for data visualization and parameter estimation according to user choices. A flexible graphical representation strategy for displaying the results is proposed, including 95% confidence intervals for the estimated parameters. The spline approach is less prone to irregular curve shapes than fitting any of the considered models or using the native PM software for calculating both point estimates and confidence intervals. These can serve as a starting point for the automated post-processing of PM data, providing much more information than the strict dichotomization into positive and negative reactions. Our results form the basis for a freely available R package for the analysis of PM data.

Estimation of an agent-based model of investor sentiment formation in financial markets

We use weekly survey data on short-term and medium-term sentiment of German investors to estimate the parameters of a stochastic model of opinion formation governed by social interactions. The bivariate nature of our data set also allows us to explore the interaction between the two hypothesized opinion formation processes, while consideration of the simultaneous weekly changes of the stock index DAX enables us to study the influence of sentiment on returns. Technically, we extend the maximum likelihood framework for parameter estimation in agent-based models introduced by Lux (2009a) by generalizing it to bivariate and tri-variate settings. As it turns out, our results are consistent with strong social interaction in short-run sentiment. While one observes abrupt changes of mood in short-run sentiment, medium-term sentiment is a more slowly moving process in which the influence of social interaction seems to be less pronounced. The tri-variate model entails a significant effect from short-run sentiment on prices in-sample, but its out-of-sample predictive performance does not beat the random walk benchmark.

An assessment framework for branch parameter estimation of power systems

This paper proposes an assessment framework for branch parameter estimation of power systems, based on which the estimation accuracy can be improved and the estimation reliability can be ensured. The proposed framework comprises three parts: 1) Parameter credibility assessment to evaluate the credibility of original parameters in the case that their true values are unknown; 2) estimation accuracy assessment to provide quantitative information about the validity of the estimation; 3) parameter dominance assessment to find out the key parameters for the estimation. This framework can be incorporated into the conventional parameter estimation process easily such that the accuracy and reliability of the estimation are improved. The implementation problems are addressed in details. Tests are carried out on the IEEE 14-bus system, IEEE 30-bus system, and IEEE 300-bus system to show the correctness and effectiveness of the proposed assessment framework.

Dynamic noise, chaos and parameter estimation in population biology

We revisit the parameter estimation framework for population biological dynamical systems, and apply it to calibrate various models in epidemiology with empirical time series, namely influenza and dengue fever. When it comes to more complex models such as multi-strain dynamics to describe the virus-host interaction in dengue fever, even the most recently developed parameter estimation techniques, such as maximum likelihood iterated filtering, reach their computational limits. However, the first results of parameter estimation with data on dengue fever from Thailand indicate a subtle interplay between stochasticity and the deterministic skeleton. The deterministic system on its own already displays complex dynamics up to deterministic chaos and coexistence of multiple attractors.

Bayesian mixture of AR models for time series clustering

In this paper, we propose a Bayesian framework for estimation of parameters of a mixture of autoregressive models for time series clustering. The proposed approach is based on variational principles and provides a tractable approximation to the true posterior density that minimizes Kullback---Liebler (KL) divergence with respect to prior distribution. This method simultaneously addresses the model complexity and parameter estimation problems. The proposed approach is applied both on simulated and real-world time series datasets. It is found to be useful in exploring and finding the true number of underlying clusters, starting from an arbitrarily large number of clusters.

Non-rigid registration of longitudinal brain tumor treatment MRI.

To evaluate changes in brain structure or function, longitudinal images of brain tumor patients must be non-rigidly registered to account for tissue deformation due to tumor growth or treatment. Most standard non-rigid registration methods will fail to align these images due to the changing feature correspondences between treatment time points and the large deformations near the tumor site. Here we present a registration method which jointly estimates a label map for correspondences to account for the substantial changes that may occur during tumor treatment. Under a Bayesian parameter estimation framework, we employ different probability distributions depending on the correspondence labels. We incorporate models for image similarity, an image intensity prior, label map smoothing, and a transformation prior that encourages deformation near the estimated tumor location. Our proposed algorithm increases registration accuracy compared to a traditional voxel-based registration method as shown using both synthetic and real patient images.

Item Response Model Parameter Estimation Based on Bayesian Joint Likelihood Langevin MCMC Method with Open Software

With the development of education and open software, item response model is more and more important in testing actual testees’ abilities. In item response model, testees’ abilities and item parameters are interrelated. Bayesian method is important estimation method in item response model, but calculation is difficulty. This paper use Scilab and Mysql database construct item response parameter estimation framework; provideBayesian joint likelihood with Langevin MCMC method. Compare three ways to calculate item parameters and testees’ abilities, including random walk sampling, simple Langevin sampling and target distribution Langevin sampling methods.Through the actual calculations, target distribution Langevin sampling improves the calculation speed and reduces the number of systematic sampling times.

A Bayesian framework for parameter estimation in dynamical models.

Mathematical models in biology are powerful tools for the study and exploration of complex dynamics. Nevertheless, bringing theoretical results to an agreement with experimental observations involves acknowledging a great deal of uncertainty intrinsic to our theoretical representation of a real system. Proper handling of such uncertainties is key to the successful usage of models to predict experimental or field observations. This problem has been addressed over the years by many tools for model calibration and parameter estimation. In this article we present a general framework for uncertainty analysis and parameter estimation that is designed to handle uncertainties associated with the modeling of dynamic biological systems while remaining agnostic as to the type of model used. We apply the framework to fit an SIR-like influenza transmission model to 7 years of incidence data in three European countries: Belgium, the Netherlands and Portugal.