Estimation Of Nonlinear Models Research Articles

Customized Steady-State Constraints for Parameter Estimation in Non-Linear Ordinary Differential Equation Models.

Ordinary differential equation models have become a wide-spread approach to analyze dynamical systems and understand underlying mechanisms. Model parameters are often unknown and have to be estimated from experimental data, e.g., by maximum-likelihood estimation. In particular, models of biological systems contain a large number of parameters. To reduce the dimensionality of the parameter space, steady-state information is incorporated in the parameter estimation process. For non-linear models, analytical steady-state calculation typically leads to higher-order polynomial equations for which no closed-form solutions can be obtained. This can be circumvented by solving the steady-state equations for kinetic parameters, which results in a linear equation system with comparatively simple solutions. At the same time multiplicity of steady-state solutions is avoided, which otherwise is problematic for optimization. When solved for kinetic parameters, however, steady-state constraints tend to become negative for particular model specifications, thus, generating new types of optimization problems. Here, we present an algorithm based on graph theory that derives non-negative, analytical steady-state expressions by stepwise removal of cyclic dependencies between dynamical variables. The algorithm avoids multiple steady-state solutions by construction. We show that our method is applicable to most common classes of biochemical reaction networks containing inhibition terms, mass-action and Hill-type kinetic equations. Comparing the performance of parameter estimation for different analytical and numerical methods of incorporating steady-state information, we show that our approach is especially well-tailored to guarantee a high success rate of optimization.

An update on modeling dose-response relationships: Accounting for correlated data structure and heterogeneous error variance in linear and nonlinear mixed models.

Advanced methods for dose-response assessments are used to estimate the minimum concentrations of a nutrient that maximizes a given outcome of interest, thereby determining nutritional requirements for optimal performance. Contrary to standard modeling assumptions, experimental data often present a design structure that includes correlations between observations (i.e., blocking, nesting, etc.) as well as heterogeneity of error variances; either can mislead inference if disregarded. Our objective is to demonstrate practical implementation of linear and nonlinear mixed models for dose-response relationships accounting for correlated data structure and heterogeneous error variances. To illustrate, we modeled data from a randomized complete block design study to evaluate the standardized ileal digestible (SID) Trp:Lys ratio dose-response on G:F of nursery pigs. A base linear mixed model was fitted to explore the functional form of G:F relative to Trp:Lys ratios and assess model assumptions. Next, we fitted 3 competing dose-response mixed models to G:F, namely a quadratic polynomial (QP) model, a broken-line linear (BLL) ascending model, and a broken-line quadratic (BLQ) ascending model, all of which included heteroskedastic specifications, as dictated by the base model. The GLIMMIX procedure of SAS (version 9.4) was used to fit the base and QP models and the NLMIXED procedure was used to fit the BLL and BLQ models. We further illustrated the use of a grid search of initial parameter values to facilitate convergence and parameter estimation in nonlinear mixed models. Fit between competing dose-response models was compared using a maximum likelihood-based Bayesian information criterion (BIC). The QP, BLL, and BLQ models fitted on G:F of nursery pigs yielded BIC values of 353.7, 343.4, and 345.2, respectively, thus indicating a better fit of the BLL model. The BLL breakpoint estimate of the SID Trp:Lys ratio was 16.5% (95% confidence interval [16.1, 17.0]). Problems with the estimation process rendered results from the BLQ model questionable. Importantly, accounting for heterogeneous variance enhanced inferential precision as the breadth of the confidence interval for the mean breakpoint decreased by approximately 44%. In summary, the article illustrates the use of linear and nonlinear mixed models for dose-response relationships accounting for heterogeneous residual variances, discusses important diagnostics and their implications for inference, and provides practical recommendations for computational troubleshooting.

042 An update on modeling dose–response relationships: Accounting for correlated data structures and heterogeneous variance in linear and nonlinear mixed models

Advanced methods for dose–response assessments are used to estimate concentrations of a nutrient that optimize a given outcome of interest, thereby determining nutritional requirements for optimal performance. Traditionally, many dose–response methods use a fixed-effects framework that assumes mutually independent observations with homogeneous variances. Yet experimental data often present a design structure that includes correlations between observations (e.g., blocking, nesting, etc.) as well as heterogeneity of variances that can mislead inference if disregarded. Our objective is to demonstrate practical implementation of computationally intensive linear and nonlinear mixed models methodology to describe dose–response relationships accounting for correlated data structure and heterogeneous variances. To illustrate, we model data from a randomized complete block design study to evaluate the standardized ileal digestible (SID) Trp:Lys dose–response on G:F of nursery pigs. A base linear mixed model was fit to explore the functional form of G:F relative to Trp:Lys and assess model assumptions, in particular residual homoscedasticity. Next, we fitted 3 competing dose–response mixed models to G:F, namely a quadratic polynomial (QP), a broken-line linear (BLL) ascending model, and a broken-line quadratic (BLQ) ascending model, all of which included heteroskedastic specifications, as dictated by the base model, and used parameter estimates from the base model as initial values. The GLIMMIX procedure of SAS (version 9.4; SAS Inst. Inc., Cary, NC) was used to fit the base and QP models and the NLMIXED procedure was used to fit the nonlinear models. We further illustrate the use of a grid-search approach to facilitate convergence and parameter estimation in nonlinear mixed models, as this seemed to be the most common implementation problem. Model fit between competing dose–response models was compared using maximum likelihood–based Bayesian information criterion (BIC). The QP, BLL, and BLQ models fitted on G:F of nursery pigs yielded BIC values of 353.7, 343.4, and 345.2, respectively, indicating a better fit of BLL followed closely by BLQ. The BLL breakpoint estimate of the SID Trp:Lys was 16.5% (95% confidence interval [CI] 16.1–17.0), whereas the BLQ estimate was 16.0% (95% CI 15.5–16.6). Importantly, accounting for heterogeneous variance enhanced inferential precision as the breadth of the CI for mean breakpoint decreased by approximately 44%, from 95% CI 15.8 to 17.4 to 95% CI 16.1 to 17.0 SID Trp:Lys. In summary, we illustrate the use of linear and nonlinear mixed models for dose–response relationships accounting for heterogeneous residual variances, discuss important diagnostics and their implications for inference, and provide practical recommendations for computational troubleshooting.

Multi-sensor marginalized particle filter based on average weight optimization in correlated noise

Particle filter is a kind of powerful and effective simulation-based method to perform optimal state estimation in nonlinear non-Gaussian state-space models. However, its main drawback is with large computational complexity and not suitable for noise correlation condition, which limits its application in the multi-sensor measurement system. Aiming at the above problem, a novel multi-sensor marginalized particle filter based on average weight optimization in correlated noise is proposed. First, marginalized particle filter is used as the basic framework of new algorithm realization by marginalizing the states appearing linearly in the dynamical system, and the objective is to reduce the calculated amount. Second, considering the rational utilization of multi-sensor measurement, the average weight optimization strategy is used to improve the adverse influence caused by random measurement noise in measuring process of particles weight. Third, combining with the model reconstruction technology, a new decoupling approach of correlated noise is designed in multi-sensor measurement. Finally, the theoretical analysis and experimental results show the feasibility and efficiency of the proposed algorithm.

Evaluation of non-linear power estimation models in a computing cluster

The data center industry is responsible for 1.5–2% of the world energy consumption. Energy management technologies have been proposed for energy-efficient scheduling of computing workloads and for allocating resources in such computing infrastructures. One of the important factors for this energy management is the estimation of power consumption as a result of the workload schedule to be carried out. The commonly used power models are a linear function of resource features. Based on measurement data sets from our cluster, we extended power model study to multiple non-linear dependencies. We provide several novel contributions: we apply neural network models and unsupervised classification models with basic OS-reported resource features for power estimation; we build and test the power estimation models in a cluster environment from a large size of measurement data; we evaluate the power estimation models in terms of not only accuracy but also portability and usability.We prove that a multiple-variable linear regression approach is more precise than a CPU-only linear approach. The neural network approaches have a slight advantage – their mean root mean square error is at most 15% less than that of the multiple-variable linear model. The neural network models have worse portability when the models generated on a node are applied on other homogeneous nodes. Gaussian Mixture Model has the highest accuracy but requires the longest training time. In the end, we prove that models trained using the system-level full features have the highest accuracy comparing to only use part of features.

Particle filtering and the laplace method for target tracking

The deterministic Laplace method is combined with particle filtering for the state estimation in nonlinear state-space models. Multidimensional Laplace approximations of the posterior moments are embedded in a particle filter. These approximations are used to design an affine transformation taking into account the observation. This transformation is applied to the particle sample in order to improve particle approximation. The resulting algorithm is called the Laplace particle filter (LPF). It is tested by simulation and exhibits good performance on three tracking problems: bearings-only target tracking, ballistic target tracking during atmospheric reentry, and neural decoding for motion analysis.

Control function instrumental variable estimation of nonlinear causal effect models

The instrumental variable method consistently estimates the effect of a treatment when there is unmeasured confounding and a valid instrumental variable. A valid instrumental variable is a variable that is independent of unmeasured confounders and affects the treatment but does not have a direct effect on the outcome beyond its effect on the treatment. Two commonly used estimators for using an instrumental variable to estimate a treatment effect are the two stage least squares estimator and the control function estimator. For linear causal effect models, these two estimators are equivalent, but for nonlinear causal effect models, the estimators are different. We provide a systematic comparison of these two estimators for nonlinear causal effect models and develop an approach to combing the two estimators that generally performs better than either one alone. We show that the control function estimator is a two stage least squares estimator with an augmented set of instrumental variables. If these augmented instrumental variables are valid, then the control function estimator can be much more efficient than usual two stage least squares without the augmented instrumental variables while if the augmented instrumental variables are not valid, then the control function estimator may be inconsistent while the usual two stage least squares remains consistent. We apply the Hausman test to test whether the augmented instrumental variables are valid and construct a pretest estimator based on this test. The pretest estimator is shown to work well in a simulation study. An application to the effect of exposure to violence on time preference is considered.

Robust and efficient parameter estimation in dynamic models of biological systems.

BackgroundDynamic modelling provides a systematic framework to understand function in biological systems. Parameter estimation in nonlinear dynamic models remains a very challenging inverse problem due to its nonconvexity and ill-conditioning. Associated issues like overfitting and local solutions are usually not properly addressed in the systems biology literature despite their importance.Here we present a method for robust and efficient parameter estimation which uses two main strategies to surmount the aforementioned difficulties: (i) efficient global optimization to deal with nonconvexity, and (ii) proper regularization methods to handle ill-conditioning. In the case of regularization, we present a detailed critical comparison of methods and guidelines for properly tuning them. Further, we show how regularized estimations ensure the best trade-offs between bias and variance, reducing overfitting, and allowing the incorporation of prior knowledge in a systematic way.ResultsWe illustrate the performance of the presented method with seven case studies of different nature and increasing complexity, considering several scenarios of data availability, measurement noise and prior knowledge. We show how our method ensures improved estimations with faster and more stable convergence. We also show how the calibrated models are more generalizable. Finally, we give a set of simple guidelines to apply this strategy to a wide variety of calibration problems.ConclusionsHere we provide a parameter estimation strategy which combines efficient global optimization with a regularization scheme. This method is able to calibrate dynamic models in an efficient and robust way, effectively fighting overfitting and allowing the incorporation of prior information.Electronic supplementary materialThe online version of this article (doi:10.1186/s12918-015-0219-2) contains supplementary material, which is available to authorized users.

Orthogonal Series Estimation in Nonlinear Cointegrating Models with Endogeneity

This paper considers a nonlinear time series model associated with both nonstationarity and endogeneity. The proposed model is then estimated by a nonparametric series method. An asymptotic theory is established in both point–wise and the space metric sense for the estimator. The Monte Carlo simulation results show that the performance of the proposed estimate is numerically satisfactory.

Midlatitude–Equatorial Dynamics of a Grounded Deep Western Boundary Current. Part II: Cross-Equatorial Dynamics

AbstractThis is Part II of a two-part theoretical study into the midlatitude–cross-equatorial dynamics of a deep western boundary current (DWBC) in an idealized meridionally aligned, differentially rotating ocean basin with zonally varying parabolic bottom topography. Part I determined the midlatitude flow across the planetary vorticity gradient and the dynamics of the DWBC as it begins to enter the equatorial region in the “intermediate equatorial region.” Part II determines the nonlinear dynamics of the DWBC as it flows across the basin along the equator in the “inner equatorial region.” The large-scale structure of the flow within the inner equatorial region corresponds to a zonally aligned nonlinear stationary planetary wave pattern that meanders about the equator in which the flow exits the equatorial region on the eastern side of the basin. In addition to numerically determining the pathlines for the large-scale equatorial flow, an approximate nonlinear model is introduced for which an analytical solution can be obtained for the nonlinear planetary wave along the equator. If the DWBC exits the equatorial region into the opposite hemisphere from its source hemisphere, the characteristic curves associated with the flow must necessarily intersect within the inner equatorial region. It is in the regions of intersecting characteristics that dissipation makes a leading-order contribution to the dynamics and induces the requisite potential vorticity adjustment permitting the cross-equatorial flow of a DWBC that is in planetary geostrophic dynamical balance in midlatitudes.

Particle-filter Multi-target Tracking Algorithm Based on Dynamic Salient Features

In order to address the problem of tracking different moving targets in image sequence against a complicated background, this paper presents a particle-filter multi-target tracking algorithm based on their dynamic salient features. By making use of the research findings on visual attention, the algorithm adopts the robust dynamic salient features as a result of combining the gray-scale and details with the motion characteristics of such targets as the state vector of particle filter. The algorithm is highly robust as it contains salient features originating from the low-level features of the targets. Meanwhile the particle filter allows optimized estimation of non-linear and non-Gaussian models. As a consequence, the algorithm is capable of managing traces in tracking different targets and dealing with their appearance, disappearance, mergence, splitting and sheltering by obstacles. Experiments show that this new algorithm enables tracking of multiple targets in complicated image sequence.

Odhady parametrů ekonomických modelů pomocí genetických algoritmů

The paper concentrates on capability of genetic algorithms for parameter estimation of non-linear economic models. In the paper we test the ability of genetic algorithms to estimate of parameters of demand function for durable goods and simultaneously search for parameters of genetic algorithm that lead to maximum effectiveness of the computation algorithm. The genetic algorithms connect deterministic iterative computation methods with stochastic methods. In the genteic aůgorithm approach each possible solution is represented by one individual, those life and lifes of all generations of individuals run under a few parameter of genetic algorithm. Our simulations resulted in optimal mutation rate of 15% of all bits in chromosomes, optimal elitism rate 20%. We can not set the optimal extend of generation, because it proves positive correlation with effectiveness of genetic algorithm in all range under research, but its impact is degreasing. The used genetic algorithm was sensitive to mutation rate at most, than to extend of generation. The sensitivity to elitism rate is not so strong.

Generalised signed-rank estimation for nonlinear models with multidimensional indices

We consider a nonlinear regression model when the index variable is multidimensional. Such models are useful in signal processing, texture modelling, and spatio-temporal data analysis. A generalised form of the signed-rank estimator of the nonlinear regression coefficients is proposed. This general form of the signed-rank (SR) estimator includes estimators and hybrid variants. Sufficient conditions for strong consistency and asymptotic normality of the estimator are given. It is shown that the rate of convergence to normality can be different from . The sufficient conditions are weak in the sense that they are satisfied by harmonic-type functions for which results in the current literature may not apply. A simulation study shows that certain generalised SR estimators (e.g. signed rank) perform better in recovering signals than others (e.g. least squares) when the error distribution is contaminated or is heavy-tailed.

Vehicle state and road friction coefficient estimation based on double cubature Kalman filter

For the estimation problem of vehicle state and road adhesion coefficient in driving, the estimation algorithm of vehicle state and road adhesion coefficient were studied in this paper based on double cubature Kalman filter. The 3-DOF nonlinear vehicle estimation model with Dugoff tire model was established. The vehicle driving state estimator and tire-road friction coefficient estimator based on double cubature Kalman filter were designed. The estimators contact with each other in the process of estimation and forms a closed loop feedback to estimate the vehicle state and road adhesion coefficient timely and accurately. Selecting the typical working condition, the algorithm was verified by driving simulator experiments in the loop. The results showed that estimation algorithm based on double cubature Kalman filter can more accurately estimate the vehicle state and road adhesion coefficient than estimation algorithm based on extend Kalman filter.

Higher-order massive neutrino perturbations in large-scale structure

We develop a higher-order perturbation theory for large-scale structure formation involving a free-streaming hot or warm dark matter species. We focus on the case of mixed cold dark matter and massive neutrinos, although our approach is applicable also to a single warm dark matter species. In order to capture the suppressed growth of neutrino density perturbations on small scales, we account for the full momentum dependence of the phase space distribution using the Vlasov equation, and derive from it a formal closed-form nonlinear equation for the neutrino density. Using a systematic perturbative expansion of this equation we compute high-order corrections to the neutrino density contrast without the explicit need to track the perturbed neutrino momentum distribution. We calculate the leading-order total matter bispectrum for several neutrino masses. Using our result as a benchmark, we test the accuracy of the fluid approximation and a linear approximation used in perturbative and N-body analyses, as well as a new hybrid approach that combines the exact linear evolution with the nonlinear structure of the fluid equations. Aiming at ≲ 1% accuracy, we find that the total matter bispectrum with a low neutrino mass m = 0.046 eV can be reproduced by all but the fluid approximation, while for larger neutrino masses m=0.46 → 0.93 eV only the hybrid approach has the desired accuracy on a large range of scales. This result serves as a cautionary note that approximate nonlinear models of neutrino clustering that reproduce the gross features of some observables may not suffice for precision calculations, nor are they guaranteed to apply to other observables. All of the approximation schemes fail to reproduce the bispectrum of the neutrino density perturbations at better than 20% accuracy across all scales, indicating that an exact treatment of nonlinear neutrino perturbations is necessary.

Best linear near unbiased estimation for nonlinear signal models via semi-infinite programming approach

When the exact unbiasedness condition is relaxed to a near unbiasedness condition, this short communication shows that the best linear near unbiased estimation problem is actually a semi-infinite programming problem. Our recently developed dual parameterization method is applied for solving the problem. Computer numerical simulation results show that the semi-infinite programming approach outperforms the least squares approach.

Split-panel Jackknife Estimation of Fixed-effect Models

Maximum-likelihood estimation of nonlinear models with fixed effects is subject to the incidental-parameter problem. This typically implies that point estimates suffer from large bias and confidence intervals have poor coverage. This article presents a jackknife method to reduce this bias and to obtain confidence intervals that are correctly centred under rectangular-array asymptotics. The method is explicitly designed to handle dynamics in the data, and yields estimators that are straightforward to implement and can be readily applied to a range of models and estimands. We provide distribution theory for estimators of model parameters and average effects, present validity tests for the jackknife, and consider extensions to higher-order bias correction and to two-step estimation problems. An empirical illustration relating to female labour-force participation is also provided.

Empirical Modeling of a Graphene Field-Effect Transistor Sensor

This theoretical nanoscience work explore the empirical method of extracting experimental radiation sensor data from published work and project the performance limit of the sensor in term of back gate voltage, resistance and radiation flux. We demonstrate an empirical regression modeling of an X-ray radiation sensor based on a graphene field effect transistor (GFET). The experimental data obtained from published work only showed the values of resistance, R versus back gate voltage, Vbg for X-ray flux, E of 0 kV, 30 kV and 40 kV. Least Squares Estimation (LSE) in nonlinear regression model is utilized to obtain the individual polynomial regression model of resistance versus back gate voltage model for each of the 3 X-ray flux. LSE with multiple nonlinear regression incorporating with Nimmo and Atkinsmethod is employ to get a unified polynomial regression based on the individual model. The model is simulated in MATLAB and provide fast execution time. The unified model for is able to predict the resistance versus back gate voltage for various X-ray radiation flux beyond these three values given values. In addition to that, we are able to find the limit of X-ray flux based on the initial characteristic. Moreover, our technique enable one to visualize the interrelationship between R, Vbg and E variables and perform multivariate analysis. The empirical regression model of GFET is shown to have good agreement with experimental data and theoretical prediction for multivariate such as back gate voltage, resistance and X-ray flux.

Estimating Non-Linear DSGES with the Approximate Bayesian Computation: An Application to the Zero Lower Bound

Non-linear model estimation is generally perceived as impractical and computationally burdensome. This perception limited the diusion on non-linear models estimation. In this paper a simple set of techniques going under the name of Approximate Bayesian Computation (ABC) is proposed. ABC is a set of Bayesian techniques based on moments matching: moments are obtained simulating the model conditional on draws from the prior distribution. An accept-reject criterion is applied on the simulations and an approximate posterior distribution is obtained by the accepted draws. A series of techniques are presented (ABC-regression, ABC-MCMC, ABC-SMC). To assess their small sample performance, Montecarlo experiments are run on AR(1) processes and on a RBC model showing that ABC estimators outperform the Limited Information Method (Kim, 2002), a GMM-style estimator. In the remainder, the estimation of a new-keynesian model with a zero lower bound on the interest rate is performed. Non-gaussian moments are exploited in the estimation procedure.

Regularization Features in the System Identification Toolbox

Regularization is a well known technique in estimation methodology. Its usefulness to assure well conditioned calculations and to handle reliable prior information about partly known parameters is a classical theme in statistics. Recently some deeper understanding about the advantages for general estimation and identification methods has been found and discussed. This is often done using the term "kernel methods". It has some links to machine learning and statistical function learning as well as to reproducing kernel Hilbert spaces. But algorithmically, it is all a question of regularization with an appropriate (quadratic norm) regularization matrix. Regularization was introduced into the MATLAB System Identification Toolbox in the 2013a version. It is a general option for all linear and nonlinear model estimation. Several specialized commands for estimation impulse responses (impulseest), tuning of kernels for ARX models (arxRegul), and general linear state space models with regularization (ssregest) have also been implemented. The paper describes and illustrates the used of these new features in the toolbox.