Estimation Of Nonlinear Models Research Articles

Bayesian Analysis of Nonlinear Exchange Rate Dynamics and the Purchasing Power Parity Persistence Puzzle

We investigate the persistence of real exchange rates using Bayesian methods. First, an algorithm for Bayesian estimation of nonlinear threshold models is developed. Unlike standard grid-based estimation, the Bayesian approach fully captures joint parameter uncertainty and uncertainty about complicated functions of the parameters, such as the half-life measure of persistence based on generalized impulse response functions. Second, model comparison is conducted via marginal likelihoods, which reflect the relative abilities of models to predict the data given prior beliefs about model parameters. This comparison is conducted for a range of linear and nonlinear models and provides a direct evaluation of the importance of nonlinear dynamics in modeling exchange rates. The marginal likelihoods also imply weights for a model-averaged measure of persistence. The empirical results for real exchange rate data from the G7 countries suggest general support for nonlinearity, but the strength of the evidence depends on which country pair is considered. However, the model-averaged estimates of half-lives are uniformly smaller than for the linear models alone, suggesting that the purchasing power parity persistence puzzle is less of a puzzle than previously thought.

Identification and estimation of nonlinear dynamic panel data models with unobserved covariates

This paper considers nonparametric identification of nonlinear dynamic models for panel data with unobserved covariates. Including such unobserved covariates may control for both the individual-specific unobserved heterogeneity and the endogeneity of the explanatory variables. Without specifying the distribution of the initial condition with the unobserved variables, we show that the models are nonparametrically identified from two periods of the dependent variable Yit and three periods of the covariate Xit. The main identifying assumptions include high-level injectivity restrictions and require that the evolution of the observed covariates depends on the unobserved covariates but not on the lagged dependent variable. We also propose a sieve maximum likelihood estimator (MLE) and focus on two classes of nonlinear dynamic panel data models, i.e., dynamic discrete choice models and dynamic censored models. We present the asymptotic properties of the sieve MLE and investigate the finite sample properties of these sieve-based estimators through a Monte Carlo study. An intertemporal female labor force participation model is estimated as an empirical illustration using a sample from the Panel Study of Income Dynamics (PSID).

Parameter estimation of nonlinear mixed-effects models using first-order conditional linearization and the EM algorithm

Nonlinear mixed-effects (NLME) models are flexible enough to handle repeated-measures data from various disciplines. In this article, we propose both maximum-likelihood and restricted maximum-likelihood estimations of NLME models using first-order conditional expansion (FOCE) and the expectation–maximization (EM) algorithm. The FOCE-EM algorithm implemented in the ForStat procedure SNLME is compared with the Lindstrom and Bates (LB) algorithm implemented in both the SAS macro NLINMIX and the S-Plus/R function nlme in terms of computational efficiency and statistical properties. Two realworld data sets an orange tree data set and a Chinese fir (Cunninghamia lanceolata) data set, and a simulated data set were used for evaluation. FOCE-EM converged for all mixed models derived from the base model in the two realworld cases, while LB did not, especially for the models in which random effects are simultaneously considered in several parameters to account for between-subject variation. However, both algorithms had identical estimated parameters and fit statistics for the converged models. We therefore recommend using FOCE-EM in NLME models, particularly when convergence is a concern in model selection.

Identification of parameter correlations for parameter estimation in dynamic biological models

BackgroundOne of the challenging tasks in systems biology is parameter estimation in nonlinear dynamic models. A biological model usually contains a large number of correlated parameters leading to non-identifiability problems. Although many approaches have been developed to address both structural and practical non-identifiability problems, very few studies have been made to systematically investigate parameter correlations.ResultsIn this study we present an approach that is able to identify both pairwise parameter correlations and higher order interrelationships among parameters in nonlinear dynamic models. Correlations are interpreted as surfaces in the subspaces of correlated parameters. Based on the correlation information obtained in this way both structural and practical non-identifiability can be clarified. Moreover, it can be concluded from the correlation analysis that a minimum number of data sets with different inputs for experimental design are needed to relieve the parameter correlations, which corresponds to the maximum number of correlated parameters among the correlation groups.ConclusionsThe information of pairwise and higher order interrelationships among parameters in biological models gives a deeper insight into the cause of non-identifiability problems. The result of our correlation analysis provides a necessary condition for experimental design in order to acquire suitable measurement data for unique parameter estimation.

Adaptive stimulus optimization for sensory systems neuroscience.

In this paper, we review several lines of recent work aimed at developing practical methods for adaptive on-line stimulus generation for sensory neurophysiology. We consider various experimental paradigms where on-line stimulus optimization is utilized, including the classical optimal stimulus paradigm where the goal of experiments is to identify a stimulus which maximizes neural responses, the iso-response paradigm which finds sets of stimuli giving rise to constant responses, and the system identification paradigm where the experimental goal is to estimate and possibly compare sensory processing models. We discuss various theoretical and practical aspects of adaptive firing rate optimization, including optimization with stimulus space constraints, firing rate adaptation, and possible network constraints on the optimal stimulus. We consider the problem of system identification, and show how accurate estimation of non-linear models can be highly dependent on the stimulus set used to probe the network. We suggest that optimizing stimuli for accurate model estimation may make it possible to successfully identify non-linear models which are otherwise intractable, and summarize several recent studies of this type. Finally, we present a two-stage stimulus design procedure which combines the dual goals of model estimation and model comparison and may be especially useful for system identification experiments where the appropriate model is unknown beforehand. We propose that fast, on-line stimulus optimization enabled by increasing computer power can make it practical to move sensory neuroscience away from a descriptive paradigm and toward a new paradigm of real-time model estimation and comparison.

Nonparametric estimation of nonlinear dynamics by metric-based local linear approximation

This paper discusses nonparametric estimation of nonlinear dynamical system models by a method of metric-based local linear approximation. We assume no functional form of a given model but estimate it from experimental data by approximating the curve implied by the function by the tangent plane around the neighborhood of a tangent point. To specify an appropriate neighborhood, we prepare a metric defined over the Euclidean space in which the curve exists and then evaluate the closeness to the tangent point according to the distances. The proposed method differs from the first order polynomial modeling in discerning the metric and the weighting function, but the first order polynomial modeling with Gaussian kernels is shown to be a special version of the proposed method. Simulation studies and application to ECG signals show the proposed method is easy to manipulate and has performance comparable to or better than the first order local polynomial modeling.

Inference for linear and nonlinear stable error processes via estimating functions

This paper describes an estimating function approach for parameter estimation in linear and nonlinear times series models with infinite variance stable errors. Joint estimates of location and scale parameters are derived for classes of autoregressive (AR) models and random coefficient autoregressive (RCA) models with stable errors, as well as for AR models with stable autoregressive conditionally heteroscedastic (ARCH) errors. Fast, on-line, recursive parametric estimation for the location parameter based on estimating functions is discussed using simulation studies. A real financial time series is also discussed in some detail.

Nonlinear Prediction Control of Synchronous Generator Excitation Based on Subsection Approximation

This paper presents a subsection approximate nonlinear model predictive control (SANMPC) method for synchronous generator excitation control in the regional power grid. The proposed SANMPC considers the voltage as the reference trajectories and the change range of active power, reactive power and voltage as unequal constraints. Segmenting for a sampling interval, it uses sampled values and the control input initial value at the current sampling moment to predict the state at each segment by the Explicit Euler method. The predictive equations of the next sampling moment can be obtained by the predictive state and the control input at the penultimate segment, and form the optimal control problem which may be solved by the interior-point method. We take advantage of a four-machine power system to verify the effectiveness of the proposed SANMPC method under MATLAB platform. The simulating results show that the SANMPC method is simple and efficient, and greatly improves the transient stability compared with the conventional controller both.

Approximate Nonlinear Modeling of Aircraft Engine Surge Margin Based on Equilibrium Manifold Expansion

Stable operation of aircraft engine compressions is constrained by rotating surge. In this paper, an approximate nonlinear surge margin model of aircraft engine compression system by using equilibrium manifold is presented. Firstly, this paper gives an overview of the current state of modeling aerodynamic flow instabilities in engine compressors. Secondly, the expansion form of equilibrium manifold is introduced, and the choosing scheduling variable method is discussed. Then, this paper also gives the identification procedure of modeling the approximate nonlinear model. Finally, the modeling and simulations with high pressure (HP) compressor surge margin of the aircraft engine show that this real-time model has the same accuracy with the thermodynamic model, but has simpler structure and shorter computation time.

Suboptimal FIR Filtering of Nonlinear Models in Additive White Gaussian Noise

The first- and second-order extended finite impulse response (EFIR1 and EFIR2, respectively) filters are addressed for suboptimal estimation of nonlinear discrete-time state-space models with additive white Gaussian noise. It is shown that, unlike the extended Kalman filter (EKF) and EFIR2 filter, the EFIR1 one does not require noise statistics and initial errors. Only within a narrow region around actual noise covariances, EFIR filters fall a bit short of EKF and they demonstrate better performance otherwise. It is shown that the optimal averaging interval for EFIR filters can be determined via measurement without a reference model in a learning cycle. We also notice that the second-order approximation can improve the local performance, but it can also deteriorate it. We thus have no recommendations about its use, at least for tracking considered as an example of applications.

Estimation and inference in unstable nonlinear least squares models

There is compelling evidence that many macroeconomic and financial variables are not generated by linear models. This evidence is based on testing linearity against either smooth nonlinearity or piece-wise linearity, but there is no framework that encompasses both. This paper provides an econometric framework that allows for both breaks and smooth nonlinearity in between breaks. We estimate the unknown break-dates simultaneously with other parameters via nonlinear least-squares. Using new central limit results for nonlinear processes, we provide inference methods on break-dates and parameter estimates and several instability tests. We illustrate our methods via simulated and empirical smooth transition models with breaks.

Nonlinear nonparametric mixed-effects models for unsupervised classification

In this work we propose a novel EM method for the estimation of nonlinear nonparametric mixed-effects models, aimed at unsupervised classification. We perform simulation studies in order to evaluate the algorithm performance and we apply this new procedure to a real dataset.

Modified D-optimal design for logistic model

Asymptotic theory of using the Fisher information matrix may provide poor approximation to the exact variance matrix of maximum likelihood estimation in nonlinear models. This may be due to not obtaining an efficient D-optimal design. In this article, we propose a modified D-optimality criterion, using a more accurate information matrix, based on the Bhattacharyya matrix. The proposed information matrix and its properties are given for two parameters simple logistic model. It is shown that the resulted modified locally D-optimal design is more efficient than the previous one; particularly, for small sample size experiments.

Modelling and Identification of Oxygen Excess Ratio of Self-Humidified PEM Fuel Cell System

One essential parameter in fuel cell operation is oxygen excess ratio which describes comparison between reacted and supplied oxygen number in cathode. Oxygen excess ratio relates to fuel cell safety and lifetime. This paper explains development of air feed model and oxygen excess ratio calculation in commercial self-humidified PEM fuel cell system with 1 kW output power. This modelling was developed from measured data which was limited in open loop system. It was carried out to get relationship between oxygen excess ratio with stack output current and fan motor voltage. It generated fourth-order 56.26% best fit ARX linear polynomial model estimation (loss function = 0.0159, FPE = 0.0159) and second-order ARX nonlinear model estimation with 75 units of wavenet estimator with 84.95% best fit (loss function = 0.0139). The second-order ARX model linearization yielded 78.18% best fit (loss function = 0.0009, FPE = 0.0009).

Real-Time Identification of Hunt–Crossley Dynamic Models of Contact Environments

Real-time estimates of environment dynamics play an important role in the design of controllers for stable interaction between robotic manipulators and unknown environments. The Hunt-Crossley (HC) dynamic contact model has been shown to be more consistent with the physics of contact, compared with the classical linear models, such as Kelvin-Voigt (KV). This paper experimentally evaluates the author's previously proposed single-stage identification method for real-time parameter estimation of HC nonlinear dynamic models. Experiments are performed on various dynamically distinct objects, including an elastic rubber ball, a piece of sponge, a polyvinyl chloride (PVC) phantom, and a PVC phantom with a hard inclusion. A set of mild conditions for guaranteed unbiased estimation of the proposed method is discussed and experimentally evaluated. Furthermore, this paper rigorously evaluates the performance of the proposed single-stage method and compares it with those of a double-stage method for the HC model and a recursive least squares method for the KV model and its variations in terms of convergence rate, the sensitivity to parameter initialization, and the sensitivity to the changes in environment dynamic properties.

On the Susceptibility of Embedded Thermal Shutdown Circuit to Radio Frequency Interference

This paper deals with the susceptibility to RF interference of a temperature sensor used in smart power system-on-chip to shut down the source of heat or the overall device in the case of overtemperature. Such a sensor is usually buried within the power section; hence, it is parasitically coupled with the power transistors and for this reason its operation can be affected by the disturbances superimposed onto the power transistors' nominal signals. In this work, the detrimental effect of the RF interference on a common thermal shutdown circuit is analyzed using an approximate nonlinear model and performing time-domain computer simulations. The results of these analyses show that RF interference can induce false fault signaling. To avoid this, a simple and effective layout solution that improves the immunity of such kind of circuits is proposed and its effectiveness is proved by experimental test results.

Estimation of nonlinear models with mismeasured regressors using marginal information

AbstractWe consider the estimation of nonlinear models with mismeasured explanatory variables, when information on the marginal distribution of the true values of these variables is available. We derive a semi‐parametric MLE that is shown to be $\sqrt{n}$ consistent and asymptotically normally distributed. In a simulation experiment we find that the finite sample distribution of the estimator is close to the asymptotic approximation. The semi‐parametric MLE is applied to a duration model for AFDC welfare spells with misreported welfare benefits. The marginal distribution of the correctly measured welfare benefits is obtained from an administrative source. Copyright © 2010 John Wiley & Sons, Ltd.

Fundamentals

The Electrophoresis series on Fundamentals has run for ten years, so this issue marks an important anniversary. Ten years ago we wondered whether a Fundamentals issue should be established; but it has since become a success story: it is among the most cited issues of Electrophoresis. Among scientists, it is the theoreticians who contribute most to Fundamentals, with the result that this special issue has established itself as a common platform where theoreticians can meet and share their results and knowledge. In almost all the papers in the Fundamentals series, you will find many formulae and mathematical equations. The beauty of mathematics when applied to physical and chemical problems lies in its ability to formulate models from first principles. The problem, of course, is the solution of the mathematical models, either analytically or numerically. If the mathematical model is correct, it explains the observed phenomena, and also predicts as yet unknown phenomena. And if the experimental results are not in accordance with the prediction of the model, either the mathematical model is incorrect (which, of course, is the less favorable case), or it is incomplete and additional principles should be considered. What will you find in this issue? The first paper is a review on high-voltage power supplies to capillary andmicrochip electrophoresis. The subsequent series of three papers in the section “Electromigration in microchip channels” analyzes various aspects of chip electrophoresis, especially the phenomena connected with electroosmosis. In spite of my long engagement with electromigration, I am still fascinated by the fact that it is a thin diffusion layer (only a few nanometers thick) at the boundary between solid and liquid phases that is responsible for such macroscopic phenomena as the electroosmotic flow. Computer simulation can still reveal various aspects of electromigration. The papers from my group describe the approximate nonlinear mathematical model of electromigration and a new version of PeakMaster software, which can now predict the shape of system zones. The new PeakMaster 5.3 remains free and publicly available. The following three papers deal with an important physicochemical interaction often used in electrophoresis: the ability of analytes to form weak complexes with other constituents of the background electrolyte. This is utilized, for example, by affinity electrophoresis or in chiral electrophoretic separations. My group interacted with the Thormann group to formulate the dynamical mathematical model of electromigration coupled with complexforming equilibria. The result was two independent simulation programs, the extended Gentrans and Simul 5 Complex, capable of explaining the phenomena observed in such systems. All five papers in the section “Theory of electromigration” deal in various ways with the most important quantity in electrophoresis – mobility. Mobility indeed deserves such attention, as it is the key parameter responsible for the discrimination of analytes during separation. Of all the papers in this section, the paper by Wernersson is worth a special mention as it utilizes molecular modeling for the prediction of electrophoretic mobility. Also the paper by Phillies is innovative and brings up an original idea of “an inversion on perspective: instead of treating the migrating species as the objects of experimental importance and the support medium as being of importance only because it facilitates separations, one treats the polymer solution as being of experimental importance, and the choice of migrating species as being of interest only because different species may probe different aspects of polymer dynamics. The last section, “New approaches in electromigration”, offers seven methodological papers on various aspects of electrophoresis. I would like to express great thanks to all the authors who contributed to this issue, as well as to the referees for their careful work. The series of Special Issues of Electrophoresis on Fundamentals will continue next year and hopefully will remain a great source of fundamental knowledge in this field.

Estimation and diagnostics for heteroscedastic nonlinear regression models based on scale mixtures of skew-normal distributions

An extension of some standard likelihood based procedures to heteroscedastic nonlinear regression models under scale mixtures of skew-normal (SMSN) distributions is developed. This novel class of models provides a useful generalization of the heteroscedastic symmetrical nonlinear regression models (Cysneiros et al., 2010), since the random term distributions cover both symmetric as well as asymmetric and heavy-tailed distributions such as skew-t, skew-slash, skew-contaminated normal, among others. A simple EM-type algorithm for iteratively computing maximum likelihood estimates of the parameters is presented and the observed information matrix is derived analytically. In order to examine the performance of the proposed methods, some simulation studies are presented to show the robust aspect of this flexible class against outlying and influential observations and that the maximum likelihood estimates based on the EM-type algorithm do provide good asymptotic properties. Furthermore, local influence measures and the one-step approximations of the estimates in the case-deletion model are obtained. Finally, an illustration of the methodology is given considering a data set previously analyzed under the homoscedastic skew-t nonlinear regression model.

Estimation in Non-Linear Non-Gaussian State Space Models with Precision-Based Methods

In recent years state space models, particularly the linear Gaussian version, have become the standard framework for analyzing macro-economic and financial data. However, many theoretically motivated models imply non-linear or non-Gaussian specifications or both. Existing methods for estimating such models are computationally intensive, and often cannot be applied to models with more than a few states. Building upon recent developments in precision-based algorithms, we propose a general approach to estimating high-dimensional non-linear non-Gaussian state space models. The baseline algorithm approximates the conditional distribution of the states by a multivariate Gaussian or t density, which is then used for posterior simulation. We further develop this baseline algorithm to construct more sophisticated samplers with attractive properties: one based on the accept-reject Metropolis-Hastings (ARMH) algorithm, and another adaptive collapsed sampler inspired by the cross-entropy method. To illustrate the proposed approach, we investigate the effect of the zero lower bound of interest rate on monetary transmission mechanism.