Nonlinear Estimation Problem Research Articles

A different approach to estimate nonlinear regression model using numerical methods

This research paper concerns with the computational methods namely the Gauss-Newton method, Gradient algorithm methods (Newton-Raphson method, Steepest Descent or Steepest Ascent algorithm method, the Method of Scoring, the Method of Quadratic Hill-Climbing) based on numerical analysis to estimate parameters of nonlinear regression model in a very different way. Principles of matrix calculus have been used to discuss the Gradient-Algorithm methods. Yonathan Bard [1] discussed a comparison of gradient methods for the solution of nonlinear parameter estimation problems. However this article discusses an analytical approach to the gradient algorithm methods in a different way. This paper describes a new iterative technique namely Gauss-Newton method which differs from the iterative technique proposed by Gorden K. Smyth [2]. Hans Georg Bock et.al [10] proposed numerical methods for parameter estimation in DAE’s (Differential algebraic equation). Isabel Reis Dos Santos et al [11], Introduced weighted least squares procedure for estimating the unknown parameters of a nonlinear regression metamodel. For large-scale non smooth convex minimization the Hager and Zhang (HZ) conjugate gradient Method and the modified HZ (MHZ) method were presented by Gonglin Yuan et al [12].

Nonlinear state estimation using neural-cubature Kalman filter

ABSTRACTThe cubature Kalman filter (CKF) has been widely used in solving nonlinear state estimation problems because of many advantages such as satisfactory filtering accuracy and easy implementation compared to extended Kalman filter and unscented Kalman filter. However, the performance of CKF may degrade due to the uncertainty of the nonlinear dynamic system model. To solve this problem, a neural-cubature Kalman filter (NCKF) algorithm containing a multilayer feed-forward neural network (MFNN) in CKF is proposed to further improve the estimation accuracy and enhance the robustness of CKF. In the proposed NCKF algorithm, the MFNN was used to modify the nonlinear state estimation of CKF as the measurements were processed, and the CKF was used as both a state estimator and an online training paradigm simultaneously. The experimental results show that the estimation accuracy and robustness of the proposed method are better than those of the CKF, square-root CKF and particle filter.

Nonlinear Estimator Design for MEMS Gyroscope with Time-varying Angular Rate

The problem of nonlinear parameter estimation of a MEMS gyroscope system is discussed. The MEMS gyroscope is considered with unknown parameters including mechanical structure imperfections and an unknown time-varying angular rate. The proposed approach uses the state estimation methodology to compensate the fabrication imperfections and to obtain a correct angular rate in real-time. The problem is formulated as a joint state and parameter estimation where the unscented Kalman filter is employed. For that purpose, a novel modeling technique approximating the angular rate using the first order Gauss-Markov process is proposed. To ensure persistent excitation of the system to achieve convergence of estimated parameters, both axes of the device are actively controlled by an appropriately designed adaptive controller. Properties of the proposed nonlinear estimation method are tested and validated through numerical simulations.

Arbitrary Polynomial Chaos for Uncertainty Propagation of Correlated Random Variables in Dynamic Systems

Dynamic simulation of stochastic systems requires uncertainty propagation. Traditional sample-based uncertainty propagation methods are often computationally intractable for online optimization-based estimation and control applications. Generalized polynomial chaos (gPC) is an efficient uncertainty propagation method that has been used for solving various nonlinear estimation and optimal control problems. However, gPC requires knowledge of the exact probability distribution of the random variables, and does not explicitly account for correlations between these random variables. This paper demonstrates the use of arbitrary polynomial chaos (aPC) for propagation of correlated multivariate random variables. aPC constructs orthogonal polynomial basis functions from only the raw moments of the random variables. Thus, aPC can be used for propagation of uncertainties with arbitrary probability distributions, even if their functional forms are unknown. The main contributions of this paper consist of presenting an algorithm for generating a set of orthogonal polynomial basis functions for correlated multivariate random variables and applying the Galerkin projection to compute closed-form expressions for the dynamics of the aPC expansion coefficients. An algorithm is also presented for efficient calculation of inner products between polynomial basis functions needed in the Galerkin projection. The error convergence properties of aPC are investigated and compared to that of gPC and Monte Carlo using a dynamic simulation case study.

Model-Based Optimal Experiment Design for Nonlinear Parameter Estimation Using Exact Confidence Regions

Optimal experiment design is usually performed as a search over a finitely-parameterized shape that (over-) approximates the confidence region of parameters of a model. In general, there exists no such shape to exactly enclose the confidence region of a nonlinear parameter estimation problem. Due to this fact, the design-of-experiment techniques are not well established for this problem and approximate designs are conducted. In this contribution, assuming Gaussian (normally distributed) noise, we propose and study (a) two schemes to over-approximate the confidence region of parameters using an ellipsoid and an orthotope and (b) a framework for optimal experiment design. We formulate the over-approximation of the confidence region as an optimization problem. The optimal experiment design is then proposed as a bi-level optimization problem. In line with the existing optimal experiment design methodology for a linear parameter estimation problem, we also propose several design criteria that optimize some measure of the over-approximated confidence region for the nonlinear case. The proposed bi-level optimization problem is solved (i) as a nonlinear programming problem using the necessary conditions for optimality or (ii) as a nested problem with globally optimized inner-level problem. We illustrate the proposed schemes on a benchmark test case.

Optimal Nonlinear Estimation in Statistical Manifolds with Application to Sensor Network Localization

Information geometry enables a deeper understanding of the methods of statistical inference. In this paper, the problem of nonlinear parameter estimation is considered from a geometric viewpoint using a natural gradient descent on statistical manifolds. It is demonstrated that the nonlinear estimation for curved exponential families can be simply viewed as a deterministic optimization problem with respect to the structure of a statistical manifold. In this way, information geometry offers an elegant geometric interpretation for the solution to the estimator, as well as the convergence of the gradient-based methods. The theory is illustrated via the analysis of a distributed mote network localization problem where the Radio Interferometric Positioning System (RIPS) measurements are used for free mote location estimation. The analysis results demonstrate the advanced computational philosophy of the presented methodology.

Estimation of zero sequence parameters of mutually coupled transmission lines from synchrophasor measurements

The authors consider the problem of estimating zero sequence parameters of a transmission line using synchrophasor data. When a set of three-phase transmission lines share, partial or complete, right of way, then their zero sequence models exhibit mutual coupling. As such the zero sequence parameters cannot be estimated by linear least squares or total least squares (TLS) technique which are the preferred methods when dealing with the positive sequence line parameter estimation problem. Further, method design has to factor the constraint of a sparse data set when dealing with zero sequence phasors. Therefore, the authors propose orthogonal distance regression approach for solving the zero sequence parameter estimation problem. This generalises the method of TLS to the non-linear parameter estimation problem considering noise in both the voltage and current synchrophasor measurements. Extensive case studies and comparative evaluations are presented to demonstrate the efficacy of the proposed approach.

A Spherical Simplex Unscented Rauch-Tung-Striebel Smoother for a Vehicle Localization Problem

The unscented Kalman filter (UKF) has become relatively a new technique used in a number of nonlinear estimation problems to overcome the limitation of Taylor series linearization. It uses a deterministic sampling approach known as sigma points to propagate nonlinear systems and has been discussed in many literature. However, a nonlinear smoothing problem has received less attention than the filtering problem. Therefore, in this article an unscented smoother based on Rauch-Tung-Striebel form is examined for discrete-time dynamic systems. It has advantages available in unscented transformation over approximation by Taylor expansion as well as its benefit in derivative free. In addition, new sampling technique known as a spherical simplex has been introduced and evaluated. To show the effectiveness of the proposed method, the unscented smoother is implemented and evaluated through a vehicle localization problem

Parameter Selection Method for Support Vector Regression Based on Adaptive Fusion of the Mixed Kernel Function

Support vector regression algorithm is widely used in fault diagnosis of rolling bearing. A new model parameter selection method for support vector regression based on adaptive fusion of the mixed kernel function is proposed in this paper. We choose the mixed kernel function as the kernel function of support vector regression. The mixed kernel function of the fusion coefficients, kernel function parameters, and regression parameters are combined together as the parameters of the state vector. Thus, the model selection problem is transformed into a nonlinear system state estimation problem. We use a 5th-degree cubature Kalman filter to estimate the parameters. In this way, we realize the adaptive selection of mixed kernel function weighted coefficients and the kernel parameters, the regression parameters. Compared with a single kernel function, unscented Kalman filter (UKF) support vector regression algorithms, and genetic algorithms, the decision regression function obtained by the proposed method has better generalization ability and higher prediction accuracy.

Localization of Near-Field Sources Based on Sparse Signal Reconstruction with Regularization Parameter Selection

Source localization using sensor array in the near-field is a two-dimensional nonlinear parameter estimation problem which requires jointly estimating the two parameters: direction-of-arrival and range. In this paper, a new source localization method based on sparse signal reconstruction is proposed in the near-field. We first utilize l1-regularized weighted least-squares to find the bearings of sources. Here, the weight is designed by making use of the probability distribution of spatial correlations among symmetric sensors of the array. Meanwhile, a theoretical guidance for choosing a proper regularization parameter is also presented. Then one well-known l1-norm optimization solver is employed to estimate the ranges. The proposed method has a lower variance and higher resolution compared with other methods. Simulation results are given to demonstrate the superior performance of the proposed method.

Численный метод оценки параметров нелинейного дифференциального оператора второго порядка

Проблема нелинейного оценивания параметров систем различной физической природы, описываемых нелинейным дифференциальным оператором, является важнейшей проблемой математического моделирования. В статье рассматривается новый численный метод оценки параметров нелинейного дифференциального оператора второго порядка с диссипативной силой, пропорциональной n-степени скорости движения. В основе численного метода лежит среднеквадратичное оценивание коэффициентов обобщенной регрессионной модели, построенной с учетом разностных уравнений, описывающих результаты измерений импульсной характеристики системы. Реализованная в методе двух этапная процедура дифференцированного оценивания параметров динамического процесса позволяет обеспечить высокую адекватность построенной модели данным эксперимента. Применение разработанного численного метода позволяет существенно (в несколько раз) повысить точность оценок параметров нелинейного дифференциального оператора по сравнению с известными методами за счет устранения смещения в оценках, обусловленного использованием аппроксимации при моделировании огибающей амплитуд колебаний.

PyParticleEst: A Python Framework for Particle-Based Estimation Methods

Particle methods such as the particle filter and particle smoothers have proven very useful for solving challenging nonlinear estimation problems in a wide variety of fields during the last decade. However, there are still very few existing tools available to support and assist researchers and engineers in applying the vast number of methods in this field to their own problems. This paper identifies the common operations between the methods and describes a software framework utilizing this information to provide a flexible and extensible foundation which can be used to solve a large variety of problems in this domain, thereby allowing code reuse to reduce the implementation burden and lowering the barrier of entry for applying this exciting field of methods. The software implementation presented in this paper is freely available and permissively licensed under the GNU Lesser General Public License, and runs on a large number of hardware and software platforms, making it usable for a large variety of scenarios.

Hierarchy cuckoo search algorithm for parameter estimation in biological systems

Mathematical models play a critical role in describing behavior of biomedical and biological processes. These models have unknown parameters that cannot be measured directly and researchers need reliable tools to estimate them from available experimental data. We present a new modification of cuckoo search (CS) algorithm, named hybrid hierarchical cuckoo search (HHCS), as a powerful computational tool to help with this task. The proposed algorithm uses an information sharing mechanism which allows cuckoo eggs to share their best search information through a heterogeneous hierarchical structure. The HHCS takes advantage of the cooperation between cuckoo eggs to maintain the diversity of solutions and to avoid premature convergence. This approach is also hybridized with a local search method in order to reduce the computational time required for determining the optimal solution. First, the generalization properties of HHCS is evaluated against a set of benchmark functions. Thereafter, we investigate the application of HHCS on two highly non-linear parameter estimation problems of biological systems, namely isomerization of α-Pinene and inhibition of HIV proteinase. Statistical analysis of experimental results shows that the proposed algorithm provides very competitive results compared to CS and several state-of-the-art algorithms in terms of solution accuracy, convergence speed, and robustness. Notably, the HHCS improves the best known solution in the literature for the inhibition of HIV proteinase.

Spectral image analysis of mutual illumination between florescent objects.

This paper proposes a method for modeling and component estimation of the spectral images of the mutual illumination phenomenon between two fluorescent objects. First, we briefly describe the bispectral characteristics of a single fluorescent object, which are summarized as a Donaldson matrix. We suppose that two fluorescent objects with different bispectral characteristics are located close together under a uniform illumination. Second, we model the mutual illumination between two objects. It is shown that the spectral composition of the mutual illumination is summarized with four components: (1) diffuse reflection, (2) diffuse-diffuse interreflection, (3) fluorescent self-luminescence, and (4) interreflection by mutual fluorescent illumination. Third, we develop algorithms for estimating the spectral image components from the observed images influenced by the mutual illumination. When the exact Donaldson matrices caused by the mutual illumination influence are unknown, we have to solve a non-linear estimation problem to estimate both the spectral functions and the location weights. An iterative algorithm is then proposed to solve the problem based on the alternate estimation of the spectral functions and the location weights. In our experiments, the feasibility of the proposed method is shown in three cases: the known Donaldson matrices, weak interreflection, and strong interreflection.

Location of Things: Geospatial Tagging for IoT Using Time-of-Arrival

We develop a new algorithm for geospatial tagging for Internet-of-Things (IoT) type applications, which we denote as location-of-things (LoT). The underlying idea of LoT applications is to use low-cost off-the-shelf two-way time-of-arrival (TW-ToA) ranging devices to perform localization of tags. We first demonstrate how conventional TW-ToA localization algorithms may experience performance degradation in cases where some of the access points (APs) are outside the communication range of the tags. We then show that we can make use of the audibility information (which indicates whether an AP is able or unable to communicate with the tags). By leveraging on this available information, we re-formulate the localization problem as a statistical nonlinear estimation problem. This information, coupled with ranging observations from audible AP leads to a new maximum likelihood estimation (MLE) algorithm for the tag’s location. Our approach provides considerable improvement of the localization performance by mitigating the well-known ambiguity problem which arises when only a few AP are audible. In addition, we derive the Cramer–Rao bound (CRB) of the source location estimate under the proposed framework.

Integrated Navigation System Design for Micro Planetary Rovers: Comparison of Absolute Heading Estimation Algorithms and Nonlinear Filtering.

This paper provides algorithms to fuse relative and absolute microelectromechanical systems (MEMS) navigation sensors, suitable for micro planetary rovers, to provide a more accurate estimation of navigation information, specifically, attitude and position. Planetary rovers have extremely slow speed (~1 cm/s) and lack conventional navigation sensors/systems, hence the general methods of terrestrial navigation may not be applicable to these applications. While relative attitude and position can be tracked in a way similar to those for ground robots, absolute navigation information is hard to achieve on a remote celestial body, like Moon or Mars, in contrast to terrestrial applications. In this study, two absolute attitude estimation algorithms were developed and compared for accuracy and robustness. The estimated absolute attitude was fused with the relative attitude sensors in a framework of nonlinear filters. The nonlinear Extended Kalman filter (EKF) and Unscented Kalman filter (UKF) were compared in pursuit of better accuracy and reliability in this nonlinear estimation problem, using only on-board low cost MEMS sensors. Experimental results confirmed the viability of the proposed algorithms and the sensor suite, for low cost and low weight micro planetary rovers. It is demonstrated that integrating the relative and absolute navigation MEMS sensors reduces the navigation errors to the desired level.

Constrained Unscented Gaussian Sum Filter for state estimation of nonlinear dynamical systems

Abstract This work presents a novel constrained Bayesian state estimation approach for nonlinear dynamical systems. The proposed approach uses the recently proposed Unscented Gaussian Sum Filter to represent the underlying non-Gaussian densities as sum of Gaussians, and explicitly incorporates constraints on states during the measurement update step. This approach, labeled Constrained-Unscented Gaussian Sum Filter (C-UGSF), can thus model non-Gaussianity in constrained, nonlinear state estimation problems. Its applicability is demonstrated using three nonlinear, constrained state estimation case studies taken from literature, namely, (i) a gas phase batch reactor, (ii) an isothermal batch process, and (iii) a continuous polymerization process. Results demonstrate superior estimation performance along with a significant improvement in computational time when compared to Unscented Recursive Nonlinear Dynamic Data Reconciliation (URNDDR), which is a popular nonlinear, constrained state estimation approach available in literature.

Adjoint multi-start-based estimation of cardiac hyperelastic material parameters using shear data.

Cardiac muscle tissue during relaxation is commonly modeled as a hyperelastic material with strongly nonlinear and anisotropic stress response. Adapting the behavior of such a model to experimental or patient data gives rise to a parameter estimation problem which involves a significant number of parameters. Gradient-based optimization algorithms provide a way to solve such nonlinear parameter estimation problems with relatively few iterations, but require the gradient of the objective functional with respect to the model parameters. This gradient has traditionally been obtained using finite differences, the calculation of which scales linearly with the number of model parameters, and introduces a differencing error. By using an automatically derived adjoint equation, we are able to calculate this gradient more efficiently, and with minimal implementation effort. We test this adjoint framework on a least squares fitting problem involving data from simple shear tests on cardiac tissue samples. A second challenge which arises in gradient-based optimization is the dependency of the algorithm on a suitable initial guess. We show how a multi-start procedure can alleviate this dependency. Finally, we provide estimates for the material parameters of the Holzapfel and Ogden strain energy law using finite element models together with experimental shear data.

Unscented Kalman Filter for Determination of Spacecraft Attitude Using Different Attitude Parameterizations and Real Data

The non-linear estimators are certainly the most important algorithms applied to real problems, especially those involving the attitude estimation of spacecraft. The purpose of this paper was to use real data of sensors to analyze the behavior of Unscented Kalman Filter (UKF) in attitude estimation problems when it is represented in different ways and compare it with the standard estimator for non-linear estimation problems. The robustness of the estimation was performed when this was subjected to imprecise initial conditions. The attitude parametrization was described in Euler angles, quaternion and quaternion incremental. The satellite China-Brazil Earth Resources Satellite and measurements provided by the Satellite Control Center of the Instituto Nacional de Pesquisas Espaciais were considered in the study. The results indicate that the behaviors for both estimators were equivalent for such parameterizations under the same conditions. However, comparing the Unscented Kalman Filter with the standard filter for non-linear systems, Extended Kalman Filter (EKF), it was observed that, in the presence of inaccurate initial conditions, the Unscented Kalman Filter presented a fast convergence whereas Extended Kalman Filter had problems and only converged later on.

Frequency-Dependent Sliding-Mode Control of Galfenol-Driven Unimorph Actuator Based on Finite-Element Model

A dynamic model is presented by coupling the structural dynamics of a Galfenol-driven unimorph beam with the magnetostriction generated by the active layer. Weak form equations are obtained by employing the virtual work principle, and finite-element representation of the model is implemented through a Galerkin discretization. In order to facilitate the design of the controller, a linearized constitutive law is used to describe the magnetostriction of Galfenol. A finite-dimensional sliding-mode controller is developed which accounts for the frequency-dependent deviations of structural damping and the piezomagnetic constant. A nonlinear switch controller, combined with the equivalent controller, is developed to guarantee Lyapunov stability at different frequencies, without changing the initial parameters of the controller. Since the model is finite-dimensional, model parameters are nonlinearly coupled in the state and output matrices in the state-space equation. A genetic algorithm is employed to solve the nonlinear estimation problem. Experimental results show that the model parameters are almost constant below 220 Hz. The model parameters are frequency dependent above 220 Hz. It is also found that significant performance enhancements are achieved by the proposed control relative to conventional Proportional-Integral (PI) control. The system remains stable for frequencies up to 400 Hz.