Full-information Estimation Research Articles

Approximating arrival costs in distributed moving horizon estimation: A recursive method

In this paper, we present a new approach to distributed moving horizon estimation for constrained nonlinear processes. The method involves approximating the arrival costs of local estimators through a recursive framework. First, distributed full-information estimation for linear unconstrained systems is presented, which serves as the foundation for deriving the analytical expression of the arrival costs for the local estimators. Subsequently, we develop a recursive arrival cost design for linear distributed moving horizon estimation. Sufficient conditions are derived to ensure the stability of the estimation error for constrained linear systems. Next, we extend the arrival cost design derived for linear systems to account for nonlinear systems, and a partition-based constrained distributed moving horizon estimation algorithm for nonlinear systems is formulated. A benchmark chemical process is used to illustrate the effectiveness and superiority of the proposed method.

Generic Stability Implication From Full Information Estimation to Moving-Horizon Estimation

Optimization-based state estimation is useful for handling of constrained linear or nonlinear dynamical systems. It has an ideal form, known as full information estimation (FIE) which uses all past measurements to perform state estimation, and also a practical counterpart, known as moving-horizon estimation (MHE) which uses most recent measurements of a limited length to perform the estimation. This work reveals a generic link from robust stability of FIE to that of MHE, showing that the former implies at least a weaker robust stability of MHE which implements a long enough horizon. The implication strengthens to strict robust stability of MHE if the corresponding FIE satisfies a mild Lipschitz continuity condition. The revealed implications are then applied to derive new sufficient conditions for robust stability of MHE, which further reveals an intrinsic relation between the existence of a robustly stable FIE/MHE and the system being incrementally input/output-to-state stable.

A Recursive Approach to Approximate Arrival Costs in Distributed Moving Horizon Estimation

In this paper, we present a novel approach to distributed moving horizon estimation for nonlinear processes. The method involves approximating the arrival costs of local estimators through a recursive framework. First, we formulate distributed full-information estimation for linear unconstrained systems, which serves as the foundation for deriving the analytical expression of the arrival cost. Subsequently, a recursive arrival cost design for linear distributed moving horizon estimation is developed. Then, we extend the arrival cost design obtained for linear systems to account for the nonlinear context, and a partition-based constrained distributed moving horizon estimation algorithm for nonlinear systems is formulated. A chemical process is introduced to illustrate the effectiveness of the proposed method.

Moving horizon estimation over relay channels: Dealing with packet losses

This paper is concerned with the moving horizon estimation problem for networked systems over relay channels. The relay channel under consideration is a wireless communication channel with a so-called relay node. Under the effects of relay node, the signal transmission between the plant and estimator is determined by a combination of three communication processes, namely the sensor-to-estimator, the sensor-to-relay and the relay-to-estimator communication processes. In order to characterize the impact of such kind of relay-channel-based communication, a three-terminal relay model is employed to describe the signal transmission behaviors between the plant and the estimator. Three Bernoulli-distributed white sequences are adopted to model the packet losses occurring in the communication processes. A full information estimation strategy is firstly developed in the sense of maximum a posteriori according to the system dynamics and transmission behaviors. Then, a moving horizon estimator is designed based on the derived full information estimation strategy. Furthermore, the solution of the proposed moving horizon estimation strategy is achieved by using the completing-the-square approach and, subsequently, sufficient conditions are derived to guarantee the mean-square ultimate boundedness of the resultant error dynamics of the state estimation. Two simulation examples are given to demonstrate the effectiveness of our developed estimation scheme.

Heywood Cases in Unidimensional Factor Models and Item Response Models for Binary Data.

Heywood cases are known from linear factor analysis literature as variables with communalities larger than 1.00, and in present day factor models, the problem also shows in negative residual variances. For binary data, factor models for ordinal data can be applied with either delta parameterization or theta parametrization. The former is more common than the latter and can yield Heywood cases when limited information estimation is used. The same problem shows up as non convergence cases in theta parameterized factor models and as extremely large discriminations in item response theory (IRT) models. In this study, we explain why the same problem appears in different forms depending on the method of analysis. We first discuss this issue using equations and then illustrate our conclusions using a small simulation study, where all three methods, delta and theta parameterized ordinal factor models (with estimation based on polychoric correlations and thresholds) and an IRT model (with full information estimation), are used to analyze the same datasets. The results generalize across WLS, WLSMV, and ULS estimators for the factor models for ordinal data. Finally, we analyze real data with the same three approaches. The results of the simulation study and the analysis of real data confirm the theoretical conclusions.

Nonlinear full information and moving horizon estimation: Robust global asymptotic stability

In this paper, we propose time-discounted schemes for full information estimation (FIE) and moving horizon estimation (MHE) that are robustly globally asymptotically stable (RGAS). We consider general nonlinear system dynamics with nonlinear process and output disturbances that are a priori unknown. For FIE being RGAS, our only assumptions are that the system is time-discounted incrementally input–output-to-state-stable (i-IOSS) and that the time-discounted FIE cost function is compatible with the i-IOSS estimate. Since for i-IOSS systems such a compatible cost function can always be designed, we show that i-IOSS is sufficient for the existence of RGAS observers. Based on the stability result for FIE, we provide sufficient conditions such that the induced MHE scheme is RGAS as well for sufficiently large horizons. For both schemes, we can guarantee convergence of the estimation error in case the disturbances converge to zero without incorporating a priori knowledge. Finally, we present explicit converge rates and show how to verify that the MHE results approach the FIE results for increasing horizons.

Estimating Joint Health State Utility Algorithms Under Partial Information

ObjectivesWe explored the performance of existing joint health state utility estimators when data are not available on utilities that isolate single-condition health states excluding any co-occurring condition. MethodsUsing data from the National Epidemiologic Survey on Alcohol and Related Conditions-III, we defined 2 information sets: (1) a full-information set that includes the narrowly defined health state utilities used in most studies that test the performance of joint health state utility estimators, and (2) a limited information set that includes only the more broadly defined health state utilities more commonly available to researchers. We used an example of alcohol use disorder co-occurring with cirrhosis of the liver, depressive disorder, or nicotine use disorder to illustrate our analysis. ResultsWe found that the performance of joint health state utility estimators is appreciably different under limited information than under full information. Full-information estimators typically overestimate the joint state utility, whereas limited-information estimators underestimate the joint state utility, except for the minimum estimator, which is overestimated in all cases. ConclusionsResearchers using joint health state utility estimators should understand the information set available to them and use methodological guidance appropriate for that information set. We recommend the minimum estimator under limited information based on its ease of use, consistency (and therefore a predictable direction of bias), and lower root mean squared error.

A variational Bayes moving horizon estimation adaptive filter with guaranteed stability

This paper addresses state estimation of linear systems with special attention on unknown process and measurement noise covariances, aiming to enhance estimation accuracy while ensuring stability. To this end, the full information estimation problem over a finite interval is first addressed. Then, a novel adaptive variational Bayesian (VB) moving horizon estimation (MHE) method is proposed, exploiting VB inference, MHE, and Monte Carlo integration with importance sampling for joint estimation of the unknown process and measurement noise covariances, along with the state trajectory over a moving window of fixed length. Further, it is proved that the proposed adaptive VB MHE filter ensures mean-square boundedness of the estimation error with any number of importance samples and VB iterations, as well as for any window length. Finally, simulation results on a target tracking example demonstrate the effectiveness of the VB MHE filter with enhanced estimation accuracy and convergence properties compared to the conventional non-adaptive Kalman filter and other existing adaptive filters.

SIMULTANEOUS EQUATIONS MODELS WITH HIGHER-ORDER SPATIAL OR SOCIAL NETWORK INTERACTIONS

This paper develops an estimation methodology for network data generated from a system of simultaneous equations, which allows for network interdependencies via spatial lags in the endogenous and exogenous variables, as well as in the disturbances. By allowing for higher-order spatial lags, our specification provides important flexibility in modeling network interactions. The estimation methodology builds, among others, on the two-step generalized method of moments estimation approach introduced in Kelejian and Prucha (1998, Journal of Real Estate Finance and Economics 17, 99–121; 1999, International Economic Review 40, 509–533; 2004, Journal of Econometrics 118, 27–50). The paper considers limited and full information estimators, and one- and two-step estimators, and establishes their asymptotic properties. In contrast to some of the earlier two-step estimation literature, our asymptotic results facilitate joint tests for the absence of all forms of network spillovers.

Minimum variance constrained estimator

This paper is concerned with the problem of state estimation for discrete-time linear systems in the presence of additional (equality or inequality) constraints on the state (or estimate). By use of the minimum variance duality, the estimation problem is converted into an optimal control problem. Two algorithmic solutions are described: the full information estimator (FIE) and the moving horizon estimator (MHE). The main result is to show that the proposed estimator is stable in the sense of an observer. The proposed algorithm is distinct from the standard algorithm for constrained state estimation based upon the use of the minimum energy duality. The two are compared numerically on the benchmark batch reactor process model.

Soft sensor for monitoring dynamic changes in cell composition

Precise and accurate online monitoring methods are needed to enable smart biomanufacturing and automation. Most of the sensors available focus on process parameters such as metabolite and dissolved gas concentrations, cell density or viability, among other variables like pH, temperature, etc. In this work, we develop a soft sensor algorithm to estimate the cell composition online, a very important aspect often overlooked in the bioprocess monitoring literature. Our strategy is based on full information estimation, an optimization-based estimator that takes into account the dynamics of the cell metabolism and considers all the available measurements from the beginning of the process, thus it has a memory effect. Being able to track dynamic changes in cell composition can open the door to promising applications, e.g., predictive control and automation of biosystems. As a case study, we consider the Escherichia coli’s metabolism growing on glycerol under different levels of oxygen supply. We compare the performance of our soft sensor method against resource balance analysis, a previously proposed estimator based on steady-state assumptions. Overall, the presented full information estimator was able to track the dynamic changes in cell composition significantly more accurately. We also discuss how our estimation strategy can be transformed into a moving horizon estimation, where only the available measurements in a fixed and moving window are considered, thereby reducing possible computational burdens.

A unified model-implied instrumental variable approach for structural equation modeling with mixed variables

The model-implied instrumental variable (MIIV) estimator is an equation-by-equation estimator of structural equation models that is more robust to structural misspecifications than full information estimators. Previous studies have concentrated on endogenous variables that are all continuous (MIIV-2SLS) or all ordinal . We develop a unified MIIV approach that applies to a mixture of binary, ordinal, censored, or continuous endogenous observed variables. We include estimates of factor loadings, regression coefficients, variances, and covariances along with their asymptotic standard errors. In addition, we create new goodness of fit tests of the model and overidentification tests of single equations. Our simulation study shows that the proposed MIIV approach is more robust to structural misspecifications than diagonally weighted least squares (DWLS) and that both the goodness of fit model tests and the overidentification equations tests can detect structural misspecifications. We also find that the bias in asymptotic standard errors for the MIIV estimators of factor loadings and regression coefficients are often lower than the DWLS ones, though the differences are small in large samples. Our analysis shows that scaling indicators with low reliability can adversely affect the MIIV estimators. Also, using a small subset of MIIVs reduces small sample bias of coefficient estimates, but can lower the power of overidentification tests of equations.

Moving Horizon H∞ Estimation of Constrained Multisensor Systems With Uncertainties and Fading Channels

This article considers the robust estimation problem of linear multisensor systems subject to constraints and additive and multiplicative uncertainties where the decaying of measurements is regarded as the multiplicative uncertainty when taking into account the fading channels between the estimator and sensors. Based on the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> performance criterion, a moving horizon H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> estimation algorithm is proposed from an approximation of the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> full information estimation, which describes the estimation problem as a minimax optimization problem. Sufficient conditions for the feasibility and stability of the proposed algorithm are discussed, under which the estimation errors are ultimately bounded. To reduce the difficulty of solving minimax problem, an approximate solution is provided. The simulation comparison results show the effectiveness of the proposed algorithm.

Evaluating Changes in the Transmission Mechanism of Government Spending Shocks

We empirically revisit the crowding-in effect of government spending on private consumption based on rolling windows of U.S. data. Results show that in earlier samples government spending is increasingly crowding in private consumption; however, this relation is reverted in the latest periods. We propose a model embedding non-separable public and private consumption in the utility function and rule-of-thumb consumers to assess the sources of non-monotonic changes in the transmission of the shock. The iterative full information estimation of the model reveals that changes in the co-movement between private and public spending is primarily driven by the fluctuations in the elasticity of substitution between private and public consumption, the share of financially constrained consumers, and the elasticity of intertemporal substitution.

Evaluating Changes in the Transmission Mechanism of Government Spending Shocks

Abstract We empirically revisit the crowding-in effect of government spending on private consumption based on rolling windows of U.S. data. Results show that in earlier samples government spending is increasingly crowding in private consumption; however, this relation is reverted in the latest periods. We propose a model embedding non-separable public and private consumption in the utility function and rule-of-thumb consumers to assess the sources of non-monotonic changes in the transmission of the shock. The iterative full information estimation of the model reveals that changes in the co-movement between private and public spending is primarily driven by the fluctuations in the elasticity of substitution between private and public consumption, the share of financially constrained consumers, and the elasticity of intertemporal substitution.

Full Information Estimation for Time-Varying Systems Subject to Round-Robin Scheduling: A Recursive Filter Approach

The full information estimation (FIE) problem is addressed for discrete time-varying systems (TVSs) subject to the effects of a round-Robin (RR) protocol. A shared communication network is adopted for data transmissions between sensor nodes and the state estimator. In order to avoid data collisions in signal transmission, only one sensor node could have access to the network and communicate with the state estimator per time instant. The so-called RR protocol, which is also known as the token ring protocol, is employed to orchestrate the access sequence of sensor nodes, under which the chosen sensor node communicating with the state estimator could be modeled by a periodic function. A novel recursive FIE scheme is developed by defining a modified cost function and using a so-called “backward-propagation-constraints.” The modified cost function represents a special global estimation performance. The solution of the proposed FIE scheme is achieved by solving a minimization problem. Then, the recursive manner of such a solution is studied for the purpose of online applications. For the purpose of ensuring the estimation performance, sufficient conditions are obtained to derive the upper bound of the norm of the state estimation error (SEE). Finally, two illustrative examples are proposed to demonstrate the effectiveness of the developed estimation algorithm.

A robust alternative estimator for small to moderate sample SEM: Bias-corrected factor score path analysis

Structural equation modeling with full information maximum likelihood estimation is the predominant method to empirically assess complex theories involving multiple latent variables in addiction research. Although full information estimators have many desirable properties including consistency, a major limitation in structural equation models is that they often sustain significant bias when implemented in small to moderate size studies (e.g., fewer than 100 or 200). Recent literature has developed a limited information estimator designed to address this limitation—conceptually implemented through a bias-corrected factor score path analysis approach—that has been shown to produce unbiased and efficient estimates in small to moderate sample settings. Despite its theoretical and empirical merits, literature has suggested that the method is underused because of three primary reasons—the methods are unfamiliar to applied researchers, there is a lack of practical and accessible guidance and software available for applied researchers, and comparisons against full information methods that are grounded in discipline-specific examples are lacking. In this study, I delineate this method through a step-by-step analysis of a sequential mediation case study involving internet addiction. I provide example R code using the lavaan package and data based on a hypothetical study of addiction. I examine the differences between the full and limited information estimators within the example data and subsequently probe the extent to which these differences are indicative of a consistent divergence between the estimators using a simulation study. The results suggest that the limited information estimator outperforms the conventional full information maximum likelihood estimator in small to moderate sample sizes in terms of bias, efficiency, and power.

Suboptimal receding horizon estimation via noise blocking

For discrete-time linear systems, we propose a suboptimal approach to constrained estimation so that the associated computation burden is reduced. This is achieved by enforcing a move blocking (MB) structure in the estimated process noise sequence (PNS). We show that full information estimation (FIE) and receding horizon estimation (RHE) with MB are both stable in the sense of an observer. The techniques in proving stability are inspired by those that have been proposed for standard RHE. To be specific, stability results are mainly achieved by (i) carefully embellishing the general assumptions for standard RHE to accommodate the MB requirement; (ii) exploiting the principle of optimality, as well as convexity of the quadratic programs (QPs) associated with FIE and RHE; (iii) relying on the fact that the Kalman filter is the best linear estimator in the least-squares sense. A crucial requirement in achieving stability for MB RHE is that the segment structure (SS) of the PNS of MB FIE for the optimization steps within the receding horizon (i.e., steps between T−N and T−1) has to be enforced in the MB RHE optimization. As a result, the MB RHE strategy becomes a dynamic estimator with a periodically varying computational complexity. The theoretical results have been illustrated with examples.

Performance of Nonrecursive Latent Variable Models under Misspecification

A problem central to structural equation modeling is measurement model specification error and its propagation into the structural part of nonrecursive latent variable models. Full-information estimation techniques such as maximum likelihood are consistent when the model is correctly specified and the sample size large enough; however, any misspecification within the model can affect parameter estimates in other parts of the model. The goals of this study included comparing the bias, efficiency, and accuracy of hypothesis tests in nonrecursive latent variable models with indirect and direct feedback loops. We compare the performance of maximum likelihood, two-stage least-squares and Bayesian estimators in nonrecursive latent variable models with indirect and direct feedback loops under various degrees of misspecification in small to moderate sample size conditions.

Simultaneous Generalized Method of Moments Estimator for Panel Data Models with Spatially Correlated Error Components

This paper develops estimators for simultaneous equations with spatial autoregressive or spatial moving average error components. We derive a limited information estimator and a full information estimator. We give the simultaneous generalized method of moments to get each component of the variance covariance of the disturbance in spatial autoregressive case as well as spatial moving average case. The results of our Monte Carlo suggest that our estimators are consistent. When we estimate the coefficient of spatial dependence it seems better to use instrumental variables estimator that takes into account simultaneity. We also apply these set of estimators on real data.