Parameter Estimation Error Research Articles

Minimum Excess Risk in Bayesian Learning

We analyze the best achievable performance of Bayesian learning under generative models by defining and upper-bounding the minimum excess risk (MER): the gap between the minimum expected loss attainable by learning from data and the minimum expected loss that could be achieved if the model realization were known. The definition of MER provides a principled way to define different notions of uncertainties in Bayesian learning, including the aleatoric uncertainty and the minimum epistemic uncertainty. Two methods for deriving upper bounds for the MER are presented. The first method, generally suitable for Bayesian learning with a parametric generative model, upper-bounds the MER by the conditional mutual information between the model parameters and the quantity being predicted given the observed data. It allows us to quantify the rate at which the MER decays to zero as more data becomes available. Under realizable models, this method also relates the MER to the richness of the generative function class, notably the VC dimension in binary classification. The second method, particularly suitable for Bayesian learning with a parametric predictive model, relates the MER to the minimum estimation error of the model parameters from data. It explicitly shows how the uncertainty in model parameter estimation translates to the MER and to the final prediction uncertainty. We also extend the definition and analysis of MER to the setting with multiple model families and the setting with nonparametric models. Along the discussions we draw some comparisons between the MER in Bayesian learning and the excess risk in frequentist learning.

Physical-model guided self-distillation network for single image dehazing.

Image dehazing, as a key prerequisite of high-level computer vision tasks, has gained extensive attention in recent years. Traditional model-based methods acquire dehazed images via the atmospheric scattering model, which dehazed favorably but often causes artifacts due to the error of parameter estimation. By contrast, recent model-free methods directly restore dehazed images by building an end-to-end network, which achieves better color fidelity. To improve the dehazing effect, we combine the complementary merits of these two categories and propose a physical-model guided self-distillation network for single image dehazing named PMGSDN. First, we propose a novel attention guided feature extraction block (AGFEB) and build a deep feature extraction network by it. Second, we propose three early-exit branches and embed the dark channel prior information to the network to merge the merits of model-based methods and model-free methods, and then we adopt self-distillation to transfer the features from the deeper layers (perform as teacher) to shallow early-exit branches (perform as student) to improve the dehazing effect. For I-HAZE and O-HAZE datasets, better than the other methods, the proposed method achieves the best values of PSNR and SSIM being 17.41dB, 0.813, 18.48dB, and 0.802. Moreover, for real-world images, the proposed method also obtains high quality dehazed results. Experimental results on both synthetic and real-world images demonstrate that the proposed PMGSDN can effectively dehaze images, resulting in dehazed results with clear textures and good color fidelity.

On the accuracy limits of misspecified delay-Doppler estimation

This work derives compact closed-form expressions of the misspecified Cramér–Rao bound and pseudo-true parameters of time-delay and Doppler for a high dynamics signal model. Those expressions are validated by analyzing the mean square error (MSE) of the misspecified maximum likelihood estimator. A noteworthy outcome of these MSE results is that, for some magnitudes of acceleration and signal-to-noise ratios, neglecting the acceleration is beneficial in the MSE sense. The variance performance improvement is obtained at the cost of a systematic error in the true parameter estimation. This can be seen as a specific case of the trade-off between bias and variance. Neglecting the acceleration can improve the Doppler estimation when the error induced on the misspecified model is less than the variance increase due to including an extra parameter to estimate. Then, for some non-zero acceleration magnitudes and short integration times, the Doppler estimation using a misspecified model outperforms a correctly specified model in the MSE sense.

A Reply to “Structural Parameters under Partial Least Squares and Covariance-based Structural Equation Modeling: A Comment on Yuan and Deng (2021)” by Schuberth, Rosseel, Rönkkö, Trichera, Kline, and Henseler (2023)

This is a follow-up reply to “Structural parameters under partial least squares and covariance-based structural equation modeling: A comment on Yuan and Deng (2021)” by Schuberth et al. (2023). In this reply, explanations are provided on why the messages in the comment article are mostly from a lack of a coherent understanding of the results in Yuan and Deng (2021, YD21). Implications of the results in YD21 are explicated to facilitate proper comparison between covariance-based structural equation modeling (SEM) and partial least-squares SEM. The meaning of bias and mean-squared errors for parameter estimates is also elaborated for models involving latent variables that have no predefined scales.

Robust adaptive parameter estimator design for a multi-sinusoidal signal with fixed-time stability and guaranteed prescribed performance boundary of estimation error

In online parameter estimation, fast and accurate estimation without dependence on the initial conditions with good transient properties is an essential issue. Hence, in this paper, a robust adaptive parameter estimation method is proposed that not only can estimate online the unknown parameters of a multi sinusoidal signal in a finite-time without dependence on the initial conditions but also can guarantee the transient and steady-state performance of the estimated parameters with a predefined boundary. At first, to estimate the frequency, offset, phase and amplitude, the multi-sinusoidal signal is represented in a linear regression form. Then, by defining an auxiliary regressor matrix and vector derived by the original regressor vector and output signal, the estimation error is computed indirectly. By employing this parameter estimation error and utilizing the fixed-time stability property of the terminal sliding mode method and Prescribed-Performance concept, robust adaptation laws are designed in such a way that the convergence time of the estimation error is finite so that its settling time is independent from the initial guess, and this time can be predefined. Moreover the estimation error boundary can be predefined. Furthermore, in this method, only the output measurement is needed and there is no need for any observer or predictor. Using the Lyapunov stability theorem it is proved that the estimation error converges to zero when the measurement is free of noise and for the noisy case the estimation error is uniformly ultimately bounded and converges to a compact set around the origin. Three illustrative examples are provided to demonstrate the advantages and effectiveness of the designed estimation laws compared to some existing methods. Simulation results show that the proposed method can estimate the unknown parameters in fixed-time with a pre-adjustable estimation error boundary, even in the presence of measurement noise and confirms the superiority of the proposed method compared to the two existing methods.

Exponential output consensus tracking for multi-agent systems with uncertain local output function via sampled communication

In this paper, the output consensus tracking problem is investigated for heterogeneous multi-agent systems (MASs) with asynchronous sampled communication, where only local output of the leader is required and the local output function contains an unknown parameter. A distributed adaptive observer is designed for recovering the leader’s state and unknown parameter, which has three features: (i) two predictors are contained in the observer to recover the output injection and the consensus error separately; (ii) by designing proper gain matrices, the state estimation is decomposed into the detectable part merely using the sampled local output of the leader and the undetectable part only using sampled information of the neighbors; (iii) the negative effect of the parameter estimation errors is offset by adding complementary terms with coupled auxiliary variables into the observer. Based on the observer, the control scheme is provided to realize the exponential output consensus tracking. It is shown that the maximal sampling period for the follower to receive local output from the leader is independent of the neighbors.

Composite adaptive attitude control for combined spacecraft with inertia uncertainties

This paper examines the attitude control problem of the combined spacecraft subject to inertia uncertainties. A composite adaptive finite-time control scheme with guaranteed parameter convergence is proposed to address this challenging problem, in the absence of persistent excitation. As a stepping stone, the attitude dynamics of the combined spacecraft is first established with explicit consideration of the center-of-mass variation and thruster reconfiguration. Then, a composite adaptive finite-time controller is developed based on a constructive time-varying sliding manifold, and in particular, the concurrent learning technique is used in conjunction with the dynamic regressor extension and mixing procedure to achieve parameter convergence under finite excitation that is strictly weaker than the persistent excitation. Lyapunov stability analysis shows that the derived adaptive controller can simultaneously guarantee finite-time convergence of the attitude tracking and parameter estimation errors; moreover, its robustness against inertia variations and external disturbances is also analyzed. Finally, a set of numerical simulations under ideal and practical scenarios are performed to validate the effectiveness and outperformance of the proposed method.

A novel optimized grey model and its application in forecasting CO[formula omitted] emissions

Carbon dioxide emissions are the main cause of global warming. At present, how to reduce carbon dioxide emissions while promoting energy savings and emission reduction is a hot research topic. Hence, China’s carbon dioxide emissions must be reasonably and accurately predicted because it is very important for the Chinese government to formulate energy and environmental policies. In this study, the classical optimization theory of the Fibonacci sequence and golden ratio were applied to the grey prediction model of an approximately inhomogeneous exponential series. Then, a new optimization model was established, and the properties of the optimization model were studied. The purpose is to reduce the parameter estimation errors of the model and improve the simulation and prediction accuracy of the model. Next, the novel model was applied to the simulation and prediction of CO2 emissions in China. The experimental results show that the effectiveness of the novel model was much better than that of the other models, which confirms the effectiveness of the new model. Based on this, China’s carbon dioxide emissions were predicted and analysed. The results show that China’s carbon dioxide emissions will still be on the rise over the next five years, and carbon dioxide emissions remain a serious problem.

Data-Driven Model-Free Adaptive Sliding Mode Control Based on FFDL for Electric Multiple Units

The electric multiple units (EMUs) have become a very convenient and powerful means of transportation in our daily life. Safe and punctual trajectory tracking control is the key to improve the performance of the EMUs system, but it is difficult to realize due to the influence of environmental uncertainty, coupling and nonlinearity. In this paper, a model-free adaptive sliding mode control (MFASMC) method is proposed for the EMUs. This method can solve the dependence of the model-based control method on the train model and eliminate the influence of external disturbances on the robust performance of the system. In this method, the running process of the EMUs is equivalent to a full format dynamic linearization (FFDL) data model, and a model-free adaptive controller (MFAC) is designed based on the data model. Then, to reduce the influence of measurement disturbance and improve the robustness of the system, a discrete sliding mode control (SMC) algorithm is introduced. Furthermore, to prevent the control input from being too large, the parameter estimation error is introduced as an additional correction term of the algorithm. In the end, the simulation experiment is carried out with CRH380A EMUs as the object. Compared with the traditional MFAC and the traditional SMC, the speed tracking effect of each power unit of the MFASMC algorithm is more effective, the change of control force is stable, the acceleration meets the requirements of driving, and has a strong inhibitory effect on external disturbances.

Track curvature detection based on bogie attitude trajectory information fusion

ABSTRACT A curvature detection method based on bogie attitude trajectory information fusion is proposed to improve the accuracy and real-time performance of track curvature detection. The running speed, yaw angular velocity, and roll angle of a bogie were measured to obtain horizontal running and roll attitude trajectories through curve fitting. Track superelevation and curvature were estimated from these trajectories, and the characteristics and mechanisms of the parameter estimation errors were analysed. Data fusion algorithms based on feature point discrimination and superelevation trend tracking were established; these were used to fuse the estimation results of superelevation and curvature from different principles to obtain curvature detection values with higher accuracies. The performance of this curvature detection method was verified through simulations comprehensively. The simulation results indicated that the proposed method effectively improved the accuracy of real-time curvature detection. The correction of the two data fusion algorithms significantly ameliorated the signal lag problem in the traditional low-pass filtering method, reducing the relative error rate to less than 50% of that in the traditional method (maximum reduction of 75%–80%). This study is expected to guide the application and development of railway vehicle active control technologies that use track curvature as a control parameter.

Quantifying how post-transcriptional noise and gene copy number variation bias transcriptional parameter inference from mRNA distributions.

Transcriptional rates are often estimated by fitting the distribution of mature mRNA numbers measured using smFISH (single molecule fluorescence in situ hybridization) with the distribution predicted by the telegraph model of gene expression, which defines two promoter states of activity and inactivity. However, fluctuations in mature mRNA numbers are strongly affected by processes downstream of transcription. In addition, the telegraph model assumes one gene copy but in experiments, cells may have two gene copies as cells replicate their genome during the cell cycle. While it is often presumed that post-transcriptional noise and gene copy number variation affect transcriptional parameter estimation, the size of the error introduced remains unclear. To address this issue, here we measure both mature and nascent mRNA distributions of GAL10 in yeast cells using smFISH and classify each cell according to its cell cycle phase. We infer transcriptional parameters from mature and nascent mRNA distributions, with and without accounting for cell cycle phase and compare the results to live-cell transcription measurements of the same gene. We find that: (i) correcting for cell cycle dynamics decreases the promoter switching rates and the initiation rate, and increases the fraction of time spent in the active state, as well as the burst size; (ii) additional correction for post-transcriptional noise leads to further increases in the burst size and to a large reduction in the errors in parameter estimation. Furthermore, we outline how to correctly adjust for measurement noise in smFISH due to uncertainty in transcription site localisation when introns cannot be labelled. Simulations with parameters estimated from nascent smFISH data, which is corrected for cell cycle phases and measurement noise, leads to autocorrelation functions that agree with those obtained from live-cell imaging.

Monitoring count time series: Robustness to nonlinearity when linear models are utilized

AbstractLinear models are typically utilized for time series analysis as these are often simple to implement and interpret, as well as being useful in modeling many practical phenomena. Hence, most of the literature on control charts for monitoring time series also consider linearity of the data generating processes (DGP). In practice, however, a nonlinear DGP can be modeled as if it is linear, either due to overlook or illusion, when it is approximately linear. This study quantifies the effects of nonlinear DGPs, misspecified and modeled as linear, on the performance of Shewhart‐type and cumulative sum (CUSUM) control charts for count time series data. Time series models for bounded and unbounded counts with several parametrizations are considered for studying the sensitivity of linear approximations to nonlinear DGPs. The Markov chain (MC) approach and simulations are used to compute the average run length (ARL) performance of the control charts. Robustness of the performance is evaluated with respect to the extent that the DGP violates linearity, with and without errors in parameter estimation. It is shown that the chart designs are, in general, robust to model misspecification when the parameters are specified. However, the CUSUM performance can be affected significantly if both the model is misspecified and the parameters are estimated. Real‐world data implementations are provided to illustrate the sensitivity of the control charts' performances to the type of model and to the estimation approach.

RISE-based adaptive tracking control for Euler–Lagrange mechanical systems with matched disturbances

The core idea of traditional adaptive control is to reconstruct parameter estimation errors with known signals and damping injection based on tracking error, while the formulation of desired damping in controller designs is usually a nontrivial task. The main contribution of this paper lies in the development of the classic RISE result and a constructive damping injection procedure for the adaptive tracking control of Euler–Lagrange mechanical systems. By utilizing generalized dynamic scaling function, scalar filtering, the improved RISE method and analyzing the existence of finite escape time of the closed-loop system, a globally asymptotically stable result is obtained with facilitative damping injection, significant order reduction and improved design efficiency when compared with the existing results. Simulations on a fully actuated 2-DOF planar robot manipulator model demonstrate the effectiveness of the proposed methods.

Finite-time parameter estimation for an online monitoring of transformer: A system identification perspective

The effective online monitoring of thermal performance is the principal criterion that determines the Loss-of-Life (LoL), ageing rate, and loading capability of the transformer. To assess transformer thermal performance and life expectancy, Top-oil Temperature (TOT) and Hot spot Temperature (HST) should be accurately estimated. In general, the first-principle approach with a thermal-electrical analogy is used for the first-order dynamical model based on the Resistance-Capacitance (RC) circuit structure to approximate the evolution of thermal performance. Traditionally, the TOT model parameters are identified by looking for the input-output data that minimize the error between estimated and actual values. The Gradient Estimator (GE) based on the least-square minimization principle ensures the parameter convergence to their actual value and the parametric estimation error to zero only when the regressor (information vector) signals fulfill the stringent Persistence of Excitation (PE) condition. As the choice of input-output data plays crucial role in parameter estimation, the Design of Experiment (DoE) is generally conducted in the laboratory to satisfy the PE condition. Therefore, the TOT model parameter estimation problem is reformulated from a system identification perspective by exploring the finite-time estimators (FTEs) that accurately identify and estimate the TOT model parameters for the streaming data from real-time working transformers without any laboratory experiments and DoE. The experimental analysis on the sufficiency of data for parameter convergence is carried out on MATLAB to demonstrate the identified PE problem, the effect of DoE, and the performance of finite time estimators for real-time scenario based non-PE data in thermal modeling of the transformer.

Parameter Estimation in Adaptive Control of Time-Varying Systems Under a Range of Excitation Conditions

This article presents a new parameter estimation algorithm for the adaptive control of a class of time-varying plants. The main feature of this algorithm is a matrix of time-varying learning rates, which enables parameter estimation error trajectories to tend exponentially fast toward a compact set whenever excitation conditions are satisfied. This algorithm is employed in a large class of problems where unknown parameters are present and are time-varying. It is shown that this algorithm guarantees global boundedness of the state and parameter errors of the system, and avoids an often used filtering approach for constructing key regressor signals. In addition, intervals of time over which these errors tend exponentially fast toward a compact set are provided, both in the presence of finite and persistent excitation. A projection operator is used to ensure the boundedness of the learning rate matrix, as compared to a time-varying forgetting factor. Numerical simulations are provided to complement the theoretical analysis.

Diffusion MRI with pulsed and free gradient waveforms: Effects of restricted diffusion and exchange.

Monitoring time dependence with diffusion MRI yields observables sensitive to compartment sizes (restricted diffusion) and membrane permeability (water exchange). However, restricted diffusion and exchange have opposite effects on the diffusion-weighted signal, which can lead to errors in parameter estimates. In this work, we propose a signal representation that incorporates the effects of both restricted diffusion and exchange up to second order in b-value and is compatible with gradient waveforms of arbitrary shape. The representation features mappings from a gradient waveform to two scalars that separately control the sensitivity to restriction and exchange. We demonstrate that these scalars span a two-dimensional space that can be used to choose waveforms that selectively probe restricted diffusion or exchange, eliminating the correlation between the two phenomena. We found that waveforms with specific but unconventional shapes provide an advantage over conventional pulsed and oscillating gradient acquisitions. We also show that parametrization of waveforms into a two-dimensional space can be used to understand protocols from other approaches that probe restricted diffusion and exchange. For example, we found that the variation of mixing time in filter-exchange imaging corresponds to variation of our exchange-weighting scalar at a fixed value of the restriction-weighting scalar. The proposed signal representation was evaluated using Monte Carlo simulations in identical parallel cylinders with hexagonal and random packing as well as parallel cylinders with gamma-distributed radii. Results showed that the approach is sensitive to sizes in the interval 4-12 and exchange rates in the simulated range of 0 to 20 , but also that there is a sensitivity to the extracellular geometry. The presented theory constitutes a simple and intuitive description of how restricted diffusion and exchange influence the signal as well as a guide to protocol design capable of separating the two effects.

Model-based kinematic and non-kinematic calibration of a 7R 6-DOF robot with non-spherical wrist

This paper presents a model-based kinematic and non-kinematic calibration method for a 7R 6-DOF robot with non-spherical wrist. By using the proposed method, kinematic errors, compliance errors from external load and self-gravity are particularly compensated. Firstly, kinematic error model is built based on modified DH (MDH) model, meanwhile calibration models for compliance and inertial parameters are derived by proper linearization of stiffness model. Secondly, to derive a suitable minimal model for practical application, model reduction and parameter identifiability analysis are conducted by considering parameter properties defined in controller. Finally, a step-by-step procedure for error parameter estimation is proposed with laser tracker as measuring device: (1) kinematic parameter errors are identified when the robot is not loaded. (2) compliance parameters are identified when the robot is loaded. (3) inertial parameters are calibrated together with kinematic errors, if deformations from self-gravity need to be compensated. Simulations and experiments on a 7R 6-DOF industrial painting robot are conducted to verify that the proposed calibration method can improve accuracy of the robot under no load and loaded conditions.

Adaptive finite-time parameter estimation for grid-connected converter with LCL filter

This study proposes an online finite-time adaptive parameter estimation method for grid-connected voltage source inverter (VSI) with LCL filter. In this framework, a time-continuous state-space system model is first developed, which is different from the nonparametric frequency-response-based method. Then, the parameter estimation error information is extracted by introducing a series of filter operations. Moreover, a sliding mode term is studied and cooperated with the obtained parameter estimation error to design the parameter estimation error-based finite-time adaptive parameter estimation algorithm. With the help of this novel adaptive law, the physical parameters of electrical components can be driven to their true values accurately and fast. Theoretical analysis is conducted in terms of the Lyapunov theory to demonstrate the convergence of the parameter estimation error. Finally, numerical simulation results are provided to illustrate the superiority of the proposed method over the conventional method.

A Numerical Algorithm for Self-Learning Model Predictive Control in Servo Systems

Model predictive control (MPC) is one of the most effective methods of dealing with constrained control problems. Nevertheless, the uncertainty of the control system poses many problems in its performance optimization. For high-precision servo systems, friction is typically the main factor in uncertainty affecting the accuracy of the system. Our work focuses on stochastic systems with unknown parameters and proposes a model predictive control strategy with machine learning characteristics that utilizes pre-estimated information to reduce uncertainty. Within this model, the parameters are obtained using the estimator. The uncertainty caused by the parameter estimation error in the system is parameterized, serving as a learning control component to reduce future uncertainty. Then, the estimated parameters and the current state of the system are used to predict the future p-step state. The control sequence is calculated under the MPC’s rolling optimization mechanism. After the system output is obtained, the new parameter value at the next moment is re-estimated. Finally, MPC is carried out to realize the dual rolling optimization mechanism. In general, the proposed strategy optimizes the control objective while reducing the system uncertainty of the future parameter and achieving better system performance. Simulation results demonstrate the effectiveness of the algorithm.

A comparison of multiple imputation strategies to deal with missing nonnormal data in structural equation modeling.

Missing data and nonnormality are two common factors that can affect analysis results from structural equation modeling (SEM). The current study aims to address a challenging situation in which the two factors coexist (i.e., missing nonnormal data). Using Monte Carlo simulation, we evaluated the performance of four multiple imputation (MI) strategies with respect to parameter and standard error estimation. These strategies include MI with normality-based model (MI-NORM), predictive mean matching (MI-PMM), classification and regression trees (MI-CART), and random forest (MI-RF). We also compared these MI strategies with robust full information maximum likelihood (RFIML), a popular (non-imputation) method to deal with missing nonnormal data in SEM. The results suggest that MI-NORM had similar performance to RFIML. MI-PMM outperformed the other methods when data were not missing on the heavy tail of a skewed distribution. Although MI-CART and MI-RF do not require any distribution assumption, they did not perform well compared with the others. Based on the results, practical guidance is provided.