Two-step Maximum Likelihood Estimation Research Articles

Approximate parameter estimation and mis-specification analysis of degradation model with asymmetric random effects

Asymmetric distributions such as the skew-normal (SN) distribution have been used for modeling the unit-to-unit variation of the degradation rate of products, and its superiority for model fitting and lifetime estimation has been demonstrated in literature. However, there still exist two issues that need to be further studied. That is, how to simply estimate the parameters of the degradation model with asymmetric random effects and whether the asymmetry or skewness affects the remaining useful life (RUL) prediction. Driven by these two issues, the parameter estimation and RUL prediction of the degradation model with general random effects are first introduced and a two-step maximum likelihood estimation (MLE) method for parameter estimation is proposed, which could greatly simplify the parameter estimation of the degradation model with asymmetric random effects. Then, the SN distribution is used as an example to analyze the second issue regarding the effect of model mis-specification on RUL prediction based on regression models and Wiener processes with different asymmetric random effects in theory. Finally, the parameter estimation and mis-specification analysis of degradation models with other asymmetric random effects are conducted through numerical examples and two case studies. The results demonstrate that the proposed two-step MLE method is satisfactory and the effect of asymmetry on RUL prediction is negligible. Therefore, there is no need to consider the asymmetry of random effects for RUL prediction.

Remaining useful life prediction of aero-engines based on random-coefficient regression model considering random failure threshold

Remaining useful life (RUL) prediction is one of the most crucial components in prognostics and health management (PHM) of aero-engines. This paper proposes an RUL prediction method of aero-engines considering the randomness of failure threshold. Firstly, a random-coefficient regression (RCR) model is used to model the degradation process of aeroengines. Then, the RUL distribution based on fixed failure threshold is derived. The prior parameters of the degradation model are calculated by a two-step maximum likelihood estimation (MLE) method and the random coefficient is updated in real time under the Bayesian framework. The failure threshold in this paper is defined by the actual degradation process of aeroengines. After that, a expectation maximization (EM) algorithm is proposed to estimate the underlying failure threshold of aeroengines. In addition, the conditional probability is used to satisfy the limitation of failure threshold. Then, based on above results, an analytical expression of RUL distribution of aero-engines based on the RCR model considering random failure threshold (RFT) is derived in a closed-form. Finally, a case study of turbofan engine is used to demonstrate the effectiveness and superiority of the RUL prediction method and the parameters estimation method of failure threshold proposed.

A degradation-shock dependent competing failure processes based method for remaining useful life prediction of drill bit considering time-shifting sudden failure threshold

• A dependent competing failure model is established for drill lifetime estimation. • The effect of wear degradation on the sudden failure threshold is considered. • Reliability functions and useful life expression of the drill are derived. • A drilling experiment is provided for validating the proposed method. Accurate remaining useful life (RUL) prediction of the drill bit is central to ensuring the processing quality of products and enhancing processing safety and reliability. However, the majority of current methods are confined to considering the wear degradation during the drilling and ignore the combined effect of wear degradation and random shock on RUL. Therefore, it is difficult to the sudden failure prediction of the drill bit. To address this limitation, a degradation-shock dependent competing failure processes (DCFPs) based method considering time-shifting sudden failure threshold is proposed in this paper for RUL prediction. First, a degradation-shock DCFPs model is established based on the processing characteristics of the drill bit. Further, the expressions of reliability function and useful life taking into account the time-shifting sudden failure threshold are derived. Then, based on the Bayesian method, a two-step maximum likelihood estimation (MLE) is designed for parameter estimation, and the crucial parameters are updated utilizing a local gaussian importance resampling particle filter. Finally, a carbon composite laminate drilling experiment is provided for demonstration. Experimental results demonstrate the superiority of the proposed method for drill bit RUL prediction, especially when the drill bit is chipped.

Exchange Rates and Macroeconomic Fundamentals: Evidence of Instabilities from Time-Varying Factor Loadings

We examine the relationship between exchange rates and macroeconomic fundamentals using a two-step maximum likelihood estimator through which we compute time-varying factor loadings. Factors are obtained as principal components, extracted from a large macro-dataset. Using 14 currencies over 1995–2018, we show that the loadings on the factors vary considerably over time with frequent sign changes. Allowing for time-varying loadings increases the percentage of explained variation in exchanges rates by an order of magnitude. Accounting for instabilities improves the predictive ability of the model globally and locally during crises, and yields better forecast of sign changes in exchange rates.

Statistical inference for ARMA time series with moving average trend

Maximum likelihood estimator (MLE) and Bayesian Information Criterion (BIC) order selection are examined for ARMA time series with slowly varying trend to validate the well-known detrending technique of moving average [Section 1.4, Brockwell, P.J., and Davis, R.A. (1991), Time Series: Theory and Methods, New York: Springer-Verlag]. In step one, a moving average equivalent to local linear regression is fitted to the raw data with a data-driven lag number, and subtracted from raw data to produce a sequence of residuals. The residuals are used in step two as substitutes of the latent ARMA series for MLE and BIC procedures. It is shown that with second order smooth trend and correctly chosen lag number, the two-step MLE is oracally efficient, i.e. it is asymptotically as efficient as the would-be MLE based on the unobserved ARMA series. At the same time, the two-step BIC consistently selects the orders as well. Simulation experiments corroborate the theoretical findings.

Inducing a desired value of correlation between two point-scale variables: a two-step procedure using copulas

Focusing on point-scale random variables, i.e. variables whose support consists of the first m positive integers, we discuss how to build a joint distribution with pre-specified marginal distributions and Pearson’s correlation rho . After recalling how the desired value rho is not free to vary between -1 and +1, but generally ranges a narrower interval, whose bounds depend on the two marginal distributions, we devise a procedure that first identifies a class of joint distributions, based on a parametric family of copulas, having the desired margins, and then adjusts the copula parameter in order to match the desired correlation. The proposed methodology addresses a need which often arises when assessing the performance and robustness of some new statistical technique, i.e. trying to build a huge number of replicates of a given dataset, which satisfy—on average—some of its features (for example, the empirical marginal distributions and the pairwise linear correlations). The proposal shows several advantages, such as—among others—allowing for dependence structures other than the Gaussian and being able to accommodate the copula parameter up to an assigned level of precision for rho with a very small computational cost. Based on this procedure, we also suggest a two-step estimation technique for copula-based bivariate discrete distributions, which can be used as an alternative to full and two-step maximum likelihood estimation. Numerical illustration and empirical evidence are provided through some examples and a Monte Carlo simulation study, involving the CUB distribution and three different copulas; an application to real data is also discussed.

Multi-objective optimization design of accelerated degradation test based on Wiener process

A multi-objective optimization method for the accelerated degradation test based on Wiener process is proposed in this article in order to solve the problem that a single objective optimization cannot solve the difficulty or even conflicting test configurations caused by different optimization objective functions. An accelerated degradation model is established based on Wiener process, and the unknown parameters are solved by a two-step maximum likelihood estimation method. Considering both the accuracies of life estimation and the model parameter estimation, a multi-objective optimization model is established with the optimization goals of the minimum asymptotic variance of P-quantile of lifetime and the maximum determinant of Fisher information matrix. The Pareto solutions are set by the multi-objective genetic algorithm, and the test configurations for multi-objectives are obtained. Under the step stress accelerated degradation test and the constant stress accelerated degradation test, the effectiveness of the proposed method is verified by an optimization example of LEDs accelerated degradation test.

State-of-Health Estimation for Lithium-Ion Batteries Based on Wiener Process With Modeling the Relaxation Effect

State-of-Health (SOH) is an intuitive reflection to the capacity during the degradation of lithium-ion batteries. Accurate SOH estimation can not only grasp the battery performance but also achieve a better balance between the safety and economic benefits for lithium-ion battery application system. Relaxation effect refers to the capacity regeneration phenomenon of lithium-ion batteries during long rest time. This paper mainly studies the impact of relaxation effect on the degradation law of lithium-ion batteries, and proposes a novel SOH estimation method based on the Wiener process. First, the life cycle of a lithium-ion battery is divided into three parts, i.e., the degradation process that eliminates the relaxation effect, the capacity regeneration process during the rest time, and the degradation process of the regenerated capacity. Next, the degradation model after eliminating the relaxation effect is established based on linear Wiener process, the capacity regenerated model is developed by the normal distribution, and the degradation model of regenerated capacity is established based on a nonlinear Wiener process. Then, for the degradation model based on nonlinear Wiener process, a two-step maximum likelihood estimation (MLE) method for prior parameters is proposed, and the random variables representing the personality features are updated online under the Bayesian framework. For the capacity regenerated model, a parameter estimation method based on MLE is proposed. In addition, a one-step and a multi-step SOH prediction method based on piecewise modeling are developed. Finally, the experiments are carried out based on the degradation data of lithium-ion batteries published by the NASA, and the results show that the method proposed in this paper can effectively improve the accuracy of the SOH estimation.

Consistent estimation of time-varying loadings in high-dimensional factor models

In this paper, we develop a two-step maximum likelihood estimator of time-varying loadings in high-dimensional factor models. We specify the loadings to evolve as stationary vector autoregressions (VAR) and show that consistent estimates of the loadings parameters can be obtained. In the first step, principal components are extracted from the data to form factor estimates. In the second step, the parameters of the loadings VARs are estimated as a set of linear regression models with time-varying coefficients. We document the finite-sample properties of the maximum likelihood estimator through an extensive simulation study and illustrate the empirical relevance of the time-varying loadings structure using a large quarterly dataset for the US economy.

Calibration Estimation of Semiparametric Copula Models with Data Missing at Random

This paper investigates the estimation of semiparametric copula models with data missing at random. The two-step maximum likelihood estimation of Genest, Ghoudi, and Rivest (1995) is infeasible if there are missing data. We propose a class of calibration estimators for the nonparametric marginal distributions and the copula parameters of interest by balancing the empirical moments of covariates between observed and whole groups. Our proposed estimators do not require the estimation of missing mechanism, and enjoy stable performance even when sample size is small. We prove that our estimators satisfy consistency and asymptotic normality. We also provide a consistent estimator for the asymptotic variance. We show via extensive simulations that our proposed method dominates existing alternatives.

Application and Simulation Study of Stress-Dependent Weibull Lifetime Models

Different Weibull lifetime models are presented whose scale, shape and minimum lifetime parameters are stress-dependent. This allows describing and predicting the lifetime of products with a Weibull distribution more accurately wherever stress-dependence applies to the failure mechanism. For instance, this is the case for failures due to fatigue, on which this paper focusses. The proposed procedure encompasses a two-step maximum likelihood estimation and a Fisher matrix (FM) confidence bounds calculation, followed by a model evaluation. This model evaluation is conducted by means of a general plausibility check (PC), likelihood ratio test (LRT) and Bayesian information criterion (BIC). Their applicability to accelerated life test data is discussed and validated using test data. Finally, a simulation study confirms a wide range of applicability.

Semiparametric Inference of the Complier Average Causal Effect with Nonignorable Missing Outcomes

Noncompliance and missing data often occur in randomized trials, which complicate the inference of causal effects. When both noncompliance and missing data are present, previous papers proposed moment and maximum likelihood estimators for binary and normally distributed continuous outcomes under the latent ignorable missing data mechanism. However, the latent ignorable missing data mechanism may be violated in practice, because the missing data mechanism may depend directly on the missing outcome itself. Under noncompliance and an outcome-dependent nonignorable missing data mechanism, previous studies showed the identifiability of complier average causal effect for discrete outcomes. In this article, we study the semiparametric identifiability and estimation of complier average causal effect in randomized clinical trials with both all-or-none noncompliance and outcome-dependent nonignorable missing continuous outcomes, and propose a two-step maximum likelihood estimator in order to eliminate the infinite dimensional nuisance parameter. Our method does not need to specify a parametric form for the missing data mechanism. We also evaluate the finite sample property of our method via extensive simulation studies and sensitivity analysis, with an application to a double-blinded psychiatric clinical trial.

A Realized Stochastic Volatility Model With Box–Cox Transformation

This article presents a new class of realized stochastic volatility model based on realized volatilities and returns jointly. We generalize the traditionally used logarithm transformation of realized volatility to the Box–Cox transformation, a more flexible parametric family of transformations. A two-step maximum likelihood estimation procedure is introduced to estimate this model on the basis of Koopman and Scharth (2013). Simulation results show that the two-step estimator performs well, and the misspecified log transformation may lead to inaccurate parameter estimation and certain excessive skewness and kurtosis. Finally, an empirical investigation on realized volatility measures and daily returns is carried out for several stock indices.

Accelerated Degradation Tests Modeling Based on the Nonlinear Wiener Process with Random Effects

Accelerated degradation tests (ADT) modeling is an important issue in lifetime assessment to the products with high reliability and long lifetime. Among the literature about the accelerated nonlinear degradation process modeling, the current methods did not consider the product-to-product variation of the products with the same type. Therefore, this paper proposes an accelerated degradation process modeling method with random effects for the nonlinear Wiener process. Firstly, we derive the lifetime distribution of the nonlinear Wiener process with random effects. Secondly, the nonlinear Wiener process is used to model the degradation process of a single stress, and the drift coefficient is considered as a random variable to describe the product-to-product variation. Using the random acceleration model, the random effects are incorporated into the constant stress ADT models and the step stress ADT models. Then, a two-step maximum likelihood estimation (MLE) method is presented to estimate the unknown parameters in the degradation models. Finally, a simulation study and a case study are provided to demonstrate the application and superiority of the proposed model.

Estimating the cost of deposit insurance with stochastic interest rates: the case of Taiwan

This study measures the deposit insurance premium under stochastic interest rates for Taiwan's banks by applying the two-step maximum likelihood estimation method. The estimation results suggest that the current premiums—charging 5, 5.5, and 6 basis points per dollar of insured deposits—are too low, but largely reflect the rank orders of the risks of the insured banks. Moreover, the regression results indicate that asset volatility dominates bank size in determining the insurance premium. When the volatility risk is decomposed into two parts, credit risk significantly dominates interest-rate risk. An examination of bank characteristics indicates that privately owned old banks are more likely to have lower levels of credit risk, asset volatility, and deposit insurance premiums than state-owned banks and newly chartered banks.

Markov-switching models with endogenous explanatory variables II: A two-step MLE procedure

This paper proposes a two-step maximum likelihood estimation (MLE) procedure to deal with the problem of endogeneity in Markov-switching regression models. A joint estimation procedure provides us with an asymptotically most efficient estimator, but it is not always feasible, due to the ‘curse of dimensionality’ in the matrix of transition probabilities. A two-step estimation procedure, which ignores potential correlation between the latent state variables, suffers less from the ‘curse of dimensionality’, and it provides a reasonable alternative to the joint estimation procedure. In addition, our Monte Carlo experiments show that the two-step estimation procedure can be more efficient than the joint estimation procedure in finite samples, when there is zero or low correlation between the latent state variables.

An alternative approach to estimation of structural vector error correction models with long-run restrictions

This paper presents an alternative approach to two-step maximum likelihood estimation of structural vector error correction models that are partially identified with long-run restrictions. We generalize the existing literature by allowing for over-identifying restrictions.

Time-Varying-Parameter Models with Endogenous Regressors: A Two-Step MLE Approach and an Augmented Kalman Filter

In this paper, we provide a unified framework for LIML (limited information maximum likelihood) IV (instrumental variables) estimation to deal with endogeneity problems in the time-varying parameter models. For this purpose, we derive a Heckman-type (1976) two-step maximum likelihood estimation (MLE) procedure. The proposed two-step procedure, based on the conventional Kalman filter, provides consistent estimates of the hyper-parameters, as well as correct inferences on the time-varying coefficients. However, the use of the conventional Kalman filter in the second step would result in an invalid conditional covariance matrix for the time-varying coefficients. The correction for the conditional covariance matrix can be made by employing an augmented Kalman filter proposed in this paper. The basic model and the two-step procedure is also extended to handle the issue of heteroscedasticity in the disturbance terms. This is done by considering a time-varying parameter model for Campbell and Mankiw's (1989) consumption function.

Two-step estimation of multivariate polychoric correlation

In this paper, a two-step procedure for estimating multivariate poiychoric currelacions is investigated. Both the generalized least squares and maximum likelihood methods are considered. Asymptotic properties of the two-step generalized least squares estimator are established, and its relations with the two-step maximum likelihood estimator are studied. Iterative procedures are developed to produce the estimates and the standard error estimates. An example is provided to illustrate the theory.