Two-step Maximum Likelihood Research Articles

A remaining useful life prediction method of aluminum electrolytic capacitor with adaptive degradation model selection

The degradation process of aluminum electrolytic capacitors(AECs) usually exhibits characteristics such as non-linearity and multi-stage. These degradation features lead to the difficulty to accurately predict the remaining useful life(RUL) of the whole degradation process of AECs. To address this, by dividing the capacitor degradation process into two stages, a two-stage RUL prediction method for AECs considering multiple degradation models is proposed in this paper.In the offline parameter estimation phase, the initial degradation model parameters of two stages are estimated using two-step maximum likelihood estimation combined with particle swarm optimization(PSO) algorithm. In the dynamic parameter updating phase, a sequential Bayesian method is used to update the model parameters. To select the optimal degradation model, an evaluation method based on historical RUL similarity is proposed to calculate the fitness of each model. Finally, the effectiveness of the method is verified on NASA’s accelerated degradation data set and several widely used methods are used for comparison. The experimental results show that the proposed method has higher accuracy, which proves the superiority of the method.

Approximate parameter estimation and mis-specification analysis of degradation model with asymmetric random effects

Asymmetric distributions such as the skew-normal (SN) distribution have been used for modeling the unit-to-unit variation of the degradation rate of products, and its superiority for model fitting and lifetime estimation has been demonstrated in literature. However, there still exist two issues that need to be further studied. That is, how to simply estimate the parameters of the degradation model with asymmetric random effects and whether the asymmetry or skewness affects the remaining useful life (RUL) prediction. Driven by these two issues, the parameter estimation and RUL prediction of the degradation model with general random effects are first introduced and a two-step maximum likelihood estimation (MLE) method for parameter estimation is proposed, which could greatly simplify the parameter estimation of the degradation model with asymmetric random effects. Then, the SN distribution is used as an example to analyze the second issue regarding the effect of model mis-specification on RUL prediction based on regression models and Wiener processes with different asymmetric random effects in theory. Finally, the parameter estimation and mis-specification analysis of degradation models with other asymmetric random effects are conducted through numerical examples and two case studies. The results demonstrate that the proposed two-step MLE method is satisfactory and the effect of asymmetry on RUL prediction is negligible. Therefore, there is no need to consider the asymmetry of random effects for RUL prediction.

Uncertainty in systemic risks rankings: Bayesian and frequentist analysis

We propose efficient Bayesian Hamiltonian Monte Carlo method for estimation of systemic risk measures, LRMES, SRISK and ΔCoVaR, and apply it for thirty global systemically important banks and for eighteen largest US financial institutions over the period of 2000–2020. The advantage of the Hamiltonian method is an efficient estimation of all parameters jointly in high dimensional models and providing posterior distributions incorporating parameter uncertainty. The systemic risk measures are computed based on the Dynamic Conditional Correlations model with generalized asymmetric volatility. We estimate the systemic risks posterior distributions and two-step maximum likelihood distributions with bootstrap simulations for LRMES. The systemic risk rankings at different quantiles of the distributions vary considerably using bootstrap approach for computation of LRMES and SRISK, and are more stable with Bayesian posterior distributions using a parametric model. A policymaker may choose to rank the firms using some quantile of their systemic risk distributions such as 90, 95, or 99% depending on risk preferences with higher quantiles being more conservative.

Remaining useful life prediction of aero-engines based on random-coefficient regression model considering random failure threshold

Remaining useful life (RUL) prediction is one of the most crucial components in prognostics and health management (PHM) of aero-engines. This paper proposes an RUL prediction method of aero-engines considering the randomness of failure threshold. Firstly, a random-coefficient regression (RCR) model is used to model the degradation process of aeroengines. Then, the RUL distribution based on fixed failure threshold is derived. The prior parameters of the degradation model are calculated by a two-step maximum likelihood estimation (MLE) method and the random coefficient is updated in real time under the Bayesian framework. The failure threshold in this paper is defined by the actual degradation process of aeroengines. After that, a expectation maximization (EM) algorithm is proposed to estimate the underlying failure threshold of aeroengines. In addition, the conditional probability is used to satisfy the limitation of failure threshold. Then, based on above results, an analytical expression of RUL distribution of aero-engines based on the RCR model considering random failure threshold (RFT) is derived in a closed-form. Finally, a case study of turbofan engine is used to demonstrate the effectiveness and superiority of the RUL prediction method and the parameters estimation method of failure threshold proposed.

A degradation-shock dependent competing failure processes based method for remaining useful life prediction of drill bit considering time-shifting sudden failure threshold

• A dependent competing failure model is established for drill lifetime estimation. • The effect of wear degradation on the sudden failure threshold is considered. • Reliability functions and useful life expression of the drill are derived. • A drilling experiment is provided for validating the proposed method. Accurate remaining useful life (RUL) prediction of the drill bit is central to ensuring the processing quality of products and enhancing processing safety and reliability. However, the majority of current methods are confined to considering the wear degradation during the drilling and ignore the combined effect of wear degradation and random shock on RUL. Therefore, it is difficult to the sudden failure prediction of the drill bit. To address this limitation, a degradation-shock dependent competing failure processes (DCFPs) based method considering time-shifting sudden failure threshold is proposed in this paper for RUL prediction. First, a degradation-shock DCFPs model is established based on the processing characteristics of the drill bit. Further, the expressions of reliability function and useful life taking into account the time-shifting sudden failure threshold are derived. Then, based on the Bayesian method, a two-step maximum likelihood estimation (MLE) is designed for parameter estimation, and the crucial parameters are updated utilizing a local gaussian importance resampling particle filter. Finally, a carbon composite laminate drilling experiment is provided for demonstration. Experimental results demonstrate the superiority of the proposed method for drill bit RUL prediction, especially when the drill bit is chipped.

Exchange Rates and Macroeconomic Fundamentals: Evidence of Instabilities from Time-Varying Factor Loadings

We examine the relationship between exchange rates and macroeconomic fundamentals using a two-step maximum likelihood estimator through which we compute time-varying factor loadings. Factors are obtained as principal components, extracted from a large macro-dataset. Using 14 currencies over 1995–2018, we show that the loadings on the factors vary considerably over time with frequent sign changes. Allowing for time-varying loadings increases the percentage of explained variation in exchanges rates by an order of magnitude. Accounting for instabilities improves the predictive ability of the model globally and locally during crises, and yields better forecast of sign changes in exchange rates.

A new INAR model based on Poisson-BE2 innovations

In this paper, two-parameter Poisson binomial-exponential 2 (PBE2) distribution is firstly reviewed, then a new integer-valued autoregressive (INAR) model with PBE2 innovations is proposed. The definition and statistical properties of the proposed model are given, including the mean, variance, covariance, strict stationarity and ergodicity. Two-step conditional least squares and conditional maximum likelihood estimation methods are considered to estimate the parameters. To assess the proposed model, a crime data set is analyzed and a comparison with other competing INAR models is given, which shows that the proposed model yields a better performance.

Inducing a desired value of correlation between two point-scale variables: a two-step procedure using copulas

Focusing on point-scale random variables, i.e. variables whose support consists of the first m positive integers, we discuss how to build a joint distribution with pre-specified marginal distributions and Pearson’s correlation rho . After recalling how the desired value rho is not free to vary between -1 and +1, but generally ranges a narrower interval, whose bounds depend on the two marginal distributions, we devise a procedure that first identifies a class of joint distributions, based on a parametric family of copulas, having the desired margins, and then adjusts the copula parameter in order to match the desired correlation. The proposed methodology addresses a need which often arises when assessing the performance and robustness of some new statistical technique, i.e. trying to build a huge number of replicates of a given dataset, which satisfy—on average—some of its features (for example, the empirical marginal distributions and the pairwise linear correlations). The proposal shows several advantages, such as—among others—allowing for dependence structures other than the Gaussian and being able to accommodate the copula parameter up to an assigned level of precision for rho with a very small computational cost. Based on this procedure, we also suggest a two-step estimation technique for copula-based bivariate discrete distributions, which can be used as an alternative to full and two-step maximum likelihood estimation. Numerical illustration and empirical evidence are provided through some examples and a Monte Carlo simulation study, involving the CUB distribution and three different copulas; an application to real data is also discussed.

A DCC-type approach for realized covariance modeling with score-driven dynamics

We propose a class of score-driven realized covariance models where volatilities and correlations are separately estimated. We can thus combine univariate realized volatility models with a recently introduced class of score-driven realized covariance models based on Wishart and matrix-F distributions. Compared to the latter, the proposed models remain computationally simple at high dimensions and allow for higher flexibility in parameter estimation. Through a Monte-Carlo study, we show that the two-step maximum likelihood procedure provides accurate parameter estimates in small samples. Empirically, we find that the proposed models outperform those based on joint estimation, with forecasting gains that become more significant as the cross-section dimension increases.

Multi-objective optimization design of accelerated degradation test based on Wiener process

A multi-objective optimization method for the accelerated degradation test based on Wiener process is proposed in this article in order to solve the problem that a single objective optimization cannot solve the difficulty or even conflicting test configurations caused by different optimization objective functions. An accelerated degradation model is established based on Wiener process, and the unknown parameters are solved by a two-step maximum likelihood estimation method. Considering both the accuracies of life estimation and the model parameter estimation, a multi-objective optimization model is established with the optimization goals of the minimum asymptotic variance of P-quantile of lifetime and the maximum determinant of Fisher information matrix. The Pareto solutions are set by the multi-objective genetic algorithm, and the test configurations for multi-objectives are obtained. Under the step stress accelerated degradation test and the constant stress accelerated degradation test, the effectiveness of the proposed method is verified by an optimization example of LEDs accelerated degradation test.

A DCC-Type Approach for Realized Covariance Modelling With Score-Driven Dynamics

We propose a class of score-driven realized covariance models where volatilities and correlations are separately estimated. We can thus combine univariate realized volatility models with a recently introduced class of score-driven realized covariance models based on Wishart and matrix-F distributions. The proposed models are computationally simple to estimate in high dimensions and allow complete flexibility in the choice of the univariate specification. Through a Monte-Carlo study, we show that the two-step maximum likelihood procedure provides accurate parameter estimates in small samples. Empirically, we find that the proposed models outperform joint estimations, with forecasting gains that become more significant as dimension increases.

State-of-Health Estimation for Lithium-Ion Batteries Based on Wiener Process With Modeling the Relaxation Effect

State-of-Health (SOH) is an intuitive reflection to the capacity during the degradation of lithium-ion batteries. Accurate SOH estimation can not only grasp the battery performance but also achieve a better balance between the safety and economic benefits for lithium-ion battery application system. Relaxation effect refers to the capacity regeneration phenomenon of lithium-ion batteries during long rest time. This paper mainly studies the impact of relaxation effect on the degradation law of lithium-ion batteries, and proposes a novel SOH estimation method based on the Wiener process. First, the life cycle of a lithium-ion battery is divided into three parts, i.e., the degradation process that eliminates the relaxation effect, the capacity regeneration process during the rest time, and the degradation process of the regenerated capacity. Next, the degradation model after eliminating the relaxation effect is established based on linear Wiener process, the capacity regenerated model is developed by the normal distribution, and the degradation model of regenerated capacity is established based on a nonlinear Wiener process. Then, for the degradation model based on nonlinear Wiener process, a two-step maximum likelihood estimation (MLE) method for prior parameters is proposed, and the random variables representing the personality features are updated online under the Bayesian framework. For the capacity regenerated model, a parameter estimation method based on MLE is proposed. In addition, a one-step and a multi-step SOH prediction method based on piecewise modeling are developed. Finally, the experiments are carried out based on the degradation data of lithium-ion batteries published by the NASA, and the results show that the method proposed in this paper can effectively improve the accuracy of the SOH estimation.

A latent class analysis towards stability and changes in breadwinning patterns among coupled households

Abstract A latent class model is proposed to examine couples’ breadwinning typologies and explain the wage differentials according to the socio-demographic characteristics of the society with data collected through surveys. We derive an ordinal variable indicating the couple’s income provision-role type and suppose the existence of an underlying discrete latent variable to model the effect of covariates. We use a two-step maximum likelihood inference conducted to account for concomitant variables, informative sampling scheme and missing responses. The weighted log-likelihood is maximised through the Expectation-Maximization algorithm and information criteria are used to develop the model selection. Predictions are made on the basis of the maximum posterior probabilities. Disposing of data collected in Japan over thirty years we compare couples’ breadwinning patterns across time. We provide some evidence of the gender wage-gap and we show that it can be attributed to the fact that, especially in Japan, duties and responsibilities for the child care are supported exclusively by women.

Consistent estimation of time-varying loadings in high-dimensional factor models

In this paper, we develop a two-step maximum likelihood estimator of time-varying loadings in high-dimensional factor models. We specify the loadings to evolve as stationary vector autoregressions (VAR) and show that consistent estimates of the loadings parameters can be obtained. In the first step, principal components are extracted from the data to form factor estimates. In the second step, the parameters of the loadings VARs are estimated as a set of linear regression models with time-varying coefficients. We document the finite-sample properties of the maximum likelihood estimator through an extensive simulation study and illustrate the empirical relevance of the time-varying loadings structure using a large quarterly dataset for the US economy.

A bivariate count model with discrete Weibull margins

Multivariate discrete data arise in many fields (statistical quality control, epidemiology, failure and reliability analysis, etc.) and modelling such data is a relevant task. Here we consider the construction of a bivariate model with discrete Weibull margins, based on Farlie–Gumbel–Morgenstern copula, analyse its properties especially in terms of attainable correlation, and propose several methods for the point estimation of its parameters. Two of them are the standard one-step and two-step maximum likelihood procedures; the other two are based on an approximate method of moments and on the method of proportion, which represent intuitive alternatives for estimating the dependence parameter. A Monte Carlo simulation study is presented, comprising more than one hundred artificial settings, which empirically assesses the performance of the different estimation techniques in terms of statistical properties and computational cost. For illustrative purposes, the model and related inferential procedures are fitted and applied to two datasets taken from the literature, concerning failure data, presenting either positive or negative correlation between the two observed variables. The applications show that the proposed bivariate discrete Weibull distribution can model correlated counts even better than existing and well-established joint distributions.

Calibration Estimation of Semiparametric Copula Models with Data Missing at Random

This paper investigates the estimation of semiparametric copula models with data missing at random. The two-step maximum likelihood estimation of Genest, Ghoudi, and Rivest (1995) is infeasible if there are missing data. We propose a class of calibration estimators for the nonparametric marginal distributions and the copula parameters of interest by balancing the empirical moments of covariates between observed and whole groups. Our proposed estimators do not require the estimation of missing mechanism, and enjoy stable performance even when sample size is small. We prove that our estimators satisfy consistency and asymptotic normality. We also provide a consistent estimator for the asymptotic variance. We show via extensive simulations that our proposed method dominates existing alternatives.

Gaussian processes with built-in dimensionality reduction: Applications to high-dimensional uncertainty propagation

Uncertainty quantification (UQ) tasks, such as model calibration, uncertainty propagation, and optimization under uncertainty, typically require several thousand evaluations of the underlying computer codes. To cope with the cost of simulations, one replaces the real response surface with a cheap surrogate based, e.g., on polynomial chaos expansions, neural networks, support vector machines, or Gaussian processes (GP). However, the number of simulations required to learn a generic multivariate response grows exponentially as the input dimension increases. This curse of dimensionality can only be addressed, if the response exhibits some special structure that can be discovered and exploited. A wide range of physical responses exhibit a special structure known as an active subspace (AS). An AS is a linear manifold of the stochastic space characterized by maximal response variation. The idea is that one should first identify this low dimensional manifold, project the high-dimensional input onto it, and then link the projection to the output. If the dimensionality of the AS is low enough, then learning the link function is a much easier problem than the original problem of learning a high-dimensional function. The classic approach to discovering the AS requires gradient information, a fact that severely limits its applicability. Furthermore, and partly because of its reliance to gradients, it is not able to handle noisy observations. The latter is an essential trait if one wants to be able to propagate uncertainty through stochastic simulators, e.g., through molecular dynamics codes. In this work, we develop a probabilistic version of AS which is gradient-free and robust to observational noise. Our approach relies on a novel Gaussian process regression with built-in dimensionality reduction. In particular, the AS is represented as an orthogonal projection matrix that serves as yet another covariance function hyper-parameter to be estimated from the data. To train the model, we design a two-step maximum likelihood optimization procedure that ensures the orthogonality of the projection matrix by exploiting recent results on the Stiefel manifold, i.e., the manifold of matrices with orthogonal columns. The additional benefit of our probabilistic formulation, is that it allows us to select the dimensionality of the AS via the Bayesian information criterion. We validate our approach by showing that it can discover the right AS in synthetic examples without gradient information using both noiseless and noisy observations. We demonstrate that our method is able to discover the same AS as the classical approach in a challenging one-hundred-dimensional problem involving an elliptic stochastic partial differential equation with random conductivity. Finally, we use our approach to study the effect of geometric and material uncertainties in the propagation of solitary waves in a one dimensional granular system.

Application and Simulation Study of Stress-Dependent Weibull Lifetime Models

Different Weibull lifetime models are presented whose scale, shape and minimum lifetime parameters are stress-dependent. This allows describing and predicting the lifetime of products with a Weibull distribution more accurately wherever stress-dependence applies to the failure mechanism. For instance, this is the case for failures due to fatigue, on which this paper focusses. The proposed procedure encompasses a two-step maximum likelihood estimation and a Fisher matrix (FM) confidence bounds calculation, followed by a model evaluation. This model evaluation is conducted by means of a general plausibility check (PC), likelihood ratio test (LRT) and Bayesian information criterion (BIC). Their applicability to accelerated life test data is discussed and validated using test data. Finally, a simulation study confirms a wide range of applicability.

Semiparametric Inference of the Complier Average Causal Effect with Nonignorable Missing Outcomes

Noncompliance and missing data often occur in randomized trials, which complicate the inference of causal effects. When both noncompliance and missing data are present, previous papers proposed moment and maximum likelihood estimators for binary and normally distributed continuous outcomes under the latent ignorable missing data mechanism. However, the latent ignorable missing data mechanism may be violated in practice, because the missing data mechanism may depend directly on the missing outcome itself. Under noncompliance and an outcome-dependent nonignorable missing data mechanism, previous studies showed the identifiability of complier average causal effect for discrete outcomes. In this article, we study the semiparametric identifiability and estimation of complier average causal effect in randomized clinical trials with both all-or-none noncompliance and outcome-dependent nonignorable missing continuous outcomes, and propose a two-step maximum likelihood estimator in order to eliminate the infinite dimensional nuisance parameter. Our method does not need to specify a parametric form for the missing data mechanism. We also evaluate the finite sample property of our method via extensive simulation studies and sensitivity analysis, with an application to a double-blinded psychiatric clinical trial.

A Realized Stochastic Volatility Model With Box–Cox Transformation

This article presents a new class of realized stochastic volatility model based on realized volatilities and returns jointly. We generalize the traditionally used logarithm transformation of realized volatility to the Box–Cox transformation, a more flexible parametric family of transformations. A two-step maximum likelihood estimation procedure is introduced to estimate this model on the basis of Koopman and Scharth (2013). Simulation results show that the two-step estimator performs well, and the misspecified log transformation may lead to inaccurate parameter estimation and certain excessive skewness and kurtosis. Finally, an empirical investigation on realized volatility measures and daily returns is carried out for several stock indices.