Classical Maximum Likelihood Estimation Research Articles

Bearing Fault Diagnosis Under Small Data Set Condition: A Bayesian Network Method With Transfer Learning for Parameter Estimation

Bearings are broadly applied in various types of industrial systems. Fault diagnosis, as a promising way for reliability of modern industrial internet of thing applications, has attracted increasing attention from both academia and industry fields. Being ideal modeling and inference tool in uncertainty situations, Bayesian network (BN) is becoming increasingly popular in many systems. However, in practical uncertain and complicated engineering surroundings, it’s difficult or expensive to collect massive labeled fault data for the sake of fault diagnosis model learning. To address the issue of BN parameter learning under small data set conditions, this paper proposes a Varying Coefficient Transfer Learning (VCTL) algorithm based on aggregation and transfer learning, that considers both knowledge from the resource domain and the resource relevance contributions. The balancing weight function is designed to determine whether the learning task in the resource domain is activated. Relevance weight factors are proposed to measure the relevance of resource and target parameters quantitatively, by combing parameter information from resource domains with those obtained from the target domain, using maximum a posterior (MAP) or maximum likelihood estimation (MLE). Finally, the target parameters are aggregated with both the target initial parameters and the parameter knowledge from the resource domain. Based on VCTL, a bearing fault diagnosis approach is proposed and verified. The experimental results show that, under the condition of the small data set, learning accuracy of VCTL algorithm with varying coefficient aggregation is better than MLE algorithm, MAP algorithm or state-of-the-art parameter transfer method, local linear pooling transfer learning (LoLP) algorithm. Under the condition of sufficient data set, learning accuracy of VCTL algorithm approaches the classical MLE or MAP, and the correctness of the proposed algorithm is verified. Moreover, we illustrate the successful application to real-world bearing fault diagnosis case with VCTL, where we had access to expert-provided resource knowledge and real fault diagnosis data.

Estimation of the Tapered Gutenberg-Richter Distribution Parameters for Catalogs with Variable Completeness: An Application to the Atlantic Ridge Seismicity

The use of the tapered Gutenberg-Richter distribution in earthquake source models is rapidly increasing, allowing overcoming the definition of a hard threshold for the maximum magnitude. Here, we expand the classical maximum likelihood estimation method for estimating the parameters of the tapered Gutenberg-Richter distribution, allowing the use of a variable through-time magnitude of completeness. Adopting a well-established technique based on asymptotic theory, we also estimate the uncertainties relative to the parameters. Differently from other estimation methods for catalogs with a variable completeness, available for example for the classical truncated Gutenberg-Richter distribution, our approach does not need the assumption on the distribution of the number of events (usually the Poisson distribution). We test the methodology checking the consistency of parameter estimations with synthetic catalogs generated with multiple completeness levels. Then, we analyze the Atlantic ridge seismicity, using the global centroid moment tensor catalog, finding that our method allows better constraining distribution parameters, allowing the use more data than estimations based on a single completeness level. This leads to a sharp decrease in the uncertainties associated with the parameter estimation, when compared with existing methods based on a single time-independent magnitude of completeness. This also allows analyzing subsets of events, to deepen data analysis. For example, separating normal and strike-slip events, we found that they have significantly different but well-constrained corner magnitudes. Instead, without distinguishing for focal mechanism and considering all the events in the catalog, we obtain an intermediate value that is relatively less constrained from data, with an open confidence region.

Handling Censoring and Censored Data in Survival Analysis: A Standalone Systematic Literature Review

The study recognized the worth of understanding the how’s of handling censoring and censored data in survival analysis and the potential biases it might cause if researchers fail to identify and handle the concepts with utmost care. We systematically reviewed the concepts of censoring and how researchers have handled censored data and brought all the ideas under one umbrella. The review was done on articles written in the English language spanning from the late fifties to the present time. We googled through NCBI, PubMed, Google scholar and other websites and identified theories and publications on the research topic. Revelation was that censoring has the potential of biasing results and reducing the statistical power of analyses if not handled with the appropriate techniques it requires. We also found that, besides the four main approaches (complete-data analysis method; imputation approach; dichotomizing the data; the likelihood-based approach) to handling censored data, there were several other innovative approaches to handling censored data. These methods include censored network estimation; conditional mean imputation method; inverse probability of censoring weighting; maximum likelihood estimation; Buckley-Janes least squares algorithm; simple multiple imputation strategy; filter algorithm; Bayesian framework;β-substitution method; search-and-score-hill-climbing algorithm and constraint-based conditional independence algorithm; frequentist; Markov chain Monte Carlo for imputed data; quantile regression; random effects hierarchical Cox proportional hazards; Lin’s Concordance Correlation Coefficient; classical maximum likelihood estimate. We infer that the presence of incomplete information about subjects does not necessarily mean that such information must be discarded, rather they must be incorporated into the study for they might carry certain relevant information that holds the key to the understanding of the research. We anticipate that through this review, researchers will develop a deeper understanding of this concept in survival analysis and select the appropriate statistical procedures for such studies devoid of biases.

On the method of moments approach applied to a (generalized) gamma population

In this article we propose for a generalized gamma population method of moment estimators for the three unknown parameters. This method of moment approach is based on estimating the unknown parameters of the log generalized gamma distribution. Contrary to the more classical method of moment approaches discussed in the literature the associated system of nonlinear equations satisfies the following properties. The system of nonlinear equations associated with these moment estimators do not have a solution if and only if the sample moment estimation of the skewness of a log generalized gamma distribution is outside a given interval. This might happen with a positive probability for small sample sizes and this probability goes to zero as the sample size increases. However, if the system of nonlinear equations has a solution the method of moment estimators are a unique solution of this system and this solution is easy to determine by applying a univariate bisection procedure. The efficiency of these methods of moment estimators are tested in the computational section against the classical maximum likelihood estimators. Based on these experiments we give some guidelines how to use these methods of moments estimators in practice.

ECM algorithm for estimating vector ARMA model with variance gamma distribution and possible unbounded density

SummaryThe simultaneous analysis of several financial time series is salient in portfolio setting and risk management. This paper proposes a novel alternating expectation conditional maximisation (AECM) algorithm to estimate the vector autoregressive moving average (VARMA) model with variance gamma (VG) error distribution in the multivariate skewed setting. We explain why the VARMA‐VG model is suitable for high‐frequency returns (HFRs) because VG distribution provides thick tails to capture the high kurtosis in the data and unbounded central density further captures the majority of near‐zero HFRs. The distribution can also be expressed in normal‐mean‐variance mixtures to facilitate model implementation using the Bayesian or expectation maximisation (EM) approach. We adopt the EM approach to avoid the time‐consuming Markov chain Monto Carlo sampling and solve the unbounded density problem in the classical maximum likelihood estimation. We conduct extensive simulation studies to evaluate the accuracy of the proposed AECM estimator and apply the models to analyse the dependency between two HFR series from the time zones that only differ by one hour.

Comparison of robust logistic regression estimators for variables with generalized extreme value distributions

The aim of this study is to compare the performance of robust estimators in the presence of explanatory variables with Generalized Extreme Value (GEV) distributions in the logistic regression model. Existence of extreme values in the logistic regression model negatively affects the bias and effectiveness of classical Maximum Likelihood (ML) estimators. For this reason, robust estimators that are less sensitive to extreme values have been developed. Random variables with extreme values may be fit in one of specific distributions. In study, the GEV distribution family was examined and five robust estimators were compared for the Fréchet, Gumbel and Weibull distributions. To the simulation results, the CUBIF estimator is prominent according to both bias and efficiency criteria for small samples. In medium and large samples, while the MALLOWS estimator has the minimum bias, the CUBIF estimator has the best efficiency. The same results apply for different contamination ratios and different scale parameter values of the distributions. Simulation findings were supported by a meteorological real data application.

Bayesian explanatory additive IRT models.

In this article we extend the framework of explanatory mixed IRT models to a more general class called explanatory additive IRT models. We do this by augmenting the linear predictors in terms of smooth functions. This development offers many new modeling options such as the inclusion of nonlinear covariate effects, the specification of various temporal and spatial dependency patterns, and parameter partitioning across covariates. We use integrated nested Laplace approximation (INLA) for accurate and computationally efficient estimation of the parameters. Uninformative, weakly informative, and informative prior settings for the hyperparameters are discussed. Running time experiments and Monte Carlo parameter recovery simulations are performed in order to study the accuracy and computational efficiency of INLA when applied to the proposed explanatory additive IRT model class. Using a real-life dataset, a variety of application scenarios is explored, and the results are compared with classical maximum likelihood estimation when possible. R code is included in the supplemental materials to allow readers to fully reproduce the examples computed in the paper.

A Framework of Learning Through Empirical Gain Maximization.

We develop in this letter a framework of empirical gain maximization (EGM) to address the robust regression problem where heavy-tailed noise or outliers may be present in the response variable. The idea of EGM is to approximate the density function of the noise distribution instead of approximating the truth function directly as usual. Unlike the classical maximum likelihood estimation that encourages equal importance of all observations and could be problematic in the presence of abnormal observations, EGM schemes can be interpreted from a minimum distance estimation viewpoint and allow the ignorance of those observations. Furthermore, we show that several well-known robust nonconvex regression paradigms, such as Tukey regression and truncated least square regression, can be reformulated into this new framework. We then develop a learning theory for EGM by means of which a unified analysis can be conducted for these well-established but not fully understood regression approaches. This new framework leads to a novel interpretation of existing bounded nonconvex loss functions. Within this new framework, the two seemingly irrelevant terminologies, the well-known Tukey's biweight loss for robust regression and the triweight kernel for nonparametric smoothing, are closely related. More precisely, we show that Tukey's biweight loss can be derived from the triweight kernel. Other frequently employed bounded nonconvex loss functions in machine learning, such as the truncated square loss, the Geman-McClure loss, and the exponential squared loss, can also be reformulated from certain smoothing kernels in statistics. In addition, the new framework enables us to devise new bounded nonconvex loss functions for robust learning.

Enhancement of light aircraft 6 DOF simulation using flight test data in longitudinal motion

Abstract This paper proposes a procedure to improve the accuracy of the light aircraft 6 DOF simulation model by implementing model tuning and aerodynamic database correction using flight test data. In this study, the full-scale flight testing of a 2-seater aircraft has been performed in specific longitudinal manoeuver for model enhancement and simulation validation purposes. The baseline simulation model database is constructed using multi-fidelity analysis methods such as wind tunnel (W/T) test, computational fluid dynamic (CFD) and empirical calculation. The enhancement process starts with identifying longitudinal equations of motion for sensitivity analysis, where the effect of crucial parameters is analysed and then adjusted using the model tuning technique. Next, the classical Maximum Likelihood (ML) estimation method is applied to calculate aerodynamic derivatives from flight test data, these parameters are utilised to correct the initial aerodynamic table. A simulation validation process is introduced to evaluate the accuracy of the enhanced 6 DOF simulation model. The presented results demonstrate that the applied enhancement procedure has improved the simulation accuracy in longitudinal motion. The discrepancy between the simulation and flight test response showed significant improvement, which satisfies the regulation tolerance.

Maximum Likelihood Estimation and Bayesian Estimation of three-parameter Weibull Distribution Based on Interval-Censored Data

The interval-censored data is that represents adjacent inspection times that surround an unknown failure time. This paper gives a review of the classical approach of the maximum likelihood estimating method to parameters of three-parameters Weibull distribution with interval-censored data. It also considers the Bayes’ estimators under asymmetric three loss functions squared error loss (SEL), linear-exponential (LINEX), and generalized entropy loss (GEL) functions. For the unknown parameters of three-parameters Weibull distributions with interval-censored data. We use Lindley’s approximation to compute the Bayes estimates. Then we will apply a Monte Carlo simulation study is carried out to compare the performances of the methods using the R programming language to compute and compare the performance of the proposed estimators. A real data application is also presented. The study observed that the Bayesian estimator under generalized entropy loss (GEL) functions is preferred over the classical maximum likelihood estimator for all parameters of scale, shape, and location.

Exponential GARCH Model with Exogenous Covariate for South Sudanese Pounds—USD Exchange Rate Volatility: On the Effects of Conflict on Volatility

The empirical models that explain the variation in exchange rate on the ground of macroeconomic fundamentals only are usually bias on the account of omitted variable hence, they cannot decently explain variations in exchange rate. However, if socio-political determinants, like civil wars, violence are incorporated in simple time series specification, the variations of exchange rate can be understood better. Apparently in developing countries like South Sudan where socio-political problems like conflict are most prevalent, the subject remains largely under-explored. This paper therefore, applies ARMA (p, q)-EGARCH (p, q) model with exogenous covariate for SSP-USD exchange rate volatility to examine the effect of conflict as an exogenous variable on exchange rate volatility. The proposed model is ARMA (1, 2)-EGARCH (1, 1) class of models with exogenous covariate in both mean and volatility equations. An empirical application of the proposed model is demonstrated by incorporating the conflict index as covariate in both mean and volatility equations of the proposed model. Parameter estimation was performed using maximum likelihood estimation method. The estimation results with classical maximum likelihood estimation method suggested that exchange rate volatility was persistent as evidenced by higher values of the coefficient of the parameter that accounts for persistence (β) in conditional volatility. Furthermore, the parameter for leverage effect in our models was found to be significant. The results showed that the effects of conflict on volatility of SSP-USD was found positive and statistically significant in both equations indicating that higher prevalence of conflict makes the exchange rate to be more volatile.

Сomparative Analysis of Polynomial Maximization and Maximum Likelihood Estimates for Data with Exponential Power Distribution

The work is devoted to the estimate accuracy comparative analysis of the experimental data parameters with exponential power distribution (EPD) using the classical Maximum Likelihood Estimation (MLE) and the original Polynomial Maximization Method (PMM). In contrast to the parametric approach of MLE, which uses the description in the form of probability density distribution, PMM is based on a partial description in the of higher-order statistics form and the mathematical apparatus of Kunchenko's stochastic polynomials. An algorithm for finding PMM estimates using 3rd order stochastic polynomials is presented. Analytical expressions allowing to determine the variance of PMM-estimates of the asymptotic case parameters and EPD parameters with a priori information are obtained. It is shown that the relative theoretical estimates accuracy of different methods significantly depends on the EPD shape parameter and matches only for a separate case of Gaussian distribution. The effectiveness of different approaches (including valuation of mean values estimates) both with and without a priori information on EPD properties was investigated by repeated statistical tests (through Monte Carlo Method). The greatest efficiency areas for each of methods depending on EPD shape parameter and sample data volume are constructed.

Back to the future: old methods for new estimation and test of the Gutenberg–Richter b-value for catalogues with variable completeness

SUMMARYIn this short paper we show how to use the classical maximum likelihood estimation procedure for the b-value of the Gutenberg–Richter law for catalogues with different levels of completeness. With a simple correction, that is subtracting the relative completeness level to each magnitude, it becomes possible to use the classical approach. Moreover, this correction allows to adopt the testing procedures, initially made for catalogues with a single level of completeness, for catalogues with different levels of completeness too.

On Examining Complex Systems Using the q-Weibull Distribution in Classical and Bayesian Paradigms

The q-Weibull distribution is a generalized form of the Weibull distribution and has potential to model complex systems and life time datasets. Bayesian inference is the modern statistical technique that can accommodate uncertainty associated with the model parameters in the form of prior distributions. This study presents Bayesian analysis of the q-Weibull distribution using uninformative and informative priors and the results are compared with those produced by the classical maximum likelihood (ML) and least-squares (LS) estimation methods. A simulation study is also made to compare the competing methods. Different model selection criteria and predicted datasets are considered to compare the inferential methods under study. Posterior analyses include evaluating posterior means, medians, credible intervals of highest density regions, and posterior predictive distributions. The entire analysis is carried out using Markov chain Monte Carlo (MCMC) setup using WinBUGS package. The Bayesian method has proved to be superior to its classical counterparts. A real dataset is used to illustrate the entire inferential procedure.

A Deep Learning Approach to Estimate Forward Default Intensities

This paper proposes a machine learning approach to estimate physical forward default intensities. Default probabilities are computed using artificial neural networks to estimate the intensities of the inhomogeneous Poisson processes governing default process. The major contribution to previous literature is to allow the estimation of non-linear forward intensities by using neural networks instead of classical maximum likelihood estimation. The model specification allows an easy replication of previous literature using linear assumption and shows the improvement that can be achieved.

On the James-Stein estimator for the poisson regression model

The Poisson regression model (PRM) aims to model a counting variable y, which is usually estimated by using maximum likelihood estimation (MLE) method. The performance of MLE is not satisfactory in the presence of multicollinearity. Therefore, we propose a Poisson James-Stein estimator (PJSE) as a solution to the problems of inflated variance and standard error of MLE with multicollinear explanatory variables. For assessing the superiority of proposed estimator, we present a theoretical comparison based on the matrix mean squared error (MMSE) and scalar mean squared error (MSE) criterions. A Monte Carlo simulation study is performed under different conditions in order to investigate the performance of the proposed estimator where MSE is considered as an evaluation criterion. In addition, an aircraft damage data is also considered to assess the superiority of proposed estimator. Based on the results of simulation and real data application, it is shown that the PJSE outperforms the classical MLE and other biased estimation methods in a sense of minimum MSE criterion.

Robustification of Linear Regression and Its Application in Genome-Wide Association Studies.

Regression analysis is one of the most popular statistical techniques that attempt to explore the relationships between a response (dependent) variable and one or more explanatory (independent) variables. To test the overall significance of regression, F-statistic is used if the parameters are estimated by the least-squares estimators (LSEs), while if the parameters are estimated by the maximum likelihood estimators (MLEs), the likelihood ratio test (LRT) statistic is used. However, both procedures produce misleading results and often fail to provide good fits to the reasonable space of the dataset in the presence of outlying observations. Moreover, outliers occur very frequently in any real datasets as well as in the molecular OMICS datasets. Hence, an effort is made in this study to robustify MLE based regression analysis by maximizing the β-likelihood function. The tuning parameter β is selected by cross-validation. For β = 0, the proposed method reduces to the classical MLE based regression analysis. We inspect the performance of the proposed method using both synthetic and real data analysis. The results of simulations indicate that the proposed method performs better than traditional methods in both outliers and high leverage points to estimate the parameters and mean square errors. The results of relative efficiency analysis show that the proposed estimator is relatively less affected than the popular estimators, including S, MM, and fast-S for normal error distribution in case high dimension and outliers. Also, real data analysis results demonstrated that the proposed method shows robust properties with respect to data contaminations, overcome the drawback of the traditional methods. Genome-wide association studies (GWAS) by the proposed method identify the vital gene influencing hypertension and iron level in the liver and spleen of mice. Furthermore, we have identified 15 and 21 significant SNPs for chalkiness degree and chalkiness percentage, respectively, by GWAS based on the proposed method. The variant of the SNPs might be provided the new resources for grain quality traits and could be used for further molecular and physiological analysis to enhance the better quality of rice grain. These results offer an important basis for further understanding of the robust regression analysis, which might be applied in various fields, including business, genetics, and bioinformatics.

Corrected Maximum Likelihood Estimations of the Lognormal Distribution Parameters

As a result of asymmetry in practical problems, the Lognormal distribution is more suitable for data modeling in biological and economic fields than the normal distribution, while biases of maximum likelihood estimators are regular of the order O ( n − 1 ) , especially in small samples. It is of necessity to derive logical expressions for the biases of the first-order and nearly consistent estimators by bias correction techniques. Two methods are adopted in this article. One is the Cox-Snell method. The other is the resampling method known as parametric Bootstrap. They can improve maximum likelihood estimators performance and correct biases of the Lognormal distribution parameters. Through Monte Carlo simulations, we obtain average root mean squared error and bias, which are two important indexes to compare the effect of different methods. The numerical results reveal that for small and medium-sized samples, the performance of analytical bias correction estimation is superior than bootstrap estimation and classical maximum likelihood estimation. Finally, an example is given based on the actual data.

Estimation of exponentiated Nadarajah-Haghighi distribution under progressively type-II censored sample with application to bladder cancer data

In this article we consider statistical inferences about the unknown parameters of the exponentiated Nadarajah-Haghighi (ENH) distribution based on progressively type-II censoring using classical and Bayesian procedures. For classical procedures, maximum likelihood (ML) and least square estimators of the unknown parameters are derived. The Bayes estimators are obtained based on both the symmetric (squared error) and asymmetric (LINEX, general entropy) loss functions. Furthermore, Markov chain Monte Carlo (MCMC) technique is used to compute the Bayes estimators and the associated credible intervals. Moreover, asymptotic confidence intervals are constructed using the normality property of the ML estimates. After that, the asymptotic confidence intervals using ML estimates and two parametric bootstrap confidence intervals are provided to compare with Bayes credible intervals. Finally, simulation study and a bladder cancer application are presented to illustrate the proposed methods of estimation.

Conditional maximum Lq-likelihood estimation for regression model with autoregressive error terms

In this article, we consider the parameter estimation of regression model with pth-order autoregressive (AR(p)) error term. We use the maximum Lq-likelihood (MLq) estimation method proposed by Ferrari and Yang (Ann Stat 38(2):753–783, 2010), as a robust alternative to the classical maximum likelihood (ML) estimation method to handle the outliers in the data. After exploring the MLq estimators for the parameters of interest, we provide some asymptotic properties of the resulting MLq estimators. We give a simulation study and three real data examples to illustrate the performance of the proposed estimators over the ML estimators and observe that the MLq estimators have superiority over the ML estimators when some outliers are present in the data.