ECM Algorithm Research Articles

Stability Grade Evaluation of Slope with Soft Rock Formation in Open-Pit Mine Based on Modified Cloud Model

In recent years, the frequent occurrence of slope failures has brought the issue of slope problems to the forefront of widespread public concern, which significantly impedes progress toward the secure and sustainable development of open-pit mines. And, high and steep slopes of weak rock strata, being a more complex type of slope, pose a greater potential for danger. In order to ensure the reliability of the safety evaluation results of the high and steep open-pit slope containing soft rocks, an evaluation index system with quantized grade intervals was created based on the thought of an analytic hierarchy process, and the MATLAB R2021a was used to calculate the numerical characteristic values of the cloud model. Then, a standard stability cloud model based on cloud theory was established. With the opening pit mine as an example, its slope stability practice cloud image was generated and the similarity between this image and the grades of the cloud model was calculated through the ECM algorithm to effectively identify the stability and verify the scientificity and validity of the model. The results show that the similarity between the practice cloud image and the standard stability cloud image for the total evaluation of the stability of an open-pit mine is 0.021, 0.279, 0.594, and 0.106, respectively. The slope stability is at grade C, which is basically consistent with the numerical simulation and the analysis results of the traditional limit equilibrium method, verifying that the model is scientific and effective to a certain extent. The method provides substantial guidance to ensure production safety in this specific open-pit mine. It provides ideas and means for other similar complex slope stability analysis and prevention. Meanwhile, it promotes the safe and sustainable development of open-pit mines.

Estimation in shape mixtures of skew-normal linear regression models via ECM coupled with Gibbs sampling

Abstract In this paper, we study linear regression models in which the error term has shape mixtures of skew-normal distribution. This type of distribution belongs to the skew-normal (SN) distribution class that can be used for heavy tails and asymmetry data. For the first time, for the classical (non-Bayesian) estimation of the parameters of the SN family, we apply the Markov chains Monte Carlo ECM (MCMC-ECM) algorithm where the samples are generated by Gibbs sampling, denoted by Gibbs-ECM, and also, we extend two other types of the EM algorithm for the above model. Finally, the proposed method is evaluated through a simulation and compared with the Numerical Math-ECM algorithm and Monte Carlo ECM (MC-ECM) using a real data set.

Mixtures of regressions using matrix-variate heavy-tailed distributions

AbstractFinite mixtures of regressions (FMRs) are powerful clustering devices used in many regression-type analyses. Unfortunately, real data often present atypical observations that make the commonly adopted normality assumption of the mixture components inadequate. Thus, to robustify the FMR approach in a matrix-variate framework, we introduce ten FMRs based on the matrix-variate t and contaminated normal distributions. Furthermore, once one of our models is estimated and the observations are assigned to the groups, different procedures can be used for the detection of the atypical points in the data. An ECM algorithm is outlined for maximum likelihood parameter estimation. By using simulated data, we show the negative consequences (in terms of parameter estimates and inferred classification) of the wrong normality assumption in the presence of heavy-tailed clusters or noisy matrices. Such issues are properly addressed by our models instead. Additionally, over the same data, the atypical points detection procedures are also investigated. A real-data analysis concerning the relationship between greenhouse gas emissions and their determinants is conducted, and the behavior of our models in the presence of heterogeneity and atypical observations is discussed.

Linear Mixed-effects Models for Censored Data with Serial Correlation Errors Using the Multivariate Student’s t-distribution

The purpose of this paper is to develop a practical framework for the analysis of the linear mixed-effects models for censored or missing data with serial correlation errors, using the multivariate Student’s t-distribution, being a flexible alternative to the use of the corresponding normal distribution. We propose an efficient ECM algorithm for computing the maximum likelihood estimates for these models with standard errors of the fixed effects and likelihood function as a by-product. This algorithm uses closed-form expressions at the E-step, which relies on formulas for the mean and variance of a truncated multivariate Student’s t-distribution. In order to illustrate the usefulness of the proposed new methodology, artificial and a real dataset are analyzed. The proposed algorithm and methods are implemented in the R package ARpLMEC.

An EM algorithm for estimating the parameters of the skew generalized t-normal distribution with application to robust finite mixture modeling

The present article describes an EM-type algorithm for estimation of the skew generalized t-normal (SGTN) distribution. The family of SGTN distributions can provide certain types of flexibility such as heavy tails and high kurtosis. The complexity of the SGTN distribution is traced to the ratio of the t density and distribution function of a normal distribution in the likelihood equations. To cope with this problem, we develop a feasible ECME algorithm for computing maximum likelihood estimates of model parameters via a selection mechanism. The proposed approach provides a robust parameter estimation method for the finite mixture model. Standard errors for the parameter estimates can be obtained via a general information-based method. Experimental results on simulated data and one real data example demonstrate the efficacy and usefulness of the proposed methodology.

A generalization of the CIRCE method for quantifying input model uncertainty in presence of several groups of experiments

The semi-empirical nature of best-estimate models closing the balance equations of thermal-hydraulic (TH) system codes is well-known as a significant source of uncertainty for accuracy of output predictions. This uncertainty, called model uncertainty, is usually represented by multiplicative (log-)Gaussian variables whose estimation requires solving an inverse problem based on a set of adequately chosen real experiments. One method from the TH field, called CIRCE,22This is a French acronym: Calcul des Incertitudes Relatives aux Corrélations Élémentaires. addresses it. We present in the paper a generalization of this method to several groups of experiments each having their own properties, including different ranges for input conditions and different geometries. An individual (log-)Gaussian distribution is therefore estimated for each group in order to investigate whether the model uncertainty is homogeneous between the groups, or should depend on the group. To this end, a multi-group CIRCE is proposed where a variance parameter is estimated for each group jointly to a mean parameter common to all the groups to preserve the uniqueness of the best-estimate model. The ECME algorithm for Maximum Likelihood Estimation developed in Celeux et al. (2010) is extended to the latter context, then applied to relevant demonstration cases. Finally, it is tested on a practical case to assess the uncertainty of critical mass flow assuming two groups due to the difference of geometry between the experimental setups.

A stochastic approximation ECME algorithm to semi-parametric scale mixtures of centred skew normal regression models

A stochastic approximation ECME algorithm to semi-parametric scale mixtures of centred skew normal regression models

High-throughput XPS spectrum modeling with autonomous background subtraction for 3 d 5/2 peak mapping of SnS

We propose a fitting model that automatically conducts the background subtraction during high-throughput peak fitting. The fitting model consists of the pseudo-Voigt mixture model and the ramp-sum background model, and each model represents the peak and background component, respectively. The optimization of the fitting model is performed by the spectrum adapted ECM algorithm that enables us to perform the peak fitting and background subtraction simultaneously through the high-throughput calculation. Application of the proposed model to the synthetic spectral data showed appropriate decomposition of the peak and background component. We also applied the proposed model to 3721 spectral data collected from the SnS sheet by X-ray photoelectron spectroscopy. The spectral data from the SnS sheet were successfully decomposed to the component of Sn 3d3/2 peak, Sn 3d5/2 peak and background, respectively. As the spectrum adapted ECM algorithm can efficiently analyze a large amount of spectral data, we can obtain the color map showing spatial distribution of Sn(II) and Sn(IV) using the parameter of Sn 3d5/2 peak. The proposed model supports us to obtain the insightful spatial distribution of peak component that has been difficult to obtain by conventional peak fitting.

Heteroscedastic partially linear model under skew-normal distribution with application in ragweed pollen concentration

We introduce a new class of heteroscedastic partially linear model (PLM) with skew-normal distribution. Maximum likelihood estimation of the model parameters by the ECM algorithm (Expectation/Conditional Maximization) as well as influence diagnostics for the new model are investigated. In addition, a Likelihood Ratio test for assessing the homogeneity of the scale parameter is presented. Simulation studies for assessing the performance of the ECM algorithm and the Likelihood Ratio test statistics for homogeneity of variance are developed. Also, a study for misspecification of the structure function is considered. Finally, an application of the new heteroscedastic PLM to a real data set on ragweed pollen concentration is presented to show that it provides a better fit than the classic homocedastic PLM. We hope that the proposed model may attract applications in different areas of knowledge.

Growth curve mixture models with unknown covariance structures

Though playing an important role in longitudinal data analysis, the uses of growth curve models are constrained by the crucial assumption that the grouping design matrix is known. In this paper we propose a Gaussian mixture model within the framework of growth curve models which handles the problem caused by the unknown grouping matrix. This allows for a greater degree of flexibility in specifying the model and freeing the response matrix from following a single multivariate normal distribution. The new model is considered under two parsimonious covariance structures together with the unstructured covariance. The maximum likelihood estimation of the proposed model is studied using the ECM algorithm, which clusters growth curve data simultaneously. Data-driving methods are proposed to find various model parameters so as to create an appropriate model for complex growth curve data. Simulation studies are conducted to assess the performance of the proposed methods and real data analysis on gene expression clustering is made, showing that the proposed procedure works well in both, model fitting and growth curve data clustering.

ECM algorithm for estimating vector ARMA model with variance gamma distribution and possible unbounded density

SummaryThe simultaneous analysis of several financial time series is salient in portfolio setting and risk management. This paper proposes a novel alternating expectation conditional maximisation (AECM) algorithm to estimate the vector autoregressive moving average (VARMA) model with variance gamma (VG) error distribution in the multivariate skewed setting. We explain why the VARMA‐VG model is suitable for high‐frequency returns (HFRs) because VG distribution provides thick tails to capture the high kurtosis in the data and unbounded central density further captures the majority of near‐zero HFRs. The distribution can also be expressed in normal‐mean‐variance mixtures to facilitate model implementation using the Bayesian or expectation maximisation (EM) approach. We adopt the EM approach to avoid the time‐consuming Markov chain Monto Carlo sampling and solve the unbounded density problem in the classical maximum likelihood estimation. We conduct extensive simulation studies to evaluate the accuracy of the proposed AECM estimator and apply the models to analyse the dependency between two HFR series from the time zones that only differ by one hour.

The multivariate tail-inflated normal distribution and its application in finance

The research objective of this paper is to handle situations where the empirical distribution of multivariate real-valued data is elliptical and with heavy tails. Many statistical models already exist that accommodate these peculiarities. This paper enriches this branch of literature by introducing the multivariate tail-inflated normal (MTIN) distribution, an elliptical heavy tails generalization of the multivariate normal (MN). The MTIN belongs to the family of MN scale mixtures by choosing a convenient continuous uniform as mixing distribution. Moreover, it has a closed-form for the probability density function characterized by only one additional ‘inflation’ parameter, with respect to the nested MN, governing the tail-weight. The moment generating function, and the first four moments, are also derived; interestingly, the latter always exist and the excess kurtosis can assume any positive value. The method of moments and maximum likelihood (ML) are considered for estimation. As concerns the latter, a direct approach, as well as a variant of the EM algorithm (namely, the ECME algorithm), are illustrated. Furthermore, a way to approximate covariance matrix of the ML estimator is suggested and the existence of the ML estimates is evaluated. Since the inflation parameter is estimated from the data, robust estimates of the mean vector of the nested MN distribution are automatically obtained by down-weighting. Simulations are performed to compare the estimation methods/algorithms, to investigate the ability of AIC and BIC to select among a set of candidate elliptical models, and to evaluate the robustness of these candidate methods when data are skewed. The findings are the following: ML is better than MM, direct ML is suggested for low dimensions, while the ECME algorithm is to be preferred when the number of variables is higher, AIC and BIC work comparably in selecting the true underlying model, and the MTIN outperforms the competing models in terms of robustness toward skew data. For illustrative purposes, the MTIN distribution is finally fitted to multivariate financial data and compared with other well-established multivariate elliptical distributions. The analysis shows how the proposed model represents a valid alternative to the considered competitors in terms of AIC and BIC, but also in reproducing the higher empirical kurtosis which is common in the financial context.

ECM Algorithm for Estimating Vector ARMA Model with Variance Gamma Distribution and Possible Unbounded Density

The simultaneous analysis of several financial time series is important in portfolio setting and risk management. This paper proposes a novel alternating Expectation conditional Maximisation (AECM) algorithm to estimate the vector autoregressive moving average (VARMA) model with variance gamma (VG) error distribution in the multivariate skewed setting. We explain why VARMA-VG model is suitable for high frequency returns (HFRs) because VG distribution provides thick tails to capture the high kurtosis in the data and the unbounded central density further captures the majority of near zero HFRs. The distribution can also be expressed in normal mean-variance mixtures which facilitate model implementation using Bayesian and expectation maximisation (EM) approaches. We adopt the EM approach which avoids the time consuming Markov chain Monto Carlo sampling and solve the unbounded density problem in the classical maximum likelihood estimation. We discuss some properties of VARMA model, conduct extensive simulation studies to evaluate the accuracy of the proposed AECM estimator and apply the models to analyse the dependency between several HFR series.

A novel MM algorithm and the mode-sharing method in Bayesian computation for the analysis of general incomplete categorical data

Incomplete categorical data often occur in the fields such as biomedicine, epidemiology, psychology, sports and so on. In this paper, we first introduce a novel minorization–maximization (MM) algorithm to calculate the maximum likelihood estimates (MLEs) of parameters and the posterior modes for the analysis of general incomplete categorical data. Although the data augmentation (DA) algorithm and Gibbs sampling as the corresponding stochastic counterparts of the expectation–maximization (EM) and ECM algorithms are developed very well, up to now, little work has been done on creating stochastic versions to the existing MM algorithms. This is the first paper to propose a mode-sharing method in Bayesian computation for general incomplete categorical data by developing a new acceptance–rejection (AR) algorithm aided with the proposed MM algorithm. The key idea is to construct a class of envelope densities indexed by a working parameter and to identify a specific envelope density which can overcome the four drawbacks associated with the traditional AR algorithm. The proposed mode-sharing based AR algorithm has three significant characteristics: (I) it can automatically establish a family of envelope densities {gλ(⋅): λ∈Sλ} indexed by a working parameter λ, where each member in the family shares mode with the posterior density; (II) with the one-dimensional grid method searching over the finite interval Sλ, it can identify an optimal working parameter λopt by maximizing the theoretical acceptance probability, yielding a best easy-sampling envelope density gλopt(⋅), which is more dispersive than the posterior density; (III) it can obtain the optimal envelope constant copt by using the mode-sharing theorem (indicating that the high-dimensional optimization can be completely avoided) or by using the proposed MM algorithm again. Finally, a toy model and three real data sets are used to illustrate the proposed methodologies.

Simultaneous Variable and Covariance Selection With the Multivariate Spike-and-Slab LASSO

We propose a Bayesian procedure for simultaneous variable and covariance selection using continuous spike-and-slab priors in multivariate linear regression models where q possibly correlated responses are regressed onto p predictors. Rather than relying on a stochastic search through the high-dimensional model space, we develop an ECM algorithm similar to the EMVS procedure of Ročková and George targeting modal estimates of the matrix of regression coefficients and residual precision matrix. Varying the scale of the continuous spike densities facilitates dynamic posterior exploration and allows us to filter out negligible regression coefficients and partial covariances gradually. Our method is seen to substantially outperform regularization competitors on simulated data. We demonstrate our method with a re-examination of data from a recent observational study of the effect of playing high school football on several later-life cognition, psychological, and socio-economic outcomes. An R package, scripts for replicating examples in this article, and results from further simulation studies are provided in the supplementary materials available online.

Compact Belief Rule Base Learning for Classification with Evidential Clustering.

The belief rule-based classification system (BRBCS) is a promising technique for addressing different types of uncertainty in complex classification problems, by introducing the belief function theory into the classical fuzzy rule-based classification system. However, in the BRBCS, high numbers of instances and features generally induce a belief rule base (BRB) with large size, which degrades the interpretability of the classification model for big data sets. In this paper, a BRB learning method based on the evidential C-means clustering (ECM) algorithm is proposed to efficiently design a compact belief rule-based classification system (CBRBCS). First, a supervised version of the ECM algorithm is designed by means of weighted product-space clustering to partition the training set with the goals of obtaining both good inter-cluster separability and inner-cluster pureness. Then, a systematic method is developed to construct belief rules based on the obtained credal partitions. Finally, an evidential partition entropy-based optimization procedure is designed to get a compact BRB with a better trade-off between accuracy and interpretability. The key benefit of the proposed CBRBCS is that it can provide a more interpretable classification model on the premise of comparative accuracy. Experiments based on synthetic and real data sets have been conducted to evaluate the classification accuracy and interpretability of the proposal.

Mixtures of general location model with factor analyzer covariance structure for clustering mixed type data

Cluster analysis is one of the most widely used method in statistical analyses, in which homogeneous subgroups are identified in a heterogeneous population. Due to the existence of the continuous and discrete mixed data in many applications, so far, some ordinary clustering methods such as, hierarchical methods, k-means and model-based methods have been extended for analysis of mixed data. However, in the available model-based clustering methods, by increasing the number of continuous variables, the number of parameters increases and identifying as well as fitting an appropriate model may be difficult. In this paper, to reduce the number of the parameters, for the model-based clustering mixed data of continuous (normal) and nominal data, a set of parsimonious models is introduced. Models in this set are extended, using the general location model approach, for modeling distribution of mixed variables and applying factor analyzer structure for covariance matrices. The ECM algorithm is used for estimating the parameters of these models. In order to show the performance of the proposed models for clustering, results from some simulation studies and analyzing two real data sets are presented.

A fast algorithm for short term electric load forecasting by a hidden semi-markov process

ABSTRACTThe paper proposes an alternative algorithm to implement the current manual choosing mechanism of energy companies whose dedicated department has a library of electric load models and one best model is chosen manually everyday for daily forecast. The proposed algorithm is a combination of an estimation of change point, and a fast ECM algorithm based on the empirical probability function, as well as methods of a hidden markov chain. We train parameters of the proposed algorithm based on a historical dataset consisting of loads, exogenous information such as temperature, and the daily recommended best model which is unavailable sometimes. Simulations and a test on a real-world dataset show that compared with other state-of-art algorithms, the proposed algorithm is fast and efficient for short-term electric load forecasting. An implement to the proposed algorithm written in Matlab is provided in supplement file.

Finite mixtures of multivariate scale-shape mixtures of skew-normal distributions

Finite mixtures of multivariate skew distributions have become increasingly popular in recent years due to their flexibility and robustness in modeling heterogeneity, asymmetry and leptokurticness of the data. This paper introduces a novel finite mixture of multivariate scale-shape mixtures of skew-normal distributions to enhance strength and flexibility when modeling heterogeneous multivariate data that contain more extreme non-normal features. A computational tractable ECM algorithm which consists of analytically simple E- and CM-steps is developed to carry out maximum likelihood estimation of parameters. The asymptotic covariance matrix of parameter estimates is derived from the observed information matrix using the outer product of expected complete-data scores. We demonstrate the utility of the proposed approach through simulated and real data examples.

Mean mixtures of normal distributions: properties, inference and application

The skew normal (SN) distribution of Azzalini (Scand J Stat 12:171–178, 1985) is one of the widely used probability distributions for modelling skewed data. In this article, we introduce a general class of skewed distributions based on mean mixtures of normal distributions, which includes the SN distribution as a special case. Some properties of this new class, such as expressions for mean, variance, skewness and kurtosis coefficients and characteristic function, are derived. Also, estimates of the model parameters are first obtained by the method of moments. Two special cases of this new class are studied in detail. It is shown that the range of skewness and kurtosis coefficients for the special cases is wider than that of the SN distribution, and in addition, unlike the SN distribution, one of these models is an infinitely divisible distribution. For carrying out the maximum likelihood (ML) estimation, an ECM algorithm is developed. This algorithm is analytically simple because closed-form expressions of conditional expectations in the E-step as well as the updating estimators in the CM-step are in explicit form. The observed information matrix is provided for approximating the asymptotic covariance matrix of the ML estimators of the parameters. The usefulness of the proposed distribution is illustrated through simulated as well as two real data sets. Next, a new extension of regression models is constructed by assuming the proposed distributions for the error term. Finally, a multivariate version of the proposed model is discussed.