Fuzzy Classification Maximum Likelihood Research Articles

Robust Switching Regressions Using the Laplace Distribution

This paper presents a robust method for dealing with switching regression problems. Regression models with switch-points are broadly employed in diverse areas. Many traditional methods for switching regressions can falter in the presence of outliers or heavy-tailed distributions because of the modeling assumptions of Gaussian errors. The outlier corruption of datasets is often unavoidable. When misapplied, the Gaussian assumption can lead to incorrect inference making. The Laplace distribution is known as a longer-tailed alternative to the normal distributions and connected with the robust least absolute deviation regression criterion. We propose a robust switching regression model of Laplace distributed errors. To advance robustness, we extend the Laplace switching model to a fuzzy class model and create a robust algorithm named FCL through the fuzzy classification maximum likelihood procedure. The robustness properties and the advance of resistance against high-leverage outliers are discussed. Simulations and sensitivity analyses illustrate the effectiveness and superiority of the proposed algorithm. The experimental results indicate that FCL is much more robust than the EM-based algorithm. Furthermore, the Laplace-based algorithm is more time-saving than the t-based procedure. Diverse real-world applications demonstrate the practicality of the proposed approach.

Clustering right-skewed data stream via Birnbaum–Saunders mixture models: A flexible approach based on fuzzy clustering algorithm

Despite the widespread use of Gaussian mixture model for clustering datasets, practical applications show that the skewed and leptokurtic mixture models can be considered as promising alternatives. This paper proposes a finite mixture of Birnbaum–Saunders (FM-BS) distributions for analyzing and clustering right-skewed, leptokurtic, and multimodal lifetime datasets. The maximum likelihood (ML) estimates of the proposed model are obtained by developing a computationally analytical expectation–maximization (EM) type algorithm, as well as a fuzzy classification maximum likelihood (FCML) type algorithm, that combines the advantages of fuzzy clustering and robust statistical estimators. Simulation studies demonstrate the accuracy and computational efficiency of the FCML algorithm to estimate parameters of the FM-BS distributions and to cluster samples drawn from the FM-BS distributions. Finally, some real datasets have been analyzed to illustrate how well the proposed FM-BS model estimates the membership values.

A fuzzy classification approach to piecewise regression models

Piecewise regression models have been widely applied to various areas in which break-point detection is important and break locations in piecewise regression models are essential to know when and how the pattern of data structure changes. One major difficulty with likelihood approaches to break-point location for regression models is the non-smoothness of the likelihood function with respect to break-points regarded as parameters. Although several non-standard methods have been proposed to overcome this problem, their applications may be limited due to additional required techniques or restrictions. Other shortcomings of existing methods are such as only a single break-point allowed, an unchanged variance or model continuity demanded, and heavy calculations involved. Locating break-points is similar to classifying data into groups of similar items. Fuzzy clustering is powerful and widely applied to substantial areas but less used in break-point detection. In this paper, we propose a new fuzzy classification maximum likelihood break-point (FCM-BP) method to simultaneously estimate break-points and regression parameters free of non-differentiability. We transform a piecewise regression model into a fuzzy class model and then employ a fuzzy classification maximum likelihood procedure to derive estimates. The non-differentiability problem is bypassed through mixture modelling with break-points regarded as class variables. The proposed FCM-BP is robust to initial values and it can detect the location and the magnitude of changes in mean, in variance, and in model coefficients simultaneously, allowing multiple break-points in all of these. FCM-BP is feasible for models with continuous or discontinuous break-points, whereas several of the well-known statistical methods are available for continuous models only. The proposed method is demonstrated and compared with other existing methods through extensive experiments using numerical and real datasets. Experimental results demonstrate the effectiveness and the superiority of FCM-BP, and real data applications reflect its broad practicability.

Detecting change-points for shifts in mean and variance using fuzzy classification maximum likelihood change-point algorithms

Knowing the time of changes, called change-point (CP), in a process is crucial for engineers to recognize the root cause fast and accurately. Since special causes may induce simultaneous changes in mean and variance, detecting changes in both at once is required. Many methodologies in quality control were developed for detecting changes in either mean or variance only, and process parameters were assumed known often. However, they are rarely known exactly and a small estimation error may lead to unfavorable CP estimates. Fuzzy partitioning is better suited to cases of vague boundaries between two segments which appear very often in reality. A new mechanism, called fuzzy classification maximum likelihood change-point (FCML-CP) algorithm, is proposed to detect shifts in mean and variance simultaneously. A CP framework is transferred into a mixture model and then a FCML-CP algorithm is created through fuzzy classification maximum likelihood procedures. The proposed FCML-CP can be applied to phase I and II processes without knowledge of in-control process parameters; it can estimate multiple CPs of process mean or/and variance simultaneously. The effectiveness and superiority of FCML-CP are shown by extensive experiments with numerical and real data sets. Specifically, the proposed FCML-CP is superior to the commonly used statistical mixture likelihood approach using expectation–maximization (EM) algorithm; it is much more time-saving especially. The remarkable performance of FCML-CP in detecting CPs for small changes is particularly important and helpful for engineers to recognize the special cause fast and correctly since an out-of-control signal resulted from small changes is usually delayed long.

Fuzzy Classification Maximum Likelihood Clustering with T-Distributions

Classification maximum likelihood (CML) procedure is a maximum likelihood mixture approach to clustering. In 1993, Yang first extended the CML to a so-called fuzzy CML (FCML), by combining fuzzy c-partitions with the CML function for a normal mixture model. However, normal distribution is not robust for outliers. In this paper we consider FCML with t-distributions and create a clustering algorithm, called FCMLT. Numerical examples and real data applications with comparisons are given to demonstrate the effectiveness and superiority of the proposed method.

A ROBUST FUZZY CLASSIFICATION MAXIMUM LIKELIHOOD CLUSTERING FRAMEWORK

In 1993, Yang first extended the classification maximum likelihood (CML) to a so-called fuzzy CML, by combining fuzzy c-partitions with the CML function. Fuzzy c-partitions are generally an extension of hard c-partitions. It was claimed that this was more robust. However, the fuzzy CML still lacks some robustness as a clustering algorithm, such as its in-ability to detect different volumes of clusters, its heavy dependence on parameter initializations and the necessity to provide an a priori cluster number. In this paper, we construct a robust fuzzy CML clustering framework that has a robust clustering method. The eigenvalue decomposition of a covariance matrix is firstly considered using the fuzzy CML model. The Bayesian information criterion (BIC) is then used for model selection, in order to choose the best model with the optimal number of clusters. Therefore, the proposed robust fuzzy CML clustering framework exhibits clustering characteristics that are robust in terms of the parameter initialization, robust in terms of the cluster number and also in terms of its capability to detect different volumes of clusters. Numerical examples and real data applications with comparisons are provided, which demonstrate the effectiveness and superiority of the proposed method.

Entropy-type classification maximum likelihood algorithms for mixture models

Mixtures of distributions are popularly used as probability models for analyzing grouped data. Classification maximum likelihood (CML) is an important maximum likelihood approach to clustering with mixture models. Yang et al. extended CML to fuzzy CML. Although fuzzy CML presents better results than CML, it is always affected by the fuzziness index parameter. In this paper, we consider fuzzy CML with an entropy-regularization term to create an entropy-type CML algorithm. The proposed entropy-type CML is a parameter-free algorithm for mixture models. Some numerical and real-data comparisons show that the proposed method provides better results than some existing methods.

Fuzzy classification maximum likelihood algorithms for mixed-Weibull distributions

In this paper we propose an efficient algorithm based on Yang’s (Fuzzy Sets Syst 57:365–337, 1993) concept, namely the fuzzy classification maximum likelihood (FCML) algorithm, to estimate the mixed-Weibull parameters. Compared with EM and Jiang and Murthy (IEEE Trans Reliab 44:477–488, 1995) methods, the proposed FCML algorithm presents better accuracy. Thus, we recommend FCML as another acceptable method for estimating the mixed-Weibull parameters.

Estimation of parameters in latent class models using fuzzy clustering algorithms

A mixture approach to clustering is an important technique in cluster analysis. A mixture of multivariate multinomial distributions is usually used to analyze categorical data with latent class model. The parameter estimation is an important step for a mixture distribution. Described here are four approaches to estimating the parameters of a mixture of multivariate multinomial distributions. The first approach is an extended maximum likelihood (ML) method. The second approach is based on the well-known expectation maximization (EM) algorithm. The third approach is the classification maximum likelihood (CML) algorithm. In this paper, we propose a new approach using the so-called fuzzy class model and then create the fuzzy classification maximum likelihood (FCML) approach for categorical data. The accuracy, robustness and effectiveness of these four types of algorithms for estimating the parameters of multivariate binomial mixtures are compared using real empirical data and samples drawn from the multivariate binomial mixtures of two classes. The results show that the proposed FCML algorithm presents better accuracy, robustness and effectiveness. Overall, the FCML algorithm has the superiority over the ML, EM and CML algorithms. Thus, we recommend FCML as another good tool for estimating the parameters of mixture multivariate multinomial models.

FUZZY CLASS LOGISTIC REGRESSION ANALYSIS

Distribution mixtures are used as models to analyze grouped data. The estimation of parameters is an important step for mixture distributions. The latent class model is generally used as the analysis of mixture distributions for discrete data. In this paper, we consider the parameter estimation for a mixture of logistic regression models. We know that the expectation maximization (EM) algorithm was most used for estimating the parameters of logistic regression mixture models. In this paper, we propose a new type of fuzzy class model and then derive an algorithm for the parameter estimation of a fuzzy class logistic regression model. The effects of the explanatory variables on the response variables are described. The focus is on binary responses for the logistic regression mixture analysis with a fuzzy class model. An algorithm, called a fuzzy classification maximum likelihood (FCML), is then created. The mean squared error (MSE) based accuracy criterion for the FCML and EM algorithms to the parameter estimation of logistic regression mixture models are compared using the samples drawn from logistic regression mixtures of two classes. Numerical results show that the proposed FCML algorithm presents good accuracy and is recommended as a new tool for the parameter estimation of the logistic regression mixture models.

Mixture Poisson regression models for heterogeneous count data based on latent and fuzzy class analysis

In this paper we propose a new approach, called a fuzzy class model for Poisson regression, in the analysis of heterogeneous count data. On the basis of fuzzy set concept and fuzzy classification maximum likelihood (FCML) procedures we create an FCML algorithm for fuzzy class Poisson regression models. Traditionally, the EM algorithm had been used for latent class regression models. Thus, the accuracy and effectiveness of EM and FCML algorithms for estimating the parameters are compared. The results show that the proposed FCML algorithm presents better accuracy and effectiveness and can be used as another good tool to regression analysis for heterogeneous count data.

On a class of fuzzy classification maximum likelihood procedures

Classification Maximum Likelihood (CML) procedure is a remarkable mixture of maximum likelihood approach to clustering. This has been well documented in the book of McLachlan and Basford. In this paper, we make the fuzzy extension of the CML procedure. Based on this class of fuzzy CML procedures, we extend the fuzzy clustering algorithms of Trauwaert, Kaufman and Rousseeuw by adding a penalty term. Especially, we derive a generalized type of fuzzy c-means (FCM) clustering algorithms, called the penalized FCM clustering algorithms. Then we create some asymptotic behaviors of these penalized FCM procedures. By doing some numerical examples we find that the penalized FCM is more meaningful and effective than FCM.