Gaussian Clustering Research Articles

A hybrid steady-state evolutionary algorithm using random swaps for Gaussian model-based clustering

A new hybrid evolutionary algorithm (EA) for Gaussian mixture model-based clustering is proposed. The EA is a steady-state method that, in each generation, selects two individuals from a population, creates two offspring using either mutation or crossover, and fine-tunes the offspring using the expectation maximization (EM) algorithm. The offspring compete with their parents for survival into the next generation. The approach proposed uses a random swap, which replaces a component mean with a randomly chosen feature vector as a mutation operator. In the crossover operator, a random component is copied from the source mixture into a destination mixture. Copying in crossover favors components of the source mixture located away from the components of the destination mixture. In computational experiments, the approach was compared to a multiple restarts EM algorithm, a random swap EM method, and a state-of-the-art hybrid evolutionary algorithm for Gaussian mixture model learning on one real and 29 synthetic datasets. The results indicate that, given the same computational budget, the proposed method usually learns mixtures with higher log-likelihoods than other benchmarks competing algorithms. The partitions of data obtained by the method correspond best to the original divisions of datasets into classes.

Corpus Statistics Empowered Document Classification

In natural language processing (NLP), document classification is an important task that relies on the proper thematic representation of the documents. Gaussian mixture-based clustering is widespread for capturing rich thematic semantics but ignores emphasizing potential terms in the corpus. Moreover, the soft clustering approach causes long-tail noise by putting every word into every cluster, which affects the natural thematic representation of documents and their proper classification. It is more challenging to capture semantic insights when dealing with short-length documents where word co-occurrence information is limited. In this context, for long texts, we proposed Weighted Sparse Document Vector (WSDV), which performs clustering on the weighted data that emphasizes vital terms and moderates the soft clustering by removing outliers from the converged clusters. Besides the removal of outliers, WSDV utilizes corpus statistics in different steps for the vectorial representation of the document. For short texts, we proposed Weighted Compact Document Vector (WCDV), which captures better semantic insights in building document vectors by emphasizing potential terms and capturing uncertainty information while measuring the affinity between distributions of words. Using available corpus statistics, WCDV sufficiently handles the data sparsity of short texts without depending on external knowledge sources. To evaluate the proposed models, we performed a multiclass document classification using standard performance measures (precision, recall, f1-score, and accuracy) on three long- and two short-text benchmark datasets that outperform some state-of-the-art models. The experimental results demonstrate that in the long-text classification, WSDV reached 97.83% accuracy on the AgNews dataset, 86.05% accuracy on the 20Newsgroup dataset, and 98.67% accuracy on the R8 dataset. In the short-text classification, WCDV reached 72.7% accuracy on the SearchSnippets dataset and 89.4% accuracy on the Twitter dataset.

Investigating Emotion Style in Human Faces Using Clustering Methods.

For the past decade, performance-driven animation has been a reality in games and movies. While capturing and transferring emotions from human beings to avatars is a reasonably solved problem, it is accepted that humans express themselves in different ways, with personal styles, even when performing the same action. This article proposes a method to extract the style of human beings' facial movement when expressing emotions in posed images. We hypothesize that personal facial styles may be detected by clustering methods based on the similarity of individuals' facial expressions. We use the K-Means and Gaussian mixture model clustering methods to group emotion styles. In addition, extracted styles are considered to generate facial expressions in virtual humans and are tested with users. After an evaluation using both quantitative and qualitative criteria, our results indicate that facial expression styles do exist and can be grouped using quantitative computational methods.

Evolving Gaussian on-line clustering in social network analysis

In this paper, we present an evolving data-based approach to automatically cluster Twitter users according to their behavior. The clustering method is based on the Gaussian probability density distribution combined with a Takagi–Sugeno fuzzy consequent part of order zero (eGauss0). This means that this method can be used as a classifier that is actually a mapping from the feature space to the class label space. The eGauss method is very flexible, is computed recursively, and the most important thing is that it starts learning “from scratch”. The structure adapts to the new data using adding and merging mechanisms. The most important feature of the evolving method is that it can process data from thousands of Twitter profiles in real time, which can be characterized as a Big Data problem. The final clusters yield classes of Twitter profiles, which are represented as different activity levels of each profile. In this way, we could classify each member as ordinary, very active, influential and unusual user. The proposed method was also tested on the Iris and Breast Cancer Wisconsin datasets and compared with other methods. In both cases, the proposed method achieves high classification rates and shows competitive results.

Industrial Image Anomaly Localization Based on Gaussian Clustering of Pretrained Feature

Anomaly localization is valuable for improvement of complex production processing in smart manufacturing system. As the distribution of anomalies is unknowable and labeled data is few, unsupervised methods based on convolutional neural network (CNN) have been studied for anomaly localization. But there are still problems for real industrial applications, in terms of localization accuracy, computation time, and memory storage. This article proposes a novel framework called as Gaussian clustering of pretrained feature (GCPF), including the clustering and inference stage, for anomaly localization in unsupervised way. The GCPF consists of three modules which include pretrained deep feature extraction (PDFE), multiple independent multivariate Gaussian clustering (MIMGC), and multihierarchical anomaly scoring (MHAS). In the clustering stage, features of normal images are extracted by pretrained CNN at the PDFE module, and then clustered at the MIMGC module. In the inference stage, features of target images are extracted and then scored for anomaly localization at the MHAS module. The GCPF is compared with the state-of-the-art methods on MVTec dataset, achieving receiver operating characteristic curve of 96.86% over all 15 categories, and extended to NanoTWICE and DAGM datasets. The GCPF outperforms the compared methods for unsupervised anomaly localization, and significantly reserves the low computation complexity and online memory storage which are important for real industrial applications.

Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering.

The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last decade. However, it is challenging to discover informative representations and define relevant articles from the rapidly growing biomedical literature due to the unsupervised nature of document clustering. Moreover, empirical investigations demonstrated that traditional text clustering methods produce unsatisfactory results in terms of non-contextualized vector space representations because that neglect the semantic relationship between biomedical texts. Recently, pre-trained language models have emerged as successful in a wide range of natural language processing applications. In this paper, we propose the Gaussian Mixture Model-based efficient clustering framework that incorporates substantially pre-trained (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) BioBERT domain-specific language representations to enhance the clustering accuracy. Our proposed framework consists of main three phases. First, classic text pre-processing techniques are used biomedical document data, which crawled from the PubMed repository. Second, representative vectors are extracted from a pre-trained BioBERT language model for biomedical text mining. Third, we employ the Gaussian Mixture Model as a clustering algorithm, which allows us to assign labels for each biomedical document. In order to prove the efficiency of our proposed model, we conducted a comprehensive experimental analysis utilizing several clustering algorithms while combining diverse embedding techniques. Consequently, the experimental results show that the proposed model outperforms the benchmark models by reaching performance measures of Fowlkes mallows score, silhouette coefficient, adjusted rand index, Davies-Bouldin score of 0.7817, 0.3765, 0.4478, 1.6849, respectively. We expect the outcomes of this study will assist domain specialists in comprehending thematically cohesive documents in the healthcare field.

Towards an Extension of the Model Conditional Processor: Predictive Uncertainty Quantification of Monthly Streamflow via Gaussian Mixture Models and Clusters

This research develops an extension of the Model Conditional Processor (MCP), which merges clusters with Gaussian mixture models to offer an alternative solution to manage heteroscedastic errors. The new method is called the Gaussian mixture clustering post-processor (GMCP). The results of the proposed post-processor were compared to the traditional MCP and MCP using a truncated Normal distribution (MCPt) by applying multiple deterministic and probabilistic verification indices. This research also assesses the GMCP’s capacity to estimate the predictive uncertainty of the monthly streamflow under different climate conditions in the “Second Workshop on Model Parameter Estimation Experiment” (MOPEX) catchments distributed in the SE part of the USA. The results indicate that all three post-processors showed promising results. However, the GMCP post-processor has shown significant potential in generating more reliable, sharp, and accurate monthly streamflow predictions than the MCP and MCPt methods, especially in dry catchments. Moreover, the MCP and MCPt provided similar performances for monthly streamflow and better performances in wet catchments than in dry catchments. The GMCP constitutes a promising solution to handle heteroscedastic errors in monthly streamflow, therefore moving towards a more realistic monthly hydrological prediction to support effective decision-making in planning and managing water resources.

Identification of Enterprise Financial Risk Based on Clustering Algorithm.

In order to solve the problem that corporate financial risks seriously affect the healthy development of enterprises, credit institutions, securities investors, and even the whole of China, the K-means clustering algorithm, the risk screening process, and the Gaussian mixture clustering algorithm, the risk screening process, are proposed; experiments have shown that although the number of high-risk companies selected by the K-means algorithm is small, only 9% of the full sample, the high-risk cluster can contain nearly 30% of the new “special treatment” companies. If the time period is extended to the next 5 years, this proportion will be higher. Finally we found that if the prediction of “special handling” events is used as the criterion for evaluating high-risk clusters, then K-means clustering can effectively screen out those risky companies that need to be treated with caution by investors. The validity of the experiment is verified.

Relative instability compensation method for hand–eye collaborative system in large-scale components measurement

In a hand–eye collaborative measurement system, the measurement error caused by the position change of an industrial robot end can create challenges during the high-precision measurement of large-scale components. The relative instability between binocular stereo vision and robot end is discussed for the first time, and a relative instability compensation (RIC) method is proposed. Firstly, using a watershed-model based on the Gaussian mixture model clustering, and locking of the region of interest are achieved. The correction of ellipse distortion combined with camera internal parameters is employed, and the 3D coordinate is reconstructed. Subsequently, the common and link points are constructed to perceive the updating of the positions. Based on the assumption of small-angle deviation, a novel model is established for compensating the relative instability. Ultimately, the experiment results indicate that the proposed method is effective for improving the measurement precision, which is superior to 0.075 mm.

Discrete Generalized Inverted Exponential Distribution: Case Study Color Image Segmentation

We present in this paper a discrete analogue of the continuous generalized inverted exponential distribution denoted by discrete generalized inverted exponential (DGIE) distribution. Since, it is cumbersome or difficult to measure a large number of observations in reality on a continuous scale in the area of reliability analysis. Yet, there are a number of discrete distributions in the literature; however, these distributions have certain difficulties in properly fitting a large amount of data in a variety of fields. The presented DGIE β , θ has shown the efficiency in fitting data better than some existing distribution. In this study, some basic distributional properties, moments, probability function, reliability indices, characteristic function, and the order statistics of the new DGIE are discussed. Estimation of the parameters is illustrated using the moment's method as well as the maximum likelihood method. Simulations are used to show the performance of the estimated parameters. The model with two real data sets is also examined. In addition, the developed DGIE is applied as color image segmentation which aims to cluster the pixels into their groups. To evaluate the performance of DGIE, a set of six color images is used, as well as it is compared with other image segmentation methods including Gaussian mixture model, K-means, and Fuzzy subspace clustering. The DGIE provides higher performance than other competitive methods.

A novel solution of deep learning for endoscopic ultrasound image segmentation: enhanced computer aided diagnosis of gastrointestinal stromal tumor

Gastrointestinal stromal tumor is one of the critical tumors that doctors do not suggest to get frequent endoscopy, so there is a need for a diagnosis system which can process ultrasound images and figure out the tumor. Many gastrointestinal tumor diagnosis methods were developed, but all of these methods used manual contour rather than automatic segmentation. The research adopts enhanced automatic segmentation to improve the diagnosis of the gastrointestinal stromal tumor with deep convolutional neural networks. This solution’s proposed system is an enhanced automated segmentation methodology using multi-scale Gaussian kernel fuzzy clustering and multi-scale vector field convolution, which segments the ultrasound image automatically into the region of interest (the infected area). Convolutional Neural Network with Class Activation Mapping is done to diagnose an image with the tumor for Four datasets, namely (USS1, SH Hospital, SNUH, BUSI). This proposed system helps to get a clearer tumor image, and the accuracy has increased from 84.275% to 88.4%, and the processing time has reduced from 28.525% to 24.575%. The proposed solution enhanced Automatic Segmentation helped to get clearer tumor image which resulted in increased accuracy and decreased performance time compared to the state-of-the-art. Automatic segmentation overcomes the dependency on the expert for drawing the Region of Interest (ROI).

Investigation of Gaussian mixture clustering model for online diagnosis of tip-wear in nanomachining

This paper studied an unsupervised machine learning method to achieve real-time diagnosis of tip wear in tip-based nanomachining with an Atomic Force Microscope (AFM). The unsupervised Gaussian Mixture Model (GMM) was applied for online pattern recognition using the collected process data. In this work, the time varying signal of machining force was collected and managed in the form of moving data windows. The characteristics of the nanomachining force signal, including the maximum force, peak to peak force, and the variance, were calculated as the data of extracted features to indicate the tip wear conditions. Outliers were identified and removed using the Mahalanobis Distance Detection. A GMM model was trained using the historical processing data, in which the tip wear states were identified by clustering the feature parameters of each data frame. The change of the tip conditions was detected and tracked through the changing numbers of the tip failure points in the data windows. Experiment results showed that the highest and the average recognition accuracy of the proposed model was 1 and 0.8367 with the longest and the average calculation time of 0.031 s and 0.025 s. Compared with supervised and semi-supervised machine learning algorithms, the GMM based on unsupervised machine learning model has slightly lower recognition accuracy, but much shorter calculation time, which is suitable for online automatic diagnosis of tool wear in large-scale nanomanufacturing in real time.

User behaviour models to forecast electricity consumption of residential customers based on smart metering data

This paper presents a novel approach to forecast day-ahead electricity consumption for residential households where highly irregular human behaviour plays a significant role. The methodology requires data from fiscal smart meters, which makes it applicable to real scenarios where personal data gathering is not feasible. These data are rarely complete; therefore, a robust combination of machine-learning techniques is used to handle missing data and outliers. The novelty of this method relies on identifying and predicting user electricity consumption behaviour as a procedure to improve the forecasting of the overall electricity consumption of each individual customer. The methodology uses Gaussian mixture clustering to identify behaviour clusters and an eXtreme Gradient Boosting classification (XGBoost) model to predict the day-ahead behaviour pattern. This predicted user behaviour cluster is fed into an Artificial Neural Network (ANN) to enable an improved capturing of the highly unpredictable user conduct for the forecast of electricity consumption. A novel metric, namely the Euclidean Distance-based Accuracy (EDA), is finally proposed to enable a more thorough evaluation of time series classification algorithms. The whole development is tested over 500 residential users placed in a southeastern region of Spain. The results showed that, when the novel approach was used, the MAPEd and NRMSEd were reduced by 7% and 9% respectively, increasing to a 20% and 17% respective reduction for the best cases according to EDA. This methodology sets the basis for massive user-centred analyses, very profitable to any electricity company.

Post-processing calcium subtraction method to minimize stenosis-overestimation by blooming artifact.

BACKGROUND: CT images are often affected by blooming artifacts during the diagnosis that facilitate an overestimation of the expression of calcification stenosis, thereby impeding the accurate diagnosis of this condition.OBJECTIVE: Arterial calcification can act as a blooming artifact in computed tomography (CT) images, leading to overestimations of the blood vessel and the size of calcified plaque. This study proposes an improved CT post-processing method that accurately measures calcium and lumen size in blood vessels.METHODS: Six hundred and thirty calcium datasets were obtained from 63 patients diagnosed with a vascular disease. Patients were grouped into three sets corresponding to each image acquisition method used: G1, for the invasive coronary angiography (ICA); G2, for multiplanar reconstruction (MPR) imaging and post-processing; and G3, for the novel method of mixed Gaussian filter and K-mean clustering (GK). Results of GK were generated by adding Gaussian and k-mean clustering algorithms to the MPR post-processing procedure. The analysis of variance (ANOVA), linear regression, and intraclass correlation coefficient (ICC) were used to compare the accuracy and sensitivity of the different methods. All measurements were performed multiple times to mitigate human error.RESULTS: The ANOVA test revealed no significant differences between the G1 and G3 groups. Hence, linear regression was used to analyze the correlation between the G1 and G3 groups ( 0.05, R2 0.885), and a higher correlation than G1 and G2 was reported ( 0.05, R2 0.432). ICC was performed for reproducibility, wherein high correlation was identified among all groups.CONCLUSIONS: Results of the study indicate that the GK method yields images that are very similar to ICA image measurements. This suggests that the GK can be used as a more effective post-processing method over the inaccurate MPR while remaining non-intrusive when determining the arterial stenosis degree, unlike the ICA.

Gaussian mixture model clustering algorithms for the analysis of high-precision mass measurements

The development of the phase-imaging ion-cyclotron resonance (PI-ICR) technique for use in Penning trap mass spectrometry (PTMS) increased the speed and precision with which PTMS experiments can be carried out. In PI-ICR, data sets of the locations of individual ion hits on a detector are created showing how ions cluster together into spots according to their cyclotron frequency. Ideal data sets would consist of a single, 2D-spherical spot with no other noise, but in practice data sets typically contain multiple spots, non-spherical spots, or significant noise, all of which can make determining the locations of spot centers non-trivial. A method for assigning groups of ions to their respective spots and determining the spot centers is therefore essential for further improving precision and confidence in PI-ICR experiments. We present the class of Gaussian mixture model (GMM) clustering algorithms as an optimal solution. We show that on simulated PI-ICR data, several types of GMM clustering algorithms perform better than other clustering algorithms over a variety of typical scenarios encountered in PI-ICR. The mass spectra of 163Gd, 163mGd, 162Tb, and 162mTb measured using PI-ICR at the Canadian Penning trap mass spectrometer were checked using GMMs, producing results that were in close agreement with the previously published values.

Gaussian Projection Deep Extreme Clustering and Chebyshev Reflective Correlation based Outlier Detection

Outlier detection or simply the task of point detection that are noticeably distinct and different from data sample is a predominant issue in deep learning. When a framework is constructed, these distinctive points can later lead to model training and compromise accurate predictions. Owing to this reason, it is paramount to recognize and eliminate them before constructing any supervised model and this is frequently the initial step when dealing with a deep learning issue. Over the recent few years, different numbers of outlier detector algorithms have been designed that ensure satisfactory results. However, their main disadvantages remain in the time and space complexity and unsupervised nature. In this work, a clustering-based outlier detection called, Random Projection Deep Extreme Learning-based Chebyshev Reflective Correlation (RPDEL-CRC) is proposed. First, Gaussian Random Projection-based Deep Extreme Learning-based Clustering model is designed. Here, by applying Gaussian Random Projection function to the Deep Extreme Learning obtains the relevant and robust clusters corresponding to the data points in a significant manner. Next, with the robust clusters, outlier detection time is said to be reduced to a greater extent. In addition, a novel Chebyshev Temporal and Reflective Correlation-based Outlier Detection model is proposed to detect outliers therefore achieving high outlier detection accuracy. The proposed approach is validated with the NIFTY-50 stock market dataset. The performance of the RPDEL-CRC method is evaluated by applying it to NIFTY-50 Stock Market dataset. Finally, we compare the results of the RPDEL-CRC method to the state-of-the-art outlier detection methods using outlier detection time, accuracy, error rate and false positive rate evaluation metrics.

Resident user load classification method based on improved Gaussian mixture model clustering

In view of the limitation of “hard assignment” of clusters in traditional clustering methods and the difficulty of meeting the requirements of clustering efficiency and clustering accuracy simultaneously in regard to massive data sets, a load classification method based on a Gaussian mixture model combining clustering and principal component analysis is proposed. The load data are fed into a Gaussian mixture model clustering algorithm after principal component analysis and dimensionality reduction to achieve classification of large-scale load datasets. The method in this paper is used to classify loads in the Canadian AMPds2 public dataset and is compared with K-Means, Gaussian mixed model clustering and other methods. The results show that the proposed method can not only achieve load classification more effectively and finely, but also save computational cost and improve computational efficiency.

Load Clustering Method Based on Improved Density Peak and Gaussian Mixture Model

A load clustering method based on improved density peak and Gaussian mixture was proposed to solve the problems that standard Gaussian mixture clustering algorithm (GMM) with random initial clustering center was easy to fall into local optimum in the clustering analysis of residential electricity behavior, and the density peak algorithm (DPC) needed to manually select the clustering center.The method combines the Cosine distance and Euclidean distance, which reflect the similarity of curve shape, and selects the clustering center automatically through the sine value relation of relative Angle between two points in γ descending graph. The clustering center selected by the improved DPC algorithm was used to initialize the parameters of the GMM model. Finally, load clustering is carried out by GMM model. The results of calculation examples show that the proposed method can effectively improve the efficiency of daily load curve clustering.

A Health Monitoring Method for Automotive Surface Mount Technologies

In this paper, we propose a two-stage online data-based diagnostic method that detects issues in the In-Circuit Test (ICT) equipment from a Surface-Mount Technology (SMT) production line. The first stage performs anomaly detection in a univariate stream of test values. The second stage achieves fault detection and isolation based on the process capability and a Gaussian mixture clustering method. Combining the two stages allows improving the online cost of the second stage and improving the confidence and interpretability of the first stage, anomaly detection. Two solutions are compared for the first stage, anomaly detection by Extreme Value Theory (EVT) and a sliding window method. The comparison is done with an automotive, industrial database and shows that using EVT delivers almost the same performance in detection with less computation time.

Damage Pattern Recognition and Crack Propagation Prediction for Crumb Rubber Concrete Based on Acoustic Emission Techniques

In this study, the clustering method of the concrete matrix rupture and rubber fracture damages as well as the prediction of the ultimate load of crumb rubber concrete using the acoustic emission (AE) technique were investigated. The loading environment of the specimens was a four-point bending load. Six clustering methods including k-means, fuzzy c-means (FCM), self-organizing mapping (SOM), Gaussian mixture model (GMM), hierarchical model, and density peak clustering method were analyzed; the results illustrated that the density peak clustering has the best performance. Next, the optimal clustering algorithm was used to cluster AE signals so as to study the evolution behavior of different damage modes, and the ultimate load of crumb rubber concrete was predicted by an artificial neural network. The results indicated that the combination of AE techniques and appropriate clustering methods such as the density peak clustering method and the artificial neural network could be used as a practical tool for structural health monitoring of crumb rubber concrete.