Semi-supervised Gaussian Mixture Model Research Articles

An Improved Semi-Supervised Gaussian Mixture Model (I-SGMM)

In the era of data-driven decision-making, the Gaussian Mixture Model (GMM) stands as a cornerstone in statistical modeling, particularly in clustering and density estimation. The Improved GMM presents a robust solution to a fundamental problem in clustering: the determination of the optimal number of clusters. Unlike its predecessor, it does not rely on a predetermined cluster count but employs model selection criteria, such as the Bayesian Information Criterion (BIC) or Akaike Information Criterion (AIC), to automatically identify the most suitable cluster count for the given data. This inherent adaptability is a hallmark of the Improved GMM, making it a versatile tool in a broad spectrum of applications, from market segmentation to image processing. Furthermore, the Improved GMM revolutionizes parameter estimation and model fitting. It leverages advanced optimization techniques, such as the Expectation-Maximization (EM) algorithm or variational inference, to achieve convergence to more favorable local optima. This results in precise and reliable parameter estimates, including cluster means, covariances, and component weights. The Improved GMM is particularly invaluable when dealing with data of varying complexities, non-standard data distributions, and clusters with differing shapes and orientations. It excels at capturing the nuanced relationships within the data, providing a powerful framework for understanding complex systems. One of the key differentiators of the Improved GMM is its accommodation of full covariance matrices for each component. This feature empowers the model to account for intricate interdependencies between variables, which is essential for modeling real-world data effectively. It is capable of handling data that exhibits non-spherical or irregular cluster shapes, a significant limitation of the traditional GMM.

Driving Style Recognition of Leading Vehicles based on Semi-supervised Gaussian Mixture Model

Driving style recognition of leading vehicles is important to ensure driving safety and improve the efficiency of road traffic. Current supervised learning based methods achieve high recognition accuracy, but their annotations are labor-intensive and time-consuming in practical scenarios. Although unsupervised learning based methods do not require manual labeling, their recognition accuracy is still low. In order to reduce the labeling cost and improve the accuracy of driving style recognition simultaneously, we propose a driving style recognition method based on semi-supervised Gaussian mixture model. Firstly, the driving data is collected and pre-processed by OBD-II vehicle recorder, and 22-dimensional driving style feature parameters are selected and constructed. Secondly, the kernel principal component analysis method is used to reduce the dimensionality and obtain 6-dimensional discriminative principal components. Finally, the Gaussian mixture model is trained by utilizing both the labeled data and unlabeled data. Concretely, the model parameters are measured by the expectation maximization algorithm and the likelihood function. The simulation experimental results demonstrate that the proposed method can effectively improve the driving style recognition accuracy, as well as reduce the reliance on labeled data.

A new froth image classification method based on the MRMR-SSGMM hybrid model for recognition of reagent dosage condition in the coal flotation process.

Intelligent separation is a core technology in the transformation, upgradation, and high-quality development of coal. Realising the intelligent recognition and accurate classification of coal flotation froth is a key technology of intelligent separation. At present, the coal flotation process relies on artificial recognition of froth features for adjusting the reagent dosage. However, owing to the low accuracy and subjectivity of artificial recognition, some problems arise, such as reagent wastage and unqualified product quality. Thus, this paper proposes a new froth image classification method based on the maximal-relevance-minimal-redundancy (MR MR)-semi-supervised Gaussian mixture model (SSGMM) hybrid model for recognition of reagent dosage condition in the coal flotation process. First, the features of morphology, colour, and texture are extracted, and the optimal froth image features are screened out using the maximal-relevance-minimal-redundancy (MRMR) feature selection algorithm based on class information. Second, the traditional GMM clusterer is improved, called SSGMM, by introducing a small number of marked samples, the traditional GMM’ problems of unclear training goals, invisible clustering results, and artificially judged clustering results are solved. Then a new hybrid classification model is proposed by combining the MRMR with the modified GMM (SSGMM) which can be named as (MRMR - SSGMM). The optimal froth image features are screened by MRMR to provide the SSGMM classifier. In the process of training and learning the feature samples, using the marked feature samples of froth images to guide the unmarked feature samples. The information of marked feature samples of froth images is mapped to the unmarked feature samples, the classification of the froth images were realised. Finally, the accuracy of the SSGMM classifier is used as the evaluation criterion for the screened features by MRMR. By automatically executing the entire learning process to find the best number of froth image features and the optimal image features, so that the classifier achieves the maximum classification accuracy. Experimental results show that the proposed classification method achieves the best results in accuracy and time, compared with other benchmark classification methods. Application results show that the method can provide reliable guidance for the adjustment of the reagent dosage, realize the accurate and timely control of the reagent dosage, reduce the consumption of the reagent and the incidence of production accidents, and stabilize the product quality in the coal flotation production process.

Soft fault diagnosis of analog circuits based on semi-supervised support vector machine

Soft fault diagnosis has been validated as a very challenging problem in analog circuits. In order to improve the generalization ability and close to the practical application of fault diagnosis models, a novel method for obtaining a large number of training samples in soft fault interval is proposed in the present work. Training samples are randomly generated in the interval of soft fault to adapt the continuously change of component parameters. Limits of experimental conditions, lead to the limited number of samples labeled by experts, so that training samples are classified using the semi-supervised support vector machine (S3VM) algorithm. Manifold learning algorithm has been used for the feature extraction and the dimension reduction of soft fault time domain response data in analog circuits. Then the semi-supervised Gaussian mixture model (SGMM) is applied to cluster decision trees. Finally, the S3VM classification is used for the soft fault diagnosis in analog circuits. Experimental results show that the proposed method can be utilized in the single and double soft fault diagnosis of analog circuits. It is observed that the S3VM method is extremely significant as a guide in actual engineering applications, although the diagnosis rate is slightly less than the fixed-parameter offset soft faults.

Optimization of Jacobian matrix annealing algorithm based on hybrid model

The annealing algorithm can effectively find the clustering distribution of network structure, but the algorithm accuracy of handling different networks needs to be further improved. In order to identify the data distribution in network structure of mixed model more accurately and solve the problems of local maximum value and convergence of mixed model, the Jacobian matrix annealing algorithm is studied. First, the model is initialized by using the inverse temperature parameter, and then the two tasks of expectation step and maximization step are performed iteratively. The posterior probability of model is calculated based on the Jacobian matrix until the algorithm reaches the set accuracy or meets the convergence condition. The Jacobian matrix annealing algorithm is compared with the inverse simulated annealing clustering algorithm under semi-supervised Gaussian mixture model on the real network, and the experimental results show that the accuracy of Jacobian matrix algorithm in the hybrid network model is better. The proposed algorithm can not only prevent the network from falling into the local optimum, but also improve the accuracy of analyzing network clustering distribution.

Towards semi-supervised and probabilistic classification in structural health monitoring

In practical applications of data-driven Structural Health Monitoring (SHM), recording labels for each of the measured signals can be infeasible and expensive. In consequence, conventional methods for (supervised) machine learning can become irrelevant in certain applications of damage classification. Semi-supervised methods, however, allow algorithms to learn from information in the available unlabelled measurements as well a limited set of labelled data. As such, this paper suggests a semi-supervised Gaussian mixture model for probabilistic damage-classification, informed by both labelled and unlabelled signals. The generative statistical model is shown to improve the classification performance, compared to supervised learning, with simulated and experimental SHM data, while requiring no further inspections of the system. Specifically, semi-supervised learning leads to 3.87% and 3.83% reductions in the classification error for the simulated and experimental datasets respectively. These results indicate that, through semi-supervised learning in SHM, the cost associated with labelling data could be managed, as the information in a small set of labelled signals can be combined with larger sets of unlabelled data.

Leveraging Frequency-Dependent Kernel and DIP-Based Clustering for Robust Speech Activity Detection in Naturalistic Audio Streams

Speech activity detection (SAD) is front-end in most speech systems, e.g., speaker verification, speech recognition etc . Supervised SAD typically leverages machine learning models trained on annotated data. For applications like zero-resource speech processing and NIST-OpenSAT-2017 public safety communications task, it might not be feasible to collect SAD annotations. SAD is challenging for naturalistic audio streams containing multiple noise-sources simultaneously. We propose a novel frequency-dependent kernel (FDK) based SAD features. FDK provides enhanced spectral decomposition from which several statistical descriptors are derived. FDK statistical descriptors are combined by principal component analysis into one-dimensional FDK-SAD features. We further proposed two decision backends: First, variable model-size Gaussian mixture model (VMGMM); and second, Hartigan dip-based robust feature clustering. While VMGMM is a model-based approach, the DipSAD is nonparametric. We used both backends for comparative evaluations in two phases: first, standalone SAD performance; and second, the effect of SAD on text-dependent speaker verification using RedDots data. The NIST-OpenSAD-2015 and NIST-OpenSAT-2017 corpora are used for standalone SAD evaluations. We establish two Center for Robust Speech Systems (CRSS) corpora namely CRSS-PLTL-II and CRSS long-duration naturalistic noise corpus. The CRSS corpora facilitate standalone SAD evaluations on naturalistic audio streams. We performed comparative studies of the proposed approaches with multiple baselines including SohnSAD, rSAD, semisupervised Gaussian mixture model, and Gammatone spectrogram features.

Manifold regularized semi-supervised Gaussian mixture model.

In the last decades, Gaussian Mixture Models (GMMs) have attracted considerable interest in data mining and pattern recognition. A GMM-based clustering algorithm models a dataset with a mixture of multiple Gaussian components and estimates the model parameters using the Expectation-Maximization (EM) algorithm. Recently, a new Locally Consistent GMM (LCGMM) has been proposed to improve the clustering performance by exploiting the local manifold structure of the data using a p nearest neighbor graph. In addition to the underlying manifold structure, many other forms of prior knowledge may guide the clustering process and improve the performance. In this paper, we introduce a Semi-Supervised LCGMM (Semi-LCGMM), where the prior knowledge is provided in the form of class labels of partial data. In particular, the new Semi-LCGMM incorporates the prior knowledge into the maximum likelihood function of the original LCGMM, and the model parameters are estimated using the EM algorithm. It is worth noting that, in our algorithm, each class may be modeled by multiple Gaussian components while in the unsupervised setting each class is modeled by a single Gaussian component. Our algorithm has shown promising results in many different applications, including clustering breast cancer data, heart disease data, handwritten digit images, human face images, and image segmentation.

A multi-manifold semi-supervised Gaussian mixture model for pattern classification

Semi-supervised Gaussian mixture model (SGMM) has been successfully applied to a wide range of engineering and scientific fields, including text classification, image retrieval, and biometric identification. Recently, many studies have shown that naturally occurring data may reside on or near manifold structures in ambient space. In this paper, we study the use of SGMM for data sets containing multiple separated or intersecting manifold structures. We propose a new multi-manifold regularized, semi-supervised Gaussian mixture model (M2SGMM) for classifying multiple manifolds. Specifically, we model the data manifold using a similarity graph with local and geometrical consistency properties. The geometrical similarity is measured by a novel application of local tangent space. We regularize the model parameters of the SGMM by incorporating the enhanced Laplacian of the graph. Experiments demonstrate the effectiveness of the proposed approach.