Traditional Clustering Methods Research Articles

Robust Spectral Clustering Incorporating Statistical Sub-Graph Affinity Model

Hyperspectral image (HSI) clustering is a challenging work due to its high complexity. Subspace clustering has been proven to successfully excavate the intrinsic relationships between data points, while traditional subspace clustering methods ignore the inherent structural information between data points. This study uses graph convolutional subspace clustering (GCSC) for robust HSI clustering. The model remaps the self-expression of the data to non-Euclidean domains, which can generate a robust graph embedding dictionary. The EKGCSC model can achieve a globally optimal closed-form solution by using a subspace clustering model with the Frobenius norm and a Gaussian kernel function, making it easier to implement, train, and apply. However, the presence of noise can have a noteworthy negative impact on the segmentation performance. To diminish the impact of image noise, the concept of sub-graph affinity is introduced, where each node in the primary graph is modeled as a sub-graph describing the neighborhood around the node. A statistical sub-graph affinity matrix is then constructed based on the statistical relationships between sub-graphs of connected nodes in the primary graph, thus counteracting the uncertainty image noise by using more information. The model used in this work was named statistical sub-graph affinity kernel graph convolutional subspace clustering (SSAKGCSC). Experiment results on Salinas, Indian Pines, Pavia Center, and Pavia University data sets showed that the SSAKGCSC model can achieve improved segmentation performance and better noise resistance ability.

Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering.

The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last decade. However, it is challenging to discover informative representations and define relevant articles from the rapidly growing biomedical literature due to the unsupervised nature of document clustering. Moreover, empirical investigations demonstrated that traditional text clustering methods produce unsatisfactory results in terms of non-contextualized vector space representations because that neglect the semantic relationship between biomedical texts. Recently, pre-trained language models have emerged as successful in a wide range of natural language processing applications. In this paper, we propose the Gaussian Mixture Model-based efficient clustering framework that incorporates substantially pre-trained (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) BioBERT domain-specific language representations to enhance the clustering accuracy. Our proposed framework consists of main three phases. First, classic text pre-processing techniques are used biomedical document data, which crawled from the PubMed repository. Second, representative vectors are extracted from a pre-trained BioBERT language model for biomedical text mining. Third, we employ the Gaussian Mixture Model as a clustering algorithm, which allows us to assign labels for each biomedical document. In order to prove the efficiency of our proposed model, we conducted a comprehensive experimental analysis utilizing several clustering algorithms while combining diverse embedding techniques. Consequently, the experimental results show that the proposed model outperforms the benchmark models by reaching performance measures of Fowlkes mallows score, silhouette coefficient, adjusted rand index, Davies-Bouldin score of 0.7817, 0.3765, 0.4478, 1.6849, respectively. We expect the outcomes of this study will assist domain specialists in comprehending thematically cohesive documents in the healthcare field.

Single vs. Multiple Signage: Applying Limited Capacity Message Processing to Fan##x2019;s Recognition and Recall of In-stadium Advertising

With the innovation of digital signage, stadiums are surrounding their field with the signage of one single sponsor and digitally rotating to the next sponsor periodically. While this rotating technique is becoming more popular, little is known about whether the technique is more effective in creating consumer recognition and recall of corporate sponsors than the traditional cluster method. Thus, the current study aims to examine the effect of these types of methods (i.e., single-signage rotation vs. traditional cluster method) on brand memory. Guided by the theoretical perspective of limited model for mediated message processing (LC3MP), it was hypothesized that single-signage rotation technique would generate greater encoding rates and retrieval rates than multiple-signage rotation. The MANOVA results showed that participants (n = 33) who were assigned to the single-signage rotation condition were more likely to recall the sponsorship brands when compared to the multiple-signages condition (n = 34). However, there were no such impacts on encoding rates. Thus, the hypothesis regarding retrieving rates was only supported, which indicates that single-signage rotation technique will help fans to recall sponsor information better than other techniques.

Robust Spectral Clustering via Low-Rank Sample Representation

Traditional clustering methods neglect the data quality and perform clustering directly on the original data. Therefore, their performance can easily deteriorate since real-world data would usually contain noisy data samples in high-dimensional space. In order to resolve the previously mentioned problem, a new method is proposed, which builds on the approach of low-rank representation. The proposed approach first learns a low-rank coefficient matrix from data by exploiting the data’s self-expressiveness property. Then, a regularization term is introduced to ensure that the representation coefficient of two samples, which are similar in original high-dimensional space, is close to maintaining the samples’ neighborhood structure in the low-dimensional space. As a result, the proposed method obtains a clustering structure directly through the low-rank coefficient matrix to guarantee optimal clustering performance. A wide range of experiments shows that the proposed method is superior to compared state-of-the-art methods.

Attributed graph clustering with multi-task embedding learning

Attributed graph clustering is challenging as it needs to effectively combine both graph structure and node feature information to accomplish node clustering. Recent studies mostly adopt graph neural networks to learn node embeddings, then apply traditional clustering methods to obtain clusters. However, their node embeddings are not specifically designed for clustering. Moreover, most of their loss functions only rely on either structure or feature information, making both kinds of information not fully retained in node embeddings. In this paper, we propose a multi-task embedding learning method (MTEL) for attributed graph clustering, which constructs two prediction tasks in terms of structure and feature based adjacency matrices respectively. To make the node embeddings helpful for the downstream clustering, in each task, we predict the minimum hop number between each pair of nodes in the adjacency matrix, so that the correlation degrees among nodes can be encoded into node embeddings. To improve the performance of the prediction task, we regularize the model parameters in these two tasks via ℓ2,1 norm, through which the model parameters can be jointly learned. Experiments on real attributed graphs show that MTEL is superior for attributed graph clustering over state-of-the-art methods.

Deep Possibilistic C-means Clustering Algorithm on Medical Datasets.

In the past, the possibilistic C-means clustering algorithm (PCM) has proven its superiority on various medical datasets by overcoming the unstable clustering effect caused by both the hard division of traditional hard clustering models and the susceptibility of fuzzy C-means clustering algorithm (FCM) to noise. However, with the deep integration and development of the Internet of Things (IoT) as well as big data with the medical field, the width and height of medical datasets are growing bigger and bigger. In the face of high-dimensional and giant complex datasets, it is challenging for the PCM algorithm based on machine learning to extract valuable features from thousands of dimensions, which increases the computational complexity and useless time consumption and makes it difficult to avoid the quality problem of clustering. To this end, this paper proposes a deep possibilistic C-mean clustering algorithm (DPCM) that combines the traditional PCM algorithm with a special deep network called autoencoder. Taking advantage of the fact that the autoencoder can minimize the reconstruction loss and the PCM uses soft affiliation to facilitate gradient descent, DPCM allows deep neural networks and PCM's clustering centers to be optimized at the same time, so that it effectively improves the clustering efficiency and accuracy. Experiments on medical datasets with various dimensions demonstrate that this method has a better effect than traditional clustering methods, besides being able to overcome the interference of noise better.

Shareability-Exclusivity Representation on Product Grassmann Manifolds for Multi-camera video clustering

With the rapid popularity of multi-camera networks, one human action is usually captured by multiple cameras located at different angles simultaneously. Multi-camera videos contain the distinct perspectives of one action, therefore multiple views can overcome the impacts of illumination and occlusion. In this paper, we propose a novel multi-camera video clustering model, named Shareability-Exclusivity Representation on Product Grassmann Manifolds (PGM-SER), to address two key issues in traditional multi-view clustering methods (MVC): (1) Most MVC methods directly construct a shared similarity matrix by fusing multi-view data or their corresponding similarity matrices, which ignores the exclusive information in each view; (2) Most MVC methods are designed for multi-view vectorial data, which cannot handle the nonlinear manifold structure hidden in multi-camera videos. The proposed PGM-SER firstly adopts Product Grassmann Manifolds to represent multi-camera videos, then simultaneously learn their shared and exclusive information in global structures to achieve multi-camera video clustering. We provide an effective optimization algorithm to solve PGM-SER and present the corresponding convergence analysis. Finally, PGM-SER is tested on three multi-camera human action video datasets and obtain satisfied experimental results.

Trip end identification based on spatial-temporal clustering algorithm using smartphone positioning data

As a widespread portable probe, smartphone equipped with Global Navigation Satellite System (GNSS) can continuously track individual’s travel trajectory, it is potential for trip end information identification. Existing studies mostly use rule-based methods with specific dwelling time and distance thresholds for trip end detection, these methods are highly dependent on expert experience and lack universality across different traffic environments. Some density-based spatial clustering methods also have issues in the case of GNSS signal missing, traffic congestion, short-time stays and repeat visits to the same place etc. Therefore, this paper proposes a two-step method for trip end identification by using smartphone GNSS positioning data. First, a spatial-temporal density-based clustering algorithm (ST-DBCA) is proposed for trip end identification, the method considers both spatial and temporal travel trajectory point density at the same time, and performs much better than traditional clustering methods. Second, three optimization models are further proposed to optimize the identification results, including 1) a short time stay optimization model, 2) a redundant stay optimization model, and 3) a traffic congestion stay optimization model. Field tests in Chengdu China are conducted to verify the feasibility and effectiveness of the proposed methods. Results show that the average trip end identification accuracy under different trip purposes reaches 92.8%, and the average errors of arrival and departure time are smaller than 150 s.

Gradient-based elephant herding optimization for cluster analysis.

Clustering analysis is essential for obtaining valuable information from a predetermined dataset. However, traditional clustering methods suffer from falling into local optima and an overdependence on the quality of the initial solution. Given these defects, a novel clustering method called gradient-based elephant herding optimization for cluster analysis (GBEHO) is proposed. A well-defined set of heuristics is introduced to select the initial centroids instead of selecting random initial points. Specifically, the elephant optimization algorithm (EHO) is combined with the gradient-based algorithm GBO for assigning initial cluster centers across the search space. Second, to overcome the imbalance between the original EHO exploration and exploitation, the initialized population is improved by introducing Gaussian chaos mapping. In addition, two operators, i.e., random wandering and variation operators, are set to adjust the location update strategy of the agents. Nine datasets from synthetic and real-world datasets are adopted to evaluate the effectiveness of the proposed algorithm and the other metaheuristic algorithms. The results show that the proposed algorithm ranks first among the 10 algorithms. It is also extensively compared with state-of-the-art techniques, and four evaluation criteria of accuracy rate, specificity, detection rate, and F-measure are used. The obtained results clearly indicate the excellent performance of GBEHO, while the stability is also more prominent.

Analyzing cancer data in North Vietnam by complex network technique

Data-clustering tools can be employed to generate new knowledge for the diagnosis and treatment of cancer. However, traditional clustering methods, such as the K-mean approach, often require the determination of input parameters such as the cluster number and initial centers to be viable. In this study, we present a network science-based clustering method with fewer parameters that were used to mine a cancer-screening dataset containing over 177,000 records. We propose an algorithm that computes the similarity between pairs of records to create a complex network in which each node represents a record, and two nodes are connected by an edge if their similarity is greater than a given threshold as determined by experimental observation. Based on the network created, we employed the network modularity optimization algorithm to detect modules (clusters) within it. Each cluster contains records that are similar to one another in terms of some attributes; therefore, we could derive rules from the cluster for insights into the cancer situation in Vietnam. These rules reveal that some cancer types are more widespread in specific families and living environments in Vietnam. Clustering data based on network science can therefore be a good option for large-scale relational data-mining problems in the future.

Statistically-Robust Clustering Techniques for Mapping Spatial Hotspots: A Survey

Mapping of spatial hotspots, i.e., regions with significantly higher rates of generating cases of certain events (e.g., disease or crime cases), is an important task in diverse societal domains, including public health, public safety, transportation, agriculture, environmental science, and so on. Clustering techniques required by these domains differ from traditional clustering methods due to the high economic and social costs of spurious results (e.g., false alarms of crime clusters). As a result, statistical rigor is needed explicitly to control the rate of spurious detections. To address this challenge, techniques for statistically-robust clustering (e.g., scan statistics) have been extensively studied by the data mining and statistics communities. In this survey, we present an up-to-date and detailed review of the models and algorithms developed by this field. We first present a general taxonomy for statistically-robust clustering, covering key steps of data and statistical modeling, region enumeration and maximization, and significance testing. We further discuss different paradigms and methods within each of the key steps. Finally, we highlight research gaps and potential future directions, which may serve as a stepping stone in generating new ideas and thoughts in this growing field and beyond.

Driver stress recognition for smart transportation: Applying multiobjective genetic algorithm for improving fuzzy c-means clustering with reduced time and model complexity

Driver stress can lead to traffic deaths and injuries which ultimately bring on world economic loss. Researchers are in full swing to develop various algorithms for driver stress recognition (DSR) to enable smart transportation. Existing methods have encountered various difficulties. To relieve the problems and improve the accuracy, a DSR algorithm has been developed using multiobjective genetic algorithm based fuzzy c-means clustering (MOGA-FCM). This algorithm adopts Electrocardiogram (ECG) signal as input which is further used to compute features. Analysis has been carried out to investigate feature selection. Attention is drawn to the reduction of time and model complexity of MOGA-FCM. To reduce the time complexity, hyper-grid scheme is introduced, and the storage of membership matrix is eliminated. To reduce the model complexity, one of the objectives of the multiobjective optimization problem is to minimize the number of clusters. Results indicate that the time complexity and model complexity have been reduced by 68 %, and a reduction of two clusters, respectively. MOGA-FCM achieves equivalent true negative rate (TNReq) and equivalent true positive rate (TPReq) of 89.9 % and 91.7 % respectively. It improves the average true negative rate by 9.8–25.3 % and average true positive rate by 9.5–26.6 % compared to traditional clustering methods. Compared to existing biometric signal-based, speech-based and image-based approaches, it improves the performance by at least 7.02 % and 12.9 % in TNReq and TPReq respectively. As a result, the proposed algorithm can give an accurate and immediate reminding to drivers to take a rest or calm down in order to prevent traffic accident and lower the risk of developing stress-related disease.

Dynamic Ensemble Learning With Multi-View Kernel Collaborative Subspace Clustering for Hyperspectral Image Classification

Recently, a series of collaborative representation (CR) methods have attracted much attention for hyperspectral images classification. In this article, two CR-based dynamic ensemble selection (DES) methods using multiview kernel collaborative subspace clustering (MVKCSC) and random subspace MVKCSC (RSMVKCSC) are proposed. In order to combine spectral and spatial information to construct a region of competence (RoC), the multiview learning strategy is used in the general DES method. Compared with traditional clustering methods, the MVC can more effectively utilize multifeature information. Moreover, a new method of constructing the Laplacian matrix using kernel CR coefficients is proposed for clustering based on subspace clustering and CR theory. This method is called MVKCSC, which can obtain the clustering results by using kernel CR self-representation coefficients. In addition, to increase the diversity of samples, the random subspace method (RSM) and MVKCSC are combined for RMVKCSC. Moreover, the algorithm can obtain better clustering results by constraining samples and features simultaneously. The effectiveness of the proposed methods is validated using three hyperspectral data sets with few samples. The experimental results show that both DES-MVKCSC and DES-RSMVKCSC outperform their single classifier counterparts. In particular, the proposed methods provide superior performance compared with the state-of-the-art DES methods.

Unsupervised Binary Protocol Clustering Based on Maximum Sequential Patterns

With the rapid development of the Internet, a large number of private protocols emerge on the network. However, some of them are constructed by attackers to avoid being analyzed, posing a threat to computer network security. The blockchain uses the P2P protocol to implement various functions across the network. Furthermore, the P2P protocol format of blockchain may differ from the standard format specification, which leads to sniffing tools such as Wireshark and Fiddler not being able to recognize them. Therefore, the ability to distinguish different types of unknown network protocols is vital for network security. In this paper, we propose an unsupervised clustering algorithm based on maximum frequent sequences for binary protocols, which can distinguish various unknown protocols to provide support for analyzing unknown protocol formats. We mine the maximum frequent sequences of protocol message sets in bytes. And we calculate the fuzzy membership of the protocol message to each maximum frequent sequence, which is based on fuzzy set theory. Then we construct the fuzzy membership vector for each protocol message. Finally, we adopt K-means++ to split different types of protocol messages into several clusters and evaluate the performance by calculating homogeneity, integrity, and Fowlkes and Mallows Index (FMI). Besides, the clustering algorithms based on Needleman–Wunsch and the fixed-length prefix are compared with the algorithm presented in this paper. Compared with these traditional clustering methods, we demonstrate a certain improvement in the clustering performance of our work.

Timeliness and stability-based operation optimization for copper flotation industrial process

Making full use of data knowledge for operation optimization is a challenging problem in industrial production, which has a decisive significance to improve the level of industrial automation. Meanwhile, the timeliness and stability of operation optimization for the copper flotation industrial process is one of the most concerned topics. In this paper, a framework of the timeliness and stability-based operation optimization for the copper flotation industrial process is constructed, which integrates data processing, category analysis, knowledge mining (KM), and operation optimization. Then, an improved density peak clustering (IDPC) strategy is investigated that the satisfactory categories can be obtained compared with traditional clustering methods. After, KM and information entropy (IE) techniques are applied to mine the valuable operation rules and construct the adjustment decision tables for each category. Next, both the strategy optimization method and the strategy priority method are investigated for operation optimization with the guarantee of the timeliness and stability of the copper flotation industrial process as much as possible. Later, the universality and computational complexity of the proposed algorithms are analyzed. Finally, the effectiveness of the proposed methods is verified by the data experiments on a real-world copper flotation industrial process.

Iterative, Deep Synthetic Aperture Sonar Image Segmentation

Synthetic aperture sonar (SAS) systems produce high-resolution images of the seabed environment. Moreover, deep learning has demonstrated superior ability in finding robust features for automating imagery analysis. However, the success of deep learning is conditioned on having lots of labeled training data, but obtaining generous pixel-level annotations of SAS imagery is often practically infeasible. This challenge has thus far limited the adoption of deep learning methods for SAS segmentation. Algorithms exist to segment SAS imagery in an unsupervised manner, but they lack the benefit of state-of-the-art learning methods and the results present significant room for improvement. In view of the above, we propose a new iterative algorithm for unsupervised SAS image segmentation combining superpixel formation, deep learning, and traditional clustering methods. We call our method Iterative Deep Unsupervised Segmentation (IDUS). IDUS is an unsupervised learning framework that can be divided into four main steps: 1) A deep network estimates class assignments. 2) Low-level image features from the deep network are clustered into superpixels. 3) Superpixels are clustered into class assignments (which we call pseudo-labels) using $k$-means. 4) Resulting pseudo-labels are used for loss backpropagation of the deep network prediction. These four steps are performed iteratively until convergence. A comparison of IDUS to current state-of-the-art methods on a realistic benchmark dataset for SAS image segmentation demonstrates the benefits of our proposal even as the IDUS incurs a much lower computational burden during inference (actual labeling of a test image). Finally, we also develop a semi-supervised (SS) extension of IDUS called IDSS and demonstrate experimentally that it can further enhance performance while outperforming supervised alternatives that exploit the same labeled training imagery.

Resident user load classification method based on improved Gaussian mixture model clustering

In view of the limitation of “hard assignment” of clusters in traditional clustering methods and the difficulty of meeting the requirements of clustering efficiency and clustering accuracy simultaneously in regard to massive data sets, a load classification method based on a Gaussian mixture model combining clustering and principal component analysis is proposed. The load data are fed into a Gaussian mixture model clustering algorithm after principal component analysis and dimensionality reduction to achieve classification of large-scale load datasets. The method in this paper is used to classify loads in the Canadian AMPds2 public dataset and is compared with K-Means, Gaussian mixed model clustering and other methods. The results show that the proposed method can not only achieve load classification more effectively and finely, but also save computational cost and improve computational efficiency.

Improved Harris Combined With Clustering Algorithm for Data Traffic Classification

Aiming at the problem that the data traffic in the intelligent wireless communication system presents complex characteristics such as burstiness and self-similarity, which leads to the low classification accuracy of the existing classification model for traffic, a data traffic classification method based on improved Harris Eagle algorithm combined with fuzzy C-means clustering is proposed. The method maps traffic samples to Harris Eagle individuals, finds the optimal position through multiple iterations of the algorithm, and uses this as the initial center point of the clustering algorithm to guide data traffic classification. The simulation shows that, compared with the traditional fuzzy clustering method, the clustering method based on the particle swarm algorithm and the gray wolf algorithm, the improved Harris Eagle combined with fuzzy clustering has better intra-class compactness and inter-class separation on the data traffic sample set. Meanwhile, the clustering accuracy and recall rate are both improved to about 90 percent.

Deep Low-Rank Graph Convolutional Subspace Clustering for Hyperspectral Image

Deep subspace clustering (DSC) has achieved considerable success in the classification task of hyperspectral image (HSI) without background (defined as noisy samples) compared with traditional subspace clustering methods. Unfortunately, directly applying DSC to classify land-cover on HSI datasets with background may suffer from the degradation of classification performance. In this paper, we propose an effective deep low-rank graph convolutional subspace clustering (DLR-GCSC) framework for improving the performance of land-cover classification on HSI datasets with background. Specifically, we design a joint spatial-spectral network to extract band-level and patch-level features simultaneously by combining 1D and 2D auto-encoder. Moreover, we construct a low-rank constrained fully connected layer as self-expression layer in the network to make the joint features more discriminative. To reduce the influence of noisy samples and obtain an informative affinity matrix, we recast the joint features into a non-Euclidean domain by introducing graph convolution. Finally, spectral clustering is applied on the informative affinity matrix to obtain the classification results. Experiments on three benchmark HSI datasets show that our proposed method achieves competitive classification performance to the state-of-the-art methods on both HSI data with background and without background.

A novel data cluster algorithm based on linear regression and residual analysis for human resource management

Human resource management has become an important part of enterprise management. How to select high-quality talent and how to allocate corresponding talent to appropriate work has become an increasingly acute problem. Traditional data cluster methods cannot effectively solve the above problem due to the high-dimensional data. Therefore, we propose a novel data cluster algorithm based on linear regression and residual analysis for human resource management. Improved hybrid entropy weight attribute similarity is adopted for measuring the similarity between objects. The proposed local density calculation method based on k-nearest neighbour (KNN) and Parzen window is used to calculate the density of each object. Then, we utilise the linear regression and residual analysis to select the clustering centre points quickly and automatically, which can eliminate the subjectivity of artificial selection. A new clustering centre objective optimisation model is proposed to determine the real clustering centre. Through theoretical analysis and comparative experiments on artificial data sets and real data sets, it shows that the proposed cluster algorithm can overcome the defects of the original algorithms, and achieve better clustering effect and lower computation time than state-of-the-art methods.