Clustering Quality Evaluation Research Articles

An integrated image visibility graph and topological data analysis for extracting time series features

A time series can often be characterized using machine learning techniques, which require feature vectors as input. The quality of the feature vectors reflects the accuracy of the utilized machine learning techniques. We propose a method for combining features extracted from two popular techniques, one from the persistence diagram, a summary of topological data analysis, and another from the visibility graph, a popular approach for constructing complex networks. We further present a method for extracting features from time series by transforming the series into a two-dimensional array and applying a 2-dimensional visibility graph. The 2-dimensional visibility graph improves the quality of the features for clustering. The features extracted from the techniques are utilized for clustering 1100 time series samples simulated from 11 classes. The two clustering quality evaluation metrics namely normalized mutual info score and adjusted rand score, which are scores given to clustering algorithms, are used to measure the clustering quality. Inclusion of 2-dimensional visibility graph features increases the values of these metrics for classifying the time series. Therefore, the results confirm the usefulness of including a 2-dimensional visibility graph from time series clustering and analysis.

An Integrated Cluster Detection, Optimization, and Interpretation Approach for Financial Data.

In many financial applications, such as fraud detection, reject inference, and credit evaluation, detecting clusters automatically is critical because it helps to understand the subpatterns of the data that can be used to infer user's behaviors and identify potential risks. Due to the complexity of human behaviors and changing social environments, the distributions of financial data are usually complex and it is challenging to find clusters and give reasonable interpretations. The goal of this study is to develop an integrated approach to detect clusters in financial data, and optimize the scope of the clusters such that the clusters can be easily interpreted. Specifically, we first proposed a new cluster quality evaluation criterion, which is free from large-scale computation and can guide base clustering algorithms such as k -Means to detect hyperellipsoidal clusters adaptively. Then, we designed a new solver for a revised support vector data description model, which efficiently refines the centroids and scopes of the detected clusters to make the clusters tighter such that the data in the clusters share greater similarities, and thus, the clusters can be easily interpreted with eigenvectors. Using ten financial datasets, the experiments showed that the proposed algorithm can efficiently find reasonable number of clusters. The proposed approach is suitable for large-scale financial datasets whose features are meaningful, and also applicable to financial mining tasks, such as data distribution interpretation and anomaly detection.

Pengelompokan Pelanggan Toko Kerajinan Menggunakan K-Means dengan Model RFM dan LRFM

Customer groups within a company or shop are important to identify for determine the right sales strategy in a competitive market. This research was conducted at one of the online handicraft shops in Bali which was affected by the intense competition in the market. The handicraft industry in Bali is considered to provide a large contribution to the community's economy. One solution that can be done is to identify the customer groups in the store using the clustering technique. This research aims to get the best customer data cluster and model. The stages in this research start from preprocessing data on historical data and customer orders to generate data model Recency, Frequency, Monetary (RFM) and Length, Recency, Frequency, Monetary (LRFM), then the clustering process with K-Means and evaluation of cluster quality with Silhouette Coefficient (SC). Results Based on the research, the RFM model becomes a data model that produces the best clustering results with an SC value of 0.545 when k = 2 which is included in the medium structure. LRFM model only produces the largest SC value when k = 3 with a value of 0.415. The SC value calculated from each data model tends to increase when the percentage of data is added. Cluster 1 has 817 members with the last transaction taking a long time, but has a below average ordering and monetary frequency. Cluster 2 has 158 members, making the last transaction at the most recent time, and has an average order and monetary frequency.
 
 Keywords: Clustering, Customer, Craft, K-Means, RFM, LRFM, Silhouette Coefficient

Trajectory clustering and query processing analysis framework for knowledge discovery in cloud environment

AbstractIn a cloud environment, resources should be acquired quickly and automatically released at runtime. Traditional trajectory data partitions, indexing and query processing techniques are extended, so that they can take advantage of the cloud of large clusters of highly parallel processing capabilities. There are ways with trajectory data in the cloud database query processing. The advanced sensing techniques available today caused the existence of many types of trajectory datasets whose study can form a strong basis for decision making regarding a data context. Trajectory data generally includes trajectory classification, trajectory clustering, and trajectory associations and so on. All these tools need trajectory similarity measures to get comparisons in trajectory data. Some of the popular existing methods of trajectory similarity include Euclidean distance, semantic distance and there near variants. Each variant of these techniques exhibits notable differences in measurement of similarity and computational difficulty. This paper targeted to deal with this situation with the goal of lower computational efforts in trajectory similarity computations. A two‐phase trajectory similarity measure is defined. The first phase does point level analysis of trajectory points representing all trajectories and the following phase use the analysis to find the similarities. Using a hybrid similarity framework, new means for trajectory clustering and trajectory query processing are developed. From trajectory analysis followed by trajectory clustering, evaluation of cluster quality and processing of trajectory query are undertaken in this paper in different ways from the existing methods. All the proposed developments are tested with trajectory datasets with multiple attributes and the results are more favour to proposed framework than the existing ones.

Cluster Analysis of Hospital Inpatient Service Efficiency Based on BOR, BTO, TOI, AvLOS Indicators using Agglomerative Hierarchical Clustering

Purpose: The research proposed an approach for grouping hospital inpatient service efficiency that have the same characteristics into certain clusters based on BOR, BTO, TOI, and AvLOS indicators using Agglomerative Hierarchical Clustering.Design/methodology/approach: Applying Agglomerative Hierarchical Clustering with dissimilarity measures such as single linkage, complete linkage, average linkage, and ward linkage.Findings/result: The experiment result has shown that ward linkage was given a quite good score of silhouette coefficient reached 0.4454 for the evaluation of cluster quality. The cluster formed using ward linkage was more proportional than the other dissimilarity measures. Ward linkage has generated cluster 0 consists of 23 members, cluster 1 consists of 34 members, while both of cluster 2 and 3 consists of only 1 member respectively. The experiment reported that each cluster had problems with inpatient indicators that were not ideal and even exceeded the ideal limit, but cluster 0 generated the ideal BOR and TOI parameters, both reached 52.17% (12 of 23 hospital inpatient) and 78.36% (18 of 23 hospital inpatient) respectively.Originality/value/state of the art: Based on previous research, this study provides an alternative to produce more proportional, representative and quality clusters in mapping hospital inpatient service efficiency that have the same characteristics into certain clusters using Agglomerative Hierarchical Clustering Method compared to the K-means Clustering Method which is often trapped in local optima.

Feature selection for unsupervised machine learning of accelerometer data physical activity clusters - A systematic review.

Identifying clusters of physical activity (PA) from accelerometer data is important to identify levels of sedentary behaviour and physical activity associated with risks of serious health conditions and time spent engaging in healthy PA. Unsupervised machine learning models can capture PA in everyday free-living activity without the need for labelled data. However, there is scant research addressing the selection of features from accelerometer data. The aim of this systematic review is to summarise feature selection techniques applied in studies concerned with unsupervised machine learning of accelerometer-based device obtained physical activity, and to identify commonly used features identified through these techniques. Feature selection methods can reduce the complexity and computational burden of these models by removing less important features and assist in understanding the relative importance of feature sets and individual features in clustering. We conducted a systematic search of Pubmed, Medline, Google Scholar, Scopus, Arxiv and Web of Science databases to identify studies published before January 2021 which used feature selection methods to derive PA clusters using unsupervised machine learning models. A total of 13 studies were eligible for inclusion within the review. The most popular feature selection techniques were Principal Component Analysis (PCA) and correlation-based methods, with k-means frequently used in clustering accelerometer data. Cluster quality evaluation methods were diverse, including both external (e.g. cluster purity) or internal evaluation measures (silhouette score most frequently). Only four of the 13 studies had more than 25 participants and only four studies included two or more datasets. There is a need to assess multiple feature selection methods upon large cohort data consisting of multiple (3 or more) PA datasets. The cut-off criteria e.g. number of components, pairwise correlation value, explained variance ratio for PCA, etc. should be expressly stated along with any hyperparameters used in clustering.

Cluster Data Analysis with a Fuzzy Equivalence Relation to Substantiate a Medical Diagnosis

This study aims to develop a methodology for the justification of medical diagnostic decisions based on the clustering of large volumes of statistical information stored in decision support systems. This aim is relevant since the analyzed medical data are often incomplete and inaccurate, negatively affecting the correctness of medical diagnosis and the subsequent choice of the most effective treatment actions. Clustering is an effective mathematical tool for selecting useful information under conditions of initial data uncertainty. The analysis showed that the most appropriate algorithm to solve the problem is based on fuzzy clustering and fuzzy equivalence relation. The methods of the present study are based on the use of this algorithm forming the technique of analyzing large volumes of medical data due to prepare a rationale for making medical diagnostic decisions. The proposed methodology involves the sequential implementation of the following procedures: preliminary data preparation, selecting the purpose of cluster data analysis, determining the form of results presentation, data normalization, selection of criteria for assessing the quality of the solution, application of fuzzy data clustering, evaluation of the sample, results and their use in further work. Fuzzy clustering quality evaluation criteria include partition coefficient, entropy separation criterion, separation efficiency ratio, and cluster power criterion. The novelty of the results of this article is related to the fact that the proposed methodology makes it possible to work with clusters of arbitrary shape and missing centers, which is impossible when using universal algorithms. Doi: 10.28991/esj-2021-01305 Full Text: PDF

A practical approach to cluster validation in the energy sector

With increasing digitization, new opportunities emerge concerning the availability and use of data in the energy sector. A comprehensive literature review shows an abundance in available unsupervised clustering algorithms as well as internal, relative and external cluster validation indices (cvi) to evaluate the results. Yet, the comparison of different clustering results on the same dataset, executed with different algorithms and a specific practical goal in mind still proves scientifically challenging. A large variety of cvi are described and consolidated in commonly used composite indices (e.g. Davies-Bouldin-Index, silhouette-Index, Dunn-Index). Previous works show the challenges surrounding these composite indices since they serve a generalized cluster quality evaluation. However, this does not suit individual clustering goals in many cases. The presented paper introduces the current state of science, existing cluster validation indices and proposes a practical method to combine them to an individual composite index, using Multi Criteria Decision Analysis (mcda). The methodology is applied on two energy economic use cases for clustering load profiles of bidirectional electric vehicles and municipalities.

Analisa Performa Klastering Data Besar pada Hadoop

Big Data is a collection of data with a large and complex size, consisting of various data types and obtained from various sources, overgrowing quickly. Some of the problems that will arise when processing big data, among others, are related to the storage and access of big data, which consists of various types of data with high complexity that are not able to be handled by the relational model. One technology that can solve the problem of storing and accessing big data is Hadoop. Hadoop is a technology that can store and process big data by distributing big data into several data partitions (data blocks). Problems arise when an analysis process requires all data spread out into one data entity, for example, in the data clustering process. One alternative solution is to do a parallel and scattered analysis, then perform a centralized analysis of the results of the scattered analysis. This study examines and analyzes two methods, namely K-Medoids Mapreduce and K-Modes without Mapreduce. The dataset used is a dataset about cars consisting of 3.5 million rows of data with 400MB distributed in a Hadoop Cluster (consisting of more than one engine). Hadoop has a MapReduce feature, consisting of 2 functions, namely map and reduce. The map function performs a selection to retrieve a key, value pairs, and returns a value in the form of a collection of key value pairs, and then the reduce function combines all key value pairs from several map functions. The results of the cluster quality evaluation are tested using the Silhouette Coefficient testing metric. The K-Medoids MapReduce algorithm for the car dataset gives a silhouette value of 0.99 with a total of 2 clusters.

Clustering assessment in weighted networks.

We provide a systematic approach to validate the results of clustering methods on weighted networks, in particular for the cases where the existence of a community structure is unknown. Our validation of clustering comprises a set of criteria for assessing their significance and stability. To test for cluster significance, we introduce a set of community scoring functions adapted to weighted networks, and systematically compare their values to those of a suitable null model. For this we propose a switching model to produce randomized graphs with weighted edges while maintaining the degree distribution constant. To test for cluster stability, we introduce a non parametric bootstrap method combined with similarity metrics derived from information theory and combinatorics. In order to assess the effectiveness of our clustering quality evaluation methods, we test them on synthetically generated weighted networks with a ground truth community structure of varying strength based on the stochastic block model construction. When applying the proposed methods to these synthetic ground truth networks’ clusters, as well as to other weighted networks with known community structure, these correctly identify the best performing algorithms, which suggests their adequacy for cases where the clustering structure is not known. We test our clustering validation methods on a varied collection of well known clustering algorithms applied to the synthetically generated networks and to several real world weighted networks. All our clustering validation methods are implemented in R, and will be released in the upcoming package clustAnalytics.

Optimizing Text Categorization for Indonesian Text Using Clustering Label Technique

Text Categorization plays an important role for clustering the rapidly growing, yet unstructured, Indonesian text in digital format. Furthermore, it is deemed even more important since access to digital format text has become more necessary and widespread. There are many clustering algorithms used for text categorization. Unfortunately, clustering algorithms for text categorization cannot easily cluster the texts due to imperfect process of stemming and stopword of Indonesian language. This paper presents an intelligent system that categorizes Indonesian text documents into meaningful cluster labels. Label Induction Grouping Algorithm (LINGO) and Bisecting K- means are applied to process it through five phases, namely the pre-processing, frequent phrase extraction, cluster label induction, content discovery and final cluster formation. The experimental result showed that the system could categorize Indonesian text and reach to 93%. Furthermore, clustering quality evaluation indicates that text categorization using LINGO has high Precision and Recall with a value of 0.85 and 1, respectively, compare to Bisecting K-means which has a value of 0.78 and 0.99. Therefore, the result shows that LINGO is suitable for categorizing Indonesian text. The main contribution of this study is to optimize the clustering results by applying and maximizing text processing using Indonesian stemmer and stopword.

Recognizing single-trial motor imagery EEG based on interpretable clustering method

This work explores an approach for single-trial motor imagery (MI) electroencephalography (EEG) classification in interpretable clustering. The tensor structured EEG data under Mu rhythm is first processed by Common Spatial Subspace Decomposition (CSSD) to obtain the multi-dimensional CSSD-mapped EEG. In the dimensional feature reduction, Fisher’s ratio is used as the cost function to automatically find the optimal projection plane with two feature vectors of CSSD-mapped EEG corresponding to the largest Fisher’s ratio. Then, Discriminative Rectangle Mixture Model (DRMM) that gives a rectangular decision rule is used to identify optimal feature vectors to realize single-trial motor imagery EEG classification in an interpretable way. This innovative data analysis model generates reasonable cluster results driven by optimal feature data. The probability density distribution functions of EEG data in two classes can effectively explain the reliability of the rectangular decision rule given by the DRMM. The proposed method has been validated using the areas under the receiver operating characteristic curve (AUC) and cluster quality evaluation metrics. Experimental results demonstrate its performance is comparable to existing clustering and gives interpretable clustering results when detecting the motor intention involving EEG signals. This paper provides a novelty method based on interpretable clustering for single-trial MI EEG classification. And it may promote the development of BCI application.

An inconsistency assessment method for backup battery packs based on time-series clustering

Abstract Slight differences in the production process and operation environment of individual cells in a battery pack result in inconsistencies from cell to cell, which become increasingly severe as the battery pack service time increases. Effectively assessing the inconsistency in a battery pack helps improve estimations of its state of charge, service condition, and degree of aging. However, it is difficult to recognize inconsistencies, especially for battery packs with low discharge frequencies, where there is no explicit parameter to directly measure inconsistency. Considering the available voltage as an evaluation factor, this paper proposes a method of inconsistency assessment for battery packs based on a clustering quality evaluation index that is applied to time-series data. First, a time-series model of an individual cell's voltage is created and the pattern distance is used to measure voltage differences between cells. Then, the k-medoids clustering algorithm is applied to realize each unit cell's cluster classification. Finally, the Davie–Bouldin clustering quality evaluation index is used to determine the inconsistency in the battery pack. To validate the proposed method, an example is demonstrated to determine the real capacity of individual cells. The results indicate that this method can accurately measure the inconsistency in a battery pack.

Learning robust affinity graph representation for multi-view clustering

Recently, an increasingly pervasive trend in real-word applications is that the data are collected from multiple sources or represented by multiple views. Owing to the powerful ability of affinity graph in capturing the structural relationships among samples, constructing a robust and meaningful affinity graph has been extensively studied, especially in spectral clustering tasks. However, conventional spectral clustering extended to multi-view scenarios cannot obtain the satisfactory performance due to the presence of noise and the heterogeneity among different views. In this paper, we propose a robust affinity graph learning framework to deal with this issue. First, we employ an improved feature selection algorithm that integrates the advantages of hypergraph embedding and sparse regression to select significant features such that more robust graph Laplacian matrices for various views on this basis can be constructed. Second, we model hypergraph Laplacians as points on a Grassmann manifold and propose a Consistent Affinity Graph Learning (CAGL) algorithm to fuse all views. CAGL aims to learn a latent common affinity matrix shared by all Laplacian matrices by taking both the clustering quality evaluation criterion and the view consistency loss into account. We also develop an alternating descent algorithm to optimize the objective function of CAGL. Experiments on five publicly available datasets demonstrate that our proposed method obtains promising results compared with state-of-the-art methods.

Sigmoid Wake Probability Model for High-Resolution Detection of Drowsiness Using Electroencephalogram.

An efficient and reliable method to detect drowsiness can reduce accidents and injuries related to drowsy driving. However, existing systems for detecting drowsiness are often of low-resolution, expensive, and dependent on external parameters. Therefore, the goal of this study is to develop a high-resolution and efficient drowsiness detection algorithm using relatively less noisy sleep study data. To this end, we recorded electroencephalogram (EEG) from 53 subjects during a sleep study and leveraged the EEG frequency band changes at sleep onset to develop a model for drowsiness detection. The model devised herein provided a likelihood of wakefulness for 3-s signal segments. By choosing appropriate thresholds of the model output, we have identified three clusters that represent wakefulness, drowsiness, and, sleep. The proposed scheme has been validated using arousals which are cases of alertness and deep sleep segments, cluster quality evaluation metrics, graphical, and statistical analyses. The results presented in this work suggest that spectral properties of EEG can be utilized for high-resolution drowsiness detection in sleep study. Upon its successful validation in a driving study, the proposed model can lead to the development of an efficient drowsy driving monitoring system.

Using Real-Time Data and Unsupervised Machine Learning Techniques to Study Large-Scale Spatio–Temporal Characteristics of Wastewater Discharges and their Influence on Surface Water Quality in the Yangtze River Basin

Most worldwide industrial wastewater, including in China, is still directly discharged to aquatic environments without adequate treatment. Because of a lack of data and few methods, the relationships between pollutants discharged in wastewater and those in surface water have not been fully revealed and unsupervised machine learning techniques, such as clustering algorithms, have been neglected in related research fields. In this study, real-time monitoring data for chemical oxygen demand (COD), ammonia nitrogen (NH3-N), pH, and dissolved oxygen in the wastewater discharged from 2213 factories and in the surface water at 18 monitoring sections (sites) in 7 administrative regions in the Yangtze River Basin from 2016 to 2017 were collected and analyzed by the partitioning around medoids (PAM) and expectation–maximization (EM) clustering algorithms, Welch t-test, Wilcoxon test, and Spearman correlation. The results showed that compared with the spatial cluster comprising unpolluted sites, the spatial cluster comprised heavily polluted sites where more wastewater was discharged had relatively high COD (>100 mg L−1) and NH3-N (>6 mg L−1) concentrations and relatively low pH (<6) from 15 industrial classes that respected the different discharge limits outlined in the pollutant discharge standards. The results also showed that the economic activities generating wastewater and the geographical distribution of the heavily polluted wastewater changed from 2016 to 2017, such that the concentration ranges of pollutants in discharges widened and the contributions from some emerging enterprises became more important. The correlations between the quality of the wastewater and the surface water strengthened as the whole-year data sets were reduced to the heavily polluted periods by the EM clustering and water quality evaluation. This study demonstrates how unsupervised machine learning algorithms play an objective and effective role in data mining real-time monitoring information and highlighting spatio–temporal relationships between pollutants in wastewater discharges and surface water to support scientific water resource management.

Automatic Determination of K in Distributed K-Means Clustering

Abstract Traditional K-Means based distributed data clustering require number of clusters as input which is difficult to obtain in case of a real life application like wireless sensor network. To mitigate this issue here an Automatic Distributed K-Means (ADK-Means) algorithm is proposed. In this algorithm cluster assignment is carried out with point symmetry based distance instead of Euclidean distance, to effectively detect arbitrary shaped clusters. The performance of proposed algorithm is demonstrated on four synthetic and one practical dataset. Three cluster quality evaluation methods have been employed and comparative analysis is done with existing distributed K-Means. The improved performance is up to 12.6% in terms of Silhouette index reported as compared to existing approach.

Parameter independent clustering based on dominant sets and cluster merging

Clustering is an important unsupervised learning approach with wide application in data mining, pattern recognition and intelligent information processing. However, existing clustering algorithms usually involve one or more user-specified parameters as input and their clustering results depend heavily on these parameters. In order to solve this problem, we present a parameter independent clustering algorithm based on the dominant sets algorithm and cluster merging. In the first step histogram equalization transformation is applied to solve the parameter dependence problem of the dominant sets algorithm. We provide the theoretic foundation of this method and discuss the implementation details. The clustering result is then refined with a cluster merging method, which is based on a new clustering quality evaluation criterion. We use extensive experiments on several datasets to validate each step and the whole procedures of our algorithm. It is shown that our parameter independent algorithm performs comparably to some existing clustering algorithms which benefit from user-specified parameters.

Shape classification using invariant features and contextual information in the bag-of-words model

In this paper, we describe a classification framework for binary shapes that have scale, rotation and strong viewpoint variations. To this end, we develop several novel techniques. First, we employ the spectral magnitude of log-polar transform as a local feature in the bag-of-words model. Second, we incorporate contextual information in the bag-of-words model using a novel method to extract bi-grams from the spatial co-occurrence matrix. Third, a novel metric termed ‘weighted gain ratio’ is proposed to select a suitable codebook size in the bag-of-words model. The proposed metric is generic, and hence it can be used for any clustering quality evaluation task. Fourth, a joint learning framework is proposed to learn features in a data-driven manner, and thus avoid manual fine-tuning of the model parameters. We test our shape classification system on the animal shapes dataset and significantly outperform state-of-the-art methods in the literature.

Document Clustering: A Review

As the internet is exploding with huge volume of text documents, the need of grouping similar documents together for versatile applications have hold the attention of researchers in this area. Document clustering can facilitate the tasks of document organization and web browsing, search engine results, corpus summarization, documents classification, information retrieval and filtering. However several attempts have been made to develop efficient document clustering algorithms but most of the clustering methods suffer from challenges in dealing with problems of high dimensionality, scalability, accuracy and meaningful cluster labels. This paper intends to provide a brief summary over methods studied and current state of documents clustering research, including basic traditional methods as well as advanced fuzzy based, GA, PSO, HS oriented techniques etc. Also document representation model and its challenges, dimensionality reduction mechanisms, issues in document clustering, and cluster quality evaluation criteria are discussed.