Good Clustering Quality Research Articles

Are heterogeinity and conflicting preferences no longer a problem? Personality-based dynamic clustering for group recommender systems

The complexity associated with groups of tourists led to the emergence of Group Recommender Systems (GRS) for tourism. But if generating recommendations for small groups is a complex task, to provide them to large and occasional groups is even more. This complexity is especially due to the group’s heterogeinity, conflicting preferences, the information overload found on the internet and the tourists’ different ways of coping with the information, hindering the recommendation process from the users’ profile construction to the final recommendation of a list of points of interest to visit. In this work, we show how we tackled the identified issues in a GRS prototype, Grouplanner, including the cold-start problem, by predicting the tourists’ preferences based only on their personality and dividing the main group into subgroups of similar personality; by using a Multi-Agent Microservice; a novel dynamic clustering algorithm, d-means, adapted from the k-means algorithm, that does not need to know the number of clusters a priori; and association rules. Using a personality dataset of n=100k users, the proposed d-means algorithm was tested against two baselines (k-means and k-means++), showing better results in the clustering quality and scalability. We were also able to determine a large set of association rules to refine the recommendations, although further improvements are needed. To test the Grouplanner prototype, a simulation with real users (n=35) was conducted. The results showed the subgroups formed were very compact, revealing a very good clustering quality, with an average silhouette of s = 0.91. 11 of the 15 proposed tourist preferences were successfully predicted and used for the preliminary recommendation lists, being 92 % of the participants satisfied with the individual recommendations and 96 % with the group recommendations.

Preprocessed Spectral Clustering with Higher Connectivity for Robustness in Real-World Applications

This paper introduces a novel model for spectral clustering to solve the problem of poor connectivity among points within the same cluster as this can negatively impact the performance of spectral clustering. The proposed method leverages both sparsity and connectivity properties within each cluster to find a consensus similarity matrix. More precisely, the proposed approach considers paths of varying lengths in the graph, computing a similarity matrix for each path, and generating a cluster for each path. By combining these clusters using multi-view spectral clustering, the method produces clusters of good quality and robustness when there are outliers and noise. The extracted multiple independent views from different paths in the graph are integrated into a consensus graph. The performance of the proposed method is evaluated on various benchmark datasets and compared to state-of-the-art techniques.

Students’ Motivation for Honors Programs in the Netherlands

This study aimed to examine student motivation for honors programs using two types of motivation (autonomous and controlled) of Dutch traditional students, Dutch non-traditional students, and international students. Data were collected using Academic Self-Regulation Questionnaire (SRQ-A). K-means cluster analysis with autonomous and controlled motivation was conducted to identify the various types of motivation clusters. Subsequently, motivational profiles were compared using Analysis of Variance between Dutch traditional students, non-traditional students, and international students. Furthermore, associations between background variables and relative autonomous motivation of students to participate in the honors program were conducted. The response rate was 33.25% ( n = 399). Four motivational profiles were identified: (1) HAMC (High Autonomous and Moderate Controlled), (2) HALC (High Autonomous and Low Controlled), (3) MALC (Moderate Autonomous and Low Controlled), (4) and MAMC (Moderate Autonomous and Moderate Controlled). Most of the honors program students in this study were represented in the good quality-HALC motivational profile. The findings also showed higher proportions of males ( n = 71, 48%) and females ( n = 120, 49.4%) in the good quality-HALC motivation profiles. Almost half of the students were found in the good quality cluster.

Automatic identification system-based trajectory clustering framework to identify vessel movement pattern

<span lang="EN-US">Automatic identification system (AIS) is a vessel radio navigation equipment that has been determined by international maritime organization (IMO). Historical AIS data can be utilized for anomaly detection, trajectory prediction, and vessel trajectory planning. These benefits can be achieved by identifying the vessel's trajectory pattern through trajectory clustering. However, more effort is needed in trajectory clustering using AIS data due to their large volume and the significant number of deficiencies. In addition, trajectory clustering cannot be directly applied to trajectory data, which also applies to vessel trajectory. Therefore, we propose a trajectory clustering framework by combining douglas peucker (DP), longest common subsequence (LCSS), multi-dimensional scaling (MDS), and density-based spatial clustering of applications with noise (DBSCAN). Our experiments, carried out with AIS data for the Lombok Strait, Indonesia, showed that the trajectory compression with DP significantly accelerates the similarity measurement process. Moreover, we found that the LCSS is the optimal algorithm for similarity measurement of vessel trajectories based on AIS data. We also applied the right combination of MDS and DBSCAN in density-based clustering. The proposed framework can distinguish trajectoriess in different directions, identify the noise, and produce good quality clusters in relatively fast total processing time.</span>

Quality Attributes Prediction of Flame Seedless Grape Clusters Based on Nutritional Status Employing Multiple Linear Regression Technique

Flame Seedless grape is considered one of the most popular and favorite grapes for consumers, since it ripens early, and has good cluster quality. Flame seedless grape marketing value depends upon its desirable appearance, berry, cluster size, and shape. Therefore, it is imperative that the cluster yield and quality are enhanced to ensure profitability. In this study, the prediction of physical characteristics of clusters and berries’ color attributes of Flame Seedless grape grown under different culture practices, in particular fertilization treatments, was carried out using nutritional status concentration (leaf mineral elements, total chlorophyll content, total carotenoids content) and multiple linear regression (MLR). The method was based on the development of two indices: the first is called index 1 (%) and was formulated by combing the mineral elements of N, P, K, Ca, and Mg concentrations; and the second is called index 2 (ppm) and was formulated by combing the elements of Fe, Cu, Mn, Zn, and B concentrations in leaf petioles. The results indicated that the established MLR models can obtain variation accuracy, based on values of coefficients of determination (R2) using the test set. The R2 values were in the range of 0.9286 to 0.9972 for cluster weight, cluster length, shoulder length, berries’ color attributes (L*, a*, b*, chroma, hue, and color index for red grapes (CIRG)). This study highlighted that during a grown season, leaf mineral elements, total chlorophyll content, and total carotenoids coupled with a MLR model can be used successfully to evaluate the physical characteristics of the cluster and berries’ color attributes of Flame seedless grape. This method is easy, fast and reliable as it retains the physical appearance of the fruits by adjusting the concentration of mineral elements, total chlorophyll content, and total carotenoids in leaves. Moreover, total chlorophyll had the greatest weight of all the predicted quality attributes.

A Cluster Analysis of Population Based Cancer Registry in Brunei Darussalam: An Exploratory

Machine learning techniques have been mostly applied in gene expression cancer data. Socio-demographic data available in cancer registries could be explored, to get further insight into relationships between cancer types and their contributing factors. Moreover, less attention has been paid to analyse the mixed demographic data (numeric and categorical) from cancer registries and its association to the cancer types. The aim of this study is to identify subgroups of patients, having similar demographics characteristics, from the population based cancer registry in Brunei Darussalam and examine the prevalent cancer types in these subgroups. Four clustering algorithms are explored in the cluster analysis of Brunei Darussalam Cancer Registry; Two-step, Partitional Around Medoid, Agglomerative Hierarchical and Model-based. Gower distance was used for measuring similarity for mixed data types. To evaluate the clusters found; cluster distribution and Silhouette index were used for cluster quality, Cohen's Kappa Index for cluster stability and Cramer's V Coefficient for clinical relevance of clusters. Six distinct demographic subgroups were consistently found by three algorithms while model-based clustering solution were not considered for deeper analysis as highly imbalanced clusters were produced. The subgroups found have good quality clusters, moderate association with cancer types and high stability. The top three prevalent cancers associated with these subgroups were consistently identified using the three algorithms. Upon comparing the subgroups’ ages during diagnosis, we identify possible screening behaviours of specific subgroups, suggesting for early screening awareness programmes. This study demonstrates the use of cluster analysis in a cancer registry to identify demographic subgroups that could suggest potential areas to develop targeted and improved healthcare management strategies.

Analysis of the Effectiveness of Online Learning Using Eda Data Science and Machine Learning

In educational institutions, it is important to evaluate the performance of lecturers. One aspect that can be used as a reference in assessing the performance of lecturers is the assessment in terms of teaching (learning - teaching process). Value data in the learning evaluation process can be further processed to classify or map the results of the learning evaluation. In this study, the authors use Data Science and Exploratory Data Analysis (EDA) techniques and apply the K-Means++ clustering algorithm using the Java programming language to determine the value of the learning process. group learning evaluation data. To determine the number of clusters to be formed, the elbow method is applied. Meanwhile, to measure the quality of each cluster formed, the silhouette coefficient method is applied. The elbow graph can visualize the Sum of Square Error (SSE) value as a way to determine the best k value in the clustering process. From the tests carried out 12 times, using 6 datasets with variations in 4 areas of assessment, 10 tests resulted in a value of k = 3, 10 tests produced a value of k = 4, and 1 test resulted in a value of k = 5 as the best number of clusters. In the cluster quality measurement process, 13 tests have good clustering quality, because on average they produce clusters that have a good structure with a Global Silhouette Coefficient > 0.65.

Proposal for a Pivot-Based Vehicle Trajectory Clustering Method

Given the large volume of georeferenced information generated and stored by many types of devices, the study and improvement of techniques capable of operating with these data is an area of great interest. The analysis of vehicular trajectories with the aim of forming clusters and identifying emerging patterns is very useful for characterizing and analyzing transportation flows in cities. This paper presents a new trajectory clustering method capable of identifying clusters of vehicular sub-trajectories in various sectors of a city. The proposed method is based on the use of an auxiliary structure to determine the correct location of the centroid of each group or set of sub-trajectories along the adaptive process. The proposed method was applied on three real databases, as well as being compared with other relevant methods, achieving satisfactory results and showing good cluster quality according to the Silhouette index.

Data normalization for training and analysis of the MaskRCNN model using the k-means method for computer vision of smart refrigerator

This paper proposes using the k-means method for the controlled adjustment of the training sample for semantic image segmentation in the artificial vision of a smart refrigerator. To solve this problem, a new two-stage architecture for computer vision is proposed. In the proposed architecture, various sets of settings for optimizing the contrast of images are used to classify pixels according to their belonging to fragments of the studied image. Extensive experimental evaluation shows that the proposed method has critical advantages over existing work. Firstly, the obtained pixel classes can be directly clustered into semantic groups using k-means. Secondly, the method can be used for additional training of artificial intelligence in solving the semantic segmentation problem. The developers propose an approach to the correct choice of the number k of centroids to obtain good quality clusters, which is difficult to determine at a high k value. To overcome the problem of initializing the k-means method, an incremental k-means clustering method is proposed, which improves the quality of clusters to reduce the sum of squared errors. Comprehensive experiments have been carried out compared to the traditional k-means algorithm and its new versions to evaluate the performance of the proposed method on synthetically generated datasets and some real-world datasets.

A Scalable Unsupervised Classification Method Using Rough Set for Remote Sensing Imagery

Reference to geographic scale and geographic space representation are characteristics of geospatial data. This work has discussed two issues related to satellite image data, namely huge size and mixed pixels. In clustering, an unsupervised classification and a set of similar objects are grouped together based on the similarity measures. The similarity between intracluster objects is high, whereas the similarity between intercluster objects is low. This paper proposes a clustering technique called spatial rough k-means that classifies the mixed pixels based on their spatial neighbourhood relationship. The authors compared the performance of different state-of-the-art clustering algorithms with that of proposed algorithms for image partitioning and map-reduce methods. The results show that the proposed algorithm has produced clusters of better quality than state-of-the-art algorithms in both the approaches used for handling the vast input data size. Experiments conducted on Landsat-TM 5 data of Delhi region demonstrate the effectiveness of the proposed work.

Density-Based Top-K Spatial Textual Clusters Retrieval

So-called spatial web queries retrieve web content representing points of interest, such that the points of interest have descriptions that are relevant to query keywords and are located close to a query location. Two broad categories of such queries exist. The first encompasses queries that retrieve single spatial web objects that each satisfy the query arguments. Most proposals belong to this category. The second category, to which this paper's proposal belongs, encompasses queries that support exploratory user behavior and retrieve sets of objects that represent regions of space that may be of interest to the user. Specifically, the paper proposes a new type of query, the top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> spatial textual cluster retrieval ( <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -STC) query that returns the top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> clusters that (i) are located close to a query location, (ii) contain objects that are relevant with regard to given query keywords, and (iii) have an object density that exceeds a given threshold. To compute this query, we propose a DBSCAN-based approach and an OPTICS-based approach that rely on on-line density-based clustering and that exploit early stop conditions. Empirical studies on real data sets offer evidence that the paper's proposals can find good quality clusters and are capable of excellent performance.

A New Validity Index Based on Fuzzy Energy and Fuzzy Entropy Measures in Fuzzy Clustering Problems.

Two well-known drawbacks in fuzzy clustering are the requirement of assigning in advance the number of clusters and random initialization of cluster centers. The quality of the final fuzzy clusters depends heavily on the initial choice of the number of clusters and the initialization of the clusters, then, it is necessary to apply a validity index to measure the compactness and the separability of the final clusters and run the clustering algorithm several times. We propose a new fuzzy C-means algorithm in which a validity index based on the concepts of maximum fuzzy energy and minimum fuzzy entropy is applied to initialize the cluster centers and to find the optimal number of clusters and initial cluster centers in order to obtain a good clustering quality, without increasing time consumption. We test our algorithm on UCI (University of California at Irvine) machine learning classification datasets comparing the results with the ones obtained by using well-known validity indices and variations of fuzzy C-means by using optimization algorithms in the initialization phase. The comparison results show that our algorithm represents an optimal trade-off between the quality of clustering and the time consumption.

Two-level clustering of UML class diagrams based on semantics and structure

The reuse of software design has been an important issue of software reuse. UML class diagrams are widely applied in software design and has become DE factor standard. As a result, the reuse of UML class diagrams has received more attention. With the increasing number of class diagrams stored in reuse repository, their retrieval becomes a time-consuming job. The clustering can narrow down retrieval range and improve the retrieval efficiency. But few efforts have been done in clustering UML class diagrams. This paper tries to propose a clustering approach for UML class diagrams. This paper proposes a two-level clustering of UML class diagrams, namely, semantic clustering and structural clustering. The UML class diagrams stored in reuse repository are clustered into a few domains based on semantics in the first level and a few categories based on structure in the second level. We propose a clustering algorithm named CUFS , in which the idea of partitioning and hierarchical clustering is combined and feature similarity is proposed for the similarity measure between two clusters in order to merge clusters. A better feature representation of a cluster, namely, feature class diagram, is proposed in this paper. In order to form each sub-cluster, the semantic and structural similarities between UML class diagrams are defined, respectively. A series of experimental results show that, the proposed feature similarity measure not only speeds up the clustering process, but also expresses the closeness degree between clusters for merging clusters. The proposed algorithm shows a good clustering quality and efficiency under the condition of different size and distribution of UML class diagrams. It is concluded that the proposed two-level clustering method considers both semantics and structure contained in a class diagram, which can flexibly adapt to different clustering requirements. Also, the proposed clustering algorithm performs better than other related algorithms, regardless of in semantic, structural and hybrid clustering.

Segmentasi Buah Mangga Menggunakan MLE dan GMM Sebagai Klasterisasi Pixel

Segmentation divides an image into parts or segments that are simpler and more meaningful so they can be analyzed further. The solution that has been found is using the Maximum Likelihood Estimation (MLE) method and the Gausian Mixture Model. GMM is a clustering method. GMM is a function consisting of several Gaussian, each identified by k ∈ {1, ..., K}, where K is the number of clusters in our dataset. Maximum Likelihood estimation is a technique used to find a certain point to maximize a function, this technique is very widely used in estimating a data distribution parameter. Tests carried out using mango images with 10 different backgrounds. GMM will cluster the pixels of the mango image to produce averages and covariates. Then the average and covariance will be used by MLE to qualify each pixel of the mango image. In this study GMM and MLE tests were carried out to segment mangoes. Based on the results obtained, the GMM and MLE methods have an error rate of 13.07% for 3 clusters, 8.06% for 4 clusters, and 6.63% for 5 clusters and good cluster quality with silhouette coefficient values ​​of 0.37686 for 3 clusters, 0.29577 for 4 clusters, and 0.26162 for 5 clusters.

Automatic clustering algorithms: a systematic review and bibliometric analysis of relevant literature

Cluster analysis is an essential tool in data mining. Several clustering algorithms have been proposed and implemented, most of which are able to find good quality clustering results. However, the majority of the traditional clustering algorithms, such as the K-means, K-medoids, and Chameleon, still depend on being provided a priori with the number of clusters and may struggle to deal with problems where the number of clusters is unknown. This lack of vital information may impose some additional computational burdens or requirements on the relevant clustering algorithms. In real-world data clustering analysis problems, the number of clusters in data objects cannot easily be preidentified and so determining the optimal amount of clusters for a dataset of high density and dimensionality is quite a difficult task. Therefore, sophisticated automatic clustering techniques are indispensable because of their flexibility and effectiveness. This paper presents a systematic taxonomical overview and bibliometric analysis of the trends and progress in nature-inspired metaheuristic clustering approaches from the early attempts in the 1990s until today’s novel solutions. Finally, key issues with the formulation of metaheuristic algorithms as a clustering problem and major application areas are also covered in this paper.

Automatic Generation of Teachers’ Course Preferences Using Document Clustering

The current study examined the automated course preferences of teachers using document clustering. Data regarding teachers’ course preferences and course outlines were collected and preprocessed for further analysis. Two separate clustering solutions were generated for teachers and courses datasets. The clustering solution for teachers contained clusters of similar faculty members grouped together on the basis of their course preferences and courses taught by them in previous years. The clustering solution generated for courses contained the list of course outlines of assigned courses. Good quality clusters for both teachers and courses were generated using K-means clustering method in CLUTO software package. The generated clustering solutions were mapped for automated generation of course preferences for each teacher in the dataset. Precision, Recall and F-measure values were also reported and they indicated promising results.

M-pSC: a manifold p-spectral clustering algorithm

Since p-spectral clustering has good performance in many practical problems, it has attracted great attention. The Cheeger cut criterion is used in p-spectral clustering to do graph partition. However, due to the improper affinity measure and outliers, the original p-spectral clustering algorithm is not effective in dealing with manifold data. To solve this problem, we propose a manifold p-spectral clustering (M-pSC) using path-based affinity measure. First, we design a path-based affinity function to describe the complex structures of manifold data. This affinity function obeys the clustering assumption that the data pairs within the manifold structure share high affinities, and the data pairs between different manifold structures share low affinities. This will help us construct a good affinity matrix, which carry more category information of the points. Then we propose a M-pSC algorithm using the path-based affinity function. In the Cheeger cut criterion, the p-Laplacian matrix are constructed based on the manifold affinity function, and the final clustering results are obtained by using the eigenvectors of graph p-Laplacian. At last, the proposed algorithm is tested on several public data sets and the experiments show that our algorithm is adaptive to different manifold data. Compared with other popular clustering algorithms, our algorithm has good clustering quality and robustness.

A density-based statistical analysis of graph clustering algorithm performance

Abstract We introduce graph clustering quality measures based on comparisons of global, intra- and inter-cluster densities, an accompanying statistical significance test and a step-by-step routine for clustering quality assessment. Our work is centred on the idea that well-clustered graphs will display a mean intra-cluster density that is higher than global density and mean inter-cluster density. We do not rely on any generative model for the null model graph. Our measures are shown to meet the axioms of a good clustering quality function. They have an intuitive graph-theoretic interpretation, a formal statistical interpretation and can be tested for significance. Empirical tests also show they are more responsive to graph structure, less likely to breakdown during numerical implementation and less sensitive to uncertainty in connectivity than the commonly used measures.

Discovery of Spatio-Temporal Patterns in Multivariate Spatial Time Series

With the advancement of the computing technology and its wide range of applications, collecting large sets of multivariate time series in multiple geographical locations introduces a problem of identifying interesting spatio-temporal patterns. We consider a new spatial structure of the data in the pattern discovery process due to the dependent nature of the data. This article presents an information-theoretic approach to detect the temporal patterns from the multivariate time series in multiple locations. Based on their occurrences of discovered temporal patterns, we propose a method to identify interesting spatio-temporal patterns by a statistical significance test. Furthermore, the identified spatio-temporal patterns can be used for clustering and classification. For evaluating the performance, a simulated dataset is tested to validate the quality of the identified patterns and compare with other approaches. The result indicates the approach can effectively identify useful patterns to characterize the dataset for further analysis in achieving good clustering quality. Furthermore, experiments on real-world datasets and case studies have been conducted to illustrate the applicability and the practicability of the proposed approach.

Improve BIRCH algorithm for big data clustering

Big Data is a collection of data with super large data volumes, has a very high diversity of data sources, so needs to be managed with methods and devices that help perform accordingly. Clustering is one of the effective techniques for dealing with Big Data. The hierarchical method with the BIRCH algorithm is able to produce a short time in data execution. The BIRCH algorithm is a matching grouping algorithm for very large data sets. In an algorithm, a CF-tree is built in which all entries in each leaf node must meet same T threshold, and the CF-tree is rebuilt at each stage with a different threshold. But using a static (fixed) threshold produces poor cluster quality, in this paper proposes a solution to this deficiency by modifying the Threshold value to dynamic so that it can produce good cluster quality and be validated using silhouette coefficient (SC). There is a very clear difference between the standard BIRCH algorithm and the BIRCH algorithm on the modified T parameter (BIRCH (CF-Leaf (modif)). The CF-Node result, the total CF-Entries and Total CF-Leaf Entries produced 60% less than CF-Node, the total CF-Entries and Total CF-Leaf Entries in the standard BIRCH algorithm.