Desired Number Of Clusters Research Articles

Application of unsupervised deep learning to image segmentation and in-situ contact angle measurements in a CO2-water-rock system

Rock surface wettability is a critical property that regulates multiphase flows in porous media, which can be quantified using the surface contact angle (CA). X-ray micro-computed tomography (μCT) provides an effective approach to in-situ measurements of surface CAs. However, the CA measurement accuracy depends significantly on the quality of CT image segmentation, which is the clustering of CT pixels into separate phases. Inspired by this, we developed a deep learning (DL)-based CA measurement workflow. Motivated by the recent tremendous progress in unsupervised learning techniques and aiming to avoid expensive manual data annotations, an unsupervised DL pipeline for CT image segmentation was proposed and implemented, which includes unsupervised model training and post-processing. The unsupervised model training was driven by a novel loss function constrained with feature similarity and spatial continuity and implemented by iterative forward and backward paths; the former clustered the pixel-wise feature vectors extracted by convolution neural networks, whereas the latter updated the parameters using gradient descent. An over-segmentation strategy was adopted for model training. The post-processing steps based on agglomerative hierarchical clustering (AHC) were implemented to further merge the over-segmented model output to the desired cluster number, which is intended to improve the efficiency of image segmentation. The developed unsupervised DL pipeline was compared with other commonly-used image segmentation methods using pixel-wise and physics-based evaluation metrics on a synthetic raw-image dataset, which had a known ground truth. The unsupervised DL pipeline showed the best performance. Next, the segmented images were input to an automatic CA measurement tool, and the results were validated by comparisons with manual measurements. The CA values from the manual and automatic measurements showed similar distributions and statistical properties. The automatic measurement demonstrated a wider spectrum because of the much larger number of measurement data points. The primary novelty of the unsupervised DL pipeline developed in this study lies in the novel loss function and the over-segmentation strategy associated with AHC post-processing. The workflow has been proven an efficient tool for pore-scale wettability characterization, which has a wide range of applications in fundamental studies of multiphase flows in natural porous media, which have critical implications to geological carbon sequestration, hydrocarbon energy recovery, and contaminant transport in groundwater.

ANALISIS CLUSTER HIERARKI PADA PERSEBARAN KASUS COVID-19 BERDASARKAN PROVINSI DI INDONESIA

This study aims to apply Hierarchical cluster analysis to the distribution of Covid-19 cases by province in Indonesia. The data used is secondary data for the 2021 period. The variables used are the number of confirmed patients, the number of recovered patients, the number of patients who died, the population, population density, the number of elderly people, and health facilities. The method in this research is hierarchical cluster analysis with agglomeration process, Beetween groups linkage, with the concept of square Euclidean distance. The research step begins with data exploration, standardization, multicollinearity assumption test and Barlett KMO test, hierarchical cluster analysis, and interpretation. The results showed that the clustering process in Hierarchical analysis can be determined based on the desired number of clusters and the results are strengthened by the Dendogram representation. The distribution of Covid-19 cases in 34 provinces is divided into 2,3, and 4 clusters based on the variables used. Prov. DKI Jakarta has different characteristics from the prov. West Java, Central Java, East Java. Likewise with the prov. Banten sd. Prov. Gorontalo has similar characteristics in the case of the spread of Covid-19.

Self Organizing Maps (Kohonen) untuk Cluster Bidang Karya Ilmiah (Skripsi) Mahasiswa Berdasarkan Nilai-Nilai Matakuliah Pendukung Machine Learning

The time for the preparation of scientific papers is considered too fast and not suitable for students due to several things in the scope of its preparation. One of the reasons is that the student is unable to complete and the supporting aspects of the field of scientific work being done, many students whose topics of scientific work are in machine learning or the like, but these students are unable to complete their scientific work because they do not understand the theoretical supporting knowledge of the machine learning field. So that it can make the student depressed or even harder to repeat his scientific work the next semester. The scope of machine learning courses are statistics and probability, matrix and linear algebra, algorithms and programming, and data structures. This research was conducted to overcome the problems faced by students, namely knowing the suitability group for the field of student scientific work they will be working on. So that in its preparation students can be responsible in their trial and are of higher quality. The clustering test with the self organizing maps (SOM) algorithm is more stable because the input according to the data owned is not random, only the weighting is done randomly but based on the uniform low (mins) and high (max) limit values. The desired number of clusters is two namely cluster 0 is able to do scientific work based on machine learning and cluster 1 vice versa. The SOM process for 40 student data with a target of two clusters and the results are cluster 0 = 14 students, cluster 1 = 26 students. The result is obtained by increasing the radius = 1, which previously this achievement was not successful if radius = 0.

A robust clustering method with noise identification based on directed K-nearest neighbor graph

Obtaining the optimal cluster number and generating reliable clustering results in nonlinear manifolds are necessary but challenging tasks. Most existing clustering algorithms have considerable limitations in dealing with local and nonlinear data patterns, while graph-based clustering has shown impressive performance in identifying clusters in such data patterns. In this paper, we propose a robust clustering method with noise cutting based on directed k-nearest neighbor graph (CDKNN) to identify the desired cluster number automatically and produce reliable clustering results simultaneously on nonlinear, non-overlapping but locally tight-connected data patterns. This method draws support from the k-nearest neighbor graph to represent the complex nonlinear datasets and applies parameter adaptive process to make the proposed clustering method better adapt to specific data patterns. The proposed method is robust to the noises of arbitrary shape datasets because it uses a directed K-nearest neighbor to cut out sparse nodes. We use simulation and UCI real-world datasets to prove the validity of the innovatory method by comparing it to k-means, DBSCAN, OPTICS, AP, SC, and CutPC algorithms in terms of clustering ACC, ARI, NMI, and FMI. The experimental results confirm that the proposed method outperforms the alternative nonlinear clustering methods.

GRADE: A Novel Gravitational Density-Based Clustering Approach in the Multi-Objective Framework

Background: Clustering analysis plays a vital role in obtaining knowledgeable data from the huge amount of data sets in knowledge discovery. Most of the traditional clustering algorithms do not work well with high dimensional data. The objective of effective clustering is to obtain well connected, compact, and separated clusters. Density-Based Clustering (DBSCAN) is one of the popular clustering algorithms that use local density information of data points to detect clusters with arbitrary shapes. The Gravitational Search Algorithm (GSA) is one of the effective approaches inspired by Newton’s law of gravitational force where every particle in the universe attracts every other particle with a force. Objectives: The primary aim of this paper is to design and develop a novel multi-objective clustering approach to produce the desired number of valid clusters. Further, these resulting clusters are to be optimized to obtain an optimal solution. Methods: In the proposed approach, a hybrid clustering algorithm based on GSA along with DBSCAN is recommended to group the data into the desired number of clusters, and in the next phase of the algorithm, Particle swarm optimization technique is applied in order to optimize the solutions using the fitness functions. Results: In the analysis of the result, we employed two objective functions namely quantization error and inter–cluster distance on four real-life data sets such as Iris, Wine, Wisconsin, and Yeast to evaluate the performance of our algorithm. Conclusion: The effectiveness of the GRADE algorithm is comprehensively demonstrated by comparing it with the well-known traditional K-mean algorithm in terms of accuracy and computational time.

Clustering Using Cyclic Spaces of Reversible Cellular Automata

This paper introduces a cycle-based clustering technique using the cyclic spaces of reversible cellular automata (CAs). Traditionally, a cluster consists of close objects, which in the case of CAs necessarily means that the objects belong to the same cycle; that is, they are reachable from each other. Each of the cyclic spaces of a cellular automaton (CA) forms a unique cluster. This paper identifies CA properties based on “reachability” that make the clustering effective. To do that, we first figure out which CA rules contribute to maintaining the minimum intracluster distance. Our CA is then designed with such rules to ensure that a limited number of cycles exist in the configuration space. An iterative strategy is also introduced that can generate a desired number of clusters by merging objects of closely reachable clusters from a previous level in the present level using a unique auxiliary CA. Finally, the performance of our algorithm is measured using some standard benchmark validation indices and compared with existing well-known clustering techniques. It is found that our algorithm is at least on a par with the best algorithms existing today on the metric of these standard validation indices.

A parameter-less algorithm for tensor co-clustering

The majority of the data produced by human activities and modern cyber-physical systems involve complex relations among their features. Such relations can be often represented by means of tensors, which can be viewed as generalization of matrices and, as such, can be analyzed by using higher-order extensions of existing machine learning methods, such as clustering and co-clustering. Tensor co-clustering, in particular, has been proven useful in many applications, due to its ability of coping with n-modal data and sparsity. However, setting up a co-clustering algorithm properly requires the specification of the desired number of clusters for each mode as input parameters. This choice is already difficult in relatively easy settings, like flat clustering on data matrices, but on tensors it could be even more frustrating. To face this issue, we propose a new tensor co-clustering algorithm that does not require the number of desired co-clusters as input, as it optimizes an objective function based on a measure of association across discrete random variables (called Goodman and Kruskal’s tau) that is not affected by their cardinality. We introduce different optimization schemes and show their theoretical and empirical convergence properties. Additionally, we show the effectiveness of our algorithm on both synthetic and real-world datasets, also in comparison with state-of-the-art co-clustering methods based on tensor factorization and latent block models.

A machine learning approach combining expert knowledge with genetic algorithms in feature selection for credit risk assessment

Most credit scoring algorithms are designed with the assumption to be executed in an environment characterized by an automatic processing of credit applications, without considering the input of expert opinions. Since in credit scoring applications expert opinions have been proved very helpful, in this work, we propose a combination strategy of integrating soft computing methods with expert knowledge. The ability of interpretation of the predictive power of each feature in the credit dataset is strengthened by the engagement of experts in the credit scoring process, and the proposed wrapper-based feature selection approach which explores how the features contributing most towards the classification of borrowers. In particular, an unsupervised machine learning algorithm allows experts to create one or more clustering scenarios regarding the features by defining a desired number of clusters per scenario. For each clustering scenario, the Analytic Hierarchy Process is applied for helping experts to make preferences for features of each cluster, set pair-wise constraints for features of equal importance, and evaluate their subjective judgments in terms of consistency. Then, expert opinions are taken into consideration for solving a credit scoring problem in the form of an optimization problem subject to constraints via soft computing methods based on supervised machine learning and evolutionary optimization algorithms. Testing instances on standard credit dataset are established to verify the effectiveness of the proposed methodology.

Second-order consensus of multiagent systems with matrix-weighted network

This paper studies second-order consensus problem on the matrix-weighted undirected network, which is a generalization of second-order scalar-weighted network. The matrix coupling can describe the interdependencies of multi-dimensional states among agents. To highlight the influence of matrix coupling, we analyze the double integral model in matrix-weighted multiagent system network. A sufficient and necessary algebraic condition based on coupling strength, matrix theory and Lyapunov stability theory is established to achieve second-order consensus. With aid of the established property, a fairly straightforward algebraic graph condition is obtained, which ensures that the matrix-weighted multiagent system achieves second-order consensus. Moreover, owing to the existence of positive semi-definite connections, the clustering phenomena naturally exists, which shows that matrix coupling plays an important role in the convergence of the matrix-weighted multiagent system network. And an algebraic graph condition for finding all clusters is offered, which can guarantee the desired number of clusters via designing matrix-weights in practical applications. Finally, four simulation examples are given to illustrate the effectiveness of the obtained results.

Pattern Classification Based on RBF Networks with Self-Constructing Clustering and Hybrid Learning

Radial basis function (RBF) networks are widely adopted to solve problems in the field of pattern classification. However, in the construction phase of such networks, there are several issues encountered, such as the determination of the number of nodes in the hidden layer, the form and initialization of the basis functions, and the learning of the parameters involved in the networks. In this paper, we present a novel approach for constructing RBF networks for pattern classification problems. An iterative self-constructing clustering algorithm is used to produce a desired number of clusters from the training data. Accordingly, the number of nodes in the hidden layer is determined. Basis functions are then formed, and their centers and deviations are initialized to be the centers and deviations of the corresponding clusters. Then, the parameters of the network are refined with a hybrid learning strategy, involving hyperbolic tangent sigmoid functions, steepest descent backpropagation, and least squares method. As a result, optimized RBF networks are obtained. With this approach, the number of nodes in the hidden layer is determined and basis functions are derived automatically, and higher classification rates can be achieved. Furthermore, the approach is applicable to construct RBF networks for solving both single-label and multi-label pattern classification problems.

A Novel Improved K-Means Algorithm Based on Parameter Adaptive Selection

As a classical clustering algorithm, K-means has been widely applied due to its features of simple mathematical thinking, fast convergence rate, less complexity, and easy to implementation. However, K-means algorithm always requires users to set the desired number of clusters in advance, and the initial cluster centers are usually generated in a random way. When dealing with unknown datasets that users do not have enough domain-assisted knowledge, such parameters setting strategies not only increases the burden on users, but also makes clustering quality difficult to guarantee. Therefore, in view of the high sensitivity of K-means clustering process to initial parameters, this paper propose an improved DDWK-means (Distance-Density-Weight K-means) algorithm. Based on the distance-density feature and the method of inertia weight of particle swarm optimization algorithm, the optimal initial cluster centers not only can be determined adaptively according to the structural characteristics of the dataset itself without introducing artificial parameters, but also can be adjusted dynamically due to the threshold change of clustering quality metric. We make an experimental study with five standard datasets from UCI (University of California Irvine), and the results indicate that the DDWK-means algorithm exhibits a significantly improvement in clustering efficiency and stability.

GPU Accelerated Fuzzy C-Means (FCM) Color Image Segmentation

In this paper, computational acceleration of color image segmentation using fuzzy c-means (FCM) algorithm has been presented. The color image is first converted from the Red Green Blue (RGB) color space to the YUV color space. Then, the luma (Y) information values are grouped according to the desired number of clusters using the FCM algorithm. The FCM algorithm is implemented on a Graphical Processing Unit (GPU) using the Compute Unified Device Library (CUDA) library which is developed by NVidia to speed up the computing time. Images used in this research are red blood cell images, geometry images and leaf images. The results of segmented images processed using GPU were seen identic to the results of segmented images processed using the Central Processing Unit (CPU). The computational time of the FCM algorithm can be accelerated by speed-up to 5,628 times faster and the average speed-up of all simulations done is 5,517 times faster.

Parallel SLINK for big data

The major strength of hierarchical clustering algorithms is that it allows visual interpretations of clusters through dendrograms. Users can cut the dendrogram at different levels to get desired number of clusters. A major problem with hierarchical algorithms is their quadratic runtime complexity, which limits the amount of data that can be clustered in reasonable amount of time. Also, due to its agglomerative merging process, each iteration depends on the data of all previous iterations, making it difficult to parallelize. Thus, there is a need for an efficient parallel implementation of SLINK algorithm which can scale to big data. We present a parallel SLINK algorithm, sGridSLINK, for shared memory architectures. sGridSLINK produces exactly the same dendrogram as the classical SLINK algorithm. We also present, hGridSLINK, a parallel algorithm which fully exploits a multi-core cluster system. To the best of our knowledge, there is no hybrid parallel algorithm for SLINK available in the literature. The proposed algorithms exploit spatial locality of data to reduce the number of distance calculations. Adaptive gridding is used to counter skewness in data and to ensure load balancing. Extensive experiments are carried out to establish the efficiency and scalability of proposed parallel algorithms. sGridSLINK is approximately 840 times faster than the state-of-the-art algorithm using 55 threads on a 48-core machine on a real dataset having 6 million data points. It also achieves a speedup of 47.93 over the best known sequential SLINK, GridSLINK, on a real dataset using 48 threads on a 48-core machine. hGridSLINK achieves a maximum speedup of 68.26 on a 32-node cluster ($$32\times 4$$ processing elements) with respect to GridSLINK. The hGridSLINK algorithm is able to cluster 200 million data points in only 1317 s (less than 22 min). No existing parallel SLINK algorithm is capable of such efficient clustering of Big Data.

Artificial intelligence for understanding concussion: Retrospective cluster analysis on the balance and vestibular diagnostic data of concussion patients.

ObjectivesWe propose a bottom-up, machine-learning approach, for the objective vestibular and balance diagnostic data of concussion patients, to provide insight into the differences in patients’ phenotypes, independent of existing diagnoses (unsupervised learning).MethodsDiagnostic data from a battery of validated balance and vestibular assessments were extracted from the database of the Swiss Concussion Center. The desired number of clusters within the patient database was estimated using Calinski-Harabasz criteria. Complex (self-organizing map, SOM) and standard (k-means) clustering tools were used, and the formed clusters were compared.ResultsA total of 96 patients (81.3% male, age (median [IQR]): 25.0[10.8]) who were expected to suffer from sports-related concussion or post-concussive syndrome (52[140] days between diagnostic testing and the concussive episode) were included. The cluster evaluation indicated dividing the data into two groups. Only the SOM gave a stable clustering outcome, dividing the patients in group-1 (n = 38) and group-2 (n = 58). A large significant difference was found for the caloric summary score for the maximal speed of the slow phase, where group-1 scored 30.7% lower than group-2 (27.6[18.2] vs. 51.0[31.0]). Group-1 also scored significantly lower on the sensory organisation test composite score (69.0[22.3] vs. 79.0[10.5]) and higher on the visual acuity (-0.03[0.33] vs. -0.14[0.12]) and dynamic visual acuity (0.38[0.84] vs. 0.20[0.20]) tests. The importance of caloric, SOT and DVA, was supported by the PCA outcomes. Group-1 tended to report headaches, blurred vision and balance problems more frequently than group-2 (>10% difference).ConclusionThe SOM divided the data into one group with prominent vestibular disorders and another with no clear vestibular or balance problems, suggesting that artificial intelligence might help improve the diagnostic process.

A General Hybrid Clustering Technique

Here, we propose a clustering technique for general clustering problems including those that have nonconvex clusters. For a given desired number of clusters K, we use three stages to find clusters. The first stage uses a hybrid clustering technique to produce a series of clusterings of various sizes (randomly selected). The key step in this stage is to find a K-means clustering using clusters where and then join these small clusters by using single linkage clustering. The second stage stabilizes the result of stage one by reclustering via the “membership matrix” under Hamming distance to generate a dendrogram. The third stage is to cut the dendrogram to get clusters where and then prune back to K to give a final clustering. A variant on our technique also gives a reasonable estimate for KT, the true number of clusters. We provide arguments to justify the steps in the stages of our methods and we provide examples involving simulated and published data to compare our technique with other techniques. An R library, GHC, implementing our method is available from Github.

An extension of PROMETHEE to hierarchical multicriteria clustering

Multicriteria clustering can be seen as a hybridisation between ranking and sorting problematic. These methods are used to build totally or partially ordered groups of alternatives based on preference relations. In the context of totally ordered clustering, two hierarchical approaches (top-down and bottom-up) based on PROMETHEE II have been developed in this paper. These methods rely on the optimisation of the clustering structure (by maximising the intra-cluster homogeneity and the inter-clusters heterogeneity). A third approach is developed as a hybrid model that merges the information obtained by both previous models. A specific quality index has been introduced to be able to evaluate the method's outputs and to choose appropriately the desired number of clusters. The three procedures have been tested on several dataset (Shanghai Ranking of World Universities, Environmental Performance Index and CPU evaluations) and the results have been compared with P2Clust.

FlashProfile: a framework for synthesizing data profiles

We address the problem of learning a syntactic profile for a collection of strings, i.e. a set of regex-like patterns that succinctly describe the syntactic variations in the strings. Real-world datasets, typically curated from multiple sources, often contain data in various syntactic formats. Thus, any data processing task is preceded by the critical step of data format identification. However, manual inspection of data to identify the different formats is infeasible in standard big-data scenarios. Prior techniques are restricted to a small set of pre-defined patterns (e.g. digits, letters, words etc.), and provide no control over granularity of profiles. We define syntactic profiling as a problem of clustering strings based on syntactic similarity, followed by identifying patterns that succinctly describe each cluster. We present a technique for synthesizing such profiles over a given language of patterns, that also allows for interactive refinement by requesting a desired number of clusters. Using a state-of-the-art inductive synthesis framework, PROSE, we have implemented our technique as FlashProfile. Across 153 tasks over 75 large real datasets, we observe a median profiling time of only ∼ 0.7s. Furthermore, we show that access to syntactic profiles may allow for more accurate synthesis of programs, i.e. using fewer examples, in programming-by-example (PBE) workflows such as Flash Fill.

A hierarchical clustering algorithm based on noise removal

Noise is irrelevant or meaningless data and hinders most types of data analysis. The existing clustering algorithms seldom take the noise points into consideration and cannot detect arbitrary-shaped clusters. This paper presents a Hierarchical Clustering algorithm Based on Noise Removal (HCBNR). It is robust against noise points and good at discovering clusters with arbitrary shapes. In this work, natural neighbor-based density is applied to remove noise points in a data set firstly. Then we construct a saturated neighbor graph on the rest points, and a novel modularity-based graph partitioning algorithm is used to divide the graph into small clusters. Finally, the small clusters are repeatedly merged according to a novel similarity metric between clusters until the desired cluster number is obtained. The experimental results on synthetic data sets and real data sets show that our method can accurately identify noise points and obtain better clustering results than existing clustering algorithms when discovering arbitrary-shaped clusters.

Clustering ensemble method

A clustering ensemble aims to combine multiple clustering models to produce a better result than that of the individual clustering algorithms in terms of consistency and quality. In this paper, we propose a clustering ensemble algorithm with a novel consensus function named Adaptive Clustering Ensemble. It employs two similarity measures, cluster similarity and a newly defined membership similarity, and works adaptively through three stages. The first stage is to transform the initial clusters into a binary representation, and the second is to aggregate the initial clusters that are most similar based on the cluster similarity measure between clusters. This iterates itself adaptively until the intended candidate clusters are produced. The third stage is to further refine the clusters by dealing with uncertain objects to produce an improved final clustering result with the desired number of clusters. Our proposed method is tested on various real-world benchmark datasets and its performance is compared with other state-of-the-art clustering ensemble methods, including the Co-association method and the Meta-Clustering Algorithm. The experimental results indicate that on average our method is more accurate and more efficient.

Uncertain fuzzy self-organization based clustering: interval type-2 fuzzy approach to adaptive resonance theory

Conventional unsupervised learning algorithms require knowledge of the desired number of clusters beforehand. Even if such knowledge is not required in advance, empirical selection of the parameter values may limit the adaptive capability of the algorithm, thereby restricting the clustering performance. An inherent uncertainty in the number and size of clusters requires integration of fuzzy sets into a clustering algorithm. In this paper, we propose a type-1 (T1) fuzzy ART method that adaptively selects the vigilance parameter value, which is critical in determining the network dynamics. This results in improved clustering performance due to the added flexibility in dynamic selection of the number of clusters with the use of fuzzy sets. To further manage the uncertainty associated with memberships, we extend the proposed T1 fuzzy ART with adaptive vigilance to an interval type-2 (IT2) fuzzy ART method. Type reduction and defuzzification are then performed using the KM algorithm to obtain a defuzzified vigilance parameter value. We evaluate our proposed methods on several data sets to validate their effectiveness.