Maximum Clustering Research Articles

How many clusters exist? Answer via maximum clustering similarity implemented in R

Finding the number of clusters in a data set is considered as one of the fundamental problems in cluster analysis. This paper integrates maximum clustering similarity (MCS), for finding the optimal number of clusters, into R statistical software through the package MCSim. The similarity between the two clustering methods is calculated at the same number of clusters, using Rand [Objective criteria for the evaluation of clustering methods. J Am Stat Assoc. 1971;66:846–850.] and Jaccard [The distribution of the flora of the alpine zone. New Phytologist. 1912;11:37–50.] indices, corrected for chance agreement. The number of clusters at which the index attains its maximum with most frequency is a candidate for the optimal number of clusters. Unlike other criteria, MCS can be used with circular data. Seven clustering algorithms, existing in R, are implemented in MCSim. A graph of the number of clusters vs. clusters similarity using corrected similarity indices is produced. Values of the similarity indices and a clustering tree (dendrogram) are produced. Several examples including simulated, real, and circular data sets are presented to show how MCSim successfully works in practice.

Improving clustering performance using independent component analysis and unsupervised feature learning

ObjectiveTo provide a parsimonious clustering pipeline that provides comparable performance to deep learning-based clustering methods, but without using deep learning algorithms, such as autoencoders.Materials and methodsClustering was performed on six benchmark datasets, consisting of five image datasets used in object, face, digit recognition tasks (COIL20, COIL100, CMU-PIE, USPS, and MNIST) and one text document dataset (REUTERS-10K) used in topic recognition. K-means, spectral clustering, Graph Regularized Non-negative Matrix Factorization, and K-means with principal components analysis algorithms were used for clustering. For each clustering algorithm, blind source separation (BSS) using Independent Component Analysis (ICA) was applied. Unsupervised feature learning (UFL) using reconstruction cost ICA (RICA) and sparse filtering (SFT) was also performed for feature extraction prior to the cluster algorithms. Clustering performance was assessed using the normalized mutual information and unsupervised clustering accuracy metrics.ResultsPerforming, ICA BSS after the initial matrix factorization step provided the maximum clustering performance in four out of six datasets (COIL100, CMU-PIE, MNIST, and REUTERS-10K). Applying UFL as an initial processing component helped to provide the maximum performance in three out of six datasets (USPS, COIL20, and COIL100). Compared to state-of-the-art non-deep learning clustering methods, ICA BSS and/or UFL with graph-based clustering algorithms outperformed all other methods. With respect to deep learning-based clustering algorithms, the new methodology presented here obtained the following rankings: COIL20, 2nd out of 5; COIL100, 2nd out of 5; CMU-PIE, 2nd out of 5; USPS, 3rd out of 9; MNIST, 8th out of 15; and REUTERS-10K, 4th out of 5.DiscussionBy using only ICA BSS and UFL using RICA and SFT, clustering accuracy that is better or on par with many deep learning-based clustering algorithms was achieved. For instance, by applying ICA BSS to spectral clustering on the MNIST dataset, we obtained an accuracy of 0.882. This is better than the well-known Deep Embedded Clustering algorithm that had obtained an accuracy of 0.818 using stacked denoising autoencoders in its model.ConclusionUsing the new clustering pipeline presented here, effective clustering performance can be obtained without employing deep clustering algorithms and their accompanying hyper-parameter tuning procedure.

ADOFL: Multi-Kernel-Based Adaptive Directive Operative Fractional Lion Optimisation Algorithm for Data Clustering

Abstract The progress of databases in fields such as medical, business, education, marketing, etc., is colossal because of the developments in information technology. Knowledge discovery from such concealed bulk databases is a tedious task. For this, data mining is one of the promising solutions and clustering is one of its applications. The clustering process groups the data objects related to each other in a similar cluster and diverse objects in another cluster. The literature presents many clustering algorithms for data clustering. Optimisation-based clustering algorithm is one of the recently developed algorithms for the clustering process to discover the optimal cluster based on the objective function. In our previous method, direct operative fractional lion optimisation algorithm was proposed for data clustering. In this paper, we designed a new clustering algorithm called adaptive decisive operative fractional lion (ADOFL) optimisation algorithm based on multi-kernel function. Moreover, a new fitness function called multi-kernel WL index is proposed for the selection of the best centroid point for clustering. The experimentation of the proposed ADOFL algorithm is carried out over two benchmarked datasets, Iris and Wine. The performance of the proposed ADOFL algorithm is validated over existing clustering algorithms such as particle swarm clustering (PSC) algorithm, modified PSC algorithm, lion algorithm, fractional lion algorithm, and DOFL. The result shows that the maximum clustering accuracy of 79.51 is obtained by the proposed method in data clustering.

Joint maximum purity forest with application to image super-resolution

We propose a random-forest scheme, namely joint maximum purity forest (JMPF), for classification, clustering, and regression tasks. In the JMPF scheme, the original feature space is transformed into a compactly preclustered feature space, via a trained rotation matrix. The rotation matrix is obtained through an iterative quantization process, where the input data, inclined to different classes, is clustered to the respective vertices of the feature space with maximum purity. In the feature space, orthogonal hyperplanes, which are employed at the split nodes of the decision trees in a random forest, can effectively tackle the clustering problems. We evaluated our proposed method on public benchmark datasets for regression and classification tasks, and experiments showed that JMPF remarkably outperforms other state-of-the-art random-forest-based approaches. Furthermore, we applied JMPF to image super-resolution (SR) specifically because the transformed, compact features are more discriminative to the clustering-regression scheme. Experimental results on several public datasets also showed that the JMPF-based image SR scheme is consistently superior to recent state-of-the-art image SR algorithms.

Lineament Mapping over Sir Creek Offshore and its Surroundings using High Resolution EGM2008 Gravity Data: An Integrated Derivative Approach

ABSTRACT The present study deals with the delineation of lineaments over Sir Creek offshore and its surroundings from EGM2008 gravity data using various derivative techniques owing to their cost-effectiveness in prospective hydrocarbon exploration. Initially, 2-D and 3-D synthetic models have been generated with vertical prismatic objects at different depths. The effectiveness of total horizontal derivative (THD) technique has been established by comparing with E-W and N-S Horizontal derivatives and First Vertical derivative techniques. The residuals of Bouguer gravity data have been estimated with different cut-off wavelengths. Further, the residual anomaly map has been enhanced by the derivative techniques for the delineation of the structural features. Possible depths of the delineated lineaments have been estimated using Euler deconvolution of the Bouguer gravity data, which indicates maximum clustering over the delineated lineaments. It is observed that most of the lineaments are in the depth range of 1.0 km to 5.5km, which correlate well with the previous seismic studies. The present study reveals that the major lineament trends in the N-S, E-W and NNW-SSE directions followed by NE-SW, NW-SE and ENE-WSW directions. These major lineament trends are due to the tectonic activities occurred during Precambrian and Cretaceous period. Different small circular features, rectangular features and shorter wavelength features have also been identified, which could be the key parameter for mapping potential location for hydrocarbon exploration.

Minimum spectral connectivity projection pursuit

We study the problem of determining the optimal low-dimensional projection for maximising the separability of a binary partition of an unlabelled dataset, as measured by spectral graph theory. This is achieved by finding projections which minimise the second eigenvalue of the graph Laplacian of the projected data, which corresponds to a non-convex, non-smooth optimisation problem. We show that the optimal univariate projection based on spectral connectivity converges to the vector normal to the maximum margin hyperplane through the data, as the scaling parameter is reduced to zero. This establishes a connection between connectivity as measured by spectral graph theory and maximal Euclidean separation. The computational cost associated with each eigen problem is quadratic in the number of data. To mitigate this issue, we propose an approximation method using microclusters with provable approximation error bounds. Combining multiple binary partitions within a divisive hierarchical model allows us to construct clustering solutions admitting clusters with varying scales and lying within different subspaces. We evaluate the performance of the proposed method on a large collection of benchmark datasets and find that it compares favourably with existing methods for projection pursuit and dimension reduction for data clustering. Applying the proposed approach for a decreasing sequence of scaling parameters allows us to obtain large margin clustering solutions, which are found to be competitive with those from dedicated maximum margin clustering algorithms.

Optimized cognitive radio network (CRN) using genetic algorithm and artificial bee colony algorithm

The spectrum or the whitespaces that are not being used by licensed or primary users (PUs) can be sensed and operated by the unlicensed or the secondary users (SUs) in the cognitive radio networks (CRN) in such a way that there is no impact on the activities of the primary users. One mechanism for topology management is referred to as clustering. This organizes the nodes in logical sets so that the performance of the network is enhanced. The objectives of clustering include supporting collaborative tasks like channel access and channel sensing alongside network stability as well as scalability which is critical for cognitive radio (CR) functioning. The channel availability being dynamic, it changes with time as well as location, thus novel clustering algorithms must be formulated to tackle these issues that are inherent to CRs. This paper proposes a lowest ID clustering algorithm which chooses a certain node as the cluster leader which has the lowest ID. This leader is referred to as cluster head (CH). The other algorithm that selects a node having the highest number of neighbouring nodes as the CH is referred to as the maximum node degree clustering algorithm. For obtaining optimal radio configurations and performance improvements, the genetic algorithm comprising selection, crossover and mutation is embedded in the CR. In order to optimize the allocation of spectrum for greater efficacy and fairness, the artificial bee colony (ABC) algorithm, a meta-heuristic optimization algorithm is incorporated. It has been shown that the proposed technique leads to a greater performance compared to the other techniques.

Multi-View Maximum Entropy Clustering by Jointly Leveraging Inter-View Collaborations and Intra-View-Weighted Attributes.

As a dedicated countermeasure for heterogeneous multi-view data, multi-view clustering is currently a hot topic in machine learning. However, many existing methods either neglect the effective collaborations among views during clustering or do not distinguish the respective importance of attributes in views, instead treating them equivalently. Motivated by such challenges, based on maximum entropy clustering (MEC), two specialized criteria-inter-view collaborative learning (IEVCL) and intra-view-weighted attributes (IAVWA)-are first devised as the bases. Then, by organically incorporating IEVCL and IAVWA into the formulation of classic MEC, a novel, collaborative multi-view clustering model and the matching algorithm referred to as the view-collaborative, attribute-weighted MEC (VC-AW-MEC) are proposed. The significance of our efforts is three-fold: 1) both IEVCL and IAVWA are dedicatedly devised based on MEC so that the proposed VC-AW-MEC is qualified to effectively handle as many multi-view data scenes as possible; 2) IEVCL is competent in seeking the consensus across all involved views throughout clustering, whereas IAVWA is capable of adaptively discriminating the individual impact regarding the attributes within each view; and 3) benefiting from jointly leveraging IEVCL and IAVWA, compared with some existing state-of-the-art approaches, the proposed VC-AW-MEC algorithm generally exhibits preferable clustering effectiveness and stability on heterogeneous multi-view data. Our efforts have been verified in many synthetic or real-world multi-view data scenes.

Bearing Fault Diagnosis Under Variable Working Conditions Based on Domain Adaptation Using Feature Transfer Learning

Bearings, as universal components, have been widely used in the important position of rotating machinery. However, due to the distribution divergence between training data and test data caused by variable working conditions, such as different rotation speeds and load conditions, most of the fault diagnosis models built during the training stage are not applicable for the detection in the test stage. The models dramatically lead to the performance degradation for fault classification. In this paper, a novel bearing fault diagnosis method, domain adaptation by using feature transfer learning (DAFTL) under variable working conditions, is proposed to solve this performance degradation issue. The dataset of normal bearings and faulty bearings are obtained via the fast Fourier transformation of raw vibration signals, under different motor speeds and load conditions. Then, the marginal and conditional distributions are reduced simultaneously between training data and test data by refining pseudo test labels based on the maximum mean discrepancy and domain invariant clustering in a common space. Ultimately, a transferable feature representation for training data and test data is achieved. With the help of the nearest-neighbor classifier built on the transferable features, bearing faults are identified in this common space. Extensive experimental results show that the DAFTL can identify the bearing fault accurately under variable working conditions and outperforms other competitive approaches.

A New Approach for Interpreting the Morisita Index of Aggregation through Quadrat Size

Spatial point pattern analysis is commonly used in ecology to examine the spatial distribution of individual organisms or events, which may shed light on the operation of underlying ecological processes driving the development of a spatial pattern. Commonly used quadrat-based methods of measuring spatial clustering or dispersion tend to be strongly influenced by the choice of quadrat size and population density. Using valley oak (Quercus lobata) stands at multiple sites, we show that values of the Morisita Index are sensitive to the choice of quadrat size, and that the comparative interpretation of the index for multiple sites or populations is problematic due to differences in scale and clustering intensity from site to site, which may call for different quadrat sizes for each site. We present a new method for analyzing the Morisita Index to estimate the appropriate quadrat size for a given site and to aid interpretation of the clustering index across multiple sites with local differences. By plotting the maximum clustering intensity (Imr) found across a range of quadrat sizes, we were able to describe how a spatial pattern changes when quadrat size varies and to identify scales of clustering and quadrat sizes for analysis of spatial patterns under different local conditions. Computing and plotting the instantaneous rate of change (first derivative of rMax), we were able to evaluate clustering across multiple sites on a standardized scale. The magnitude of the rMax first derivative is a useful tool to quantify the degree of crowding, dispersion, or random spatial distribution as a function of quadrat size.

Large values of the clustering coefficient

A prominent parameter in the context of network analysis, originally proposed by Watts and Strogatz (1998), is the clustering coefficient of a graph G. It is defined as the arithmetic mean of the clustering coefficients of its vertices, where the clustering coefficient of a vertex u of G is the relative density m(G[NG(u)])∕dG(u)2 of its neighborhood if dG(u) is at least 2, and 0 otherwise. It is unknown which graphs maximize the clustering coefficient among all connected graphs of given order and size.We determine the maximum clustering coefficients among all connected regular graphs of a given order, as well as among all connected subcubic graphs of a given order. In both cases, we characterize all extremal graphs. Furthermore, we determine the maximum increase of the clustering coefficient caused by adding a single edge.

Sensor selection based on maximum entropy fuzzy clustering for target tracking in large‐scale sensor networks

This study proposes a sensor selection approach based on maximum entropy fuzzy clustering to address the target tracking problem in large-scale sensor networks. The authors try to deal with this problem at two levels: (i) sensor-level tracking: data association problem and sensor-level tracking are carried out at the local level, and only the track outputs are transmitted to the fusion centre for data fusion; (ii) global-level fusion: two sensor selection strategies are adopted at the fusion centre, which seek to only choose a subset of reliable sensors for track-to-track fusion and bias registration. In addition, an improved sensor selection approach is proposed for data fusion in both sparse and dense target environments, and a new fuzzy membership reconstruction strategy is introduced for data association in dense target environments. Furthermore, the proposed sensor selection strategy is also effective in the presence of the possible changing sensor biases. Simulation results are given to evaluate the performance of the proposed approaches.

Analysis of the clustering of inertial particles in turbulent flows

An asymptotic solution is derived for the motion of inertial particles exposed to Stokes drag in an unsteady random flow. This solution provides the finite-time Lyapunov exponents as a function of Stokes number and Lagrangian strain- and rotation-rates autocovariances. The sum of these exponents, which corresponds to a concentration-weighted divergence of particle velocity field, is considered as a measure of clustering. For inertial particles dispersed in an isotropic turbulent flow our analysis predicts maximum clustering at an intermediate Stokes number and minimal clustering at small and large Stokes numbers. Direct numerical simulations are performed for quantitative validation of our analysis, showing a reasonable agreement between the two.

Differential Flattening

Amulti-layer graphconsists of multiple layers of weighted graphs, where the multiple layers represent the different aspects of relationships. Considering multiple aspects (i.e., layers) together is essential to achieve a comprehensive and consolidated view. In this article, we propose a novel framework ofdifferential flattening, which facilitates the analysis of multi-layer graphs, and apply this framework to community detection. Differential flattening merges multiple graphs into a single graph such that the graph structure with the maximum clustering coefficient is obtained from the single graph. It has two distinct features compared with existing approaches. First, dealing with multiple layers is doneindependentlyof a specific community detection algorithm, whereas previous approaches rely on a specific algorithm. Thus, any algorithm for a single graph becomes applicable to multi-layer graphs. Second, the contribution of each layer to the single graph is determinedautomaticallyfor the maximum clustering coefficient. Since differential flattening is formulated by an optimization problem, the optimal solution is easily obtained by well-known algorithms such as interior point methods. Extensive experiments were conducted using the Lancichinetti-Fortunato-Radicchi (LFR) benchmark networks as well as the DBLP, 20 Newsgroups, and MIT Reality Mining networks. The results show that our approach of differential flattening leads to discovery of higher-quality communities than baseline approaches and the state-of-the-art algorithms.

Likelihood-Based Inference of B Cell Clonal Families.

The human immune system depends on a highly diverse collection of antibody-making B cells. B cell receptor sequence diversity is generated by a random recombination process called “rearrangement” forming progenitor B cells, then a Darwinian process of lineage diversification and selection called “affinity maturation.” The resulting receptors can be sequenced in high throughput for research and diagnostics. Such a collection of sequences contains a mixture of various lineages, each of which may be quite numerous, or may consist of only a single member. As a step to understanding the process and result of this diversification, one may wish to reconstruct lineage membership, i.e. to cluster sampled sequences according to which came from the same rearrangement events. We call this clustering problem “clonal family inference.” In this paper we describe and validate a likelihood-based framework for clonal family inference based on a multi-hidden Markov Model (multi-HMM) framework for B cell receptor sequences. We describe an agglomerative algorithm to find a maximum likelihood clustering, two approximate algorithms with various trade-offs of speed versus accuracy, and a third, fast algorithm for finding specific lineages. We show that under simulation these algorithms greatly improve upon existing clonal family inference methods, and that they also give significantly different clusters than previous methods when applied to two real data sets.

DOFL: Kernel Based Directive Operative Fractional Lion Optimisation Algorithm for Data Clustering

In recent years, clustering finds various applications in most of the fields like networking, telecommunications and medical domain. So, various clustering algorithms are developed by researchers to improve the clustering performance. Among this, optimisation based clustering algorithm is one of the recently developed algorithms for clustering process to discover the optimal clusters based on the objective function. Accordingly, in this paper, we have proposed directive operative fractional lion optimization using kernel based clustering algorithm. Initially, three objective functions are developed based on the kernel function. Then, the result of objective function is selected as the fitness function for the proposed optimisation algorithm called directive operative based fractional lion algorithm to select the rapid centroid of the data cluster. Finally, the performance of the proposed kernel based directive operative fractional lion algorithm (DOFL) algorithm is evaluated based on the various evaluation metrics such as cluster accuracy, jaccard coefficient and rand coefficient using the both benchmarked iris and wine data sets. From the research outcome, we can prove that, the maximum clustering accuracy of 79% is obtained by the proposed optimization clustering algorithm compared to other existing clustering algorithms such as Particle Swarm clustering algorithm (PSC), modified Particle Swarm clustering algorithm (mPSC), lion algorithm and fractional lion algorithm.

Stepwise iterative maximum likelihood clustering approach.

BackgroundBiological/genetic data is a complex mix of various forms or topologies which makes it quite difficult to analyze. An abundance of such data in this modern era requires the development of sophisticated statistical methods to analyze it in a reasonable amount of time. In many biological/genetic analyses, such as genome-wide association study (GWAS) analysis or multi-omics data analysis, it is required to cluster the plethora of data into sub-categories to understand the subtypes of populations, cancers or any other diseases. Traditionally, the k-means clustering algorithm is a dominant clustering method. This is due to its simplicity and reasonable level of accuracy. Many other clustering methods, including support vector clustering, have been developed in the past, but do not perform well with the biological data, either due to computational reasons or failure to identify clusters.ResultsThe proposed SIML clustering algorithm has been tested on microarray datasets and SNP datasets. It has been compared with a number of clustering algorithms. On MLL datasets, SIML achieved highest clustering accuracy and rand score on 4/9 cases; similarly on SRBCT dataset, it got for 3/5 cases; on ALL subtype it got highest clustering accuracy for 5/7 cases and highest rand score for 4/7 cases. In addition, SIML overall clustering accuracy on a 3 cluster problem using SNP data were 97.3, 94.7 and 100 %, respectively, for each of the clusters.ConclusionsIn this paper, considering the nature of biological data, we proposed a maximum likelihood clustering approach using a stepwise iterative procedure. The advantage of this proposed method is that it not only uses the distance information, but also incorporate variance information for clustering. This method is able to cluster when data appeared in overlapping and complex forms. The experimental results illustrate its performance and usefulness over other clustering methods. A Matlab package of this method (SIML) is provided at the web-link http://www.riken.jp/en/research/labs/ims/med_sci_math/.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-016-1184-5) contains supplementary material, which is available to authorized users.

Hierarchical Maximum Likelihood Clustering Approach.

In this paper, we focused on developing a clustering approach for biological data. In many biological analyses, such as multiomics data analysis and genome-wide association studies analysis, it is crucial to find groups of data belonging to subtypes of diseases or tumors. Conventionally, the k-means clustering algorithm is overwhelmingly applied in many areas including biological sciences. There are, however, several alternative clustering algorithms that can be applied, including support vector clustering. In this paper, taking into consideration the nature of biological data, we propose a maximum likelihood clustering scheme based on a hierarchical framework. This method can perform clustering even when the data belonging to different groups overlap. It can also perform clustering when the number of samples is lower than the data dimensionality. The proposed scheme is free from selecting initial settings to begin the search process. In addition, it does not require the computation of the first and second derivative of likelihood functions, as is required by many other maximum likelihood-based methods. This algorithm uses distribution and centroid information to cluster a sample and was applied to biological data. A MATLAB implementation of this method can be downloaded from the web link http://www.riken.jp/en/research/labs/ims/med_sci_math/.

Selection of the Maximum Spatial Cluster Size of the Spatial Scan Statistic by Using the Maximum Clustering Set-Proportion Statistic.

Spatial scan statistics are widely used in various fields. The performance of these statistics is influenced by parameters, such as maximum spatial cluster size, and can be improved by parameter selection using performance measures. Current performance measures are based on the presence of clusters and are thus inapplicable to data sets without known clusters. In this work, we propose a novel overall performance measure called maximum clustering set–proportion (MCS-P), which is based on the likelihood of the union of detected clusters and the applied dataset. MCS-P was compared with existing performance measures in a simulation study to select the maximum spatial cluster size. Results of other performance measures, such as sensitivity and misclassification, suggest that the spatial scan statistic achieves accurate results in most scenarios with the maximum spatial cluster sizes selected using MCS-P. Given that previously known clusters are not required in the proposed strategy, selection of the optimal maximum cluster size with MCS-P can improve the performance of the scan statistic in applications without identified clusters.

Rainfall-runoff Modeling Using Dynamic Evolving Neural Fuzzy Inference System with Online Learning

Neuro-Fuzzy Systems (NFS) are computational intelligence tools that have recently been employed in hydrological modeling. In many of the common NFS the learning algorithms used are based on batch learning where all the parameters of the fuzzy system are optimized off-line. Although these models have frequently been used, there is a criticism on such learning process as the number of rules are needed to be predefined by the user. This will reduce the flexibility of the NFS architecture while dealing with different data with different level of complexity. On the other hand, online or local learning evolves through local adjustments in the model as new data is introduced in sequence. In this study, dynamic evolving neural fuzzy inference system (DENFIS) is used in which an evolving, online clustering algorithm called the Evolving Clustering Method (ECM) is implemented. ECM is an online, maximum distance-based clustering method which is able to estimate the number of clusters in a data set and find their current centers in the input space through its fast, one-pass algorithm. The 10-minutes rainfall-runoff time series from a small (23.22 km2) tropical catchment named Sungai Kayu Ara in Selangor, Malaysia, was used in this study. Out of the 40 major events, 12 were used for training and 28 for testing. Results obtained by DENFIS were then compared with the ones obtained by physically-based rainfall-runoff model HEC-HMS and a regression model ARX. It was concluded that DENFIS results were comparable to HEC-HMS and superior to ARX model. This indicates a strong potential for DENFIS to be used in rainfall-runoff modeling.