Standard Clustering Algorithms Research Articles

Community detection in networks using graph embeddings.

Graph embedding methods are becoming increasingly popular in the machine learning community, where they are widely used for tasks such as node classification and link prediction. Embedding graphs in geometric spaces should aid the identification of network communities as well because nodes in the same community should be projected close to each other in the geometric space, where they can be detected via standard data clustering algorithms. In this paper, we test the ability of several graph embedding techniques to detect communities on benchmark graphs. We compare their performance against that of traditional community detection algorithms. We find that the performance is comparable, if the parameters of the embedding techniques are suitably chosen. However, the optimal parameter set varies with the specific features of the benchmark graphs, like their size, whereas popular community detection algorithms do not require any parameter. So, it is not possible to indicate beforehand good parameter sets for the analysis of real networks. This finding, along with the high computational cost of embedding a network and grouping the points, suggests that, for community detection, current embedding techniques do not represent an improvement over network clustering algorithms.

Novel electricity pattern identification system based on improved I-nice algorithm

As a result of the rapid increase in smart-metre popularity, a large volume of smart electricity data has been generated. The underlying information contained within such data sets is very helpful and useful to businesses and power companies. However, effective mining of this valuable knowledge is challenging for both industry and academia. Current research mainly focuses on application of statistical analyses to available smart electricity data rather than construction of an advanced learning system. This paper reports development of an intelligent system to identify electricity patterns within industrial electricity data. The core component of this novel identification system is the I-nice-α clustering algorithm, which is a variant of the I-nice algorithm. I-nice-α utilises kernel density estimation technology and the minimum mean discrepancy to optimise the cluster number and determine the cluster centres, respectively, of the target data. Our theoretical analysis proves that I-nice-α has lower computational complexity than I-nice. We also compare I-nice-α with I-nice and four other standard clustering algorithms (i.e., the Gaussian mixture model (GMM), density-based spatial clustering of applications with noise (DBSCAN), the fuzzy c-means (FCM) algorithm, and balanced iterative reducing and clustering using hierarchies (BIRCH)), based on 15 benchmark data sets and electricity consumption data. The experiment results show that I-nice-α achieves superior clustering performance; thus, the feasibility and effectiveness of the improved algorithm are demonstrated. In addition, hierarchical (i.e., daily and annual) electricity patterns are determined by analysing industrial electricity data with I-nice-α; this result elucidates the production organism and power assignment. Thus, the proposed electricity pattern identification system has considerable potential application among power companies and businesses.

Identification of Fuzzy Rule-Based Models With Output Space Knowledge Guidance

In this article, we advocate that a knowledge tidbit residing in the output space could be helpful in improving the performance (accuracy) of the fuzzy rule-based model. It states that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">if two outputs are far apart from each other</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it is advisable to place their corresponding inputs in different clusters when forming subspaces of the input space</i> . Considering this knowledge guidance mechanism, we propose two different methods to partition the input space. In the first method, input data are first partitioned with the use of the standard clustering algorithm, say fuzzy C-means; here, a constructed partition matrix is reflective of the structure present in the input space. Then, the knowledge tidbit is used to adjust the entries of the original partition matrix in such a way that those input data whose corresponding output data are far apart from each other are assigned with low values of proximity. In the second method, we propose two strategies to modify the distance between input data and a prototype (cluster center) identified in the input space. The crux of this method is that if there are many input data (which, in virtue of the knowledge tidbit, are regarded as being far-apart from the input data of interest) around a certain prototype, the distance between the input data of interest and this prototype should be penalized. Thus, the membership of these input data to the prototype is reduced. The comprehensive experimental studies carried out on both synthetic and publicly available data are used to examine the usefulness of the proposed methods.

S-RASTER: contraction clustering for evolving data streams

Contraction Clustering (RASTER) is a single-pass algorithm for density-based clustering of 2D data. It can process arbitrary amounts of data in linear time and in constant memory, quickly identifying approximate clusters. It also exhibits good scalability in the presence of multiple CPU cores. RASTER exhibits very competitive performance compared to standard clustering algorithms, but at the cost of decreased precision. Yet, RASTER is limited to batch processing and unable to identify clusters that only exist temporarily. In contrast, S-RASTER is an adaptation of RASTER to the stream processing paradigm that is able to identify clusters in evolving data streams. This algorithm retains the main benefits of its parent algorithm, i.e. single-pass linear time cost and constant memory requirements for each discrete time step within a sliding window. The sliding window is efficiently pruned, and clustering is still performed in linear time. Like RASTER, S-RASTER trades off an often negligible amount of precision for speed. Our evaluation shows that competing algorithms are at least 50% slower. Furthermore, S-RASTER shows good qualitative results, based on standard metrics. It is very well suited to real-world scenarios where clustering does not happen continually but only periodically.

An empirical evaluation of random transformations applied to ensemble clustering

Ensemble clustering techniques have improved in recent years, offering better average performance between domains and data sets. Benefits range from finding novelty clustering which are unattainable by any single clustering algorithm to providing clustering stability, such that the quality is little affected by noise, outliers or sampling variations. The main clustering ensemble strategies are: to combine results of different clustering algorithms; to produce different results by resampling the data, such as in bagging and boosting techniques; and to execute a given algorithm multiple times with different parameters or initialization. Often ensemble techniques are developed for supervised settings and later adapted to the unsupervised setting. Recently, Blaser and Fryzlewicz proposed an ensemble technique to classification based on resampling and transforming input data. Specifically, they employed random rotations to improve significantly Random Forests performance. In this work, we have empirically studied the effects of random transformations based in rotation matrices, Mahalanobis distance and density proximity to improve ensemble clustering. Our experiments considered 12 data sets and 25 variations of random transformations, given a total of 5580 data sets applied to 8 algorithms and evaluated by 4 clustering measures. Statistical tests identified 17 random transformations that are viable to be applied to ensembles and standard clustering algorithms, which had positive effects on cluster quality. In our results, the best performing transforms were Mahalanobis-based transformations.

Relevance- and interface-driven clustering for visual information retrieval

Search results of spatio-temporal data are often displayed on a map, but when the number of matching search results is large, it can be time-consuming to individually examine all results, even when using methods such as filtered search to narrow the content focus. This suggests the need to aggregate results via a clustering method. However, standard unsupervised clustering algorithms like K-means (i) ignore relevance scores that can help with the extraction of highly relevant clusters, and (ii) do not necessarily optimize search results for purposes of visual presentation. In this article, we address both deficiencies by framing the clustering problem for search-driven user interfaces in a novel optimization framework that (i) aims to maximize the relevance of aggregated content according to cluster-based extensions of standard information retrieval metrics and (ii) defines clusters via constraints that naturally reflect interface-driven desiderata of spatial, temporal, and keyword coherence that do not require complex ad-hoc distance metric specifications as in K-means. After comparatively benchmarking algorithmic variants of our proposed approach – RadiCAL – in offline experiments, we undertake a user study with 24 subjects to evaluate whether RadiCAL improves human performance on visual search tasks in comparison to K-means clustering and a filtered search baseline. Our results show that (a) our binary partitioning search (BPS) variant of RadiCAL is fast, near-optimal, and extracts higher-relevance clusters than K-means, and (b) clusters optimized via RadiCAL result in faster search task completion with higher accuracy while requiring a minimum workload leading to high effectiveness, efficiency, and user satisfaction among alternatives.

Spatial Correlation Based Clustering with dynamic CH Selection Scheme towards Enhanced QoS Metrics in Wireless Sensor Networks

Resource constrained wireless sensor nodes are generally randomly distributed in a given area of interest to sense required information and the sensed data is transmitted to the sink station or Base Station (BS) through various clustering and data routing algorithms. The standard clustering algorithms that are aimed at efficient data routing techniques are Low Energy Adaptive Clustering Hierarchy (LEACH) algorithm , Distributed Energy Efficient Clustering (DEEC) algorithm, Stable Election Protocol (SEP) and various others. Most of these algorithms are the various variants of LEACH. Our proposed scheme of data gathering and routing is based on a two hop structure wherein the Cluster Leader (CL) forwards the aggregated received data from the Cluster members to the Sink directly. The Chance-value that determines the Cluster Leader in a round is decided by a combination of parameters specific to the Sensor nodes in the cluster. In our simulation approach we have also tried to analyze the effect of changing density of sensor nodes in the select area. Thus in our proposed scheme, we embark on a fixed Clustering scheme where the CL is selected dynamically so as to extend the network’s lifetime and achieve enhanced throughput in comparison to the standard algorithms like DEEC, SEP and LEACH.

Characterization of entanglements in glassy polymeric ensembles using the Gaussian linking number.

We propose a method for enumerating entanglements between long chained, linear polymers that is based on the Gaussian linking number. The linking number is calculated between closely approaching segments of the macromolecular chains. Topological features of an entanglement, i.e., the extent to which one open segment winds around another, are reflected by the linking number. We show that using this measure, we can track disentanglement events through a deformation history and gain insights into how large scale disentanglements lead to failure. Incorporating an additional step where the topological entanglements identified along each chain are optimally clustered using standard clustering algorithms, we can also obtain a measure of the average number of rheological constraints that exist along each chain in an ensemble. Comparisons with other methods of enumerating entanglements, especially the primitive path analysis, are also made. Our results indicate that the linking number between two entangled segments in the undeformed state is a good indicator of the strength of the entanglement. Also, disentanglements occurring overwhelmingly around chain ends are an important cause of failure when a triaxial stress state exists in the polymer.

Object Boundary Identification using Two-phase Incremental Clustering

Abstract In order to perform the required task, understanding its environment is important for a service robot employed in indoor and outdoor environments. Learning the environment can be done using characteristic features of the environment such as the set of objects present there. Object identification and environment mapping is done using on board sensing devices like ultrasonic sensors, Lidars, and cameras. In this paper, we have employed a novel two-phase incremental clustering based approach to map object boundaries. Tracing of points on object boundaries has been done by exploring the environment in multiple paths using inexpensive ultrasonic sensors. The proposed two-phase algorithm performs much faster compared to the standard hierarchical agglomerative clustering algorithm. Effectiveness of the proposed algorithm is analyzed using a data set created with the help of a simulated environment. Analysis of the results obtained confirms that our approach is capable of capturing object boundaries that can be used as features to identify robotic environments.

Familial Clustering for Weakly-Labeled Android Malware Using Hybrid Representation Learning

Labeling malware or malware clustering is important for identifying new security threats, triaging and building reference datasets. The state-of-the-art Android malware clustering approaches rely heavily on the raw labels from commercial AntiVirus (AV) vendors, which causes misclustering for a substantial number of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">weakly-labeled malware</i> due to the inconsistent, incomplete and overly generic labels reported by these closed-source AV engines, whose capabilities vary greatly and whose internal mechanisms are opaque (i.e., intermediate detection results are unavailable for clustering). The raw labels are thus often used as the only important source of information for clustering. To address the limitations of the existing approaches, this paper presents Andre, a new ANDroid Hybrid REpresentation Learning approach to clustering weakly-labeled Android malware by preserving heterogeneous information from multiple sources (including the results of static code analysis, the meta-information of an app, and the raw-labels of the AV vendors) to jointly learn a hybrid representation for accurate clustering. The learned representation is then fed into our outlier-aware clustering to partition the weakly-labeled malware into known and unknown families. The malware whose malicious behaviours are close to those of the existing families on the network, are further classified using a three-layer Deep Neural Network (DNN). The unknown malware are clustered using a standard density-based clustering algorithm. We have evaluated our approach using 5,416 ground-truth malware from Drebin and 9,000 malware from VirusShare (uploaded between Mar. 2017 and Feb. 2018), consisting of 3324 weakly-labeled malware. The evaluation shows that Andre effectively clusters weakly-labeled malware which cannot be clustered by the state-of-the-art approaches, while achieving comparable accuracy with those approaches for clustering ground-truth samples.

Uto-Aztecan Lexicostatistics 2.0

Uto-Aztecan subgrouping has long been the subject of debate. We aim to establish a more up-to-date foundation for Uto-Aztecan lexicostatistics by reexamining Wick Miller’s influential lexicostatistic classification. Miller’s cognate density measure yields a symmetrical table based on the number of cognates each language pairing shares on a modified Swadesh-100 wordlist. However, no language has cognate sets for all word meanings (glosses) on that list. We offer an improved metric, relative cognate density, for analyzing an updated database of Kenneth Hill’s Uto-Aztecan cognate sets. We generate an asymmetrical table dividing the number of pairwise shared cognates (“match counts”) by the number of cognate sets each comparison language actually has available. Employing more standard distance-based clustering algorithms (UPGMA, Neighbor Joining, and NeighborNet), our results align with Miller’s in some respects (e.g., in identifying a Southern-Uto-Aztecan branch but not a Northern-Uto-Aztecan one) but differ in others (e.g., in not identifying “Sonoran”).

Time series classification through visual pattern recognition

In this paper, a new approach to time series classification is proposed. It transforms the scalar time series into a two-dimensional space of amplitude (time series values) and a change of amplitude (increment). Subsequently, it uses this representation to plot the data. One figure is produced for each time series. In consequence, the time series classification problem is converted into the visual pattern recognition problem. This transformation allows applying a wide range of algorithms for standard pattern recognition – in this domain, there are more options to choose from than in the domain of time series classification. In this paper, we demonstrated the high effectiveness of the new method in a series of experiments on publicly available time series. We compare our results with several state-of-the-art approaches dedicated to time series classification. The new method is robust and stable. It works for time series of differing lengths and is easy to extend and alter. Even with a baseline variant presented in an empirical study in this paper, it achieves a satisfying classification accuracy. Furthermore, the proposed conversion of raw time series into images that are subjected to feature extraction opens the possibility to apply standard clustering algorithms.

Classification of close binary stars using recurrence networks.

Close binary stars are binary stars where the component stars are close enough such that they can exchange mass and/or energy. They are subdivided into semidetached, overcontact, or ellipsoidal binary stars. A challenging problem in the context of close binary stars is their classification into these subclasses based solely on their light curves. Conventionally, this is done by observing subtle features in the light curves like the depths of adjacent minima, which is tedious when dealing with large datasets. In this work, we suggest the use of machine learning algorithms applied to quantifiers derived from recurrence networks to differentiate between classes of close binary stars. We show that overcontact binary stars occupy a region different from semidetached and ellipsoidal binary stars in a plane of characteristic path length and average clustering coefficient, computed from their recurrence networks. We use standard clustering algorithms and report that the clusters formed correspond to the standard classes with a high degree of accuracy.

Communication-Optimal Distributed Dynamic Graph Clustering

We consider the problem of clustering graph nodes over large-scale dynamic graphs, such as citation networks, images and web networks, when graph updates such as node/edge insertions/deletions are observed distributively. We propose communication-efficient algorithms for two well-established communication models namely the message passing and the blackboard models. Given a graph with n nodes that is observed at s remote sites over time [1,t], the two proposed algorithms have communication costs Õ(ns) and Õ(n + s) (Õ hides a polylogarithmic factor), almost matching their lower bounds, Ω(ns) and Ω(n + s), respectively, in the message passing and the blackboard models. More importantly, we prove that at each time point in [1,t] our algorithms generate clustering quality nearly as good as that of centralizing all updates up to that time and then applying a standard centralized clustering algorithm. We conducted extensive experiments on both synthetic and real-life datasets which confirmed the communication efficiency of our approach over baseline algorithms while achieving comparable clustering results.

SEAT-DSR: Spatial and energy aware trusted dynamic distance source routing algorithm for secure data communications in wireless sensor networks

This paper propose a novel secure routing mechanism called Spatial and Energy Aware Trusted Dynamic Distance Source Routing (SEAT-DSR) algorithm for enhancing the network life time of wireless sensor networks. Here, the spatial information, energy level, and the effectiveness of data quality are equalized by the Quality of Service (QoS) based energy aware routing algorithms. In addition to this approach, a standard clustering algorithm is also incorporates for grouping the wireless sensor nodes based on the trust score, spatial information, energy level and the distance between the nodes. In this SEAT-DSR is also capable of making decision over the evaluation metrics that are decided and expressed the QoS. Moreover, a new hierarchical trust mechanism is also introduced in this model which adopts multi-attributes of many wireless sensor nodes according to the data communication speed, data size, energy consumption, and the recommendation. This new hierarchical trust method relies over an improved the sliding window time by considering the presence of various attacks frequency to identify the attackers by discovering their anomalous behaviour. The proposed SEAT-DSR is evaluated by conducting many experiments in a simulation environment that creates by using Network Simulator-2 (NS2). The experimental results of the proposed algorithm are proved that the average packet transfer rate is increased drastically than the existing secure routing methodologies.

Testing the rogue taxa hypothesis for clustering instability

There have been longstanding concerns about the stability of hierarchical clustering. A suggested explanation for this instability is the presence of “rogue taxa”, i.e. taxa whose removal from a data set can apparently restore stability. In this study, the rogue taxa hypothesis is tested by partitioning a large data set into many smaller ones and checking for rogue behavior. The checking was performed with a standard hierarchical clustering algorithm and with a novel algorithm designed to have greater stability. It was found that rogue taxa cannot reasonably be said to exist because the status of being a rogue taxon depends on the data partition in which the taxon is embedded. In addition to the choice of data used, the choice of algorithm and algorithm parameters can have a large effect on the degree to which a taxon appears rogue. Instability in hierarchical clustering can be increased by problematic data points, but the status of data points being problematic depends not on their biological antecedents, but on their position in the local geometry of the data. The results of this study strongly suggest that instability in traditional hierarchical clustering routines is primarily a problem with the algorithm design.

DHC: A Distributed Hierarchical Clustering Algorithm for Large Datasets

Hierarchical clustering is a classical method to provide a hierarchical representation for the purpose of data analysis. However, in practical applications, it is difficult to deal with massive datasets due to their high computation complexity. To overcome this challenge, this paper presents a novel distributed storage and computation hierarchical clustering algorithm, which has a lower time complexity than the standard hierarchical clustering algorithms. Our proposed approach is suitable for hierarchical clustering on massive datasets, which has the following advantages. First, the algorithm is able to store massive dataset exceeding the main memory space by using distributed storage nodes. Second, the algorithm is able to efficiently process nearest neighbor searching along parallel lines by using distributed computation at each node. Extensive experiments are carried out to validate the effectiveness of the DHC algorithm. Experimental results demonstrate that the algorithm is 10 times faster than the standard hierarchical clustering algorithm, which is an effective and flexible distributed algorithm of hierarchical clustering for massive datasets.

Semisoft clustering of single-cell data

Motivated by the dynamics of development, in which cells of recognizable types, or pure cell types, transition into other types over time, we propose a method of semisoft clustering that can classify both pure and intermediate cell types from data on gene expression from individual cells. Called semisoft clustering with pure cells (SOUP), this algorithm reveals the clustering structure for both pure cells and transitional cells with soft memberships. SOUP involves a two-step process: Identify the set of pure cells and then estimate a membership matrix. To find pure cells, SOUP uses the special block structure in the expression similarity matrix. Once pure cells are identified, they provide the key information from which the membership matrix can be computed. By modeling cells as a continuous mixture of K discrete types we obtain more parsimonious results than obtained with standard clustering algorithms. Moreover, using soft membership estimates of cell type cluster centers leads to better estimates of developmental trajectories. The strong performance of SOUP is documented via simulation studies, which show its robustness to violations of modeling assumptions. The advantages of SOUP are illustrated by analyses of two independent datasets of gene expression from a large number of cells from fetal brain.

A Methodology for Brand Name Hierarchical Clustering Based on Social Media Data

Clustering of brand names is a common technique for identification of different groups of brand names with maximum effectiveness and similarities. In this study, we proposed a methodology for brand name clustering, based on data from social media like Twitters. Dataset consist of tweets mentioning various garment brands in them. In order to cluster the brand names, we have proposed an algorithm, named BNACA (Brand Names Agglomerate Clustering Algorithm), an extension to the standard hierarchical clustering algorithm. In the proposed algorithm, we used single linkage as a similarity measure. The proposed clustering algorithm provides consistent clustered results for various sets of brand names of garment industry. Finally, clusters of garment brand names are visualized through dendogram. The dendogram clearly shows the maximum similarities among garment brand names.

A Methodology for Brand Name Hierarchical Clustering Based on Social Media Data

Clustering of brand names is a common technique for identification of different groups of brand names with maximum effectiveness and similarities. In this study, we proposed a methodology for brand name clustering, based on data from social media like Twitters. Dataset consist of tweets mentioning various garment brands in them. In order to cluster the brand names, we have proposed an algorithm, named BNACA (Brand Names Agglomerate Clustering Algorithm), an extension to the standard hierarchical clustering algorithm. In the proposed algorithm, we used single linkage as a similarity measure. The proposed clustering algorithm provides consistent clustered results for various sets of brand names of garment industry. Finally, clusters of garment brand names are visualized through dendogram. The dendogram clearly shows the maximum similarities among garment brand names.