Traditional Hierarchical Clustering Algorithm Research Articles

Boosting cluster tree with reciprocal nearest neighbors scoring

Clustering plays a pivotal role in knowledge processing, knowledge bases, and expert systems, enabling AI systems to acquire knowledge effectively. Hierarchical clustering, in particular, offers an intelligent approach to represent knowledge hierarchically by transforming raw data into one/multiple tree-shaped components. However, a notable difficulty arises when attempting to pinpoint appropriate representative points within lower levels of the cluster tree. These points are of paramount importance, as they serve as the roots for subsequent aggregation within the upper levels of the cluster tree. Traditional hierarchical clustering algorithms have relied on rudimentary techniques to select these representative points, which may not provide an adequate representation. Consequently, the resulting cluster tree often falls short in terms of empirical performance. To address this shortcoming, we proposed an innovative hierarchical clustering algorithm in this paper. The proposed algorithm is designed to efficiently identify the representative point within each sub-minimum-spanning-tree during the construction of the cluster tree, achieved by topology-based scoring the reciprocal nearest data points. Rigorous testing on UCI datasets has demonstrated the superior clustering accuracy (measured by Rand Index and Normalized Mutual Information) of our proposed algorithm compared to other benchmark algorithms. Further analysis reveals that our algorithm boasts a O(nlogn) time-complexity and a O(logn) space-complexity, indicating its scalability and efficiency in handling large-scale data with minimal time and storage costs. Importantly, our algorithm’s ability to process up to two million data points on a standard personal computer underscores its cost-effectiveness.

An Improved Hierarchical Clustering Algorithm Based on the Idea of Population Reproduction and Fusion

Aiming to resolve the problems of the traditional hierarchical clustering algorithm that cannot find clusters with uneven density, requires a large amount of calculation, and has low efficiency, this paper proposes an improved hierarchical clustering algorithm (referred to as PRI-MFC) based on the idea of population reproduction and fusion. It is divided into two stages: fuzzy pre-clustering and Jaccard fusion clustering. In the fuzzy pre-clustering stage, it determines the center point, uses the product of the neighborhood radius eps and the dispersion degree fog as the benchmark to divide the data, uses the Euclidean distance to determine the similarity of the two data points, and uses the membership grade to record the information of the common points in each cluster. In the Jaccard fusion clustering stage, the clusters with common points are the clusters to be fused, and the clusters whose Jaccard similarity coefficient between the clusters to be fused is greater than the fusion parameter jac are fused. The common points of the clusters whose Jaccard similarity coefficient between clusters is less than the fusion parameter jac are divided into the cluster with the largest membership grade. A variety of experiments are designed from multiple perspectives on artificial datasets and real datasets to demonstrate the superiority of the PRI-MFC algorithm in terms of clustering effect, clustering quality, and time consumption. Experiments are carried out on Chinese household financial survey data, and the clustering results that conform to the actual situation of Chinese households are obtained, which shows the practicability of this algorithm.

Functional distributional clustering using spatio-temporal data

This paper presents a new method called the functional distributional clustering algorithm (FDCA) that seeks to identify spatially contiguous clusters and incorporate changes in temporal patterns across overcrowded networks. This method is motivated by a graph-based network composed of sensors arranged over space where recorded observations for each sensor represent a multi-modal distribution. The proposed method is fully non-parametric and generates clusters within an agglomerative hierarchical clustering approach based on a measure of distance that defines a cumulative distribution function over temporal changes for different locations in space. Traditional hierarchical clustering algorithms that are spatially adapted do not typically accommodate the temporal characteristics of the underlying data. The effectiveness of the FDCA is illustrated using an application to both empirical and simulated data from about 400 sensors in a 2.5 square miles network area in downtown San Francisco, California. The results demonstrate the superior ability of the the FDCA in identifying true clusters compared to functional only and distributional only algorithms and similar performance to a model-based clustering algorithm.

An improved agglomerative hierarchical clustering anomaly detection method for scientific data

SummaryAnomaly detection tries to find out the data that disobeys the rule of majority data or expected patterns. The traditional hierarchical clustering algorithms have been adopted to detect anomaly, but have the disadvantages of low effectiveness and unstability. So we propose an improved agglomerative hierarchical clustering method for anomaly detection. It dynamically adjusts the optimum clustering number according to the self‐defined criterion to save the trouble of manually picking clustering number, and determines the optimum clustering distance mode according to cophenetic correlation coefficient to reduce the procedure of manually testing the suitable distance mode in each iteration. The performances of proposed method are verified on tensile test, HTRU2 and credit card dataset. Compared with the traditional methods, our method possesses the most comprehensive performance (the highest F‐measure with less iterations), which shows effectiveness of anomaly detection. And compared with the traditional methods (single, complete, average, and centroid mode), our method achieves the best performance on tensile test and HTRU2 dataset, showing stronger generalization. Compared with other methods (Decision + Gradient Boosted Tree, Decision Trees + Decision Stump, etc) on credit card dataset, our method obtains similar accuracy, and ranks in the top level in the aspect of sensitivity.

A Linguistic Large Group Decision Making Method Based on the Cloud Model

Serving as an effective tool for describing both randomness and fuzziness of qualitative concepts, the cloud model has become a common topic of research. By realizing the uncertain transformation between qualitative concepts and quantitative data, the cloud model provides a new way to deal with group decision making problems in the linguistic environment. However, classical methods based on the cloud model mainly focus on addressing problems with a small number of decision makers. In this paper, we apply the cloud model to the decision making problems that involve a large group of decision makers. Specifically, we first define a new measure of fuzzy distance for clouds based on the $\boldsymbol { \alpha -cuts}$ . On the basis of the proposed fuzzy distance measure, we then present a new similarity measure between clouds. Next, we construct an improved clustering approach based on the traditional hierarchical clustering algorithm. Furthermore, we develop a hybrid weight scheme to obtain the cluster weight vector, which takes both the subgroup size and the variance into consideration. Moreover, we present a consensus-based method based on the cloud model for large group decision making with linguistic information. Finally, in order to confirm the validity and effectiveness of the proposed method, we give an application in the context of the Belt and Road Initiative of China, and perform some detailed comparisons to show the advantages of the proposed method.

Clustering Bathymetric Data for Electronic Navigational Charts

An electronic navigational chart is a major source of information for the navigator. The component that contributes most significantly to the safety of navigation on water is the information on the depth of an area. For the purposes of this article, the authors use data obtained by the interferometric sonar GeoSwath Plus. The data were collected in the area of the Port of Szczecin. The samples constitute large sets of data. Data reduction is a procedure to reduce the size of a data set to make it easier and more effective to analyse. The main objective of the authors is the compilation of a new reduction algorithm for bathymetric data. The clustering of data is the first part of the search algorithm. The next step consists of generalisation of bathymetric data. This article presents a comparison and analysis of results of clustering bathymetric data using the following selected methods:K-means clustering algorithm, traditional hierarchical clustering algorithms and self-organising map (using artificial neural networks).

A clustering-based ontology matching technique using cognitive theory and concept importance

Ontology matching techniques play a crucial role in solving semantic heterogeneity problem. These techniques have gained more importance because of the development of numerous ontologies developed for the same domain by different domain experts. The perception of concepts, relationships and context varies from one expert to other. This paper proposes a method of ontology matching which is based on the clustering method. The proposed ontology matching algorithm partitions the large ontologies into clusters using traditional hierarchical agglomerative clustering (HAC) and concept algebra-based clustering algorithm. The work reported in this paper is threefold: i) reduction of ontology concept match by formation of ontology clusters using HAC and formation of ontology clusters using three dimensions of similarity viz. structural similarity, attribute similarity and semantic comprehensive correlation degree which is perception conceived from cognitive algebra theory; ii) identification of quality clusters using the concept importance measure which quantifies importance of concepts using concept types and relationship types; iii) a refactoring technique is used to analyse how the precision of the ontology matching system improves by implementing renaming operation (RN). The proposed ontology matching system is evaluated using OAEI benchmark datasets.

Shortest-linkage-based parallel hierarchical clustering on main-belt moving objects of the solar system

Data clustering is an important data preparation process in many scientific analysis researches. In astronomy, although the distributed environments and modern observation techniques enable users to collect and access huge amounts of data, the corresponding clustering process may become very costly. One of the challenges is that the sequential clustering algorithms, that can be applied to cluster hundreds of thousand main-belt asteroids to reason about the origins of the main-belt asteroids, may not be used in the distributed environment directly. Therefore, this study focuses on the problem of parallelizing the traditional hierarchical agglomerative clustering algorithm using shortest-linkage. We propose a new parallel hierarchical agglomerative clustering algorithm based on the master–worker model. The master process divides the whole computation into several small tasks, and distributes the tasks to the worker processes for parallel processing. Then, the master process merges the results from the worker processes to form a hierarchical data structure. The proposed algorithm uses a pruning threshold to reduce the execution time and the storage requirement during the computation. It also supports fast incremental update that merges new data items into a constructed hierarchical tree in seconds, given a tree of about 550,000 data items. To evaluate the performance of our algorithm, this study has conducted several experiments using the MPCORB dataset and a dataset from the DVO database. The results confirm the efficiency of our proposed methodology. Compared with prior similar studies, the proposed algorithm is more flexible and practical in the problem of distributed hierarchical agglomerative clustering.

Genomic and proteomic analysis with dynamically growing self organising tree (DGSOT) for measuring clinical outcomes of cancer

Genomics and proteomics microarray technologies are used for analysing molecular and cellular expressions of cancer. This creates a challenge for analysis and interpretation of the data generated as it is produced in large volumes. The current review describes a combined system for genetic, molecular interpretation and analysis of genomics and proteomics technologies that offers a wide range of interpreted results. Artificial neural network systems technology has the type of programmes to best deal with these large volumes of analytical data. The artificial system to be recommended here is to be determined from the analysis and selection of the best of different available technologies currently being used or reviewed for microarray data analysis. The system proposed here is a tree structure, a new hierarchical clustering algorithm called a dynamically growing self-organizing tree (DGSOT) algorithm, which overcomes drawbacks of traditional hierarchical clustering algorithms. The DGSOT algorithm combines horizontal and vertical growth to construct a mutlifurcating hierarchical tree from top to bottom to cluster the data. They are designed to combine the strengths of Neural Networks (NN), which have speed and robustness to noise, and hierarchical clustering tree structure which are minimum prior requirement for number of clusters specification and training in order to output results of interpretable biological context. The combined system will generate an output of biological interpretation of expression profiles associated with diagnosis of disease (including early detection, molecular classification and staging), metastasis (spread of the disease to non-adjacent organs and/or tissues), prognosis (predicting clinical outcome) and response to treatment; it also gives possible therapeutic options ranking them according to their benefits for the patient. Key words : Genomics, proteomics, microarray, dynamically growing self-organizing tree (DGSOT).

带信息反馈的凝聚层次聚类算法

The paper proposes an hierarchical agglomerative clustering with information feedback, aiming to solve the problems (the absence of direction information in similarity measurement and relatively poor adaptability) in traditional hierarchical clustering algorithms based on similarity. In the algorithm, the unpredictable complicated data structure is described with three basic structural feature units, and then it is modeled to construct a general framework of similarity measurement and agglomerative hierarchical clustering algorithm. The algorithm adds information feedback into the agglomerative hierarchical clustering, and specifies similarity measurement according to different stages, so as to make full use of the direction and distance information between pairs of points in clustering. The advantages of the algorithm are as follows: (i) the powerful adaptability allows the algorithm to process complicated data sets without any priors or assumption; (ii) the powerful robustness to noises permits the algorithm to recognize noises without any pretreatment of the data set. The superiority of the algorithm is proved by the application on synthetic and real data sets.

Network Based Anomaly Intrusion Detection System Using SVM

The security and integrity of a computer system is compromised when an intrusion occurs. It becomes impossible for legitimate users to access different network services when network-based attacks purposely occupy or sabotage network resources and services. Our proposed method is a scalable detection method for network based anomalies. We use Support Vector Machines (SVM) for classification. This paper presents a method for enhancing the training time of SVM, particularly when dealing with large data sets, using hierarchical clustering technique. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the problems of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps to find the boundary points, which are the most qualified data points to train SVM, between any two classes. We present a new approach of combination of SVM and DGSOT, which begins with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We show that our proposed variations contribute significantly in improving the training process of SVM with high percentage of detection accuracy.

Network based anomaly intrusion detection system using SVM

The security and integrity of a computer system is compromised when an intrusion occurs. It becomes impossible for legitimate users to access different network services when network-based attacks purposely occupy or sabotage network resources and services. Our proposed method is a scalable detection method for network based anomalies. We use Support Vector Machines (SVM) for classification. This paper presents a method for enhancing the training time of SVM, particularly when dealing with large data sets, using hierarchical clustering technique. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the problems of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps to find the boundary points, which are the most qualified data points to train SVM, between any two classes. We present a new approach of combination of SVM and DGSOT, which begins with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We show that our proposed variations contribute significantly in improving the training process of SVM with high percentage of detection accuracy. Keywords: SVM, classification, Intrusion detection, Intrusion detection System, Network Security

Projective ART with buffers for the high dimensional space clustering and an application to discover stock associations

Unlike to traditional hierarchical and partitional clustering algorithms which always fail to deal with very large databases, a neural network architecture, projective adaptive resonance theory (PART), is developed for the high dimensional space clustering. However, the success of the PART algorithm depends on both accurate parameters and satisfied orders of input data sets. These disadvantages prevent PART from being applied to realtime databases. In this paper, we propose an improved method, PART with buffer management, to overcome these disadvantages. The major contributions of our method are introducing a buffer management and a new similar degree function and buffer checkout process. The buffer management mechanism allows data sets not to be immediately clustered to one cluster. The purpose of the average similar degree is to successfully work with high similar noise data sets and partly achieve an order-independent objective without correct parameters. And the average similar degree has a good attribute, the parameter-tolerance. Namely, the clustering result does not depend on the precise choice of input parameters, and different parameter values have close clustering results including dimensions associated with clusters. The buffer checkout process can handle a huge amount of input data sets by a small buffer space. Also, simulations and comparisons in high dimensional spaces are reported, and an application by using our algorithm to find stock concurrence association rules is given finally.

Rough clustering of sequential data

This paper presents a new indiscernibility-based rough agglomerative hierarchical clustering algorithm for sequential data. In this approach, the indiscernibility relation has been extended to a tolerance relation with the transitivity property being relaxed. Initial clusters are formed using a similarity upper approximation. Subsequent clusters are formed using the concept of constrained-similarity upper approximation wherein a condition of relative similarity is used as a merging criterion. We report results of experimentation on msnbc web navigation dataset that are intrinsically sequential in nature. We have compared the results of the proposed approach with that of the traditional hierarchical clustering algorithm using vector coding of sequences. The results establish the viability of the proposed approach. The rough clusters resulting from the proposed algorithm provide interpretations of different navigation orientations of users present in the sessions without having to fit each object into only one group. Such descriptions can help web miners to identify potential and meaningful groups of users.

Comparing SOM neural network with Fuzzy c-means, K-means and traditional hierarchical clustering algorithms

In this paper we present a comparison among some nonhierarchical and hierarchical clustering algorithms including SOM (Self-Organization Map) neural network and Fuzzy c-means methods. Data were simulated considering correlated and uncorrelated variables, nonoverlapping and overlapping clusters with and without outliers. A total of 2530 data sets were simulated. The results showed that Fuzzy c-means had a very good performance in all cases being very stable even in the presence of outliers and overlapping. All other clustering algorithms were very affected by the amount of overlapping and outliers. SOM neural network did not perform well in almost all cases being very affected by the number of variables and clusters. The traditional hierarchical clustering and K-means methods presented similar performance.

A new intrusion detection system using support vector machines and hierarchical clustering

Whenever an intrusion occurs, the security and value of a computer system is compromised. Network-based attacks make it difficult for legitimate users to access various network services by purposely occupying or sabotaging network resources and services. This can be done by sending large amounts of network traffic, exploiting well-known faults in networking services, and by overloading network hosts. Intrusion Detection attempts to detect computer attacks by examining various data records observed in processes on the network and it is split into two groups, anomaly detection systems and misuse detection systems. Anomaly detection is an attempt to search for malicious behavior that deviates from established normal patterns. Misuse detection is used to identify intrusions that match known attack scenarios. Our interest here is in anomaly detection and our proposed method is a scalable solution for detecting network-based anomalies. We use Support Vector Machines (SVM) for classification. The SVM is one of the most successful classification algorithms in the data mining area, but its long training time limits its use. This paper presents a study for enhancing the training time of SVM, specifically when dealing with large data sets, using hierarchical clustering analysis. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the drawbacks of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps find the boundary points, which are the most qualified data points to train SVM, between two classes. We present a new approach of combination of SVM and DGSOT, which starts with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We compare our approach with the Rocchio Bundling technique and random selection in terms of accuracy loss and training time gain using a single benchmark real data set. We show that our proposed variations contribute significantly in improving the training process of SVM with high generalization accuracy and outperform the Rocchio Bundling technique.

Exploiting homogeneity in protein sequence clusters for construction of protein family hierarchies

In the field of proteomics, protein hierarchies based on sequence analysis have been extensively applied to automate the annotations of new proteins and facilitate the discovery and analysis of protein families. However, the presence of ambiguous similarities in large databases increases the difficulty of delivering protein family hierarchies with favorable sensitivity and specificity. This work develops the HomoClust algorithm that exploits the homogeneity of protein sequences in generating protein family hierarchies. HomoClust improves the clustering quality of traditional hierarchical clustering algorithms by adopting different clustering mechanisms for different levels of sequence similarity. With considering homogeneity detection during clustering process, HomoClust increases the sensitivity of protein clusters without a drop in high specificity.

Hierarchical clustering algorithm for comprehensive orthologous-domain classification in multiple genomes

Ortholog identification is a crucial first step in comparative genomics. Here, we present a rapid method of ortholog grouping which is effective enough to allow the comparison of many genomes simultaneously. The method takes as input all-against-all similarity data and classifies genes based on the traditional hierarchical clustering algorithm UPGMA. In the course of clustering, the method detects domain fusion or fission events, and splits clusters into domains if required. The subsequent procedure splits the resulting trees such that intra-species paralogous genes are divided into different groups so as to create plausible orthologous groups. As a result, the procedure can split genes into the domains minimally required for ortholog grouping. The procedure, named DomClust, was tested using the COG database as a reference. When comparing several clustering algorithms combined with the conventional bidirectional best-hit (BBH) criterion, we found that our method generally showed better agreement with the COG classification. By comparing the clustering results generated from datasets of different releases, we also found that our method showed relatively good stability in comparison to the BBH-based methods.

HIERARCHICAL CLUSTERING FOR COMPLEX DATA

In this paper we introduce a new tree-structured self-organizing neural network called a dynamical growing self-organizing tree (DGSOT). This DGSOT algorithm constructs a hierarchy from top to bottom by division. At each hierarchical level, the DGSOT optimizes the number of clusters, from which the proper hierarchical structure of the underlying data set can be found. We propose a K-level up distribution (KLD) mechanism. This KLD scheme increases the scope for data distribution in the hierarchy, which allows the data mis-clustered in the early stages to be re-evaluated at a later stage increasing the accuracy of the final clustering result. The DGSOT algorithm, combined with the KLD mechanism, overcomes the drawbacks of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). The DGSOT algorithm has been tested on two benchmark data sets including gene expression complex data set and we observe that our algorithm extracts patterns with different levels of abstraction. Furthermore, our approach is useful on recognizing features in complex gene expression data. As a dendrogram, these results can be easily displayed for visualization.

RECENT RESULTS IN HIERARCHICAL CLUSTERING: I–THE REDUCIBLE NEIGHBORHOODS CLUSTERING ALGORITHM

The clustering of large data sets is of great interest in fields such as pattern recognition, numerical taxonomy, image or speech processing. The traditional Ascendant Hierarchical Algorithm (AHC) cannot be run for sets of more than a few thousand elements. The reducible neighborhoods clustering algorithm, which is presented in this paper, has overtaken the limits of the traditional hierarchical clustering algorithm by generating an exact hierarchy on a large data set. The theoretical justification of this algorithm is the so-called Bruynooghe reducibility principle, that lays down the condition under which the exact hierarchy may be constructed locally, by carrying out aggregations in restricted regions of the representation space. As for the Day and Edelsbrunner algorithm, the maximum theoretical time complexity of the reducible neighborhoods clustering algorithm is O(n2 log n), regardless of the chosen clustering strategy. But the reducible neighborhoods clustering algorithm uses the original data table and its practical performances are by far better than Day and Edelsbrunner’s algorithm, thus allowing the hierarchical clustering of large data sets, i.e. composed of more than 10 000 objects.