Robust Hierarchical Clustering Research Articles

Robust Hierarchical Clustering for Directed Networks: An Axiomatic Approach

We provide a complete taxonomic characterization of robust hierarchical clustering methods for directed networks following an axiomatic approach. We begin by introducing three practical properties ...

Detection of Auction Fraud in Commercial Sites

Online auctions have become one of the most convenient ways to commit fraud due to a large amount of money being traded every day. Shill bidding is the predominant form of auction fraud, and it is also the most difficult to detect because it so closely resembles normal bidding behavior. Furthermore, shill bidding does not leave behind any apparent evidence, and it is relatively easy to use to cheat innocent buyers. Our goal is to develop a classification model that is capable of efficiently differentiating between legitimate bidders and shill bidders. For our study, we employ an actual training dataset, but the data are unlabeled. First, we properly label the shill bidding samples by combining a robust hierarchical clustering technique and a semi-automated labeling approach. Since shill bidding datasets are imbalanced, we assess advanced over-sampling, under-sampling and hybrid-sampling methods and compare their performances based on several classification algorithms. The optimal shill bidding classifier displays high detection and low misclassification rates of fraudulent activities.

Robust hierarchical clustering for novelty identification in sensor networks: With applications to industrial systems

The paper proposes a new, robust cluster-based classification technique for Novelty Identification in sensor networks that possess a high degree of correlation among data streams. During normal operation, a uniform cluster across objects (sensors) is generated that indicates the absence of novelties. Conversely, in presence of novelty, the associated sensor is clustered distinctly from the remaining sensors, thereby isolating the data stream which exhibits the novelty. It is shown how small perturbations (stemming from noise, for instance) can affect the performance of traditional clustering methods, and that the proposed variant exhibits a robustness to such influences. Moreover, the proposed method is compared with a recently reported technique, and shown that it performs 365% faster computationally. To provide an application case study, the technique is used to identify emerging fault modes in a sensor network on a sub-15MW industrial gas turbine in presence of other abrupt, but normal changes that visually might otherwise be interpreted as malfunctions.

Assessing Side-Chain Perturbations of the Protein Backbone: A Knowledge-Based Classification of Residue Ramachandran Space

Grouping the 20 residues is a classic strategy to discover ordered patterns and insights about the fundamental nature of proteins, their structure, and how they fold. Usually, this categorization is based on the biophysical and/or structural properties of a residue's side-chain group. We extend this approach to understand the effects of side chains on backbone conformation and to perform a knowledge-based classification of amino acids by comparing their backbone ϕ,ψ distributions in different types of secondary structure. At this finer, more specific resolution, torsion angle data are often sparse and discontinuous (especially for nonhelical classes) even though a comprehensive set of protein structures is used. To ensure the precision of Ramachandran plot comparisons, we applied a rigorous Bayesian density estimation method that produces continuous estimates of the backbone ϕ,ψ distributions. Based on this statistical modeling, a robust hierarchical clustering was performed using a divergence score to measure the similarity between plots. There were seven general groups based on the clusters from the complete Ramachandran data: nonpolar/β-branched (Ile and Val), AsX (Asn and Asp), long (Met, Gln, Arg, Glu, Lys, and Leu), aromatic (Phe, Tyr, His, and Cys), small (Ala and Ser), bulky (Thr and Trp), and, lastly, the singletons of Gly and Pro. At the level of secondary structure (helix, sheet, turn, and coil), these groups remain somewhat consistent, although there are a few significant variations. Besides the expected uniqueness of the Gly and Pro distributions, the nonpolar/β-branched and AsX clusters were very consistent across all types of secondary structure. Effectively, this consistency across the secondary structure classes implies that side-chain steric effects strongly influence a residue's backbone torsion angle conformation. These results help to explain the plasticity of amino acid substitutions on protein structure and should help in protein design and structure evaluation.

Hierarchical clustering based on ordinal consistency

Hierarchical clustering is the grouping of objects of interest according to their similarity into a hierarchy, with different levels reflecting the degree of inter-object resemblance. It is an important area in data analysis and pattern recognition. In this paper, we propose a new approach for robust hierarchical clustering based on possibly incomplete and noisy similarity data. Our approach uses a novel perspective in finding the object hierarchy by trying to optimize ordinal consistency between the available similarity data and the hierarchical structure. Using experiments we show that our approach is able to perform more effectively than similar algorithms when there are substantial noises in the data. Furthermore, when similarity-ordering information is only available in the form of incomplete pairwise similarity comparisons, our approach can still be applied directly. We illustrate this by applying our approach to randomly generated hierarchies and phylogenetic tree construction from quartets, an important area in computational biology.

Rock: A robust clustering algorithm for categorical attributes

Clustering, in data mining, is useful to discover distribution patterns in the underlying data. Clustering algorithms usually employ a distance metric based (e.g., euclidean) similarity measure in order to partition the database such that data points in the same partition are more similar than points in different partitions. In this paper, we study clustering algorithms for data with boolean and categorical attributes. We show that traditional clustering algorithms that use distances between points for clustering are not appropriate for boolean and categorical attributes. Instead, we propose a novel concept of links to measure the similarity/proximity between a pair of data points. We develop a robust hierarchical clustering algorithm ROCK that employs links and not distances when merging clusters. Our methods naturally extend to non-metric similarity measures that are relevant in situations where a domain expert/similarity table is the only source of knowledge. In addition to presenting detailed complexity results for ROCK, we also conduct an experimental study with real-life as well as synthetic data sets to demonstrate the effectiveness of our techniques. For data with categorical attributes, our findings indicate that ROCK not only generates better quality clusters than traditional algorithms, but it also exhibits good scalability properties.