Clustering Of Large Databases Research Articles

PHD: an efficient data clustering scheme using partition space technique for knowledge discovery in large databases

Rapid technological advances imply that the amount of data stored in databases is rising very fast. However, data mining can discover helpful implicit information in large databases. How to detect the implicit and useful information with lower time cost, high correctness, high noise filtering rate and fit for large databases is of priority concern in data mining, specifying why considerable clustering schemes have been proposed in recent decades. This investigation presents a new data clustering approach called PHD, which is an enhanced version of KIDBSCAN. PHD is a hybrid density-based algorithm, which partitions the data set by K-means, and then clusters the resulting partitions with IDBSCAN. Finally, the closest pairs of clusters are merged until the natural number of clusters of data set is reached. Experimental results reveal that the proposed algorithm can perform the entire clustering, and efficiently reduce the run-time cost. They also indicate that the proposed new clustering algorithm conducts better than several existing well-known schemes such as the K-means, DBSCAN, IDBSCAN and KIDBSCAN algorithms. Consequently, the proposed PHD algorithm is efficient and effective for data clustering in large databases.

PATTERN BASED SUBSPACE CLUSTERING: A REVIEW

The task of biclustering or subspace clustering is a data mining technique that allows simultaneous clustering of rows and columns of a matrix. Though the definition of similarity varies from one biclustering model to another, in most of these models the concept of similarity is often based on such metrics as Manhattan distance, Euclidean distance or other Lp distances. In other words, similar objects must have close values in at least a set of dimensions. Pattern-based clustering is important in many applications, such as DNA micro-array data analysis, automatic recommendation systems and target marketing systems. However, pattern-based clustering in large databases is challenging. On the one hand, there can be a huge number of clusters and many of them can be redundant and thus makes the pattern-based clustering ineffective. On the other hand, the previous proposed methods may not be efficient or scalable in mining large databases. The objective of this paper is to perform a comparative study of all subspace clustering algorithms in terms of efficiency, accuracy and time complexity.

Hierarchical Clustering of Large Databases and Classification of Antibiotics at High Noise Levels

A new algorithm for divisive hierarchical clustering of chemical compounds based on 2D structural fragments is suggested. The algorithm is deterministic, and given a random ordering of the input, will always give the same clustering and can process a database up to 2 million records on a standard PC. The algorithm was used for classification of 1,183 antibiotics mixed with 999,994 random chemical structures. Similarity threshold, at which best separation of active and non active compounds took place, was estimated as 0.6. 85.7% of the antibiotics were successfully classified at this threshold with 0.4% of inaccurate compounds. A .sdf file was created with the probe molecules for clustering of external databases.

A search space reduction methodology for data mining in large databases

Given the present need for Customer Relationship and the increased growth of the size of databases, many new approaches to large database clustering and processing have been attempted. In this work, we propose a methodology based on the idea that statistically proven search space reduction is possible in practice. Two clustering models are generated: one corresponding to the full data set and another pertaining to the sampled data set. The resulting empirical distributions were mathematically tested to verify a tight non-linear significant approximation.

A General Approach to Clustering in Large Databases with Noise

Several clustering algorithms can be applied to clustering in large multimedia databases. The effectiveness and efficiency of the existing algorithms, however, are somewhat limited, since clustering in multimedia databases requires clustering of high-dimensional feature vectors and because multimedia databases often contain large amounts of noise. In this paper, we therefore introduce a new Kernel Density Estimation-based algorithm for clustering in large multimedia databases called DENCLUE (DENsity-based CLUstEring). Kernel Density Estimation (KDE) models the overall point density analytically as the sum of kernel (or influence) functions of the data points. Clusters can then be identified by determining density attractors and clusters of arbitrary shape can be easily described by a simple equation of the overall density function. The advantages of our KDE-based DENCLUE approach are: (1) it has a firm mathematical basis; (2) it has good clustering properties in data sets with large amounts of noise; (3) it allows a compact mathematical description of arbitrarily shaped clusters in high-dimensional data sets; and (4) it is significantly faster than existing algorithms. To demonstrate the effectiveness and efficiency of DENCLUE, we perform a series of experiments on a number of different data sets from CAD and molecular biology. A comparison with k-Means, DBSCAN, and BIRCH shows the superiority of our new algorithm.

Multi-dimensional semantic clustering of large databases for association rule mining

Clustering is an activity of finding abstractions from data [1]. These abstractions are mainly used for: (i) identifying outliers and (ii) other decision making activities. In this paper, we propose a novel application, association rule mining (ARM), based on abstractions/cluster descriptions obtained using clustering. A majority of ARM algorithms proposed in the literature are used for mining intra-transaction association rules. Typically, the ARM algorithms find large itemsets from the transaction database based on the frequency of co-occurrence of items. ARM involves two main steps [3. A.K.H. Tung, H. Lu, J. Han, L. Feng, Breaking the barrier of transactions: mining inter-transaction association rules, Proceedings of the 1999 International Conference on Knowledge Discovery and Data Mining, August 1999.3]: (i) generating large itemsets which are having the frequency of co-occurrence (support) greater than or equal to the user defined minimum support (σ) and (ii) generation of association rules from the large itemsets which satisfy the user-defined minimum confidence (c). Mining for complete set of inter-transaction association rules along a single dimension, the time axis, is proposed in [3]. In this paper, we propose a clustering scheme based on multiple dimensions for mining a complete set of inter-transaction association rules. This scheme has two components: (i) generation of descriptions of clusters based on multi-dimensional semantic grouping; our algorithm needs at most two database scans to do this step, and (ii) exploring associations between cluster descriptions/abstractions.2. Clustering process Clustering is a subjective process. It employs knowledge to group data. Knowledge is in the form of the similarity measure used, the values assigned to parameters like the number of clusters and assumptions on the nature of clusters, and structures that capture explicit knowledge [1]. We call clustering based on knowledge, semantic clustering. In this paper, we discuss clustering based on multiple dimensions including size of transaction, cost of transaction, and their combinations to generate inter-transaction associations.

An Index Structure for Data Mining and Clustering

In this paper we present an index structure, called MetricMap, that takes a set of objects and a distance metric and then maps those objects to a k-dimensional space in such a way that the distances among objects are approximately preserved. The index structure is a useful tool for clustering and visualization in data-intensive applications, because it replaces expensive distance calculations by sum-of-square calculations. This can make clustering in large databases with expensive distance metrics practical. We compare the index structure with another data mining index structure, FastMap, recently proposed by Faloutsos and Lin, according to two criteria: relative error and clustering accuracy. For relative error, we show that (i) FastMap gives a lower relative error than MetricMap for Euclidean distances, (ii) MetricMap gives a lower relative error than FastMap for non-Euclidean distances (i.e., general distance metrics), and (iii) combining the two reduces the error yet further. A similar result is obtained when comparing the accuracy of clustering. These results hold for different data sizes. The main qualitative conclusion is that these two index structures capture complementary information about distance metrics and therefore can be used together to great benefit. The net effect is that multi-day computations can be done in minutes.

ChemInform Abstract: Clustering of Large Databases of Compounds: Using the MDL “Keys” as Structural Descriptors

AbstractChemInform is a weekly Abstracting Service, delivering concise information at a glance that was extracted from about 100 leading journals. To access a ChemInform Abstract of an article which was published elsewhere, please select a “Full Text” option. The original article is trackable via the “References” option.

Clustering of Large Databases of Compounds: Using the MDL “Keys” as Structural Descriptors

An analysis of chemical structures from several commercially available libraries of compounds is presented with a view of acquiring compounds for screening. The Jarvis−Patrick clustering method has been applied, using the MDL “keys” as structural descriptors. The nature of the MDL keys is examined in this context, some features of the clustering algorithm are discussed, and clustering statistics are presented.