Handling Datasets in a Multi-Relational Environment: Cluster Dispersion vs Cluster Purity

Abstract

Clustering multiple-instances in a multi-relational environment requires data transformations (e.g. data aggregation) from datasets stored in multiple tables into a single table. Unfortunately, most relational databases are limited to a few basic methods of aggregation (e.g. max, min, sum, count, ave) to aggregate continuous and categorical values. Therefore, data transformation is limited only to aggregation of continuous and categorical values. In this paper, to get the best number of clusters, we propose a genetic semi-supervised clustering technique as a means of aggregating data stored in multiple tables. This algorithm is suitable for classification of datasets with a high degree of one-to-many associations, in which a single record has multiple instances that are associated with it. The clustering algorithm can be used in two ways. One is the unsupervised clustering, where the user may control the result of clustering by optimizing the value of cluster dispersion. The other is a semi-supervised clustering, where the user uses an unsupervised clustering method optimized with a genetic algorithm incorporating a measure of classification accuracy used in decision tree algorithm, the GINI index. In this paper, we examine both methods to dynamically cluster multiple instances, as a means of aggregating them, and illustrate the effectiveness of the semi-supervised genetic algorithm-based clustering technique.