Comparison of clustering methods for clinical databases

Abstract

Clustering methods can be viewed as unsupervised learning from a given dataset. Even without domain knowledge or labels such as the names of diseases given by medical experts, these methods generate partition of datasets. In some cases, these new generated classes lead to discovery of a new disease or new concept. This paper discusses how clustering methods work on a practical medical data set. For comparison, the following four clustering methods were selected and evaluated on a dataset on meningoencephalitis: single- and complete-linkage agglomerative hierarchical clustering, Ward’s method and rough clustering. For comparison, a single similarity measure, a linear combination of the Mahalanobis distance between numerical attributes and the Hamming distance between nominal attributes was given to each clustering method. Usefulness of the clustering methods was evaluated from the following viewpoints: (1) the quality of generated clusters, (2) correspondence between the attributes used to generate the high-quality clusters and clinical knowledge. The experimental results showed that the best clusters were obtained using Ward’s method where the clinically reasonable attributes were selected, which also suggested that this similarity measure would be applicable to the medical data sets.