Improving density-based methods for hierarchical clustering of web pages

Abstract

The rapid increase of information on the web makes it necessary to improve information management techniques. One of the most important techniques is clustering web data. In this paper, we propose a new 3-phase clustering method that finds dense units in a data set using density-based algorithms. The distances in the dense units are stored in order in structures such as a min heap. In the extraction stage, these distances are extracted one by one, and their effects on the clustering process are examined. Finally, in the combination stage, clustering is completed using improved versions of well-known single and average linkage methods. All steps of the methods are performed in O(n log n) time complexity. The proposed methods have the benefit of low complexity, and experimental results show they generate clusters with high quality. Other experiments also show that they provide additional advantages, such as clustering by sampling.