Efficient $$k$$ k -closest pair queries in general metric spaces

Abstract

Given two object sets $$P$$P and $$Q$$Q, a k-closest pair$$(k\hbox {CP})$$(kCP)query finds $$k$$k closest object pairs from $$P\times Q$$P×Q. This operation is common in many real-life applications such as GIS, data mining, and recommender systems. Although it has received much attention in the Euclidean space, there is little prior work on the metric space. In this paper, we study the problem of kCP query processing in general metric spaces, namely Metric kCP$$(\hbox {M}k\hbox {CP})$$(MkCP)search, and propose several efficient algorithms using dynamic disk-based metric indexes (e.g., M-tree), which can be applied to arbitrary type of data as long as a certain metric distance is defined and satisfies the triangle inequality. Our approaches follow depth-first and/or best-first traversal paradigm(s), employ effective pruning rules based on metric space properties and the counting information preserved in the metric index, take advantage of aggressive pruning and compensation to further boost query efficiency, and derive a node-based cost model for $$\hbox {M}k\hbox {CP}$$MkCP retrieval. In addition, we extend our techniques to tackle two interesting variants of $$\hbox {M}k\hbox {CP}$$MkCP queries. Extensive experiments with both real and synthetic data sets demonstrate the performance of our proposed algorithms, the effectiveness of our developed pruning rules, and the accuracy of our presented cost model.