Top-N Query Processing with Real-time Entity Resolution

Abstract

Entity Resolution (ER) is the process of identifying records in a database that refer to the same real-world entity. A top-N query in a database is to find a sorted set of N tuples that best, but not necessarily completely, satisfy the query condition. Traditional techniques of top-N query processing have not yet integrated with ER, and as a result, they will become invalid for dirty datasets with duplicate tuples. In this paper, we propose two methods for processing top-N queries with real-time ER. One employs learning mechanisms and utilizes the Select-From-Where SQL statements based on a relational database management system. The second method, while not using such SQL statements, is a database-friendly threshold algorithm using simple lists. Extensive experiments are conducted to measure the performance of the methods for both clean and dirty datasets.