Parker: Data fusion through consistent repairs using edit rules under partial keys

Abstract

Data integration is the problem of consolidating information provided by multiple sources. After schema mapping and duplicate detection have been dealt with, the problem consists in fusing disparate data that comes from the sources. In this work, we treat this problem as a data cleaning problem where consistency within and between sources is modelled by respectively edit rules and a partial key. The combination of these constraints leads to the definition of edit rules under a partial key (EPKs). We show that we can adapt the well-known set cover method for edit rules to the setting of EPKs and this yields an efficient algorithm to find minimal-cost repairs. These repairs then provide solutions for the initial data fusion problem. The methodology is implemented in a repair engine called Parker. Empirical results show that Parker is several orders of magnitude faster than state-of-the-art repair tools. At the same time, the quality of the repairs in terms of F1-score ranges from comparable to better compared to these tools.