Bagging, bumping, multiview, and active learning for record linkage with empirical results on patient identity data

Abstract

Record linkage or deduplication deals with the detection and deletion of duplicates in and across files. For this task, this paper introduces and evaluates two new machine-learning methods (bumping and multiview) together with bagging, a tree-based ensemble-approach. Whereas bumping represents a tree-based approach as well, multiview is based on the combination of different methods and the semi-supervised learning principle. After providing a theoretical background of the methods, initial empirical results on patient identity data are given. In the empirical evaluation, we calibrate the methods on three different kinds of training data. The results show that the smallest training data set, which is obtained by a simple active learning strategy, leads to the best results. Multiview can outperform the other methods only when all are calibrated on a randomly sampled training set; in all other cases, it performs worse. The results of bumping do not differ significantly from the overall best performing method bagging. We cautiously conclude that tree-based record linkage methods are likely to produce similar results because of the low-dimensionality (p≪n) and straightforwardness of the underlying problem. Multiview is possibly rather suitable for problems that are more sophisticated.