Abstract
Entity resolution (ER) is a problem of matching records from different datasets that refer to the same entities. Accurate and fast entity resolution has broad applications in the field of data management. Although many ER approaches have been proposed, it is still a challenge to design a scalable algorithm to cope with massive datasets. In this paper, we propose a parallel entity resolution algorithm with Spark programming model, which can be executed on the computer cluster in parallel. In our approach, we adopt inverted indices and attribute-related TF-IDF weights of tokens to improve the precision and efficiency of the algorithm. Experimental results show that the proposed parallel ER algorithm achieves good accuracy and performance, and the algorithm is scalable for processing entity resolution of large datasets.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: DEStech Transactions on Engineering and Technology Research
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
