Q-grams-imp: an improved q-grams algorithm aimed at edit similarity join

Abstract

Similarity join is more and more important in many applications and has attracted wide-spread attention from scholars and communities. Similarity join has been used in many applications, such as spell checking, copy detection, entity linking, pattern recognition and so on. Actually, in many web and enterprise scenarios, where typos and misspellings often occur, we need to find an efficient algorithm to handle these situations. In this paper, we propose an improved algorithm on q-grams called q-grams-imp that is aimed at solving edit similarity join. We use this algorithm in order to reduce the number of tokens and thus reduce space costs, so it is fit best for same size strings. But for different sizes of strings, we need to handle these strings in order to fit for the algorithm. Finally, we conclude and get the results that our proposed algorithm is better than the traditional method.