Document representation and clustering models for bilingual documents clustering

Abstract

ABSTRACTCurrently, the internet has created many documents in languages other than English. People face challenges when seeking and using information; for example, non‐native English‐speaking students tend to have problems when utilizing libraries in North American universities. To help people efficiently organize information, bilingual documents clustering has advantages for practical utilization, it can divide documents into groups with the same topic and there is no need for a training dataset. Document representation and clustering models are two important parts in clustering. This paper compares four popular representation methods, vector space model (VSM), latent semantic indexing (LSI), latent Dirichlet allocation (LDA) and doc2vec (D2V), together with four different types of clustering algorithms, K‐means++, BIRCH, DBSCAN and affinity propagation (AP) to identify appropriate combinations for bilingual documents clustering. Parallel corpus and comparable corpus are all used for the bilingual datasets. Experimental results show that, clustering performance varies when combining different representation methods with clustering algorithms. It's important to make good choice of models for better documents organization.