Bahasa Indonesia pre-trained word vector generation using word2vec for computer and information technology field

Abstract

Words embedding or distributed representations is a popular method for representing words. In this method, the resulting vector value is a set of real values with specific dimensions that are more effective than the Bag of Word (BoW) method. Also, the advantages of distributed representations can produce word vectors that contain semantic and syntactic information, so that word vectors with close meanings will have close word vectors. However, distributed representation requires a huge corpus with a long training time. For this reason, many researchers have created trained word vectors that can be used repeatedly. The problem is that the available trained word vectors are usually general domain word vectors. This study aims to form pre-trained word vectors for specific domains, namely computers and information technology. Researchers used a dataset of student scientific papers from the Universitas Sumatera Utara (USU) repository. Researchers used the word2vec model, where the model has two architectures, namely the Continuous Bag-of-Words (CBOW) and Skip-gram. This research’s result is word2vec model with the CBOW method is more effective than the Skip-gram method.