Abstract
Progression in machine learning and statistical inference are facilitating the advancement of domains like computer vision, natural language processing (NLP), automation & robotics, and so on. Among the different persuasive improvements in NLP, word embedding is one of the most used and revolutionary techniques. In this paper, we manifest an open-source library for Bangla word extraction systems named BnVec which expects to furnish the Bangla NLP research community by the utilization of some incredible word embedding techniques. The BnVec is splitted up into two parts, the first one is the Bangla suitable defined class to embed words with access to the six most popular word embedding schemes (CountVectorizer, TF-IDF, Hash Vectorizer, Word2vec, fastText, and Glove). The other one is based on the pre-trained distributed word embedding system of Word2vec, fastText, and GloVe. The pre-trained models have been built by collecting content from the newspaper, social media, and Bangla wiki articles. The total number of tokens used to build the models exceeds 395,289,960. The paper additionally depicts the performance of these models by various hyper-parameter tuning and then analyzes the results.
Highlights
Word embedding refers to the vector representation of linguistic or phonetic information
It is one of the most popular document representation models that is being comprehensively used in multiple domains of Natural language processing (NLP) application including named entity recognition [7], sentiment analysis, part of speech tagging, and so forth [20]
We have presented two methodologies for Bengali word embedding
Summary
Word embedding refers to the vector representation of linguistic or phonetic information. It is one of the most popular document representation models that is being comprehensively used in multiple domains of Natural language processing (NLP) application including named entity recognition [7], sentiment analysis, part of speech tagging, and so forth [20]. Changing over data into lower-dimensional vectors is the primary objective of this research Throughout these years various strategies for word clustering or embedding have been presented. We tried three separate techniques to choose the strategy that works best for Bangla word-embedding. These days word vector portrayal has been the most widely recognized strategy for word group creation.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: International Journal of Engineering & Technology
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
