Part of Speech Tagging in Bengali Using Support Vector Machine

Abstract

Part of speech (POS) tagging is the task of labeling each word in a sentence with its appropriate syntactic category called part of speech. POS tagging is a very important preprocessing task for language processing activities. This paper reports about task of POS tagging for Bengali using support vector machine (SVM). The POS tagger has been developed using a tagset of 26 POS tags, defined for the Indian languages. The system makes use of the different contextual information of the words along with the variety of features that are helpful in predicting the various POS classes. The POS tagger has been trained, and tested with the 72,341, and 20 K wordforms, respectively. Experimental results show the effectiveness of the proposed SVM based POS tagger with an accuracy of 86.84%. Results show that the lexicon, named entity recognizer and different word suffixes are effective in handling the unknown word problems and improve the accuracy of the POS tagger significantly. Comparative evaluation results have demonstrated that this SVM based system outperforms the three existing systems based on the hidden markov model (HMM), maximum entropy (ME) and conditional random field (CRF).