Part-of-Speech Tagging for low resource languages: Activation function for deep learning network to work with Minimal Training Data

Abstract

Numerous natural language processing (NLP) applications exist today, especially for the most commonly spoken languages like English, Chinese, and Spanish. Popular traditional methods like Naive Bayes classifiers, Hidden Markov models, Conditional Random field-based classifiers, and other stochastic methods have contributed to this improvement over the last three decades. Recently, deep learning has led to exciting breakthroughs in several areas of artificial intelligence, including image processing and natural language processing. It is important to label words as parts of speech to begin developing most of the NLP applications. A deep study in this area reveals that these approaches require massive training data. Therefore, these approaches have not been helpful for languages not rich in digital resources. Applying these methods with very little training data prompts the need for innovative problem-solving. This paper describes our research, which examines the strengths and weaknesses of well-known approaches, such as conditional random fields and state-of-the-art deep learning models, when applied for part-of-speech tagging using minimal training data for Assamese and English. We also examine the factors affecting them. We discuss our deep learning architecture and the proposed activation function, which shows promise with little training data. The activation function categorizes words belonging to different classes with more confidence by using the outcomes of statistical methods. With minimal training, our deep learning architecture using the proposed PSM-Taylor SoftMax improves accuracy by 4%–9%, This technique is a combination of SMTaylor SoftMax and probability distribution.