Abstract
Over the past few decades, text classification problems have been widely utilized in many real time applications. Leveraging the text classification methods by means of developing new applications in the field of text mining and Natural Language Processing (NLP) is very important. In order to accurately classify tasks in many applications, a deeper insight into deep learning methods is required as there is an exponential growth in the number of complex documents. The success of any deep learning algorithm depends on its capacity to understand the nonlinear relationships of the complex models within data. Thus, a huge challenge for researchers lies in the development of suitable techniques, architectures, and models for text classification. In this paper, hybrid deep learning models, with an emphasis on positioning of attention mechanism analysis, are considered and analyzed well for text classification. The first hybrid model proposed is called convolutional Bidirectional Long Short-Term Memory (Bi-LSTM) with attention mechanism and output (CBAO) model, and the second hybrid model is called convolutional attention mechanism with Bi-LSTM and output (CABO) model. In the first hybrid model, the attention mechanism is placed after the Bi-LSTM, and then the output Softmax layer is constructed. In the second hybrid model, the attention mechanism is placed after convolutional layer and followed by Bi-LSTM and the output Softmax layer. The proposed hybrid models are tested on three datasets, and the results show that when the proposed CBAO model is implemented for IMDB dataset, a high classification accuracy of 92.72% is obtained and when the proposed CABO model is implemented on the same dataset, a high classification accuracy of 90.51% is obtained.
Highlights
A representation topic in Natural Language Processing (NLP) analysis is text classification
A hybrid model utilizing Convolutional Neural Networks (CNNs) and LSTM with the effective use of normalization techniques, dropout techniques, and rectified linear units was proposed by Rehman et al, and the results show a high accuracy and precision rate [43]
Work e technique of categorizing text into a group of words is called text classification. e analysis of text can be done automatically by text classification using NLP by means of assigning a set of predefined categories depending on its context. us, NLP which is classified into rule-based systems, machine-based systems, and hybrid systems is utilized for various applications such as topic detection, sentiment analysis, and natural language inference purposes
Summary
A representation topic in NLP analysis is text classification. For managing tremendous amount of text documents in the fields of web mining, information retrieval, and NLP, text classification plays a vital role [1]. e knowledge gained from text expression is employed in an efficient manner; the assignment of one or more predefined topics to a natural text document is done in text classification. E simplest deep learning models utilized for text representation are feedforward neural networks [12]. Transformers were utilized for text classification as they allow for more parallelization than other deep learning models, so that very huge models can be trained quite efficiently [34]. Based on word2vec’s Continuous Bag of Words (CBOW) design, a Bi-LSTM neural network architecture was designed by Melamud et al where, for the variable length sentence contexts, the general context embedding function was learned very efficiently [41]. By combining different word embedding with different learning approaches like LSTM, BiLSTM, CNN, and GRU, a normal hybrid model was proposed by Salur and Aydin [47]. Two hybrid models with respect to the positioning of the attention mechanism is utilized and developed for efficient text classification.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
