DEEP QUALITY-CONSTRAINED LSTM FOR TEXTUAL DATA ANALYSIS

Abstract

The creation of language models is a vital object of natural language processing because it creates a possible distribution in a sequence of words and offers a unique representation of each occurrence of a word. Therefore, an intelligent language model can differentiate the subtle nuances of language and model its syntax and semantics. It also allows the acquisition of high-level representations. In addition, it serves as an initial model with helpful information, which can be transferred to various other word processing methods for a deeper and more complete understanding of the language. This work proposes an innovative Long Short-Term Memory (LSTM) neural network whose implementation is based on the use of an attention mechanism as a measure of alignment of the output with the input sequence and a corresponding coverage optimization mechanism. The optimization mechanism informs the model about the possible outcomes at each step which have already been produced in previous actions. The adaptation and evaluation of the system’s design parameters were tested in multidimensional and challenging data sets We were considering parametric and heuristic procedures for finding the optimal combination of hyper-parameters. Experimental results demonstrate the quality of the proposed system, revealing essential directions for the design of similar systems.