Advanced Explainable AI: Self Attention Deep Neural Network of Text Classification

Abstract

The classification of texts is a crucial component of the data retrieval mechanism. By utilizing semantic details representation, and the text vector sequence is condensed, resulting in a reduction in the temporal and spatial order of the memory pattern. This process helps to clarify the context of the text, extract crucial feature information, and fuse these features to determine the classification outcome. This approach represents the preprocessed text data using character-level vectors. The self-attention mechanism is used to understand the interdependence of words in a text, allowing for the extraction of internal structure-related data. Furthermore, the semantic characteristics of text data have been extracted independently using Deep Convolutional Neural Network (DCNN) and Bi-directional Gated Recurrent Unit (BiGRU) using a Soft-Attention mechanism. These two distinct feature extraction outcomes are then merged. The Softmax layer is employed to categorize the deep-extracted attributes, hence enhancing the accuracy of the classification model. This improvement is achieved by including a uniform distribution component into the cross-entropy loss function. Our results demonstrate that our suggested method for explainability outperforms the model that was suggested in terms of accuracy and computing efficiency. For the purpose of assessing the effectiveness of our suggested approach, we developed many baseline models and performed an evaluation their studies.