Long Short-Term Memory Language Model Research Articles

Assessment of bidirectional transformer encoder model and attention based bidirectional LSTM language models for fake news detection

Fake news arouses to be untrue with the point of deceiving it openly which is now viewed as the greatest threat to society by cultivating the political division and doubts in government. Since this kind of news is disseminated in sheer volume through social media, driving the improvement of strategies for the recognizable proof of false news is necessary. Therefore, this study focuses on text analytics to derive the hidden properties of stylistic content to detect fake and real news. An erudite literature study of fake news detection diverted towards issues such as attention, context, and parallelization. In this same direction, the assessment evaluates the sequential memory-based deep learning model in comparison to the parallel memory-based deep learning model. For sequential, Long Short-Term Memory (LSTM), Bi-Directional LSTM, and Attention-based Bi-directional LSTM are taken into consideration. Besides, for parallel, the transformer-based BERT model is examined. To identify the efficacy of applied approaches, four datasets are taken from diverse domains such as political news, entertainment news, satire news, conspiracy news, and global pandemic news. The experimental analysis of real-world information demonstrates that the pre-trained transformer encoder-based BERT model outperforms with a quite significant margin of improvement. Also, as inspected Attention-based Bi-directional approach provides state-of-the-art results with good training accuracy.

COVID-19 sentiment analysis via deep learning during the rise of novel cases.

Social scientists and psychologists take interest in understanding how people express emotions and sentiments when dealing with catastrophic events such as natural disasters, political unrest, and terrorism. The COVID-19 pandemic is a catastrophic event that has raised a number of psychological issues such as depression given abrupt social changes and lack of employment. Advancements of deep learning-based language models have been promising for sentiment analysis with data from social networks such as Twitter. Given the situation with COVID-19 pandemic, different countries had different peaks where rise and fall of new cases affected lock-downs which directly affected the economy and employment. During the rise of COVID-19 cases with stricter lock-downs, people have been expressing their sentiments in social media. This can provide a deep understanding of human psychology during catastrophic events. In this paper, we present a framework that employs deep learning-based language models via long short-term memory (LSTM) recurrent neural networks for sentiment analysis during the rise of novel COVID-19 cases in India. The framework features LSTM language model with a global vector embedding and state-of-art BERT language model. We review the sentiments expressed for selective months in 2020 which covers the major peak of novel cases in India. Our framework utilises multi-label sentiment classification where more than one sentiment can be expressed at once. Our results indicate that the majority of the tweets have been positive with high levels of optimism during the rise of the novel COVID-19 cases and the number of tweets significantly lowered towards the peak. We find that the optimistic, annoyed and joking tweets mostly dominate the monthly tweets with much lower portion of negative sentiments. The predictions generally indicate that although the majority have been optimistic, a significant group of population has been annoyed towards the way the pandemic was handled by the authorities.

A Bidirectional LSTM Language Model for Code Evaluation and Repair

Programming is a vital skill in computer science and engineering-related disciplines. However, developing source code is an error-prone task. Logical errors in code are particularly hard to identify for both students and professionals, and a single error is unexpected to end-users. At present, conventional compilers have difficulty identifying many of the errors (especially logical errors) that can occur in code. To mitigate this problem, we propose a language model for evaluating source codes using a bidirectional long short-term memory (BiLSTM) neural network. We trained the BiLSTM model with a large number of source codes with tuning various hyperparameters. We then used the model to evaluate incorrect code and assessed the model’s performance in three principal areas: source code error detection, suggestions for incorrect code repair, and erroneous code classification. Experimental results showed that the proposed BiLSTM model achieved 50.88% correctness in identifying errors and providing suggestions. Moreover, the model achieved an F-score of approximately 97%, outperforming other state-of-the-art models (recurrent neural networks (RNNs) and long short-term memory (LSTM)).

Biden vs Trump: Modeling US General Elections Using BERT Language Model

Social media has played a crucial role in shaping the worldview during election campaigns. It has been used as a medium for political campaigns and a tool for organizing protests; some of which have been peaceful, while others have led to riots. Previous research indicates that understanding user behaviour, particularly in terms of sentiments expressed during elections, can indicate the election outcome. Recently, there has been tremendous progress in language modelling with deep learning via long short-term memory (LSTM) models and variants known as Bidirectional Encoder Representations from Transformers (BERT). Motivated by these innovations, we develop a framework to model the US general elections. We investigate if sentiment analysis can provide a means to predict election outcomes. We use the LSTM and BERT language models for Twitter sentiment analysis leading to the US 2020 presidential elections. Our results show that sentiment analysis can form a general basis for modelling election outcomes where the BERT model indicates Biden winning the elections.

Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

The rate of software development has increased dramatically. Conventional compilers cannot assess and detect all source code errors. Software may thus contain errors, negatively affecting end-users. It is also difficult to assess and detect source code logic errors using traditional compilers, resulting in software that contains errors. A method that utilizes artificial intelligence for assessing and detecting errors and classifying source code as correct (error-free) or incorrect is thus required. Here, we propose a sequential language model that uses an attention-mechanism-based long short-term memory (LSTM) neural network to assess and classify source code based on the estimated error probability. The attentive mechanism enhances the accuracy of the proposed language model for error assessment and classification. We trained the proposed model using correct source code and then evaluated its performance. The experimental results show that the proposed model has logic and syntax error detection accuracies of 92.2% and 94.8%, respectively, outperforming state-of-the-art models. We also applied the proposed model to the classification of source code with logic and syntax errors. The average precision, recall, and F-measure values for such classification are much better than those of benchmark models. To strengthen the proposed model, we combined the attention mechanism with LSTM to enhance the results of error assessment and detection as well as source code classification. Finally, our proposed model can be effective in programming education and software engineering by improving code writing, debugging, error-correction, and reasoning.