Combination Of Long Short-term Memory Research Articles

CNN and LSTM-Based Emotion Charting Using Physiological Signals.

Novel trends in affective computing are based on reliable sources of physiological signals such as Electroencephalogram (EEG), Electrocardiogram (ECG), and Galvanic Skin Response (GSR). The use of these signals provides challenges of performance improvement within a broader set of emotion classes in a less constrained real-world environment. To overcome these challenges, we propose a computational framework of 2D Convolutional Neural Network (CNN) architecture for the arrangement of 14 channels of EEG, and a combination of Long Short-Term Memory (LSTM) and 1D-CNN architecture for ECG and GSR. Our approach is subject-independent and incorporates two publicly available datasets of DREAMER and AMIGOS with low-cost, wearable sensors to extract physiological signals suitable for real-world environments. The results outperform state-of-the-art approaches for classification into four classes, namely High Valence—High Arousal, High Valence—Low Arousal, Low Valence—High Arousal, and Low Valence—Low Arousal. Emotion elicitation average accuracy of is achieved with ECG right-channel modality, 76.65% with EEG modality, and 63.67% with GSR modality for AMIGOS. The overall highest accuracy of 99.0% for the AMIGOS dataset and 90.8% for the DREAMER dataset is achieved with multi-modal fusion. A strong correlation between spectral- and hidden-layer feature analysis with classification performance suggests the efficacy of the proposed method for significant feature extraction and higher emotion elicitation performance to a broader context for less constrained environments.

Discovering software developer's coding expertise through deep learning

The field of software development is growing rapidly and prevailing in every walk of life. The role of software developers in such a challenging and complex activity is very much important. The allocation of right software developers (i.e. who possesses appropriate coding skills) to projects is one of the crucial factors for successful software development. The problem is that it is very difficult for a client, project manager, as well as for software development organisations to find out an appropriate developer and assign him/her to a particular project. To achieve this, there is a need for such a sound mechanism that could detect the level of software developer coding expertise. This study has formulated criteria for novice and expert developers and carried out such criteria to discover the level of coding expertise of software developers using three different models of deep learning. These models include long short-term memory (LSTM), convolution 1D and hybrid (a combination of LSTM and convolution 1D). The deep learning models have analysed software developers’ previously written source code collected from the GitHub repository. An experiment was conducted to evaluate the performance of models. The results showed that the LSTM model performed better in comparison to other models by achieving 96.25% accuracy.

Self-Attention Network for Partial-Discharge Diagnosis in Gas-Insulated Switchgear

Detecting, measuring, and classifying partial discharges (PDs) are important tasks for assessing the condition of insulation systems used in different electrical equipment. Owing to the implementation of the phase-resolved PD (PRPD) as a sequence input, an existing method that processes sequential data, e.g., the recurrent neural network, using a long short-term memory (LSTM) has been applied for fault classification. However, the model performance is not further improved because of the lack of supporting parallel computation and the inability to recognize the relevance of all inputs. To overcome these two drawbacks, we propose a novel deep-learning model in this study based on a self-attention mechanism to classify the PD patterns in a gas-insulated switchgear (GIS). The proposed model uses a self-attention block that offers the advantages of simultaneous computation and selective focusing on parts of the PRPD signals and a classification block to finally classify faults in the GIS. Moreover, the combination of LSTM and self-attention is considered for comparison purposes. The experimental results show that the proposed method achieves performance superiority compared with the previous neural networks, whereas the model complexity is significantly reduced.

Time series prediction: A combination of Long Short-Term Memory and structural time series models

Stock market is an important capital mobilization channel for economy. However, the market has potential loss due to fluctuations of stock prices to reflect uncertain events such as political news, supply and demand of daily trading volume. There are many approaches to reduce risk such as portfolio construction and optimization, hedging strategies. Hence, it is critical to leverage time series prediction techniques to achieve higher performance in stock market. Recently, Vietnam stock markets have gained more and more attention as their performance and capitalization improvement. In this work, we use market data from Vietnam’s two stock market to develop an incorporated model that combines Sequence to Sequence with Long-Short Term Memory model of deep learning and structural models time series. We choose 21 most traded stocks with over 500 trading days from VN-Index of Ho Chi Minh Stock Exchange and HNX-Index of Hanoi Stock Exchange (Vietnam) to perform the proposed model and compare their performance with pure structural models and Sequence to Sequence. For back testing, we use our model to decide long or short position to trade VN30F1M (VN30 Index Futures contract settle within one month) that are traded on HNX exchange. Results suggest that the Sequence to Sequence with LSTM model of deep learning and structural models time series achieve higher performance with lower prediction errors in terms of mean absolute error than existing models for stock price prediction and positive profit for derivative trading. This work significantly contribute to literature of time series prediction as our approach can relax heavy assumptions of existing methodologies such as Auto-regressive–moving-average model, Generalized Auto-regressive Conditional Heteroskedasticity. In practical, investors from Vietnam stock market can use the proposed model to develop trading strategies.

Sentence Level Sentiment Analysis using Deep Learning Method

An Emoji is a small image representing facial expression, entity or a concept that can be either static or animated. In this paper, Emojis are used to study both cross-language and language based sentiment patterns. All the languages do not come with fair amount of labels. Emojis are useful signs of sentiment analysis in cross-lingual tweets. In this paper, an approach is proposed to extend the existing binary sentiment classification using multi-way classification. A novel Long Short-Term Memory (LSTM) - Convolutional Neural Networks (CNN) based model is proposed to obtain sentiments from emojis. The sentiments are classified using Deep Learning method like CNN. The proposed system outperforms the existing system in terms of Accuracy, Precision, Recall, F-Measure and Time Period. Finally the researcher manifests the fact that the CNN and LSTM combination as model shows an immense improvement to detecting the sentiment targets.

Identification Technology of Grid Monitoring Alarm Event Based on Natural Language Processing and Deep Learning in China

Power dispatching systems currently receive massive, complicated, and irregular monitoring alarms during their operation, which prevents the controllers from making accurate judgments on the alarm events that occur within a short period of time. In view of the current situation with the low efficiency of monitoring alarm information, this paper proposes a method based on natural language processing (NLP) and a hybrid model that combines long short-term memory (LSTM) and convolutional neural network (CNN) for the identification of grid monitoring alarm events. Firstly, the characteristics of the alarm information text were analyzed and induced and then preprocessed. Then, the monitoring alarm information was vectorized based on the Word2vec model. Finally, a monitoring alarm event identification model based on a combination of LSTM and CNN was established for the characteristics of the alarm information. The feasibility and effectiveness of the method in this paper were verified by comparison with multiple identification models.

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

LSTM-Guided Coaching Assistant for Table Tennis Practice.

Recently, wearable devices have become a prominent health care application domain by incorporating a growing number of sensors and adopting smart machine learning technologies. One closely related topic is the strategy of combining the wearable device technology with skill assessment, which can be used in wearable device apps for coaching and/or personal training. Particularly pertinent to skill assessment based on high-dimensional time series data from wearable sensors is classifying whether a player is an expert or a beginner, which skills the player is exercising, and extracting some low-dimensional representations useful for coaching. In this paper, we present a deep learning-based coaching assistant method, which can provide useful information in supporting table tennis practice. Our method uses a combination of LSTM (Long short-term memory) with a deep state space model and probabilistic inference. More precisely, we use the expressive power of LSTM when handling high-dimensional time series data, and state space model and probabilistic inference to extract low-dimensional latent representations useful for coaching. Experimental results show that our method can yield promising results for characterizing high-dimensional time series patterns and for providing useful information when working with wearable IMU (Inertial measurement unit) sensors for table tennis coaching.