Deep Recurrent Neural Network Research Articles

Estimation and early prediction of grip force based on sEMG signals and deep recurrent neural networks

Hands are used for communicating with the surrounding environment and have a complex structure that enables them to perform various tasks with their multiple degrees of freedom. Hand amputation can prevent a person from performing their daily activities. In that event, finding a suitable, fast, and reliable alternative for the missing limb can affect the lives of people who suffer from such conditions. As the most important use of the hands is to grasp objects, the purpose of this study is to accurately predict gripping force from surface electromyography (sEMG) signals during a pinch-type grip. In that regard, gripping force and sEMG signals are derived from 10 healthy subjects. Results show that for this task, recurrent networks outperform nonrecurrent ones, such as a fully connected multilayer perceptron (MLP) network. Gated recurrent unit (GRU) and long short-term memory (LSTM) networks can predict the gripping force with R-squared values of 0.994 and 0.992, respectively, and a prediction rate of over 1300 predictions per second. The predominant advantage of using such frameworks is that the gripping force can be predicted straight from preprocessed sEMG signals without any form of feature extraction, not to mention the ability to predict future force values using larger prediction horizons adequately. The methods presented in this study can be used in the myoelectric control of prosthetic hands or robotic grippers.

Learning task-agnostic and interpretable subsequence-based representation of time series and its applications in fMRI analysis

The recent success of sequential learning models, such as deep recurrent neural networks, is largely due to their superior representation-learning capability for learning the informative representation of a targeted time series. The learning of these representations is generally goal-directed, resulting in their task-specific nature, giving rise to excellent performance in completing a single downstream task but hindering between-task generalisation. Meanwhile, with increasingly intricate sequential learning models, learned representation becomes abstract to human knowledge and comprehension. Hence, we propose a unified local predictive model based on the multi-task learning paradigm to learn the task-agnostic and interpretable subsequence-based time series representation, allowing versatile use of learned representations in temporal prediction, smoothing, and classification tasks. The targeted interpretable representation could convey the spectral information of the modelled time series to the level of human comprehension.Through a proof-of-concept evaluation study, we demonstrate the empirical superiority of learned task-agnostic and interpretable representation over task-specific and conventional subsequence-based representation, such as symbolic and recurrent learning-based representation, in solving temporal prediction, smoothing, and classification tasks. These learned task-agnostic representations can also reveal the ground-truth periodicity of the modelled time series. We further propose two applications of our unified local predictive model in functional magnetic resonance imaging (fMRI) analysis to reveal the spectral characterisation of cortical areas at rest and reconstruct more smoothed temporal dynamics of cortical activations in both resting-state and task-evoked fMRI data, giving rise to robust decoding.

A fuzzy extension of MEREC method using parabolic measure and its applications

Human qualitative judgments are often characterized by uncertainty and predictability. Decision-makers tend to be more confident in making linguistic decisions than in crisp value judgments. MEREC is capable of achieving relative objective weights of several conflicting criteria. This paper contains two parts, first, the extension of MEREC method in fuzzy circumstances based on linguistic terms in which a parabolic measure has been used to calculate the overall performance of alternatives as it is able to work according the definition of TFNs and to show the applicability, a simple decision matrix is ​​analysed in a fuzzy environment. Second, a new hybrid ranking methodology Fuzzy MEREC-TOPSIS for multi criteria decision making. Further, to illustrate the credibility and effectiveness of the proposed hybrid ranking method, a real-world example of stock selection has been used. The portfolio is constructed using ranking of the stocks received through the proposed method and capital is allocated according to the order of preference of stocks. To validate the proposed ranking model, the next 30 days closing price of each stock is predicted by a deep recurrent neural network and the portfolio for future investments is analysed. The results of the future analysis validate the credibility of the portfolio.

DeepArr: An investigative tool for arrhythmia detection using a contextual deep neural network from electrocardiograms (ECG) signals

In the context of Cardiovascular Diseases, arrhythmia is one of the causes of sudden death, which is related to abnormal electrical activities of the heart that can be reflected by the electrocardiogram (ECG) which plays the main role in heart disease analysis. However, it is still a challenge to detect arrhythmia based on ECG basic characteristics because of the non-stationary nature of ECG signal even cardiologists faced challenges in arrhythmia diagnosis. Therefore, automatic arrhythmia detection-based ECG signals with height accuracy is a serious and indispensable task. Hence In this paper, we propose a new deep learning-based approach called “DeepArr” that uses a sequential fusion method to combine feed-forward and recurrent deep neural networks to exploit relevant features representation of arrhythmia from electrocardiograms (ECG) signals. A comprehensive experimental study has been made in this research, which shows that the proposed approach offers the most efficient tool for accurate classification and ranks top of the list of recently published algorithms on the MIT-BIH arrhythmia dataset. 10-fold cross-validation is carried out. The proposed DeepArr model achieved an accuracy, specificity, sensitivity, precision, and F1-score of 99.46%, 99.57%, 97.01%, 98.26%, and 97.63%, respectively. The proposed model provides a robust tool for the early detection of Arrhythmia.

Deep learning hyperparameter optimization: Application to electricity and heat demand prediction for buildings

Optimal planning and operation studies of modern energy systems are tied up with medium to long-term predictions of energy demand. Deep learning algorithms have recently become significantly useful in this regard because of their capacity to learn load patterns with a high degree of fluctuation and uncertainty. Yet, empirical evaluation of different configurations of deep learning models, for enhancing prediction accuracy, has been mostly ignored in the preceding literature. In this paper, different architectures of deep recurrent neural networks (DRNNs) are explored and adjusted specifically for the purpose of performing medium- and long-term predictions of energy demands. We intend to create a bespoke DRNN for heating and electricity consumption prediction with a 1-h resolution. Moreover, hyperparameter optimization, which is a time-consuming and rigorous task in deep learning algorithms due to their abundance, dependence on the particular application, and empirical nature, is studied comprehensively. In this respect, the space of possible configuration variables is thoroughly explored to identify a collection of hyperparameters that enables the DRNN model to more accurately forecast energy consumption. We also benchmark the proposed algorithm’s performance against other data-driven models, namely support vector machine (SVM) and gradient boosting (GB) regression. The proposed model outperforms SVM by 5.4% and GB regression by 7.0% in terms of energy forecasting accuracy.

Volatility forecasting using deep recurrent neural networks as GARCH models

Volatility forecasting using deep recurrent neural networks as GARCH models

The prediction of collective Economic development based on the PSO-LSTM model in smart agriculture.

Smart agriculture can promote the rural collective economy's resource coordination and market access through the Internet of Things and artificial intelligence technology and guarantee the collective economy's high-quality, sustainable development. The collective agricultural economy (CAE) is non-linear and uncertain due to regional weather, policy and other reasons. The traditional statistical regression model has low prediction accuracy and weak generalization ability on such issues. This article proposes a production prediction method using the particle swarm optimization-long short term memory (PSO-LSTM) model to predict CAE. Specifically, the LSTM method in the deep recurrent neural network is applied to predict the regional CAE. The PSO algorithm is utilized to optimize the model to improve global accuracy. The experimental results demonstrate that the PSO-LSTM method performs better than LSTM without parameter optimization and the traditional machine learning methods by comparing the RMSE and MAE evaluation index. This proves that the proposed model can provide detailed data references for the development of CAE.

A secured and optimized deep recurrent neural network (DRNN) scheme for remote health monitoring system with edge computing

Patients now want a contemporary, advanced healthcare system that is faster and more individualized and that can keep up with their changing needs. An edge computing environment, in conjunction with 5G speeds and contemporary computing techniques, is the solution for the latency and energy efficiency criteria to be satisfied for a real-time collection and analysis of health data. The feature of optimum computing approaches, including encryption, authentication, and classification that are employed on the devices deployed in an edge-computing architecture, has been ignored by previous healthcare systems, which have concentrated on novel fog architecture and sensor kinds. To avoid this problem in this paper, an Optimized Deep Recurrent Neural Network (O-DRNN) model is used with a multitier secured architecture. Initially, the data obtained from the patient are sent to the healthcare server in edge computing and the processed data are stored in the cloud using the Elliptic Curve Key Agreement Scheme (ECKAS) security model. The data is pre-processed and optimal features are selected using the Particle Swarm Optimization (PSO) algorithm. O-DRNN algorithm hyper-parameters are optimized using Bayesian optimization for better diagnosis. The proposed work offers superior outcomes in terms of accuracy and encryption latency while using computational cloud services.

Deep Bidirectional Recurrent Neural Networks Ensemble for Remaining Useful Life Prediction of Aircraft Engine.

Remaining useful life (RUL) prediction of aircraft engine (AE) is of great importance to improve its reliability and availability, and reduce its maintenance costs. This article proposes a novel deep bidirectional recurrent neural networks (DBRNNs) ensemble method for the RUL prediction of the AEs. In this method, several kinds of DBRNNs with different neuron structures are built to extract hidden features from sensory data. A new customized loss function is designed to evaluate the performance of the DBRNNs, and a series of the RUL values is obtained. Then, these RUL values are reencapsulated into a predicted RUL domain. By updating the weights of elements in the domain, multiple regression decision tree (RDT) models are trained iteratively. These models integrate the predicted results of different DBRNNs to realize the final RUL prognostics with high accuracy. The proposed method is validated by using C-MAPSS datasets from NASA. The experimental results show that the proposed method has achieved more superior performance compared with other existing methods.

AACO: Aquila Anti-Coronavirus Optimization-Based Deep LSTM Network for Road Accident and Severity Detection

Globally, traffic accidents are of main concern because of more death rates and economic losses every year. Thus, road accident severity is the most important issue of concern, mainly in the undeveloped countries. Generally, traffic accidents result in severe human fatalities and large economic losses in real-world circumstances. Moreover, appropriate, precise prediction of traffic accidents has a high probability with regard to safeguarding public security as well as decreasing economic losses. Hence, the conventional accident prediction techniques are usually devised with statistical evaluations, which identify and evaluate the fundamental relationships among human variability, environmental aspects, traffic accidents and road geometry. However, the conventional approaches have major restrictions based on the assumptions regarding function kind and data distribution. In this paper, Aquila Anti-Coronavirus Optimization-based Deep Long Short-Term Memory (AACO-based Deep LSTM) is developed for road accident severity detection. Spearman’s rank correlation coefficient and Deep Recurrent Neural Network (DRNN) are utilized for the feature fusion process. Data augmentation method is carried out to improve the detection performance. Deep LSTM detects the road accident and its severity, where Deep LSTM is trained by the designed AACO algorithm for better performance. The developed AACO-based Deep LSTM model outperformed other existing methods with the Mean Square Error (MSE), Root-Mean-Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) of 0.0145, 0.1204 and 0.075%, respectively.

Short-Term Energy Load Forecasting Model with Sample Filtering

Short-term energy load forecasting is a crucial task in the power smart grid, which enables the power utilities to understand the future energy demands and plans to attain the demand and supply equilibrium, thereby optimizing power deployment and reducing power losses. Several techniques have been implemented to enhance energy load forecasting. However, the nonlinear nature of the data collected in the smart grid makes it difficult to attain 100% energy load forecasting accuracy. For instance, the Deep Feedforward Neural Networks model based on Input Attention Mechanism and Hidden Connection Mechanism has a mean absolute percentage error of 3.17%; model based on Sequence to Sequence Recurrent Neural Network with Attention had a mean absolute percentage error of 2.7%. The model based on Deep Recurrent Neural Networks with Levenberg–Marquardt backpropagation algorithm had a mean absolute percentage error of 0.58; and Deep Feedforward Neural Network with sample weights model had 3.22 % as root mean squared error. To improve energy load forecasting accuracy, this work proposed a model based on Deep Recurrent Neural Networks and sample filtering, which provides an exhaustive elucidation for modelling a sophisticated stochastic relationship between the input and output features. Deep Recurrent Neural Networks have proven to be good at modelling the nonlinearities in data of different fields and are mostly used in energy load forecasting to reduce forecasting error and a high degree of overfitting. Sample filtering is achieved through the use of K-Means clustering which determines the number of clusters to be used in the model. Findings from the study showed that by employing Deep Recurrent Neural Networks and sample filtering, the short-term energy load forecasting accuracy is improved in reference to mean absolute percentage error and root mean squared error of 0.31% and 1.014, respectively. As a result of the reduction in error, the energy demand and supply chain equilibrium are enhanced, thereby optimizing power deployment and reducing power losses.
 Keywords: Machine learning, Neural networks, Sample filtering, Smart grid, Short-term energy forecasting

Credit Card Fraud Detection Using LSTM

As credit card becomes the most popular payment mode particularly in the online sector, the fraudu- lent activities using credit card payment technologies are rapidly increasing as a result. The purpose of this work is to develop a novel system for credit card fraud detection based on sequential modeling of data, using attention mechanism Long Short Term Mem- ory(LSTM) deep Recurrent Neural Networks(RNN) and Synthetic Minority Oversampling Technique Edited Nearest Neighbour(SMOTE-ENN). The pro- posed approach aims to capture the historic purchase behavior of credit card holders with the goal of im- proving fraud detection accuracy on new incoming transactions. Experiments show that our proposed model gives strong results and its accuracy is quite high. keywords: Credit Card, SMOTE-ENN, FraudDetection, Machine learning,LSTM.

Ship Anomalous Behavior Detection Using Clustering and Deep Recurrent Neural Network

In this study, we propose a real-time ship anomaly detection method driven by Automatic Identification System (AIS) data. The method uses ship trajectory clustering classes as a normal model and a deep learning algorithm as an anomaly detection tool. The method is divided into three main steps: (1) quality maintenance of the original AIS data, (2) extraction of normal ship trajectory clusters using Density-Based Spatial Clustering of Applications with Noise (DBSCAN), in which a segmented improved Dynamic Time Warping (DTW) algorithm is used to measure the degree of trajectory similarity, (3) the clustering results are used as a normative model to train a Bi-directional Gated Recurrent Unit (BiGRU) recurrent neural network, which is used as a trajectory predictor to achieve real-time ship anomaly detection. Experiments were conducted using real AIS data from the port of Tianjin, China. The experimental results are manifold. Firstly, the data pre-processing process effectively improves the quality of raw AIS data. Secondly, the ship trajectory clustering model can accurately classify the traffic flow of different modes in the sea area. Moreover, the trajectory prediction result of the BiGRU model has the smallest error with the actual ship trajectory and has a better trajectory prediction performance compared with the Long Short-Term Memory Network model (LSTM) and Gated Recurrent Unit (GRU). In the final anomaly detection experiment, the detection accuracy and timeliness of the BiGRU model are also higher than LSTM and GRU. Therefore, the proposed method can achieve effective and timely detection of ship anomalous behaviors in terms of position, heading and speed during ship navigation, which provides insight to enhance the intelligence of marine traffic supervision and improve the safety of marine navigation.

Phish-armour: phishing detection using deep recurrent neural networks

Phish-armour: phishing detection using deep recurrent neural networks

Integration of the latent variable knowledge into deep image captioning with Bayesian modeling

AbstractAutomatic image captioning systems assign one or more sentences to images to describe their visual content. Most of these systems use attention‐based deep convolutional neural networks and recurrent neural networks (CNN‐RNN‐Att). However, they must optimally use latent variables and side information within the image concepts. This study aims to integrate a latent variable into image captioning using CNN‐RNN‐Att. A Bayesian modeling framework is used for this work. As an instance of a latent variable, High‐Level Semantic Concepts (HLSCs) of tags are used to implement the proposed model. The Bayesian model output interpretation is to localize the entire image description process and breaks it down into sub‐problems. Thus, a baseline descriptor subnet is trained independently for each sub‐problem, and it is the only expert in captioning for a given HLSC. The final output is the caption derived from the subnet; its HLSC is closest to the image content. The results indicate that CNN‐RNN‐Att applied to data localized using HLSCs improves the captioning accuracy of the proposed method, which can be compared to the latest state‐of‐the‐art and most accurate captioning systems.

A novel cyber intrusion detection model based on improved hybrid sampling

The class imbalance of samples of network traffic will cause the poor classification performance of intrusion detection models based on machine learning. To solve this problem, this paper researches sampling algorithm and deep learning for intrusion detection in imbalanced network traffic. This paper proposes a deep recurrent neural network (delayed long short-term memory (DLSTM)) intrusion detection model based on the balanced samples. First, an improved hybrid sampling (IHS) method based on chaotic particle swarm optimization (CPSO) algorithm is proposed as the sampling algorithm to balance the imbalanced samples. Next, a DLSTM with long short-term memory (LSTM) function is proposed to realize high-precision classification of intrusion behaviours. Finally, the method is validated on the standard network traffic dataset. The experimental results show that the DLSTM intrusion detection model based on the IHS method outperforms other comparative models at accuracy. The model is available to the computer network information security defence system.

Improving the Robustness of Loanword Identification in Social Media Texts

As a potential bilingual resource, loanwords play a very important role in many natural language processing tasks. If loanwords in a low-resource language can be identified effectively, the generated donor-receipt word pairs will benefit many cross-lingual natural language processing tasks. However, most studies on loanword identification mainly focus on formal texts such as news and government documents. Loanword identification in social media texts is still an under-studied field. Since it faces many challenges and can be widely used in several downstream tasks, more efforts should be put on loanword identification in social media texts. In this study, we present a multi-task learning architecture with deep bi-directional recurrent neural networks for loanword identification in social media texts, where different task supervision can happen at different layers. The multi-task neural network architecture learns higher-order feature representations from word and character sequences along with basic spell error checking, part-of-speech tagging, and named entity recognition information. Experimental results on Uyghur loanword identification in social media texts in five donor languages (Chinese, Arabic, Russian, Turkish, and Farsi) show that our method achieves the best performance compared with several strong baseline systems. We also combine the loanword detection results into the training data of neural machine translation for low-resource language pairs. Experiments show that models trained on the extended datasets achieve significant improvements compared with the baseline models in all language pairs.

Intelligent hybrid hand gesture recognition system using deep recurrent neural network with chaos game optimisation

ABSTRACT Hand gesture recognition is considered an essential task in various human-computer interaction (HCI) applications. Therefore, developing a robust system for hand gesture recognition is a challenge. This work proposed a new hybrid approach named hybrid deep recurrent neural network (RNN) incorporated with a chaos game optimisation (CGO) algorithm for efficiently recognising hand gestures. The main objective of a hybrid recurrent neural network with chaos game optimisation (RNN-CGO) is to achieve the recognition of alphabet signs from 2D gesture images. The different stages included in this technique are pre-processing, feature extraction, feature selection, and classification. The proposed approach is executed using American Sign Language (ASL) dataset with the help of the PYTHON platform. The precision, accuracy, f1-score, and recall obtained for the proposed approach are 99.28%, 99.96%, 99.25%, and 99.28%, respectively. The inference time required for the proposed approach is 0.121 s. The proposed technique has less computational complexity. The results showed that the proposed approach is better than the existing hand gesture recognition model approaches.

LSTM and GRU based Accurate Water Quality Prediction for Smart Aquaculture

The fast and precise water quality prediction in smart aquaculture can assist the farmers in taking corrective actions before the ecological environment deteriorates. However, the empirical methods take high computation time and provide low accuracy in water quality prediction due to the non-linear and dynamic nature of water quality parameters. This research work proposes long short-term memory (LSTM) and gated recurrent unit (GRU) deep learning recurrent neural network (DL-RNN) models for aquaculture water quality prediction (A-WQP). This work also presents an extensive study about the impact of hyper-parameters (hp ) on the performance of the proposed DL-RNN models using two different water quality datasets. The experimental evaluations show that for the optimal sets of hyper-parameters, the proposed DL models offer superior prediction accuracy and computation efficiency.

Physically constrained deep recurrent neural network for stiffness computation of plate structures

AbstractIn the present study, we introduce two Neural Network (NN) enhanced methods to approximate the local tangent stiffness matrix and the internal force computation for a 2D Finite Element. The proposed model is based on Long‐Short Term Memory (LSTM), which inherently captures the required path‐dependent behavior through its internal parameters. Furthermore, we propose an enhanced training algorithm where an additional loss term corresponding to the derivative of the NN following the Sobolev training procedure is introduced. Such a learning algorithm combines the data‐driven approach with the necessary physical constraint to train the NN. Thus, the present work focuses on introducing the NN at an element level for plate structures taking physical non‐linearities into account. The performance of the proposed methods is demonstrated in an academic example showing a maximum of 90.564% boost in simulation speed.