Effectiveness Of Recurrent Neural Networks Research Articles

MUSIC GENERATION USING NEURAL NETWORK

This study investigates the utilization of recurrent neural networks (RNNs) for generating music through MIDI files. By encoding musical data into sequences, RNN models are trained to learn patterns and structures inherent in compositions. Through the analysis of MIDI data and the evaluation of generated sequences, the effectiveness of RNNs in autonomously creating cohesive musical pieces is explored, advancing the frontier of AI-driven musical composition. Keywords—Music Generation, Long Short-Term Memory, Recurrent Neural Network, MIDI data .

Neural network-based measure of consonant Lenition in Parkinson's disease

This study investigates the effects of Parkinson's disease (PD) and various linguistic factors on the degree of lenition in Spanish stops. The lenition is estimated from posterior probabilities calculated by recurrent neural networks trained to recognize sonorant and continuant phonological features. First, individuals with PD exhibit a higher degree of lenition in their stops compared to healthy controls, suggesting that PD has a significant impact on the articulatory control of stops, resulting in more pronounced lenition. Second, stress influences the degree of lenition, with stressed syllables showing less weakening compared to unstressed syllables. Thirdly, place of articulation plays a role in lenition, but the effects differ between PD and healthy control groups. PD patients demonstrate greater weakening in bilabial and velar stops compared to dental stops, while healthy controls exhibit more lenition in velar stops, with no significant difference between bilabial and dental stops. These findings suggest abnormalities in lip movement control among PD patients align with previous research on reduced lip velocities in Parkinsonian patients. Importantly, the study highlights the effectiveness of recurrent neural networks in quantifying lenition in PD patients.

An RNN-Based Performance Identification Model for Multi-Agent Containment Control Systems

In the containment control problem of multi-agent systems (MASs), the convergence of followers is always a potential threat to the security of system operations. From the perspective of system topology, the inherently non-linear properties of the algebraic connectivity of the follower2follower (F2F) network, combined with the influence of the leader2follower (L2F) topology on the system, make it difficult to design the convergence positions of the followers through mere mathematical analysis. Therefore, in the background of temporary networking tasks for large-scale systems, to achieve the goal of forecasting the performance of the whole system when networking is only completed with local information, this paper investigates the application and effectiveness of recurrent neural networks (RNNs) in the containment control system performance identification, thus improving the efficiency of system networking while ensuring system security. Two types of identification models based on two types of neural networks (NNs), MLP and standard RNN are developed, according to the range of information required for performance identification. Evaluation of the models is carried out by means of the coefficient of determination (R2) as well as the root-mean-square error (RMSE). The results show that each model may produce a better forecasting accuracy than the other models in specific cases, with models based on the standard RNN possessing smaller errors. With the proposed method, model identification can be achieved, but in-depth development of the model in further studies is still necessary to the extent the accuracy of the model.

Application of Recurrent Neural Networks to Model Bias Correction: Idealized Experiments With the Lorenz‐96 Model

AbstractSystematic biases in numerical weather prediction models cause forecast deviation from reality. While model biases also affect data assimilation and degrade the analysis accuracy, observation information incorporated through data assimilation can provide information for detecting and alleviating such biases. In this study, the application of machine learning to model bias correction is demonstrated, emphasizing the effectiveness of recurrent neural networks. Idealized experiments are performed using the two‐scale coupled Lorenz‐96 model as the true system and single Lorenz‐96 model as the imperfect forecast model, to compare the effectiveness of bias correction methods based on various architectures of neural networks and simple linear regression. The neural networks generally outperformed linear regression, and recurrent neural networks showed the best ability in finding the systematic bias component from the analysis increment data. Bias correction using the recurrent neural networks also gives the most significant improvement in reducing the error growth rate in extended range forecasts. The results suggest that including past time series of the forecast variables improve model bias correction when limited information of the observation is incorporated through data assimilation.

SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks

The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as Automatic Speech Recognition has fostered interest in RNN inference acceleration. Due to the recurrent nature and data dependencies of RNN computations, prior work has designed customized architectures specifically tailored to the computation pattern of RNN, getting high computation efficiency for certain chosen model sizes. However, given that the dimensionality of RNNs varies a lot for different tasks, it is crucial to generalize this efficiency to diverse configurations. In this work, we identify adaptiveness as a key feature that is missing from today’s RNN accelerators. In particular, we first show the problem of low resource utilization and low adaptiveness for the state-of-the-art RNN implementations on GPU, FPGA, and ASIC architectures. To solve these issues, we propose an intelligent tiled-based dispatching mechanism for increasing the adaptiveness of RNN computation, in order to efficiently handle the data dependencies. To do so, we propose Sharp as a hardware accelerator, which pipelines RNN computation using an effective scheduling scheme to hide most of the dependent serialization. Furthermore, Sharp employs dynamic reconfigurable architecture to adapt to the model’s characteristics. Sharp achieves 2×, 2.8×, and 82× speedups on average, considering different RNN models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and GPU implementations, respectively. Furthermore, we provide significant energy reduction with respect to the previous solutions, due to the low power dissipation of Sharp (321 GFLOPS/Watt).

Recurrent neural networks in cyber security

The recent increase in cyber-attacks and malware clearly demonstrates that current countermeasures do not seem to be enough to protect against it, as hackers become more cautious in their approach with the cunning of developing systems that automatically rewrite and reorder their malicious software to avoid detection. Typical machine learning approaches that learn a classifier based on a manually created feature vector are not robust enough to such reordering. Hence, the need for an effective automated cyber security solution using deep neural networks. In this article, we demonstrate research on the effectiveness of recurrent neural networks (RNNs) for combat in cyberspace. The conducted experiment shows that RNN with Long Short Term Memory (LSTM) performs much better than classical machine learning algorithms (SVM and Random Forest) with accuracy of 99.70%, 98.55% and 99.42%, respectively. This is possible because RNNs have a built-in memory that can remember multiple previous states and implicitly extract distinctive features, hidden complex structure, and complex sequential relationships in the data, which helps achieve better accuracy. This paper describes an investigation of a recurrent neural network (RNN) model for cyber security using domain malware detection as an application area. The performance of RNN and other classical machine learning classifiers is studied and evaluated for malware classification in the cyber security usage domain and compared. From this study, it can be seen that RNN has better accuracy than classical machine learning classifiers (SVM and Random forest). This is possible because RNNs have a built-in memory capability that can store and replay multiple previous states, and implicitly extract salient features, hidden/underlying complex structure, and complex sequential relationships in the data, which help achieve better accuracy. Thus, it will be useful for creating a real-time application for analyzing malicious activities on the network.

Stacked recurrent neural network for botnet detection in smart homes

Internet of Things (IoT) devices in Smart Home Network (SHN) are highly vulnerable to complex botnet attacks. In this paper, we investigate the effectiveness of Recurrent Neural Network (RNN) to correctly classify network traffic samples in the minority classes of highly imbalanced network traffic data. Multiple layers of RNN are stacked to learn the hierarchical representations of highly imbalanced network traffic data with different levels of abstraction. We evaluate the performance of Stacked RNN (SRNN) model with Bot-IoT dataset. Results show that SRNN outperformed RNN in all classification scenarios. Specifically, SRNN model learned the discriminating features of highly imbalanced network traffic samples in the training set with better representations than RNN model. Also, SRNN model is more robust and it demonstrated better capability to effectively handle over-fitting problem than RNN model. Furthermore, SRNN model achieved better generalization ability in detecting network traffic samples of the minority classes.

Recurrent Neural Networks Based Online Behavioural Malware Detection Techniques for Cloud Infrastructure

Several organizations are utilizing cloud technologies and resources to run a range of applications. These services help businesses save on hardware management, scalability and maintainability concerns of underlying infrastructure. Key cloud service providers (CSPs) like Amazon, Microsoft and Google offer Infrastructure as a Service (IaaS) to meet the growing demand of such enterprises. This increased utilization of cloud platforms has made it an attractive target to the attackers, thereby, making the security of cloud services a top priority for CSPs. In this respect, malware has been recognized as one of the most dangerous and destructive threats to cloud infrastructure (IaaS). In this paper, we study the effectiveness of Recurrent Neural Networks (RNNs) based deep learning techniques for detecting malware in cloud Virtual Machines (VMs). We focus on two major RNN architectures: Long Short Term Memory RNNs (LSTMs) and Bidirectional RNNs (BIDIs). These models learn the behavior of malware over time based on run-time fine-grained processes system features such as CPU, memory, and disk utilization. We evaluate our approach on a dataset of 40,680 malicious and benign samples. The process level features were collected using real malware running in an open online cloud environment with no restrictions, which is important to emulate practical cloud provider settings and also capture the true behaviour of stealth and sophisticated malware. Both our LSTM and BIDI models achieve high detection rates over 99% for different evaluation metrics. In addition, an analysis study is conducted to understand the significance of input data representations. Our results suggest that in particular cases, input ordering does have some affect on the performance of the trained RNN models.

A study of effectiveness of deep learning in locating real faults

Abstract Context: The recent progress of deep learning has shown its promising learning ability in making sense of data, and many fields have utilized this learning ability to learn an effective model, successfully solving their problems. Fault localization has explored and used deep learning to server an aid in debugging, showing the promising results on fault localization. However, as far as we know, there is no detailed studies on evaluating the benefits of using deep learning for locating real faults present in programs. Objective: To understand the benefits of deep learning in locating real faults, this paper explores more about deep learning by studying the effectiveness of fault localization using deep learning for a set of real bugs reported in the widely used programs. Method: We use three representative deep learning architectures (i.e. convolutional neural network, recurrent neural network and multi-layer perceptron) for fault localization, and conduct large-scale experiments on 8 real-world programs equipped with all real faults to evaluate their effectiveness on fault localization. Results: We observe that the localization effectiveness varies considerably among three neural networks in the context of real faults. Specifically, convolutional neural network performs the best in locating real faults, showing an average of 38.97% and 26.22% saving over multi-layer perceptron and recurrent neural network respectively; recurrent neural network and multi-layer perceptron yield comparable effectiveness even if the effectiveness of recurrent neural network is marginally higher than multi-layer perceptron. Conclusion: In context of real faults, convolutional neural network is the most effective for fault localization among the investigated architectures, and we suggest potential factors of deep learning for improving fault localization.

Recurrent Neural Networks for Meteorological Time Series Imputation

The aim of the work presented in this paper is to analyze the effectiveness of recurrent neural networks in imputation processes of meteorological time series, for this six different models based on recurrent neural networks such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) are implemented and it is experimented with hourly meteorological time series such as temperature, wind direction and wind velocity. The implemented models have architectures of 2, 3 and 4 sequential layers and their results are compared with each other, as well as with other imputation techniques for univariate time series mainly based on moving averages. The results show that for temperature time series on average the recurrent neural network achieve better results than the imputation techniques based on moving averages; in the case of wind direction time series, on average only one model based on RNN manages to exceed the models based on moving averages; and finally, for wind velocity time series on average, no RNN-based model manages to exceed the results achieved by moving averages-based models.

Enhance Medical Sentiment Vectors through Document Embedding using Recurrent Neural Network

Adverse Drug Reaction (ADR) extraction is the process of identifying drug implications mentioned in social posts. Handling medical text for the identification of ADR is vital to research in terms of configuring the side effect and other medical-related entities within any medical text. However, investigating the role of such effect in the context of positive and negative is the responsibility of sentiment classification task where every medical review document would be categorized into its polarity, this is known as Medical Sentiment Analysis (MSA). Several studies have presented various techniques for MSA. Most of the recent studies have concentrated on architectures such as the Convolutional Neural Network (CNN) to get the document embedding. Yet, such architecture focuses only on the input without considering the previous or latter input. This might lead to weaker embedding for the document where some terms would not be considered. Hence, this paper proposes a new document embedding approach based on the Recurrent Neural Network (RNN) to improve the sentiment classification. Using a benchmark dataset of medical sentiments, the proposed method showed greater performance of sentiment classification accuracy. Such finding proves the effectiveness of RNN in producing document embedding.

Analysis of Recurrent Neural Networks for Probabilistic Modeling of Driver Behavior

The validity of any traffic simulation model depends on its ability to generate representative driver acceleration profiles. This paper studies the effectiveness of recurrent neural networks in predicting the acceleration distributions for car following on highways. The long short-term memory recurrent networks are trained and used to propagate the simulated vehicle trajectories over 10-s horizons. On the basis of several performance metrics, the recurrent networks are shown to generally match or outperform baseline methods in replicating driver behavior, including smoothness and oscillatory characteristics present in real trajectories. This paper reveals that the strong performance is due to the ability of the recurrent network to identify recent trends in the ego-vehicle's state, and recurrent networks are shown to perform as, well as feedforward networks with longer histories as inputs.