Elman-type Recurrent Neural Network Research Articles

DEA-RNN: A HYBRID DEEP LEARNING APPROACH FOR CYBERBULLYING DETECTION IN TWITTER SOCIAL MEDIA PLATFORM

Cyberbullying has emerged as a pervasive issue within social media platforms, posing significant risks to users' well-being. To address this growing concern and enhance platform safety, this paper proposes a novel hybrid deep learning model, termed DEA-RNN, designed specifically for detecting cyberbullying on Twitter. The DEA-RNN model integrates Elman-type Recurrent Neural Networks (RNN) with an optimized Dolphin Echolocation Algorithm (DEA) to fine-tune RNN parameters and expedite training. Through comprehensive evaluation using a dataset comprising 10,000 tweets, we compared DEA-RNN's performance with state-of-theart algorithms such as Bi-directional Long Short Term Memory (Bi-LSTM), SVM, Multinomial Naive Bayes (MNB), and Random Forests (RF). Our experimental findings demonstrate the superiority of DEA-RNN across all scenarios, showcasing its effectiveness in cyberbullying detection on the Twitter platform. Particularly noteworthy is DEA-RNN's exceptional performance in scenario 3, achieving an average accuracy of 90.45%, precision of 89.52%, recall of 88.98%, F1-score of 89.25%, and specificity of 90.94%.

Language Inference Using Elman Networks with Evolutionary Training

In this paper, a novel Elman-type recurrent neural network (RNN) is presented for the binary classification of arbitrary symbol sequences, and a novel training method, including both evolutionary and local search methods, is evaluated using sequence databases from a wide range of scientific areas. An efficient, publicly available, software tool is implemented in C++, accelerating significantly (more than 40 times) the RNN weights estimation process using both simd and multi-thread technology. The experimental results, in all databases, with the hybrid training method show improvements in a range of 2% to 25% compared with the standard genetic algorithm.

DEA-RNN: A Hybrid Deep Learning Approach for Cyberbullying Detection in Twitter Social Media Platform

Cyberbullying (CB) has become increasingly prevalent in social media platforms. With the popularity and widespread use of social media by individuals of all ages, it is vital to make social media platforms safer from cyberbullying. This paper presents a hybrid deep learning model, called DEA-RNN, to detect CB on Twitter social media network. The proposed DEA-RNN model combines Elman type Recurrent Neural Networks (RNN) with an optimized Dolphin Echolocation Algorithm (DEA) for fine-tuning the Elman RNN’s parameters and reducing training time. We evaluated DEA-RNN thoroughly utilizing a dataset of 10000 tweets and compared its performance to those of state-of-the-art algorithms such as Bi-directional long short term memory (Bi-LSTM), RNN, SVM, Multinomial Naive Bayes (MNB), Random Forests (RF). The experimental results show that DEA-RNN was found to be superior in all the scenarios. It outperformed the considered existing approaches in detecting CB on Twitter platform. DEA-RNN was more efficient in scenario 3, where it has achieved an average of 90.45% accuracy, 89.52% precision, 88.98% recall, 89.25% F1-score, and 90.94% specificity.

Improved Chicken Swarm Optimized Recurrent Neural Networks for Big Data Intrusion Detection System

Cyber security threats are an ever increasing, frequent and complex issue in the modern information era. With the advent of big data, incremental increase of huge amounts of data has further increased the security problems. Intrusion Detection Systems (IDS) were been developed to monitor and secure the cyber data systems and networks from any intrusions. However, the intrusion detection is difficult due to the rapid evolution of security attacks and the high volume, variety and speed of big data. In addition, the shallow architectures of existing IDS models lead to high computation cost and high memory requirements, thus further diminishing the efficiency of intrusion detection. The recent studies have suggested the use of data analytics and the deep learning algorithms can be effective in improving the IDS. An efficient IDS model is developed in this study by using the improved Elman-type Recurrent Neural Networks (RNN) in which the Improved Chicken Swarm Optimization (ICSO) optimally determines RNN parameters. RNN is an efficient method for classifying network traffic data but its traditional training algorithms are slow in convergence and faces local optimum problem. The introduction of ICSO with enhanced global search ability significantly avoids those limitations and improves the training process of RNN. This optimized deep learning algorithm of RNN, named as ICSO-RNN, is employed in the IDS with Intuitionistic Fuzzy Mutual Information feature selection to analyze larger network traffic datasets. The proposed IDS model using ICSO-RNN is tested on UNSW NB15 dataset. The final outcomes suggested that ICSO-RNN model has high performance in intrusion detection, with minimum training time and is proficient for big data.

Improvement of Prosody Modeling Using Semantic Role Labeling, Hybrid Feature Selection and BPSO–PSO–Optimized RNN

A modified Elman-type recurrent neural network (RNN) is used in this study for prosody generation in natural speech synthesis application. To improve the performance of generator, three modifications are applied to the system as compared to a traditional neural model: (a) using semantic role labeling to enrich the word-level features, (b) using hybrid of genetic algorithm and ant colony optimization algorithm for feature selection to reduce the number of input features, and (c) using hybrid of particle swarm optimization (PSO) and binary PSO algorithms for optimization of weights and structure of RNN-based prosody generator model, respectively. Experimental results show the superior performance of proposed model in predicting prosodic information with lower root mean squared error and a lighter model as compared to a traditional neural model.

Modelling of lead removal from battery industrial wastewater treatment sludge leachate on cement kiln dust by using Elman's RNN

In this study, an Elman type recurrent neural network (RNN) was employed to develop a prediction model for lead removal from industrial sludge leachate using cement kiln dust. The leaching characteristics of industrial sludge were observed through the toxicity characteristics leaching procedure (TCLP). Dosage, contact time, and temperature were considered as independent experimental factors. A comparison between the model results and experimental data showed that the Elman's RNN model is able to predict lead removal from industrial sludge leachate. The outcomes of suggested Elman's RNN modelling were then compared to batch experimental studies. The results show that industrial sludge leachate using cement kiln dust is an efficient sorbent, and Elman's RNN is dynamic in nature and is able to model the batch experimental system.

A hybrid optimization-based recurrent neural network for real-time data prediction

This paper proposes a novel hybrid optimization algorithm for simultaneous structure and parameter learning of Elman-type recurrent neural networks (RNNs) combining the advantages of discrete particle swarm optimization (DPSO) algorithm and improved particle swarm optimization (IPSO) algorithm. The proposed hybrid algorithm, named HDIPSO algorithm, is developed in such a way that DPSO algorithm is adopted to determine the structure of a RNN, while IPSO algorithm is used to adjust the parameters for each of the structures present in the DPSO. The IPSO algorithm employs a new velocity update strategy and a mutation operation to improve convergence and increase diversity of the swarm. The aim is to generate an automatic process for obtaining RNNs with high generalization performance and low architectural complexity. The effectiveness of the proposed method is evaluated on three prediction problems: real-time data of thermal system in a 600-MW power plant, Mackey–Glass time series and CATS time series. Results show that HDIPSO-RNN has better prediction accuracy and generalization performance, as well as generates more compact networks in comparison with some methods from the literature.

Prediction of β-turns and β-turn types by a novel bidirectional Elman-type recurrent neural network with multiple output layers (MOLEBRNN)

Prediction of β-turns from amino acid sequences has long been recognized as an important problem in structural bioinformatics due to their frequent occurrence as well as their structural and functional significance. Because various structural features of proteins are intercorrelated, secondary structure information has been often employed as an additional input for machine learning algorithms while predicting β-turns. Here we present a novel bidirectional Elman-type recurrent neural network with multiple output layers (MOLEBRNN) capable of predicting multiple mutually dependent structural motifs and demonstrate its efficiency in recognizing three aspects of protein structure: β-turns, β-turn types, and secondary structure. The advantage of our method compared to other predictors is that it does not require any external input except for sequence profiles because interdependencies between different structural features are taken into account implicitly during the learning process. In a sevenfold cross-validation experiment on a standard test dataset our method exhibits the total prediction accuracy of 77.9% and the Mathew's Correlation Coefficient of 0.45, the highest performance reported so far. It also outperforms other known methods in delineating individual turn types. We demonstrate how simultaneous prediction of multiple targets influences prediction performance on single targets. The MOLEBRNN presented here is a generic method applicable in a variety of research fields where multiple mutually depending target classes need to be predicted. Availability: http://webclu.bio.wzw.tum.de/predator-web/.

Combining prediction of secondary structure and solvent accessibility in proteins

Owing to the use of evolutionary information and advanced machine learning protocols, secondary structures of amino acid residues in proteins can be predicted from the primary sequence with more than 75% per-residue accuracy for the 3-state (i.e., helix, beta-strand, and coil) classification problem. In this work we investigate whether further progress may be achieved by incorporating the relative solvent accessibility (RSA) of an amino acid residue as a fingerprint of the overall topology of the protein. Toward that goal, we developed a novel method for secondary structure prediction that uses predicted RSA in addition to attributes derived from evolutionary profiles. Our general approach follows the 2-stage protocol of Rost and Sander, with a number of Elman-type recurrent neural networks (NNs) combined into a consensus predictor. The RSA is predicted using our recently developed regression-based method that provides real-valued RSA, with the overall correlation coefficients between the actual and predicted RSA of about 0.66 in rigorous tests on independent control sets. Using the predicted RSA, we were able to improve the performance of our secondary structure prediction by up to 1.4% and achieved the overall per-residue accuracy between 77.0% and 78.4% for the 3-state classification problem on different control sets comprising, together, 603 proteins without homology to proteins included in the training. The effects of including solvent accessibility depend on the quality of RSA prediction. In the limit of perfect prediction (i.e., when using the actual RSA values derived from known protein structures), the accuracy of secondary structure prediction increases by up to 4%. We also observed that projecting real-valued RSA into 2 discrete classes with the commonly used threshold of 25% RSA decreases the classification accuracy for secondary structure prediction. While the level of improvement of secondary structure prediction may be different for prediction protocols that implicitly account for RSA in other ways, we conclude that an increase in the 3-state classification accuracy may be achieved when combining RSA with a state-of-the-art protocol utilizing evolutionary profiles. The new method is available through a Web server at http://sable.cchmc.org.

10204 動的環境における移動ロボットのSensor-Based Controlに関する研究(OS15 知能ロボット)

Acquisition of actions based on prediction, which needs some context information, is important for a robot in dynamic environment. In this paper, task of avoiding moving object is employed first. The appropriate actions for this task are learned based on the combination of neural network and reinforcement learning. Architecture with Elman-type recurrent neural network and that without it are compared. Simulation result is shown the effectivity of recurrent neural network for this task. What is more, task of going to goal avoiding moving object is employed, and effect of learning system with Actor-Q architecture which could choose an action from plural that is verified for this task by simulation.

Predictions of Missing Wave Data by Recurrent Neuronets

Real time wave measurements in Turkey are often interrupted because of operational difficulties encountered. Therefore, the lacking significant wave height, period and directions were simultaneously estimated from the dynamic Elman type recurrent neural networks. Their predictions were compared with the commonly applied static feed-forward multilayer neural networks and with the stochastic Auto Regressive (AR) and Exogenous Input Auto Regressive (ARX) models. Two distinct learning algorithms, the steepest descent with momentum and the conjugate gradient methods were employed to train the neural networks. It was concluded that, the recurrent neural network generally showed better performance than the feed-forward neural network in the concurrent forecasting of multiple wave parameters. Both artificial intelligence techniques demonstrated a good performance when compared to the predictions of AR and ARX models. Prediction methods are also compared using continuous artificial data generated with known properties by measuring their performance in predicting the removed segments of various lengths. The multivariate ENN model successfully predicted the removed segments of artificially generated wave data. Hence, the learning ability of artificial intelligence techniques was verified signifying the robustness and fault-failure tolerance of neural networks.

Dynamics of a recurrent neural network acquired through learning a context-based attention task

Selective attention is a very important function for robots acting in the real world. In this function, not only attention itself, but also context extraction and retention are very intelligent processes and are not easily realized. In this article, an attention task is presented in which context information must be extracted from the first pattern presented, and using the context information, a recognition response must be generated from the second pattern presented. An Elman-type recurrent neural network is used to extract and retain the context information. The reinforcement signal that indicates whether the response is correct or not is the only signal given to the system during learning. By this simple learning process, the necessary context information got to be extracted and retained, and then the system changed to generate the correct responses. The function of associative memory was also observed in the feedback-loop in the Elman-type neural network. Furthermore the adaptive formation of basins was examined by varying the learning conditions.

Multivariable adaptive control using an observer based on a recurrent neural network

A real-time learning control technique for a general non-linear multivariable process is presented and applied to a laboratory plant. The proposed technique is a hybrid approach, which combines the ability of a recurrent neural network for modelling purposes and a linear pole placement control law to design the controller, providing a bridge between the field of neural networks and the well-known linear adaptive control methods. An Elman-type recurrent neural network strategy is introduced to model the behaviour of the non-linear plant, using available input–output data (an unmeasurable state problem is assumed). Following a linearization technique a linear time-varying state-space model is obtained, which allows simultaneous estimation of parameters and states. Once the neural model is linearized, some well-established standard linear control strategies can be applied. With simultaneous online training of the neural network and controller synthesis, the resulting structure is an indirect adaptive self-tuning strategy. The identification and control performances of the proposed approach are investigated on a non-linear multivariable three-tank laboratory system. Experimental results show the effectiveness of the proposed hybrid structure. Copyright © 1999 John Wiley & Sons, Ltd.

Identification of linear and nonlinear dynamic systems using recurrent neural networks

This paper describes the use of Elman-type recurrent neural networks to identify dynamic systems. Networks as originally designed by Elman ( Cognitive Sci., 1990, 14, 179–211) and also those in which self-connections are made to the context units were employed to identify a variety of linear and nonlinear systems. It was found that the latter networks were more versatile than the basic Elman nets in being able to model the dynamic behaviour of high order linear and nonlinear systems.