Recurrent Neural Network Approach Research Articles

Designing Graded Fuel Cell Electrodes for Proton Exchange Membrane (Pem) Fuel Cells with Recurrent Neural Network (Rnn) Approaches

Designing Graded Fuel Cell Electrodes for Proton Exchange Membrane (Pem) Fuel Cells with Recurrent Neural Network (Rnn) Approaches

Identification of Oscillatory Modes in Power System Using Deep Learning Approach

The increase in electric power demand pushes the modern power system for more interconnected networks. It leads to a lack of inertia and creates more critical disturbances in the power system. When this oscillation isn’t damped out, it results in cascade tripping. Immediate detection of low-frequency oscillatory modes and their parameters will help the power system operator to act on a particular event without consuming much time. This research paper proposes novel strategies for identifying low-frequency modes using deep learning techniques, and the model can predict the LFO modes in different topologies. This work presents the Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) approach to predict the instantaneous mode oscillatory parameters in the power system. Once the LSTM-RNN model is trained for different power disturbance situations, it can be used for any events associated with the system. Simulation results are verified using two area Kundur systems at various disturbance conditions. The simulations are performed using MATLAB software and python tensor flow library. The results are validated using statistical methods, and it confirms the superior viability and adaptability of the proposed approach in predicting the instantaneous mode parameters.

An Intelligent Disease Prediction System for Psychological Diseases by Implementing Hybrid Hopfield Recurrent Neural Network Approach

An Intelligent Disease Prediction System for Psychological Diseases by Implementing Hybrid Hopfield Recurrent Neural Network Approach

Long-Time Prediction of Arrhythmic Cardiac Action Potentials Using Recurrent Neural Networks and Reservoir Computing.

The electrical signals triggering the heart's contraction are governed by non-linear processes that can produce complex irregular activity, especially during or preceding the onset of cardiac arrhythmias. Forecasts of cardiac voltage time series in such conditions could allow new opportunities for intervention and control but would require efficient computation of highly accurate predictions. Although machine-learning (ML) approaches hold promise for delivering such results, non-linear time-series forecasting poses significant challenges. In this manuscript, we study the performance of two recurrent neural network (RNN) approaches along with echo state networks (ESNs) from the reservoir computing (RC) paradigm in predicting cardiac voltage data in terms of accuracy, efficiency, and robustness. We show that these ML time-series prediction methods can forecast synthetic and experimental cardiac action potentials for at least 15–20 beats with a high degree of accuracy, with ESNs typically two orders of magnitude faster than RNN approaches for the same network size.

Recurrent neural network-based multiaxial plasticity model with regularization for physics-informed constraints

A recurrent neural network (RNN) based model is developed as a surrogate to predict nonlinear plastic response under multiaxial loading. The RNN-based model is trained and tested on stress versus strain curves generated using a numerical solution based on the classical radial return method. Besides simply learning the basic constitutive relationship, a novel approach is taken to enforce certain physical conditions. Specifically, regularization is employed to maintain non-negative plastic power density throughout the loading history thereby ensuring monotonically increasing plastic work and thermodynamic consistency. Enforcing physics in this manner permits coupling of the data-driven RNN approach with physics-based knowledge and laws. This has the effect of reducing the necessary amount of data and ensuring known physical laws are not violated. Since, once trained, the model need not perform the expensive task of solving nonlinear equations, its efficiency is orders of magnitude greater than its numerical counterpart. The RNN-based model has been trained on varied sets of data and the accuracy on test datasets validated. The developed model is general and robust and has widespread application such as in the simulation of metal forming, large scale plasticity, and part life prediction.

Channel state information estimation for 5G wireless communication systems: recurrent neural networks approach.

In this study, a deep learning bidirectional long short-term memory (BiLSTM) recurrent neural network-based channel state information estimator is proposed for 5G orthogonal frequency-division multiplexing systems. The proposed estimator is a pilot-dependent estimator and follows the online learning approach in the training phase and the offline approach in the practical implementation phase. The estimator does not deal with complete a priori certainty for channels’ statistics and attains superior performance in the presence of a limited number of pilots. A comparative study is conducted using three classification layers that use loss functions: mean absolute error, cross entropy function for kth mutually exclusive classes and sum of squared of the errors. The Adam, RMSProp, SGdm, and Adadelat optimisation algorithms are used to evaluate the performance of the proposed estimator using each classification layer. In terms of symbol error rate and accuracy metrics, the proposed estimator outperforms long short-term memory (LSTM) neural network-based channel state information, least squares and minimum mean square error estimators under different simulation conditions. The computational and training time complexities for deep learning BiLSTM- and LSTM-based estimators are provided. Given that the proposed estimator relies on the deep learning neural network approach, where it can analyse massive data, recognise statistical dependencies and characteristics, develop relationships between features and generalise the accrued knowledge for new datasets that it has not seen before, the approach is promising for any 5G and beyond communication system.

Exploring the Adaptation of Recurrent Neural Network Approaches for Extracting Drug–Drug Interactions from Biomedical Text

Information extraction (IE) is the process of automatically identifying structured information from unstructured or partially structured text. IE processes can involve several activities, such as named entity recognition, event extraction, relationship discovery, and document classification, with the overall goal of translating text into a more structured form. Information on the changes in the effect of a drug, when taken in combination with a second drug, is known as drug–drug interaction (DDI). DDIs can delay, decrease, or enhance absorption of drugs and thus decrease or increase their efficacy or cause adverse effects. Recent research trends have shown several adaptation of recurrent neural networks (RNNs) from text. In this study, we highlight significant challenges of using RNNs in biomedical text processing and propose automatic extraction of DDIs aiming at overcoming some challenges. Our results show that the system is competitive against other systems for the task of extracting DDIs.

Spatiotemporal modeling of land subsidence using a geographically weighted deep learning method based on PS-InSAR

The demand for water resources during urbanization forces the continuous exploitation of groundwater, resulting in dramatic piezometric drawdown and inducing regional land subsidence (LS). This has greatly threatened sustainable development in the long run. LS modeling helps understanding the factors responsible for the ongoing loss of land elevation and hence enhances the development of prevention strategies. Data-driven LS models perform well with fewer variables and faster convergence than physically-based hydrogeological models. However, the former models often cannot simultaneously reflect the temporal nonlinearity and spatial correlation (SC) characteristics of LS under complex variables. We proposed a LS spatiotemporal model which considers both nonlinear and spatial correlations between LS and groundwater level change of exploited aquifers. It is based on deep learning method and LS time series detected by permanent scatterer-interferometric synthetic aperture radar (PS-InSAR). The LS time series and hydrogeological properties are constructed as a spatiotemporal dataset for model training. The spatiotemporal LS model, geographically weighted long short-term memory (GW-LSTM), is constructed by integrating SC with LSTM. This latter is a deep recurrent neural network approach incorporating sequential data. The model is validated by a case study in the Beijing plain. The results show that the accuracy of the proposed model can be greatly improved considering the spatial correlation between LS and influencing factors. Furthermore, the comparison between the LSTM and GW-LSTM models reveals that groundwater level variation is not a unique causation of LS in the study area. The developed model deals with the spatiotemporal characteristics of LS under multiple variables and can be used to predict LS under different scenarios of groundwater level variations for the purpose of monitoring and providing evidence to support the prevention of future LS.

Characterizing learner behavior from touchscreen data

As educational approaches increasingly adopt digital formats, data logs create a newfound wealth of information about student behaviors in those learning environments for educators and researchers. Yet making sense of that information, particularly to inform pedagogy, remains a significant challenge. Data from digital sensors that sample at the millisecond level of granularity, such as computer mouses or touchscreens, is notoriously difficult to computationally process and mine for patterns. Adding to the challenge is the limited domain knowledge of this biological sensor level of interaction which prohibits a comprehensive manual feature engineering approach to utilize those data streams. In this paper, we attempt to enhance the assessment capability of a touchscreen-based tutoring system by using Recurrent Neural Networks (RNNs) to predict students’ strategies from their 60hz data streams. We hypothesize that the ability of neural networks to learn representations automatically, instead of relying on human feature engineering, may benefit this classification task. Our classification models (including majority class) were trained and cross-validated at several levels on historical data which had been human coded with learners’ strategies. Our RNN approach to this difficult classification task moderately advances performance above logistic regression. We discuss the implications of this performance for enabling greater tutoring system autonomy. We also present visualizations that illustrate how this neural network approach to modeling sensor data can reveal patterns detected by the RNN. The surfaced patterns, regularized from a larger superset of mostly uncoded data, underscore the mix of normative and seemingly idiosyncratic behavior that characterizes the state space of learning at this high frequency level of observation.

Day-Ahead Solar Irradiance Forecasting Using Hybrid Recurrent Neural Network with Weather Classification for Power System Scheduling

At the present time, power-system planning and management is facing the major challenge of integrating renewable energy resources (RESs) due to their intermittent nature. To address this problem, a highly accurate renewable energy generation forecasting system is needed for day-ahead power generation scheduling. Day-ahead solar irradiance (SI) forecasting has various applications for system operators and market agents such as unit commitment, reserve management, and biding in the day-ahead market. To this end, a hybrid recurrent neural network is presented herein that uses the long short-term memory recurrent neural network (LSTM-RNN) approach to forecast day-ahead SI. In this approach, k-means clustering is first used to classify each day as either sunny or cloudy. Then, LSTM-RNN is used to learn the uncertainty and variability for each type of cluster separately to predict the SI with better accuracy. The exogenous features such as the dry-bulb temperature, dew point temperature, and relative humidity are used to train the models. Results show that the proposed hybrid model has performed better than a feed-forward neural network (FFNN), a support vector machine (SVM), a conventional LSTM-RNN, and a persistence model.

Modelling non-Markovian dynamics in photonic crystals with recurrent neural networks

We develop a recurrent neural network framework to model the non-Markovian dynamics exhibited by two-level atoms interacting with the radiation reservoir of a photonic crystal. Despite the strong non-Markovianity of the atomic dynamics induced by the rapid spectral variation in photonic density of states of the photonic reservoir, our recurrent neural network approach is able to capture precise details in the atomic evolution, including the fractional steady-state atomic population inversion and spectral splitting of the atomic transition. We demonstrate the robustness of the recurrent neural network setup against reduced data sets and its effectiveness to deal with systems of increased complexity.

A Recurrent Neural Network approach for whole genome bacteria identification

ABSTRACT The identification of bacteria plays an essential role in multiple areas of research. Those areas include experimental biology, food and water industries, pathology, microbiology, and evolutionary studies. Although there exist methodologies for identification, a transition to a whole-genome sequence-based taxonomy is already undergoing. Next-Generation Sequencing helps the transition by producing DNA sequence data efficiently. However, the rate of DNA sequence data generation and the high dimensionality of such data need faster computer methodologies. Machine learning, an area of artificial intelligence, has the ability to analyze high dimensional data in a systematic, fast, and efficient way. Therefore, we propose a sequential deep learning model for bacteria identification. The proposed neural network exploits the vast amounts of information generated by Next-Generation Sequencing, in order to extract an identification model for whole-genome bacteria sequences. After validating the identification model, the bidirectional recurrent neural network outperformed other classification approaches.

Ethanol Fuel Demand Forecasting in Brazil Using a LSTM Recurrent Neural Network Approach

Ethanol is a biofuel widely consumed in Brazil, which functions as a substitute for gasoline since the late 1970s. Due to several fluctuations in the characteristics of the Brazilian vehicle fleet and the political-economic conditions of the country, forecasting ethanol consumption has become a difficult task to perform. Under this scenario, the aim of this paper was to forecast ethanol consumption in Brazil using an approach of Long-Short Term Memory (LSTM) Recurrent Neural Networks (RNN) and Autoregressive Integrated Moving Average (ARIMA) models. The above, taking into consideration univariate and multivariate models for each case. Likewise, single-layer and multi-layer topologies of LSTM RNN were explored in this study. The results show that LSTM models overperformed ARIMA models even working with a relatively small training dataset of just 180 instances. This, for both univariate and multivariate models. A novel approach for searching suitable LSTM Neural Network topologies is proposed in this paper.

Deep Learning-Based Ground Vibration Monitoring: Reinforcement Learning and RNN–CNN Approach

This letter studies deep learning-based efficient ground vibration monitoring systems. In this work, artificial intelligence (AI) techniques are adopted to effectively deal with practical issues of data collection and classification. Specifically, we develop a novel energy-efficient data collection scheme by adopting deep Q-network-based reinforcement learning. Also, we propose an enhanced joint recurrent neural network (RNN) and convolutional neural network (CNN) approach for ground vibration classification. The performance of the proposed scheme is evaluated using real-world ground vibration data. The experimental results show that the proposed classification scheme outperforms the best existing scheme with CNN by more than 13% in terms of classification accuracy. It is also shown that the proposed energy management scheme can improve the accuracy of the proposed ground vibration monitoring system by 7.6% over the comparable scheme using equal power allocation.

A novel recurrent neural network approach in forecasting short term solar irradiance

Forecasting solar irradiance is of utmost importance in supplying renewable energy efficiently and timely. This paper aims to experiment five variants of recurrent neural networks (RNN), and develop effective and reliable 5-minute short term solar irradiance prediction models. The 5 RNN classes are long–short term memory (LSTM), gated recurrent unit (GRU), Simple RNN, bidirectional LSTM (Bi-LSTM), and bidirectional GRU (Bi-GRU); the first 3 classes are unidirectional and the last two are bidirectional RNN models. The 26 months data under consideration, exhibits extremely volatile weather conditions in Jinju city, South Korea. Therefore, after different experimental processes, 5 hyper-parameters were selected for each model cautiously. In each model, different levels of depth and width were tested; moreover, a 9-fold cross validation was applied to distinguish them against high variability in the seasonal time-series dataset. Generally the deeper architectures of the aforementioned models had significant outcomes; meanwhile, the Bi-LSTM and Bi-GRU provided more accurate predictions as compared to the unidirectional ones. The Bi-GRU model provided the lowest RMSE and highest R2 values of 46.1 and 0.958; additionally, it required 5.25*10−5 seconds per trainable parameter per epoch, the lowest incurred computational cost among the mentioned models. All 5 models performed differently over the four seasons in the 9-fold cross validation test. On average, the bidirectional RNNs and the simple RNN model showed high robustness with less data and high temporal data variability; although, the stronger architectures of the bidirectional models, deems their results more reliable.

Deep Recurrent Neural Networks Approach to Sedimentary Facies Classification Using Well Logs

Determination of lithofacies is one of the most important steps for reservoir characterization. Well log curves are not always sufficient to determine lithology as some times the signals are similar for different lithologies. We believe that the sequence of sedimentary patterns that follows the general geologic rules can be essential to help this disambiguation. This work aims to present a computational system based on deep recurrent neural networks (RNNs) as an effective method to automatically identify lithofacies patterns from well logs. We show that bidirectional long-short-term memory (BiLSTM) RNNs can learn long-term dependencies between time steps of sequence data improving the context available to facies classification. We validated our method by applying it to a real case study from Rio Bonito Formation (Paraná Basin, Brazil) and the proposed method is compared with XGBoost, Random Forest, Naïve Bayes, and support vector machine (SVM) learning approaches. The results indicate that the performance of the proposed method for lithology identification is higher when compared with these other methods.

A review on Video Classification with Methods, Findings, Performance, Challenges, Limitations and Future Work

In recent years, there has been a rapid development in web users and sufficient bandwidth. Internet connectivity, which is so low cost, makes the sharing of information (text, audio, and videos) more common and faster. This video content needs to be analyzed for prediction it classes in different purpose for the users. Many machines learning approach has been developed for the classification of video to save people time and energy. There are a lot of existing review papers on video classification, but they have some limitations such as limitation of the analysis, badly structured, not mention research gaps or findings, not clearly describe advantages, disadvantages, and future work. But our review paper almost overcomes these limitations. This study attempts to review existing video-classification procedures and to examine the existing methods of video-classification comparatively and critically and to recommend the most effective and productive process. First of all, our analysis examines the classification of videos with taxonomical details, the latest application, process, and datasets information. Secondly, overall inconvenience, difficulties, shortcomings and potential work, data, performance measurements with the related recent relation in science, deep learning, and the model of machine learning. Study on video classification systems using their tools, benefits, drawbacks, as well as other features to compare the techniques they have used also constitutes a key task of this review. Lastly, we also present a quick summary table based on selected features. In terms of precision and independence extraction functions, the RNN (Recurrent Neural Network), CNN (Convolutional Neural Network) and combination approach performs better than the CNN dependent method.

On the way to fault detection method in moving load dynamics problem by modified recurrent neural networks approach

Parameters identification on structure subjected to moving load can be predicted by using the accurate and reliable data. The concepts of recurrent neural networks (RNNs) approach have been used in parameters (crack locations and severities) identifications in structure subjected to moving load in the present methodology. This methodology has incorporated the knowledge based Elman's recurrent neural networks (ERNNs) and Jordan's recurrent neural networks (JRNNs) jointly for the identification of parameters. This approach has been addressed as the inverse problem for predicting the locations and quantification of cracks in the structure in a supervised manner. The Levenberg-Marquardt's back propagation algorithm is implemented to train the proposed networks. To check the robustness of the present method, Numerical studies followed by Finite Element Analysis (FEA) and experimental verifications (Forward problems) are presented as a case study by considering a multi-cracked simply supported structure under a moving mass. The estimated crack locations and severities obtained from the proposed RNNs model converge well with those of FEA and experiments. From the demonstration of the case study, it concludes that the proposed analogy can identify and quantify the crack locations and severities effectively.

Finite-time event-triggered approach for recurrent neural networks with leakage term and its application

This work investigates the finite-time event-triggered approach for recurrent neural networks with leakage term and its application. Here, decentralized event-triggered framework is recommended where event is checked at every sensor node related to local information for available triggering and the updated control is done whenever a centralized event is triggered. By handling the Lyapunov–Krasovskii functional (LKF) method together with novel inequality techniques like Wirtinger single and double integral inequality (WSI,WDI) technique, delay productive term (DPT), and a few adequate conditions are acquired to ensure the finite-time stability (FTS) analysis for the considered system, which is expressed with respect to linear matrix inequalities (LMIs). At last, numerical simulations are provided to indicate the efficiency of the expected results, two of these examples were supported by genuine use of the benchmark issue that correlates with sensible concerns under finite-time execution.

Neural machine translation with a polysynthetic low resource language

Low-resource languages (LRL) with complex morphology are known to be more difficult to translate in an automatic way. Some LRLs are particularly more difficult to translate than others due to the lack of research interest or collaboration. In this article, we experiment with a specific LRL, Quechua, that is spoken by millions of people in South America yet has not undertaken a neural approach for translation until now. We improve the latest published results with baseline BLEU scores using the state-of-the-art recurrent neural network approaches for translation. Additionally, we experiment with several morphological segmentation techniques and introduce a new one in order to decompose the language’s suffix-based morphemes. We extend our work to other high-resource languages (HRL) like Finnish and Spanish to show that Quechua, for qualitative purposes, can be considered compatible with and translatable into other major European languages with measurements comparable to the state-of-the-art HRLs at this time. We finalize our work by making our best two Quechua–Spanish translation engines available on-line.