Training Recurrent Neural Networks Research Articles

Task-Aware Verifiable RNN-Based Policies for Partially Observable Markov Decision Processes

Partially observable Markov decision processes (POMDPs) are models for sequential decision-making under uncertainty and incomplete information. Machine learning methods typically train recurrent neural networks (RNN) as effective representations of POMDP policies that can efficiently process sequential data. However, it is hard to verify whether the POMDP driven by such RNN-based policies satisfies safety constraints, for instance, given by temporal logic specifications. We propose a novel method that combines techniques from machine learning with the field of formal methods: training an RNN-based policy and then automatically extracting a so-called finite-state controller (FSC) from the RNN. Such FSCs offer a convenient way to verify temporal logic constraints. Implemented on a POMDP, they induce a Markov chain, and probabilistic verification methods can efficiently check whether this induced Markov chain satisfies a temporal logic specification. Using such methods, if the Markov chain does not satisfy the specification, a byproduct of verification is diagnostic information about the states in the POMDP that are critical for the specification. The method exploits this diagnostic information to either adjust the complexity of the extracted FSC or improve the policy by performing focused retraining of the RNN. The method synthesizes policies that satisfy temporal logic specifications for POMDPs with up to millions of states, which are three orders of magnitude larger than comparable approaches.

Machine learning astrophysics from 21 cm lightcones: impact of network architectures and signal contamination

ABSTRACT Imaging the cosmic 21 cm signal will map out the first billion years of our Universe. The resulting 3D lightcone (LC) will encode the properties of the unseen first galaxies and physical cosmology. Here, we build on previous work using neural networks (NNs) to infer astrophysical parameters directly from 21 cm LC images. We introduce recurrent neural networks (RNNs), capable of efficiently characterizing the evolution along the redshift axis of 21 cm LC images. Using a large database of simulated cosmic 21 cm LCs, we compare the relative performance in parameter estimation of different network architectures. These including two types of RNNs, which differ in their complexity, as well as a more traditional convolutional neural network (CNN). For the ideal case of no instrumental effects, our simplest and easiest to train RNN performs the best, with a mean squared parameter estimation error (MSE) that is lower by a factor of ≳2 compared with the other architectures studied here, and a factor of ≳8 lower than the previously-studied CNN. We also corrupt the cosmic signal by adding noise expected from a 1000 h integration with the Square Kilometre Array, as well as excising a foreground-contaminated ‘horizon wedge’. Parameter prediction errors increase when the NNs are trained on these contaminated LC images, though recovery is still good even in the most pessimistic case (with R2 ≳ 0.5−0.95). However, we find no notable differences in performance between network architectures on the contaminated images. We argue this is due to the size of our data set, highlighting the need for larger data sets and/or better data augmentation in order to maximize the potential of NNs in 21 cm parameter estimation.

Multitask neural networks for predicting bladder pressure with time series data

Multitask learning (MTL) can improve accuracy over vanilla neural networks in modeling population level time series data. This can be accomplished by assigning the prediction for each individual in the population as a separate task, thereby leveraging the heterogeneity of population level data. Here, we investigate a novel approach by training recurrent neural networks (RNNs) in a multitask setting. We apply this new methodology to experimental data for predicting bladder pressure, and then bladder contractions, from an external urethral sphincter electromyograph (EUS EMG) signal. We found that the multitask models make more accurate individual level predictions than their single tasking counterparts. We observed that, for bladder pressure prediction, either incorporating multitask learning or RNN structure generalized best to out of sample test data and multitasking RNNs had high out of sample correlation coefficients. These results suggest that MTL models could be used to leverage heterogeneous population time series data for making individualized predictions. From these bladder pressure predictions, we predicted the onset of bladder contractions. Our results indicate that the MTL RNN model was superior in both intra- and inter-individual bladder contraction predictions as measured by sensitivity (85.7%), specificity (98.7%) and precision (73.5%).

Evaluation of Recurrent Neural Network Models for Parkinson's Disease Classification Using Drawing Data.

Parkinson's disease is a disorder that affects the neurons in the human brain. The various symptoms include slowness of motor functions (bradykinesia), motor instability, speech impairment and in some cases, psychiatric effects such as hallucinations. Most of these, however, are also common side effects of natural aging. This makes an accurate diagnosis of Parkinson's disease a challenging task. Some breakthroughs have been made in recent years with the help of deep learning. This work aims at considering figure drawing data as a time series of coordinates, angles and pressure readings to train recurrent neural network models. In addition, the work compares two recurrent network models, Long Short-Term Memory and Echo State Networks, to explore the advantages and disadvantages of both architectures.

Imaging subsurface orebodies with airborne electromagnetic data using a recurrent neural network

Conventional interpretation of airborne electromagnetic data has been conducted by solving the inverse problem. However, with recent advances in machine learning (ML) techniques, a 1D deep neural network inversion that predicts a 1D resistivity model using multifrequency vertical magnetic fields and sensor height information at one location has been applied. Nevertheless, since the final interpretation of this 1D approach relies on connecting 1D resistivity models, 1D ML interpretation has low accuracy for the estimation of an isolated anomaly, as in conventional 1D inversion. Thus, we have developed a 2D interpretation technique that can overcome the limitations of 1D interpretation and consider spatial continuity by using a recurrent neural network (RNN). We generate various 2D resistivity models, calculate the ratio of primary and induced secondary magnetic fields of the vertical direction in ppm scale using a vertical magnetic dipole source, and then train the RNN using the resistivity models and the corresponding electromagnetic (EM) responses. To verify the validity of 2D RNN inversion, we apply the trained RNN to synthetic and field data. Through application of the field data, we demonstrate that the design of the training data set is crucial to improve prediction performance in a 2D RNN inversion. In addition, we investigate changes in the RNN inversion results of field data dependent on the data preprocessing. We demonstrate that using two types of data, logarithmic transformed data and linear-scale data, having different patterns of input information can enhance the prediction performance of the EM inversion results.

Efficient Kinect Sensor-based Kurdish Sign Language Recognition Using Echo System Network

Sign language assists in building communication and bridging gaps in understanding. Automatic sign language recognition (ASLR) is a field that has recently been studied for various sign languages. However, Kurdish sign language (KuSL) is relatively new and therefore researches and designed datasets on it are limited. This paper has proposed a model to translate KuSL into text and has designed a dataset using Kinect V2 sensor. The computation complexity of feature extraction and classification steps, which are serious problems for ASLR, has been investigated in this paper. The paper proposed a feature engineering approach on the skeleton position alone to provide a better representation of the features and avoid the use of all of the image information. In addition, the paper proposed model makes use of recurrent neural networks (RNNs)-based models. Training RNNs is inherently difficult, and consequently, motivates to investigate alternatives. Besides the trainable long short-term memory (LSTM), this study has proposed the untrained low complexity echo system network (ESN) classifier. The accuracy of both LSTM and ESN indicates they can outperform those in state-of-the-art studies. In addition, ESN which has not been proposed thus far for ASLT exhibits comparable accuracy to the LSTM with a significantly lower training time.

Multi-time-scale input approaches for hourly-scale rainfall–runoff modeling based on recurrent neural networks

Abstract This study proposes two effective approaches to reduce the required computational time of the training process for time-series modeling through a recurrent neural network (RNN) using multi-time-scale time-series data as input. One approach provides coarse and fine temporal resolutions of the input time-series data to RNN in parallel. The other concatenates the coarse and fine temporal resolutions of the input time-series data over time before considering them as the input to RNN. In both approaches, first, the finer temporal resolution data are utilized to learn the fine temporal scale behavior of the target data. Then, coarser temporal resolution data are expected to capture long-duration dependencies between the input and target variables. The proposed approaches were implemented for hourly rainfall–runoff modeling at a snow-dominated watershed by employing a long short-term memory network, which is a type of RNN. Subsequently, the daily and hourly meteorological data were utilized as the input, and hourly flow discharge was considered as the target data. The results confirm that both of the proposed approaches can reduce the required computational time for the training of RNN significantly. Lastly, one of the proposed approaches improves the estimation accuracy considerably in addition to computational efficiency.

Learning various length dependence by dual recurrent neural networks

Recurrent neural networks (RNNs) are widely used as a memory model for sequence-related problems. Many variants of RNN have been proposed to solve the gradient problems of training RNNs and process long sequences. Although some classical models have been proposed, capturing long-term dependence while responding to short-term changes remains a challenge. To address this problem, we propose a new model named Dual Recurrent Neural Networks (DuRNN). The DuRNN consists of two parts to learn the short-term dependence and progressively learn the long-term dependence. The first part is a recurrent neural network with constrained full recurrent connections to deal with short-term dependence in sequence and generate short-term memory. Another part is a recurrent neural network with independent recurrent connections which helps to learn long-term dependence and generate long-term memory. A selection mechanism is added between two parts to transfer the needed long-term information to the independent neurons. Multiple modules can be stacked to form a multi-layer model for better performance. Our contributions are: 1) a new recurrent model developed based on the divide-and-conquer strategy to learn long and short-term dependence separately, and 2) a selection mechanism to enhance the separating and learning of different temporal scales of dependence. Both theoretical analysis and extensive experiments are conducted to validate the performance of our model. Experimental results indicate that the proposed DuRNN model can handle not only very long sequences (over 5,000 time steps), but also short sequences very well.

Emergence of prefrontal neuron maturation properties by training recurrent neural networks in cognitive tasks

SummaryWorking memory and response inhibition are functions that mature relatively late in life, after adolescence, paralleling the maturation of the prefrontal cortex. The link between behavioral and neural maturation is not obvious, however, making it challenging to understand how neural activity underlies the maturation of cognitive function. To gain insights into the nature of observed changes in prefrontal activity between adolescence and adulthood, we investigated the progressive changes in unit activity of recurrent neural networks as they were trained to perform working memory and response inhibition tasks. These included increased delay period activity during working memory tasks and increased activation in antisaccade tasks. These findings reveal universal properties underlying the neuronal computations behind cognitive tasks and explicate the nature of changes that occur as the result of developmental maturation.

Diagnosis of Heart Disease Using Feature Selection Methods Based On Recurrent Fuzzy Neural Networks

The World Health Organization (WHO) estimated one-third of all global deaths reason by cardiovascular diseases. Nowadays, artificial intelligence attracts many considerations in diagnosing heart disease. This study used trained recurrent fuzzy neural networks (RFNN) for diagnosing heart disease. This study also used five kinds of feature selection and extraction models for comparing the action of a model, such as data envelopment analysis (DEA), Linear Discriminative Analysis (LDA), Principle Component Analysis (PCA), Correlation Feature Selection (CFS), and Relief. By using these methods, this paper diagnosed whether the patient has a heart disease problem or not. The results showed that Correlation feature selection has the best operation in feature selection in RFNN by accuracy of 98.4%.

Faster Region-Convolutional Neural network oriented feature learning with optimal trained Recurrent Neural Network for bone age assessment for pediatrics

This paper tactics to develop the novel Tanner-Whitehouse 3 (TW3)-based automated Bone Age Assessment (BAA) model for children with the assistance of Faster Region- Convolutional Neural Network (Faster R-CNN) and Optimal trained Recurrent Neural Network (RNN). The proposed model covers three main stages: (a) Segmentation, (b) Faster R-CNN-based feature learning, and (c) optimal classification. Further, the segmentation of those regions is performed by adaptive Otsu thresholding. A Faster R-CNN is built to learn the features from the segmented regions. Once the features are extracted from the pooling layers of Faster R-CNN, it is subjected to the RNN. As a modification to RNN, the training weight is optimized by the Average Fitness-based Sun Flower Optimization (AF-SFO), and the optimized network predicts the age of the bone. The experimental evaluation of the proposed model over a set of images collected manually and publically shows its superior performance when compared to the state-of-the-art techniques.

Deep learning for prediction of depressive symptoms in a large textual dataset

Depression is a common illness worldwide with potentially severe implications. Early identification of depressive symptoms is a crucial first step towards assessment, intervention, and relapse prevention. With an increase in data sets with relevance for depression, and the advancement of machine learning, there is a potential to develop intelligent systems to detect symptoms of depression in written material. This work proposes an efficient approach using Long Short-Term Memory (LSTM)-based Recurrent Neural Network (RNN) to identify texts describing self-perceived symptoms of depression. The approach is applied on a large dataset from a public online information channel for young people in Norway. The dataset consists of youth’s own text-based questions on this information channel. Features are then provided from a one-hot process on robust features extracted from the reflection of possible symptoms of depression pre-defined by medical and psychological experts. The features are better than conventional approaches, which are mostly based on the word frequencies (i.e., some topmost frequent words are chosen as features from the whole text dataset and applied to model the underlying events in any text message) rather than symptoms. Then, a deep learning approach is applied (i.e., RNN) to train the time-sequential features discriminating texts describing depression symptoms from posts with no such descriptions (non-depression posts). Finally, the trained RNN is used to automatically predict depression posts. The system is compared against conventional approaches where it achieved superior performance than others. The linear discriminant space clearly reveals the robustness of the features by generating better clustering than other traditional features. Besides, since the features are based on the possible symptoms of depression, the system may generate meaningful explanations of the decision from machine learning models using an explainable Artificial Intelligence (XAI) algorithm called Local Interpretable Model-Agnostic Explanations (LIME). The proposed depression symptom feature-based approach shows superior performance compared to the traditional general word frequency-based approaches where frequency of the features gets more importance than the specific symptoms of depression. Although the proposed approach is applied on a Norwegian dataset, a similar robust approach can be applied on other depression datasets developed in other languages with proper annotations and symptom-based feature extraction. Thus, the depression prediction approach can be adopted to contribute to develop better mental health care technologies such as intelligent chatbots.

Short-term memory by transient oscillatory dynamics in recurrent neural networks

Despite the significance of short-term memory in cognitive function, the process of encoding and sustaining the input information in neural activity dynamics remains elusive. Herein, we unveiled the significance of transient neural dynamics to short-term memory. By training recurrent neural networks to short-term memory tasks and analyzing the dynamics, the characteristics of the short-term memory mechanism were obtained in which the input information was encoded in the amplitude of transient oscillations, rather than the stationary neural activities. This transient trajectory was attracted to a slow manifold, which permitted the discarding of irrelevant information. Additionally, we investigated the process by which the dynamics acquire robustness to noise. In this transient oscillation, the robustness to noise was obtained by a strong contraction of the neural states after perturbation onto the manifold. This mechanism works for several neural network models and tasks, which implies its relevance to neural information processing in general.

Statistical Machine Learning in Model Predictive Control of Nonlinear Processes

Recurrent neural networks (RNNs) have been widely used to model nonlinear dynamic systems using time-series data. While the training error of neural networks can be rendered sufficiently small in many cases, there is a lack of a general framework to guide construction and determine the generalization accuracy of RNN models to be used in model predictive control systems. In this work, we employ statistical machine learning theory to develop a methodological framework of generalization error bounds for RNNs. The RNN models are then utilized to predict state evolution in model predictive controllers (MPC), under which closed-loop stability is established in a probabilistic manner. A nonlinear chemical process example is used to investigate the impact of training sample size, RNN depth, width, and input time length on the generalization error, along with the analyses of probabilistic closed-loop stability through the closed-loop simulations under Lyapunov-based MPC.

RNN-Based Counterfactual Prediction, With an Application to Homestead Policy and Public Schooling

Abstract This paper proposes a method for estimating the effect of a policy intervention on an outcome over time. We train recurrent neural networks (RNNs) on the history of control unit outcomes to learn a useful representation for predicting future outcomes. The learned representation of control units is then applied to the treated units for predicting counterfactual outcomes. RNNs are specifically structured to exploit temporal dependencies in panel data and are able to learn negative and non-linear interactions between control unit outcomes. We apply the method to the problem of estimating the long-run impact of US homestead policy on public school spending.

Online Fall Detection Using Recurrent Neural Networks on Smart Wearable Devices

Unintentional falls can cause severe injuries and even death, especially if no immediate assistance is given. A fall detection system aims to detect a fall as soon as it occurs, therefore issuing an automatic assistance request. Wearable embedded sensors are emerging as the most viable solution for continuous monitoring since they are more effective, less intrusive and less expensive than other systems. Tailoring a deep learning method to the requirements of microcontrollers entails matching very stringent constraints in terms of both memory and computational power. In addition, datasets acquired with wearable devices are relatively scarce and not necessarily devised for supervised learning. In this work, we discuss the design of a software architecture based on recurrent neural networks which can be effective for fall detection while running entirely onboard a wearable device. The well-known and publicly-available SisFall dataset was adopted and extended with fine-grained temporal annotations to cope with the supervised training of recurrent neural networks. We then show that an appropriate process of architectural minimization together with accurate hyperparameters selection leads to a workable model which compares favorably with other detection techniques. The embedding of the resulting architecture has been validated using a state-of-art hardware device.

(Invited) Application of Machine Learning for High-Performance Multicrystalline Materials

Many practical materials are multicrystalline to contain complex microstructures and crystal defects. Owing to its complexity, universal guideline regarding ideal microstructures to maximize macroscopic performance has not been fully established. Therefore, we attempt to pioneer “multicrystalline informatics” through collaboration of experiments, theory, computation, and machine learning to show how we can obtain high-performance multicrystalline materials. We employ silicon as a model material, and prepare various useful machine learning tools for efficient materials development.One example is to predict distribution of crystal orientations in a large sample (e.g. whole 6-inch wafer) from optical images. We started with surface treatment of multicrystalline silicon wafers so that the reflection become sensitive to the crystal orientations. Multiple optical images were captured from each wafer with white-light illumination from different orientations. In addition, orientations of each crystal grain for a couple of wafers were measured by X-ray Laue scanner. The obtained data sets of reflection patterns and crystal orientations were used for training, validation, and test of a long short-term memory recurrent neural network. The trained model was applied to a test wafer to contain ~1,000 crystal grains, and the median of the estimated errors was ~3 degree.Another example is to predict probability of generation of dislocation clusters. For this purpose, a large quantity of photoluminescence (PL) images was collected from wafers obtained from a same ingot. Three-dimensional reconstruction of PL images after extracting dislocation clusters by image processing successfully visualized distribution of dislocation clusters. Therefore, we can collect many small PL images as positive samples to act as the source of dislocations as well as negative samples. The obtained data sets were used for transfer learning of pre-trained convolutional neural network for image classification. The final output layer was changed to contain two outputs if the input image acts as the source of dislocations or not, and weights for the last edges were determined using the data sets. The network was applied to test wafers so that we can visualize probability distribution of generation of dislocation clusters. A part of sources was used for multiscale structural characterizations to clarify microscopic origins.In addition, we are working on various machine learning models and statistical analysis for multicrystalline materials. Integration of such models is believed to be beneficial to show how we can maximize macroscopic performance of multicrystalline materials.This work was supported by JST/CREST, Grant No. JPMJCR17J1 (2017-2023).

Optimization of a Fuzzy Automatic Voltage Controller Using Real-Time Recurrent Learning

The automatic voltage regulator is an important component in energy generation systems; therefore, the tuning of this system is a fundamental aspect for the suitable energy conversion. This article shows the optimization of a fuzzy automatic voltage controller for a generation system using real-time recurrent learning, which is a technique conventionally used for the training of recurrent neural networks. The controller used consists of a compact fuzzy system based on Boolean relations, designed having equivalences with PI, PD, PID, and second order controllers. For algorithm implementation, the training equations are deduced considering the structure of the second order compact fuzzy controller. The results show that a closed-loop fuzzy control strategy was successfully implemented using real-time recurrent learning. In order to implement the controllers optimization, different weighting values for error and control action are used. The results show the behavior of the configurations used and its performance considering the steady state error, overshoot, and settling time.

A recurrent neural network framework with an adaptive training strategy for long-time predictive modeling of nonlinear dynamical systems

Long-time prediction of future states has been challenging in data-driven modeling of nonlinear dynamical systems as the prediction error accumulates over the prediction horizon. One of the potential reasons is the lack of robustness for the data-driven model. In this study we present a recurrent neural network (RNN) framework with an adaptive training strategy to model nonlinear dynamical systems from data for long-time prediction of future states. Specifically, we exploit the recurrence of network to improve the model robustness by explicitly incorporating the multi-step prediction with error accumulation into model training. Furthermore, we introduce an adaptive training strategy, where the prediction horizon gradually increases from a small value to facilitate the RNN training. We demonstrate the proposed approach on a family of Duffing oscillators, including autonomous and non-autonomous systems with various attractors, and discuss its advantages and limitations.

Encoding-based memory for recurrent neural networks

Learning to solve sequential tasks with recurrent models requires the ability to memorize long sequences and to extract task-relevant features from them. In this paper, we study memorization from the point of view of the design and training of recurrent neural networks. We study how to maximize the short-term memory of recurrent units, an objective difficult to achieve using backpropagation. We propose a new model, the Linear Memory Network, which features an encoding-based memorization component built with a linear autoencoder for sequences. Additionally, we provide a specialized training algorithm that initializes the memory to efficiently encode the hidden activations of the network. Experimental results on synthetic and real-world datasets show that the chosen encoding mechanism is superior to static encodings such as orthogonal models and the delay line. The method also outperforms RNN and LSTM units trained using stochastic gradient descent. Experiments on symbolic music modeling show that the training algorithm specialized for the memorization component improves the final performance compared to stochastic gradient descent.