Temporal Neural Network Model Research Articles

Gas leakage detection using spatial and temporal neural network model

Natural gas leakage can impose significant danger on a facility and its surrounding communities. Methods for early detection and diagnosis of such leakages have been developed and widely used for gas pipelines and storage tanks. Most techniques include inspection of sensor-aided mathematical models. Application of machine learning techniques to gas leakage detection has been rarely explored. In the present work, convolutional network (to model spatial likelihood of leak) is combined with bi-directional long short-term memory layer network, or BiLSTM (to model temporal dependence of leak likelihood) to perform leak detection and diagnosis. The developed model was trained and tested using sequence of concentration profiles generated using open-source simulated data. The model learned successfully to predict gas leakage and classify its size. The study also explores the flexibility of this network to perform quick detection and diagnose with the limited data. While the networks did not require parameter adjustments to achieve high prediction accuracy, further optimization is possible through data selection and pre-processing. The model needs to be further tested for wide range of leak scenarios. At its present condition, the combined application of convolutional network and BiLSTM shows promising results for early and accurate leak detection in natural gas facilities. Experimental results are needed to confirm the effectiveness of the model and data uncertainty.

Genes Expression Classification Through Histone Modification Using Temporal Neural Network

Background: Genes expression is high dimensional data, so it is very difficult to classify high dimensional data through traditional machine learning approaches. In this work we have proposed a model based on combined approach of Convolutional Neural Network and Recurrent Neural Network, both belong to deep learning model. The prediction has shown improved result than other machine learning algorithms. Expressions are generated through histone modification. Methods: To improve the accuracy deep learning model is proposed i.e. based on Convolutional and Recurrent neural network. This proposed model uses filter, causal convolutional layers and Residual Block for predictions. Results: In this work we have implemented the machine learning algorithms and deep learning algorithms like Logistic Regression, SVM, CNN, Deep Chrome and the proposed Temporal Neural Network. The performance is measured on the basis of parameters like accuracy, precision and AUC on the training and testing set. Conclusion: The proposed Temporal Neural Network model has shown better performance than other machine learning and deep learning algorithms. Due to this proposed deep learning algorithm can be successfully applied on the genes expression dataset.

A Temporal Neural Network Model for Probabilistic Multi-Period Forecasting of Distributed Energy Resources

Probabilistic forecasts of electrical loads and photovoltaic generation provide a family of methods able to incorporate uncertainty estimations in predictions. This paper aims to extend the literature on these methods by proposing a novel deep-learning model based on a mixture of convolutional neural networks, transformer models and dynamic Bayesian networks. Further, the paper also illustrates how to utilize Stochastic Variational Inference for training output distributions that allow time series sampling, a possibility not given for most state-of-the-art methods which do not use distributions. On top of this, the model also proposes an encoder-decoder topology that uses matrix transposes in order to both train on the sequential and the feature dimension. The performance of the work is illustrated on both load and generation time series obtained from a site representative of distributed energy resources in Norway and compared to state-of-the-art methods such as long-short-term memory. With a single-minute prediction resolution and a single-second computation time for an update with a batch size of 100 and a horizon of 24 hours, the model promises performance capable of real-time application. In summary, this paper provides a novel model that allows generating future scenarios for time series of distributed energy resources in real-time, which can be used to generate profiles for control problems under uncertainty.

A Spatiotemporal Neural Network Modeling Method for Nonlinear Distributed Parameter Systems

Neural network (NN) has been widely used in the field of modeling of lumped parameter systems. However, an NN approach cannot be used to model complex nonlinear distributed parameter systems (DPSs) because it does not account for this type of system's relationship with space. In this article, we propose a novel spatiotemporal NN (SNN) method to model nonlinear DPSs, which considers not only nonlinear dynamics regarding time, but also a nonlinear relationship with space. A temporal NN model was first constructed to represent the nonlinear temporal dynamics of each sensor's position. A spatial distribution function was then developed to represent the nonlinear relationship between spatial points. This strategy results in inherent consideration of any spatial dynamics. Finally, by integrating both the temporal NN model and the spatial distribution function, a novel SNN model was created to represent the spatiotemporal dynamics of the nonlinear DPSs. A two-step solving approach was further developed to learn the model. Additional analysis and proof of concept showed the effectiveness of this proposed method. This proposed method is different from traditional data-driven modeling methods in that it uses full information from all sensors and does not require model reduction technology. Case studies not only demonstrate the effectiveness of this proposed method, but also its superior modeling performance as compared with several commonly used methods.

From Regional to Global Brain: A Novel Hierarchical Spatial-Temporal Neural Network Model for EEG Emotion Recognition

In this paper, we propose a novel Electroencephalograph (EEG) emotion recognition method inspired by neuroscience with respect to the brain response to different emotions. The proposed method, denoted by R2G-STNN, consists of spatial and temporal neural network models with regional to global hierarchical feature learning process to learn discriminative spatial-temporal EEG features. To learn the spatial features, a bidirectional long short term memory (BiLSTM) network is adopted to capture the intrinsic spatial relationships of EEG electrodes within brain region and between brain regions, respectively. Considering that different brain regions play different roles in the EEG emotion recognition, a region-attention layer into the R2G-STNN model is also introduced to learn a set of weights to strengthen or weaken the contributions of brain regions. Based on the spatial feature sequences, BiLSTM is adopted to learn both regional and global spatial-temporal features and the features are fitted into a classifier layer for learning emotion-discriminative features, in which a domain discriminator working corporately with the classifier is used to decrease the domain shift between training and testing data. Finally, to evaluate the proposed method, we conduct both subject-dependent and subject-independent EEG emotion recognition experiments on SEED database, and the experimental results show that the proposed method achieves state-of-the-art performance.

MULTI-TEMPORAL LAND COVER CLASSIFICATION WITH LONG SHORT-TERM MEMORY NEURAL NETWORKS

Abstract. Land cover classification (LCC) is a central and wide field of research in earth observation and has already put forth a variety of classification techniques. Many approaches are based on classification techniques considering observation at certain points in time. However, some land cover classes, such as crops, change their spectral characteristics due to environmental influences and can thus not be monitored effectively with classical mono-temporal approaches. Nevertheless, these temporal observations should be utilized to benefit the classification process. After extensive research has been conducted on modeling temporal dynamics by spectro-temporal profiles using vegetation indices, we propose a deep learning approach to utilize these temporal characteristics for classification tasks. In this work, we show how long short-term memory (LSTM) neural networks can be employed for crop identification purposes with SENTINEL 2A observations from large study areas and label information provided by local authorities. We compare these temporal neural network models, i.e., LSTM and recurrent neural network (RNN), with a classical non-temporal convolutional neural network (CNN) model and an additional support vector machine (SVM) baseline. With our rather straightforward LSTM variant, we exceeded state-of-the-art classification performance, thus opening promising potential for further research.

Crop Yield Forecasting Using Artificial Neural Networks: A Comparison between Spatial and Temporal Models

Our recent study using historic data of wheat yield and associated plantation area, rainfall, and temperature has shown that incorporating statistics and artificial neural networks can produce highly satisfactory forecasting of wheat yield. However, no comparison has been made between the outcomes from the spatial neural network model and commonly used temporal neural network models in crop forecasting. This paper presents the latest research outcomes from using both the spatial and temporal neural network models in crop forecasting. Our simulation shows that the spatial NN model is able to predict the wheat yield with respect to a given plantation area with a high accuracy compared with the temporal NARNN and NARXNN models. However, the high accuracy of the spatial NN model in crop yield forecasting is limited to the forecasting of crop yield only within normal ranges. Users must be cautious when using either NARNN or NARXNN for crop yield forecasting due to their inconsistency between the results of training and forecasting.

Dynamic Emulations of Surface Radiation Components During Day and Night Under all Sky and Surface Conditions Using Temporal Neural Networks

Modeling of land surface radiation budget and its components is essential to a better understanding of soil-vegetation-atmosphere interactions. Time Delay Neural Network (TDNN) and Time Lag Recurrent Network (TLRN) models were used to emulate all the hourly surface radiation components for a temperate peatland during day and night under all-sky and -surface conditions. Sensitivity analyses of full versus reduced models, daytime versus nighttime periods, and TDNN versus TLRN models were carried out using training-, cross-validation, and testing-derived metrics of root mean squared error (RMSE), mean absolute error (MAE), and coefficient of determination (R2) for each of the components. The full daytime temporal neural network models performed best based on RMSE of 2.3 W m−2 in downwelling longwave radiation to 112.2 W m−2 in upwelling shortwave radiation; R2 of 0.52 in downwelling longwave radiation to 0.88 in net shortwave radiation and net radiation; and MAE of 1.73 W m−2 in downwelling longwave radiation to 90.57 W m−2 in upwelling shortwave radiation. The best nighttime TDNN models led to RMSE values that ranged from 4.4 W m−2 in downwelling longwave radiation to 9.3 W m−2 in upwelling longwave radiation; R2 values that ranged from 0.38 in net longwave radiation to 0.60 in downwelling longwave radiation; and MAE values that ranged from 4.1 W m−2 in downwelling longwave radiation to 8.1 W m−2 in upwelling longwave radiation. Temporal neural networks used in this study appear to be a promising approach to predict nonlinear behaviors of the daytime and nighttime surface radiation components.

Temporal neural networks for downscaling climate variability and extremes

This paper presents an application of temporal neural networks for downscaling global climate models (GCMs) output. Because of computational constraints, GCMs are usually run at coarse grid resolution (in the order of 100s of kilometres) and as a result they are inherently unable to present local sub-grid scale features and dynamics. Consequently, outputs from these models cannot be used directly in many climate change impact studies. This research explored the issues of ‘downscaling’ the outputs of GCMs using a temporal neural network (TNN) approach. The method is proposed for downscaling daily precipitation and temperature series for a region in northern Quebec, Canada. The downscaling models are developed and validated using large-scale predictor variables derived from the National Center for Environmental Prediction (NCEP) reanalysis data set. The performance of the temporal neural network downscaling model is also compared to a regression-based statistical downscaling model with emphasis on their ability in reproducing the observed climate variability and extremes. The downscaling results for the base period (1961–2000) suggest that the TNN is an efficient method for downscaling both daily precipitation as well as daily maximum and minimum temperature series. Furthermore, the different model test results indicate that the TNN model mostly outperforms the statistical models for the downscaling of daily precipitation extremes and variability.

IDENTIFICATION AND PREDICTION OF UNSTEADY TRANSONIC AERODYNAMIC LOADS BY MULTI-LAYER FUNCTIONALS

Nonlinear unsteady aerodynamic effects present major modelling difficulties in the analysis and control of aeroelastic response. A rigorous mathematical framework, that can account for the complex nonlinearities and time-history effects of the unsteady aerodynamic response, is provided by the use of functional representations. A recent development, based on functional approximation theory, has achieved a new functional form; namely, multi-layer functionals. The development of a multi-layer functional for discrete-time, finite memory, causal systems has been shown to be realizable via finite impulse response neural networks. Identification of an appropriate temporal neural network model of the nonlinear transonic aerodynamic response is facilitated via a supervised training process using multiple input–output sets, with data obtained by an Euler CFD code. The training process is based on a genetic algorithm to optimize the network architecture, combined with a random search algorithm to update weight and bias values. The approach is examined for two different multiple aerodynamic input–output data sets, and in both cases, the prediction properties of the network model establish the multi-layer functional as a suitable representation of unsteady aerodynamic response.