Variant Of Recurrent Neural Network Research Articles

Handrim Reaction Force and Moment Assessment Using a Minimal IMU Configuration and Non-Linear Modeling Approach during Manual Wheelchair Propulsion.

Manual wheelchair propulsion represents a repetitive and constraining task, which leads mainly to the development of joint injury in spinal cord-injured people. One of the main reasons is the load sustained by the shoulder joint during the propulsion cycle. Moreover, the load at the shoulder joint is highly correlated with the force and moment acting at the handrim level. The main objective of this study is related to the estimation of handrim reactions forces and moments during wheelchair propulsion using only a single inertial measurement unit per hand. Two approaches are proposed here: Firstly, a method of identification of a non-linear transfer function based on the Hammerstein-Wiener (HW) modeling approach was used. The latter represents a typical multi-input single output in a system engineering modeling approach. Secondly, a specific variant of recurrent neural network called BiLSTM is proposed to predict the time-series data of force and moments at the handrim level. Eleven subjects participated in this study in a linear propulsion protocol, while the forces and moments were measured by a dynamic platform. The two input signals were the linear acceleration as well the angular velocity of the wrist joint. The horizontal, vertical and sagittal moments were estimated by the two approaches. The mean average error (MAE) shows a value of 6.10 N and 4.30 N for the horizontal force for BiLSTM and HW, respectively. The results for the vertical direction show a MAE of 5.91 N and 7.59 N for BiLSTM and HW, respectively. Finally, the MAE for the sagittal moment varies from 0.96 Nm (BiLSTM) to 1.09 Nm for the HW model. The approaches seem similar with respect to the MAE and can be considered accurate knowing that the order of magnitude of the uncertainties of the dynamic platform was reported to be 2.2 N for the horizontal and vertical forces and 2.24 Nm for the sagittal moments. However, it should be noted that HW necessitates the knowledge of the average force and patterns of each subject, whereas the BiLSTM method do not involve the average patterns, which shows its superiority for time-series data prediction. The results provided in this study show the possibility of measuring dynamic forces acting at the handrim level during wheelchair manual propulsion in ecological environments.

A multi-modal Parkinson’s disease diagnosis system from EEG signals and online handwritten tasks using grey wolf optimization based deep learning model

Parkinson’s disease (PD) is a neurodegenerative disorder characterized by the gradual deterioration of motor function, affecting speech, writing, muscle control, and mobility. The existing studies have not utilized both the electroencephalography (EEG) signals and online handwritten tasks together to diagnose PD. The studies have also not explored the EEG signals collected from specific brain regions like the substansia niagra (SN) and ventral tegmental area (VTA), crucial for dopamine production linked to PD. This article proposes a multi-modal PD diagnosis system from EEG signals (collected from SN and VTA regions of the brain), collected during performing online handwritten tasks, using grey wolf optimization (GWO) algorithm. Mel-frequency cepstral coefficients (MFCC) features have been generated from the EEG signals and optimized by the GWO algorithm. The classification (diagnosis) experiments on the optimal number of feature values, obtained from GWO algorithm, have been carried out using bidirectional long short-term memory (BLSTM) variant of recurrent neural network (RNN). The classification experiments have also been conducted using support vector machine (SVM), bagged random forest (BRF), and long short-term memory (LSTM) variant of RNN classifier to have a performance comparison with the proposed method. The effectiveness of the introduced PD diagnosis system has been analyzed on a self-generated dataset named EEG signal based on online handwriting (ESOH). A maximum classification accuracy of 99.30% has been achieved from the proposed PD diagnosis system. The experimental outcomes illustrate that the introduced PD diagnosis system outperforms the state-of-the-art PD diagnosis systems relying on the EEG signals for diagnosing the PD.

Mutation-Based Multivariate Time-Series Anomaly Generation on Latent Space with an Attention-Based Variational Recurrent Neural Network for Robust Anomaly Detection in an Industrial Control System

Anomaly detection involves identifying data that deviates from normal patterns. Two primary strategies are used: one-class classification and binary classification. In Industrial Control Systems (ICS), where anomalies can cause significant damage, timely and accurate detection is essential, often requiring analysis of time-series data. One-class classification is commonly used but tends to have a high false alarm rate. To address this, binary classification is explored, which can better differentiate between normal and anomalous data, though it struggles with class imbalance in ICS datasets. This paper proposes a mutation-based technique for generating ICS time-series anomalies. The method maps ICS time-series data into a latent space using a variational recurrent autoencoder, applies mutation operations, and reconstructs the time-series, introducing plausible anomalies that reflect multivariate correlations. Evaluations of ICS datasets show that these synthetic anomalies are visually and statistically credible. Training a binary classifier on data augmented with these anomalies effectively mitigates the class imbalance problem.

Photonic deep residual time-delay reservoir computing

Time-delay reservoir computing (TDRC) represents a simplified variant of recurrent neural networks, employing a nonlinear node with a feedback mechanism to construct virtual nodes. The capabilities of TDRC can be enhanced by transitioning to a deep architecture. In this work, we propose a novel photonic deep residual TDRC (DR-TDRC) with augmented capabilities. The additional time delay added to the residual structure enables DR-TDRC superior to traditional deep structures across various benchmark tasks, especially in memory capability and almost an order of magnitude improvement in nonlinear channel equalization. Additionally, a specifically designed clipping algorithm is utilized to counteract the damage of redundant layers in deep structures, enabling the extension of the deep TDRC to dozens rather than just a few layers, with higher performance. We experimentally demonstrate the proof-of-concept with a 4-layer DR-TDRC containing 960 interrelated neurons (240 neurons per layer), based on four injection-locked distributed feedback lasers. We confirm the potential for scalable deep RC with elevated performance. Our results provide a feasible approach for expanding deep photonic computing to satisfy the boosting demand for artificial intelligence.

Cloud-based machine learning algorithms for anomalies detection

Gradient boosting machines harnesses the inherent capabilities of decision trees and meticulously corrects their errors in a sequential fashion, culminating in remarkably precise predictions. Word2Vec, a prominent word embedding technique, occupies a pivotal role in natural language processing (NLP) tasks. Its proficiency lies in capturing intricate semantic relationships among words, thereby facilitating applications such as sentiment analysis, document classification, and machine translation to discern subtle nuances present in textual data. Bayesian networks introduce probabilistic modeling capabilities, predominantly in contexts marked by uncertainty. Their versatile applications encompass risk assessment, fault diagnosis, and recommendation systems. Gated recurrent units (GRU), a variant of recurrent neural networks, emerges as a formidable asset in modeling sequential data. Both training and testing are crucial to the success of an intrusion detection system (IDS). During the training phase, several models are created, each of which can recognize typical from anomalous patterns within a given dataset. To acquire passwords and credit card details, "phishing" usually entails impersonating a trusted company. Predictions of student performance on academic tasks are improved by hyper parameter optimization of the gradient boosting regression tree using the grid search approach.

Domain knowledge-driven variational recurrent networks for drought monitoring

In the context of climate change, droughts, increasingly frequent and severe, necessitate effective monitoring. Existing methods, such as drought indices and data-driven models, face important limitations. Drought indices are built on prior expert knowledge but lack calibration based on actual drought events, while data-driven models prioritize goodness of fit over real event identification, undermining their credibility and generalization, and also struggling to generalize from regional to large-scale contexts. To address these challenges, here we introduce a hybrid machine learning framework for time series that combines domain knowledge and observational data in a variational recurrent neural network. The network models the joint distribution of total precipitation, air temperature, and real drought events, providing accurate predictions and uncertainty estimates. Extensive experiments focusing on a wide range of European drought events from 2011 to 2018 consistently show that our hybrid model surpasses both drought indices and data-driven models in terms of accuracy in drought detection, underlining its effectiveness, robustness, and stability. Our model achieves the best ROC-AUC (%) results in Afghanistan (79.7 ± 0.5), Italy (84.3 ± 0.6), Russia (89.4 ± 0.2), Europe-0 (84.3 ± 0.1), and Europe-1 (82.8 ± 0.4), effectively capturing the starting and ending times of drought events with lower uncertainty, and also generalizing better for unseen locations.

SERT-StructNet: Protein secondary structure prediction method based on multi-factor hybrid deep model

Protein secondary structure prediction (PSSP) is a pivotal research endeavour that plays a crucial role in the comprehensive elucidation of protein functions and properties. Current prediction methodologies are focused on deep-learning techniques, particularly focusing on multi-factor features. Diverging from existing approaches, in this study, we placed special emphasis on the effects of amino acid properties and protein secondary structure propensity scores (SSPs) on secondary structure during the meticulous selection of multi-factor features. This differential feature-selection strategy results in a distinctive and effective amalgamation of the sequence and property features. To harness these multi-factor features optimally, we introduced a hybrid deep feature extraction model. The model initially employs mechanisms such as dilated convolution (D-Conv) and a channel attention network (SENet) for local feature extraction and targeted channel enhancement. Subsequently, a combination of recurrent neural network variants (BiGRU and BiLSTM), along with a transformer module, was employed to achieve global bidirectional information consideration and feature enhancement. This approach to multi-factor feature input and multi-level feature processing enabled a comprehensive exploration of intricate associations among amino acid residues in protein sequences, yielding a Q3 accuracy of 84.9% and an Sov score of 85.1%. The overall performance surpasses that of the comparable methods. This study introduces a novel and efficient method for determining the PSSP domain, which is poised to deepen our understanding of the practical applications of protein molecular structures.

Prediction of ground water quality in western regions of Tamilnadu using LSTM network

Assessing and safeguarding groundwater quality is critical for sustaining life in water-scarce regions like western Tamil Nadu. The motivation behind this study stems from the pressing need to address water quality challenges in a region grappling with scarcity. Despite existing efforts, a notable research gap exists in predictive tools that comprehensively capture the nuanced temporal variations and trends in groundwater quality. This is where the LSTM network steps in, showcasing exceptional accuracy in short-term predictions and discerning long-term trends. This research uses Long Short-Term Memory (LSTM) networks, a variant of recurrent neural networks, to predict groundwater quality in South Indian Regions, especially in Tamil Nadu. Extensive data, encompassing parameters such as pH, dissolved oxygen, turbidity, and various chemical constituents, were gathered over an extended timeframe. The LSTM model was then trained on this historical dataset, factoring in temporal dependencies and seasonality inherent in groundwater quality data. The validation process rigorously tests the LSTM model against actual groundwater quality measurements. The results were impressive, as the model demonstrated a remarkable ability to unravel the complex variations in groundwater quality.

Speaker identification under noisy conditions using hybrid convolutional neural network and gated recurrent unit

<p><span>Speaker identification is biometrics that classifies or identifies a person from other speakers based on speech characteristics. Recently, deep learning models outperformed conventional machine learning models in speaker identification. Spectrograms of the speech have been used as input in deep learning-based speaker identification using clean speech. However, the performance of speaker identification systems gets degraded under noisy conditions. Cochleograms have shown better results than spectrograms in deep learning-based speaker recognition under noisy and mismatched conditions. Moreover, hybrid convolutional neural network (CNN) and recurrent neural network (RNN) variants have shown better performance than CNN or RNN variants in recent studies. However, there is no attempt conducted to use a hybrid CNN and enhanced RNN variants in speaker identification using cochleogram input to enhance the performance under noisy and mismatched conditions. In this study, a speaker identification using hybrid CNN and the gated recurrent unit (GRU) is proposed for noisy conditions using cochleogram input. VoxCeleb1 audio dataset with real-world noises, white Gaussian noises (WGN) and without additive noises were employed for experiments. The experiment results and the comparison with existing works show that the proposed model performs better than other models in this study and existing works</span><span>.</span></p>

Nonlinear Spiking Neural Systems With Autapses for Predicting Chaotic Time Series.

Spiking neural P (SNP) systems are a class of distributed and parallel neural-like computing models that are inspired by the mechanism of spiking neurons and are 3rd-generation neural networks. Chaotic time series forecasting is one of the most challenging problems for machine learning models. To address this challenge, we first propose a nonlinear version of SNP systems, called nonlinear SNP systems with autapses (NSNP-AU systems). In addition to the nonlinear consumption and generation of spikes, the NSNP-AU systems have three nonlinear gate functions, which are related to the states and outputs of the neurons. Inspired by the spiking mechanisms of NSNP-AU systems, we develop a recurrent-type prediction model for chaotic time series, called the NSNP-AU model. As a new variant of recurrent neural networks (RNNs), the NSNP-AU model is implemented in a popular deep learning framework. Four datasets of chaotic time series are investigated using the proposed NSNP-AU model, five state-of-the-art models, and 28 baseline prediction models. The experimental results demonstrate the advantage of the proposed NSNP-AU model for chaotic time series forecasting.

Enhancing Electrical Power Demand Prediction Using LSTM-Based Deep Learning Models for Local Energy Communities

The pursuit of accurate electrical power demand forecasting has led to the application of deep learning algorithms, notably demonstrating promising outcomes despite the prerequisite of substantial training data. This study pioneers a new learning paradigm, employing sophisticated deep learning models specifically, Long Short-Term Memory networks and recurrent neural networks (RNNs). Leveraging historical load data, temperature, wind speed, and day-ahead predicted spot prices, this approach follows a structured flow involving data preprocessing, sequence generation, model training, and subsequent prediction of future load demand using LSTM-based RNN variants. The study’s paramount findings underscore the substantial advancement achieved by this proposed methodology over prevailing techniques. The method significantly improves prediction accuracy by over 11%, demonstrating the efficacy of deep learning models and a significant leap forward in forecasting precision. Beyond its superior predictive capabilities, this novel strategy serves as a catalyst for enhancing energy distribution management in local energy communities. Its effectiveness lies not only in its precision but also in enabling the optimization and cost-effective control of energy distribution, vital for sustainable energy management in these communities. Ultimately, this pioneering approach presents a robust solution poised to revolutionize the landscape of electrical power demand forecasting and its practical application in local energy systems.

Movie sentiment analysis based on Long Short-Term Memory Network

An important task in the study of Natural Language Processing (NLP) is the analysis of movie reviews. It finishes the task of classifying movie review texts into sentiment, such as positive, negative or neutral sentiment. Previous works mainly follow the pipeline of LSTM (Long Short-Term Memory Network). The network model is a variant of Recurrent Neural Network (RNN) and particularly suitable for processing natural language texts. Though existing LSTM-based works have improved the performance significantly, we argue that most of them deal with the problem of analyzing the sentiment of movie reviews while ignore analyze the model performance in different application scenarios, such as different lengths of the reviews and the frequency of sentiment adverbs in the reviews. To alleviate the above issue, in this paper, we constructed a simple LSTM model containing an embedding layer, a batch normalization layer, a dropout layer, a one-dimensional convolutional layer, a maximal pooling layer, a bi-directional LSTM layer and a fully connected layer. We used the existing IMDB movie review dataset to train the model, and selected two research scenarios of movie review length and frequency of occurrence of sentiment adverbs to test the model, respectively. From the experimental results, we proposed a model for the scenarios in which the LSTM model handles the problem of sentiment analysis with respect to the dataset construction, model stability and generalization ability, text fragment processing, data preprocessing and feature extraction, model optimization and improvement.

Multi-task learning using non-linear autoregressive models and recurrent neural networks for tide level forecasting

Tide level forecasting plays an important role in environmental management and development. Current tide level forecasting methods are usually implemented for solving single task problems, that is, a model built based on the tide level data at an individual location is only used to forecast tide level of the same location but is not used for tide forecasting at another location. This study proposes a new method for tide level prediction at multiple locations simultaneously. The method combines nonlinear autoregressive moving average with exogenous inputs (NARMAX) model and recurrent neural networks (RNNs), and incorporates them into a multi-task learning (MTL) framework. Experiments are designed and performed to compare single task learning (STL) and MTL with and without using non-linear autoregressive models. Three different RNN variants, namely, long short-term memory (LSTM), gated recurrent unit (GRU) and bidirectional LSTM (BiLSTM) are employed together with non-linear autoregressive models. A case study on tide level forecasting at many different geographical locations (5 to 11 locations) is conducted. Experimental results demonstrate that the proposed architectures outperform the classical single-task prediction methods.

A model-free method based on RDPG for fiber diameter steady control

Optical fibers have made outstanding contributions to optical instruments such as micro-channel plates (MCPs) and micro-pore optics (MPOs). As the stricter performance requirements for the MCP and MPO, traditional diameter control methods are incapable. Additionally, the partial observation issue of fiber drawing process further limits the outcome of the existing control approaches. Innovative control algorithms are vital for the diameter precision enhancement. We investigate the significance of recurrent deterministic policy gradient (RDPG) on diameter control. RDPG consists of deep deterministic policy gradient (DDPG) and recurrent neural network (RNN) where DDPG is a model-free reinforcement learning algorithm for continuous control and RNN for addressing partial observation. The impact of RNN variants selection on control performance is further discussed. Results illustrate that the application of RNN substantially solves the partial observation and varying RNN variants play a positive role in diameter control, but bidirectional gated recurrent unit (BiGRU) reaches the best accuracy. Comparison experiment results indicate that RDPG is able to learn reasonable control strategies and achieves significantly more accurate and robust diameter control in contrast with the existing methods.

Echo state network structure optimization algorithm based on correlation analysis

Echo State Network (ESN) is an effective variant of Recurrent Neural Network (RNN). However, it is difficult for traditional ESN to determine the reservoir size that matches a given task. In this paper, an ESN structure optimization pruning algorithm based on correlation analysis, called PCESN, is proposed to design the size of the reservoir automatically. First, a characteristic matrix is constructed utilizing probability theory and information theory to measure the correlation between each neuron in the reservoir and the output neuron. On this basis, a pruning criterion is proposed to achieve a sparse reservoir structure by dynamically removing neurons with low correlation. Second, in order to retain the sample information of the neurons in the removed reservoir during the network reduction, the input weights of the retained reservoir are updated by means of the average transverse propagation of the weights. Finally, the performance of PCESN is tested on multiple time series. Simulation results show that the proposed PCESN outperforms some fixed size ESN and other dynamic ESN in terms of prediction accuracy, generalization performance and model complexity.

Precipitation forecast using RNN variants by analyzing Optimizers and Hyperparameters for Time-series based Climatological Data

Flood is a significant problem in many regions of the world for the catastrophic damage it causes to both property and human lives; excessive precipitation being the major cause. The AI technologies, Deep Learning Neural Networks and Machine Learning algorithms attempt realistic solutions to numerous disaster management challenges. This paper works on RNN- based rainfall/ precipitation forecasting models by investigating the performances of various Recurrent Neural Network (RNN) architectures, Bidirectional RNN (BRNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU) and ensemble models such as BRNN-GRU, BRNN-LSTM, LSTM-GRU, BRNN-LSTM-GRU using NASAPOWER datasets of Andhra Pradesh (AP) and Tamil Nadu (TN) in India. The different stages in the workflow of the methodology are Data collection, Data pre-processing, Data splitting, Defining hyperparameters, Model building and Performance evaluation. Experiments for identifying improved optimizers and hyperparameters for the time-series climatological data are investigated for accurate precipitation forecast. The metrics: Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Square Error (RMSE) and Root Mean Squared Logarithmic Error (RMSLE) values are used to compare the precipitation predictions of different models. The RNN variants and ensemble models, BRNN, LSTM, GRU, BRNN-GRU, BRNN-LSTM, LSTM-GRU, BRNN-LSTM-GRU produce predictions with RMSLE values of 2.448, 0.555, 0.255, 1.305, 1.383, 0.364, 1.740 for AP and 1.735, 0.663, 0.152, 0.889, 1.118, 0.379, 1.328 for TN respectively. The best performing RNN model, GRU when ensembled with the existing statistical model SARIMA produces an RMSLE value of 0.754 and 1.677 respectively for AP and TN.

Physical reservoirs based on MoS2-HZO integrated ferroelectric field-effect transistors for reservoir computing systems.

Reservoir computing (RC), a variant of recurrent neural networks (RNNs), is well-known for its reduced energy consumption through exclusive focus on training the output weight and its superior performance in handling spatiotemporal information. Implementing these networks in hardware requires devices with superior fading memory behavior. Unlike filament-based two-terminal devices, those relying on ferroelectric switching demonstrate improved voltage reliability, while three-terminal transistors provide additional active control. HfO2-based ferroelectric materials such as Hf0.5Zr0.5O2 (HZO), have garnered attention for their scalability and seamless integration with CMOS technology. This study implements a RC hardware based on MoS2-HZO integrated device structure with enhanced spontaneous polarization field. By adjusting the oxygen vacancy concentration, the devices exhibit consistent responses to both identical and nonidentical voltages, making them suitable for diverse RC applications. The high accuracy of MNIST handwritten digits recognition highlights the rich reservoir states of the traditional RC architecture. Additionally, the impact of masks on RC implementation is assessed, showcasing the device's capability for spatiotemporal signal analysis. This development paves the way for implementing energy-efficient and high-performance computing solutions.

Deep Learning-Based Bitcoin Price Prediction Using Long Short-Term Memory Networks

Cryptocurrencies, notably Bitcoin, have surged in popularity and importance, attracting widespread attention due to their volatile nature and potential for significant financial gains. Predicting Bitcoin prices accurately has emerged as a challenging yet indispensable task for investors, traders, and policymakers seeking to navigate the complexities of the cryptocurrency market. Traditional forecasting methods often struggle to capture the intricate temporal patterns inherent in this domain, prompting the exploration of advanced techniques such as deep learning. This paper presents a comprehensive study on employing Long Short-Term Memory (LSTM) networks for Bitcoin price prediction, a variant of Recurrent Neural Networks (RNNs). Our research encompasses the entire process, from data collection to model evaluation, focusing on addressing the unique challenges posed by cryptocurrency market data. We begin by discussing the methodology involved in data acquisition, preprocessing, and feature engineering, which are crucial steps for ensuring the quality and relevance of input data. Next, we delve into the intricacies of LSTM model selection, hyperparameter tuning, and training strategies, aiming to optimize predictive performance while mitigating overfitting. Through extensive experimentation on real-world Bitcoin price datasets spanning multiple periods, we rigorously evaluate the effectiveness of LSTM networks in capturing long-term dependencies and forecasting future price movements. Comparative analyses against traditional time-series forecasting methods underscore the superior predictive capabilities of deep learning approaches in this context. Furthermore, we investigate the interpretability of LSTM-based models, examining the insights gained from analyzing learned representations and hidden states. These analyses provide valuable perspectives for market participants seeking to understand the driving factors behind Bitcoin price dynamics. Our findings demonstrate the practical utility of LSTM networks in Bitcoin price prediction and contribute to advancing the broader discourse on deep learning applications in financial forecasting. By shedding light on the potential of deep learning techniques to enhance predictive accuracy and robustness in cryptocurrency markets, our research offers valuable insights for stakeholders navigating this rapidly evolving landscape.

Deep Learning Techniques for Identifying Poets in Arabic Poetry: A Focus on LSTM and Bi-LSTM

In this paper, we introduce a comprehensive approach to the classification of Arabic poetry using deep learning techniques. We utilized a dataset comprising nearly one million records of Arabic poetry verses, each labeled with nine different poets encompassing both classical and modern poetic styles. We explored various algorithms to identify the most effective models for accurately determining the poet's identity from the verses. Based on our analysis, we selected LSTM (Long Short-Term Memory) and Bi-LSTM (Bidirectional LSTM) as our primary baseline models, given the demonstrated efficacy of RNN (Recurrent Neural Network) variants in text classification. LSTM, in particular, has shown significant proficiency in analyzing sequential data across multiple languages. Our results indicate a promising average classification accuracy of 92.35%, highlighting the potential for automating the classification of texts in morphologically complex languages such as Arabic. Notably, Bi-LSTM marginally outperformed the standard LSTM, achieving average accuracy of 92.56%. We further discuss the implications of our findings for Arabic literature, particularly in the realm of poetry, and address the challenges encountered during the study. These findings represent a significant advancement in Arabic NLP, offering a powerful framework for poet identification and paving the way for future applications of deep learning in processing Arabic literary texts.

A Multitask Integrated Deep-Learning Probabilistic Prediction for Load Forecasting

Spinning reserve without accurate load forecasting can lead to automatic disconnection of critical loads by under-frequency load shedding devices. Such a predicament poses a grave threat to the economic and social welfare of the affected region, and in extreme scenarios, can result in a debilitating grid collapse. However, existing models lack the deep feature extraction capability and cannot accurately predict the uncertainty of electricity demand. Thus, a multitask integrated deep-learning probabilistic prediction with multidimensional feature extraction based on granularity information and quantile regression is explored in this study to solve the insufficient feature extraction problem in probabilistic prediction. In the proposed scheme, we designed a multi-layer feature extraction framework, which information granularity based on fuzzy rough sets to extract time-domain features, multilayer Laplace operators to extract features globally, and quantile regression recurrent neural network variants to extract time context information. In addition, the Pareto integration method is extended to capture better individuals. The experimental results demonstrate that the proposed system can effectively improve deterministic forecasting and uncertainty analysis of electricity demand and manage daily fluctuations.