Recurrent Learning Research Articles

Comparative analysis of RNN versus IIR digital filtering to optimize resilience to dynamic perturbations in pH sensing for vertical farming

AbstractVertical farming (VF) refers to systems of agriculture where crops are grown in trays stacked vertically by exposing them to artificial light and using sensing technology to improve product quality and yield. In this work, we propose an advanced filtering scheme based on recurrent neural networks (RNNs) and deep learning to enable efficient control strategies for VF applications. We demonstrate that the best RNN model incorporates five neuron layers, with the first and second containing 90 long short‐term memory neurons. The third layer implements one gated recurrent units neuron. The fourth segment incorporates one RNN network, while the output layer is designed by using a single neuron exhibiting a rectified linear activation function. By utilizing this RNN digital filter, we introduce two variations: (1) a scaled RNN model to tune the filter to the signal of interest, and (2) a moving average filter to eliminate harmonic oscillations of the output waveforms. The RNN models are contrasted with conventional digital Butterworth, Chebyshev I, Chebyshev II, and elliptic infinite impulse response (IIR) configurations. The RNN digital filtering schemes avoid introducing unwanted oscillations, which makes them more suitable for VF than their IIR counterparts. Finally, by utilizing the advanced features of scaling of the RNN model, we demonstrate that the RNN digital filter can be pH selective, as opposed to conventional IIR filters.

Research on Fault Prediction of Power Devices in Rod Control Power Cabinets Based on BiTCN-Attention Transfer Learning Model

The Insulated Gate Bipolar Transistor (IGBT) is the key power device in the rod control power cabinet of nuclear power plants; its reliable operation is of great significance for ensuring the safe and economical operation of the nuclear power plants. Therefore, it is necessary to conduct fault prediction research on IGBT to achieve better condition-based maintenance and improve its operational reliability. However, power cabinets often operate under multiple, complex working conditions, so predicting IGBT faults from single working condition data usually has limitations and low accuracy. Its failure probability has an important relationship with the actual operating conditions of the cabinet. In order to improve the reliability and maintainability of the control power cabinet in nuclear power plants, this paper takes IGBTs in the rod control power cabinet as the object and makes full use of the data of IGBTs under multiple working conditions to carry out research on the cross-condition fault prediction of IGBTs under multiple-source working conditions. A transfer learning (TL) model based on a bidirectional time convolutional network (BiTCN) combined with attention was proposed to solve the problem of low accuracy of cross-operating fault prediction in a multi-source domain. Firstly, an IGBT fault simulation model was built to collect the life cycle state data of the module under different working conditions. Then, after pre-processing such as removing outliers, kernel principal component analysis (KPCA) was used to integrate all source domain data, obtain source domain characterization data, and train the BiTCN-attention model. Finally, the BiTCN-attention model trained in the source domain was transferred, and the model was fine-tuned according to the target domain data. Simulation results show that the accuracy of the proposed BiTCN-attention transfer learning prediction method can reach more than 99%, which is significantly better than that of the recurrent neural network transfer learning (RNN-TL) model, long short-term memory network transfer learning (LSTM-TL) model, gated cyclic unit transfer learning (GRU-TL) model, and time convolutional network transfer learning (TCN-TL) model. This method can not only reduce the inconsistency of fault characteristic values caused by changes in working conditions but also accurately predict the degradation trend when only early fault data are available, providing an effective solution for IGBT fault prediction across working conditions in multi-source domains.

A multi-objective indirect neural adaptive processes control design for minimization of energy consumption: An experimental validation on a transesterification reactor

Nowadays, energy management environment is a very important issue that technologies have focused on in order to save costs and minimize energy waste. This objective can be achieved by means of an energy resource management approach through an appropriate optimization technique. However, energy savings can conflict with other objective functions, to solve the problem a multi-objective optimization that considers the minimization of the control energy can be adopted. In this paper, we propose to use a multi-objective indirect neural adaptive control concept for nonlinear systems with unknown dynamics. The control scheme consists of an adaptive instantaneous neural emulator and neural controller built on fully connected real-time recurrent learning networks (RTRL). The multi-objective particle swarm optimization (MOPSO) algorithm is used as a mechanism to find NE and NC adaptive learning rates that optimize the proposed objective functions: control energy minimization and closed-loop preservation. Comparative studies and experimental validation on a semi-batch reactor, used to generate cleaner biofuels, are performed to confirm the effectiveness of the proposed strategy.

Deep anomaly detection: A linear one-class SVM approach for high-dimensional and large-scale data

Anomaly detection plays a crucial role in various fields including cyber security, finance, healthcare, and industrial monitoring. Traditional methods in anomaly detection often face several challenges such as scalability, adaptability, and difficulty in handling high-dimensional data. So a novel Recurrent Extreme Learning based –Boosted Chimp (REL-BC) algorithm is proposed for anomaly detection. The REL-BC model involves a data pre-processing phase and an anomaly detection phase. The data pre-processing phase involves three stages namely one-hot-encoding, outlier disposal, and min-max normalization. In this study, a Recurrent Neural Network is utilized to seize the temporal dependencies and traffic data in the network. Also, the Extreme Learning Machine (ELM) is employed in distinguishing normal as well as anomalous patterns. Further Chimp Optimization is employed for optimizing hyperparametersto improve the efficiency of the REL-BC approach. The outcome of the experimentation revealed that it demonstrated the improvement of performance for the REL-BC method in detecting anomalies based on various measures.

State estimation for nonlinear systems using a recurrent neural network learning algorithm and an event-triggered state observer

State estimation for nonlinear systems using a recurrent neural network learning algorithm and an event-triggered state observer

HCO-RLF: Hybrid classification optimization using recurrent learning and fuzzy for COVID-19 detection on CT images

COVID-19 infection detection through initial lesion classification provides early diagnosis and prevents breathing difficulties. Detecting the infectious part of the lungs using computerized tomography (CT) images has become an instantaneous detection method. In this article, a Hybrid Classification Optimization (HCO) using Recurrent Leaning and Fuzzy (RLF) is proposed. The neural network classifies infected and non-infected regions using pixel distributions and their variations. This is performed by identifying missing features and training the recurrent network using regional differences. Based on the feature availability, recurrent learning classifies the region through the input dataset and recurrent training correlation. The fuzzy predicts missing features through substituted derivatives obtained between wide ranges of variations. In this process, the maximum fuzzy derivatives for feature substitution are used for infected region prediction. The least fuzzy derivatives are prevented from the training layer to reduce false rates in region classification. This joint process improves the training consistency to leverage the detection and region classification accuracies. The proposed HCO-RLF improves detection and classification accuracy, and precision by 11.96 %, 9.98 %, and 13.42 % for the varying classification rates. Besides, the results are obtained in comparison with the existing DR-MIL, DSAE, and BS-FSA methods discussed later in the article.

A Lightweight Recurrent Learning Network for Sustainable Compressed Sensing

Recently, deep learning-based compressed sensing (CS) has achieved great success in reducing the sampling and computational cost of sensing systems and improving the reconstruction quality. These approaches, however, largely overlook the issue of the computational cost; they rely on complex structures and task-specific operator designs, resulting in extensive storage and high energy consumption in CS imaging systems. In this article, we propose a lightweight but effective deep neural network based on recurrent learning to achieve a sustainable CS system; it requires a smaller number of parameters but obtains high-quality reconstructions. Specifically, our proposed network consists of an initial reconstruction sub-network and a residual reconstruction sub-network. While the initial reconstruction sub-network has a hierarchical structure to progressively recover the image, reducing the number of parameters, the residual reconstruction sub-network facilitates recurrent residual feature extraction via recurrent learning to perform both feature fusion and deep reconstructions across different scales. In addition, we also demonstrate that, after the initial reconstruction, feature maps with reduced sizes are sufficient to recover the residual information, and thus we achieved a significant reduction in the amount of memory required. Extensive experiments illustrate that our proposed model can achieve a better reconstruction quality than existing state-of-the-art CS algorithms, and it also has a smaller number of network parameters than these algorithms. Our source codes are available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/C66YU/CSRN</uri> .

Reflection Removal via Recurrent Learning Guided by Physics Prior and Focal Perceptual Loss

Reflection Removal via Recurrent Learning Guided by Physics Prior and Focal Perceptual Loss

Multimodal biometric integration: Trends and insights from the past quinquennial

Recent years have seen a notable breakthrough in multimodal biometric systems, which combine numerous biometric modalities for increased security and accuracy. This paper offers a thorough analysis of recent advances in the field, addressing the many multimodal biometric strategies that have been put out by different researchers. Numerous modalities are covered by the evaluation, such as palm print, speech, iris, facial features, palm print, fingerprint, body form, gait, and more. To obtain amazing results in terms of accuracy and security, researchers have used cutting-edge methods like convolutional neural networks (CNNs), support vector machines (SVM), recurrent neural networks (RNNs), deep learning, and optimization algorithms. Many fusion strategies have been investigated to efficiently merge data from many modalities, such as feature-level fusion, decision-level fusion, and score-level fusion. Developments in template protection systems have also addressed security issues related to transmission and storage of biometric data. Even though a lot of the suggested systems have shown great accuracy rates, there are still issues with hardware restrictions, dataset biases, privacy problems, and computational costs. Prospective study avenues encompass investigating more extensive and heterogeneous datasets, crafting resilient fusion algorithms, and amalgamating cutting-edge technologies like deep learning-based biometric cryptosystems and federated learning. All things considered, the literature study demonstrates the enormous potential of multimodal biometric systems in offering safe and dependable authentication solutions for a wide range of applications, from on-line student authentication and proctoring systems to customized healthcare networks.

Enhancing streamflow simulations with gated recurrent units deep learning models in the flood prone region with low-convergence streamflow data

Highly accurate streamflow simulations are essential for water resources and river management. However, there are longstanding challenges for streamflow modeling in the hydrology field. As a result, deep learning methods have been introduced as novel tools and are being used in simulation. This paper proposes a novel approach that utilizes unconventional data preprocessing techniques to improve the accuracy of daily streamflow predictions in flood-prone regions. We applied this approach to a flood-prone area in western Iran and simulated daily streamflow using a deep learning Gated Recurrent Unit (GRU) method under various data selection scenarios. These scenarios focused on identifying the optimal combination of input variables, time steps, and outlier removal techniques. The outlier removal methods investigated in this study include Mahalanobis distance, critical section removal, Z-score, and no removal. The average rainfall of the area, data driven precipitation, Normalized Difference Vegetation Index (NDVI), surface soil moisture, groundwater baseflows, and streamflow in the hydrometric station were evaluated using correlation control method, and inputs with the lowest correlation were removed. Based on the results obtained from the deep learning models produced in the research, it was found that the GRU model, with various modified inputs using Z-score removal, had the best performance. The model had an average of Root Mean Squared Error (RMSE) at 5.24 mm and R2 (Coefficient of Determination) at 0.91 during training, while during validation, it had an RMSE of 7.74 mm and R2 of 0.83. Considering using relatively low convergence data in streamflow simulation in this study, it can be said that the listed scenarios showed an appropriate result in dealing with the data and recognizing the complex pattern of the daily streamflow, and the future studies will show the improvement of the GRU models if higher convergence data will be used.

Solar energy prediction in IoT system based optimized complex-valued spatio-temporal graph convolutional neural network

The accurate prediction of solar energy generation is significant for efficient energy management in Internet of Things (IoT) devices. However, current forecasting models frequently fail to account for the dynamic nature of weather conditions. Also, traditional methods often have limited accuracy and scalability. This paper proposes Solar Energy Prediction in IoT system based optimized Complex-Valued Spatio-Temporal Graph Convolutional Neural Network (SEP-CVSGCNN-IoT) to overcome the limitations of existing models. Initially, the data are collected from solar panel and weather forecast. The collected data is given to the pre-processing using Data-Adaptive Gaussian Average Filtering (DAGAF) to remove the unwanted data and replace missing data. The pre-processed data is given into Nutcracker Optimization (NCO) algorithm for selecting optimal features. Then, the selected features are given to the Complex-Valued Spatio-Temporal Graph Convolutional Neural Network (CVSGCNN) for solar energy prediction. Finally, Dipper Throated Optimization Algorithm (DTOA) is proposed to enhance the weight parameter of CVSGCNN classifier, which precisely predicts solar energy in IoT. The proposed SEP-CVSGCNN-IoT method provides 18.46%, 26.34, 15.69 and 20.84% higher accuracy and 18.24%, 23.77, 24.34 and 16.29% lower mean absolute error when analyzed with existing techniques, such as deep learning enhanced solar energy prediction and AI-driven IoT (DL-ESEF-AI), towards efficient renewable energy prediction using deep learning (TEE-REP-DL), a new deep learning method for effectual forecasting of short-term PV energy production (DL-EF-SPEP) and metaheuristic-dependent hyper parameter tuning for recurrent deep learning: application to the solar energy generation prediction (HT-RDL-PSEG) respectively.

Recurrent and Concurrent Prediction of Longitudinal Progression of Stargardt Atrophy and Geographic Atrophy towards Comparative Performance on Optical Coherence Tomography as on Fundus Autofluorescence

Stargardt atrophy and geographic atrophy (GA) represent pivotal endpoints in FDA-approved clinical trials. Predicting atrophy progression is crucial for evaluating drug efficacy. Fundus autofluorescence (FAF), the standard 2D imaging modality in these trials, has limitations in patient comfort. In contrast, spectral-domain optical coherence tomography (SD-OCT), a 3D imaging modality, is more patient friendly but suffers from lower image quality. This study has two primary objectives: (1) develop an efficient predictive modeling for the generation of future FAF images and prediction of future Stargardt atrophic (as well as GA) regions and (2) develop an efficient predictive modeling with advanced 3D OCT features at ellipsoid zone (EZ) for the comparative performance in the generation of future enface EZ maps and prediction of future Stargardt atrophic regions on OCT as on FAF. To achieve these goals, we propose two deep neural networks (termed ReConNet and ReConNet-Ensemble) with recurrent learning units (long short-term memory, LSTM) integrating with a convolutional neural network (CNN) encoder–decoder architecture and concurrent learning units integrated by ensemble/multiple recurrent learning channels. The ReConNet, which incorporates LSTM connections with CNN, is developed for the first goal on longitudinal FAF. The ReConNet-Ensemble, which incorporates multiple recurrent learning channels based on enhanced EZ enface maps to capture higher-order inherent OCT EZ features, is developed for the second goal on longitudinal OCT. Using FAF images at months 0, 6, and 12 to predict atrophy at month 18, the ReConNet achieved mean (±standard deviation, SD) and median Dice coefficients of 0.895 (±0.086) and 0.922 for Stargardt atrophy and 0.864 (±0.113) and 0.893 for GA. Using SD-OCT images at months 0 and 6 to predict atrophy at month 12, the ReConNet-Ensemble achieved mean and median Dice coefficients of 0.882 (±0.101) and 0.906 for Stargardt atrophy. The prediction performance on OCT images is comparably good to that on FAF. These results underscore the potential of SD-OCT for efficient and practical assessment of atrophy progression in clinical trials and retina clinics, complementing or surpassing the widely used FAF imaging technique.

Transforming Agriculture with Advanced Robotic Decision Systems via Deep Recurrent Learning

Global farming methods have improved owing to supply demand, human resource shortages, and seasonal fluctuations. Integrating automated mechanics, smart controllers, and artificial intelligence has become imperative to tackle these problems. Through self-learning and wise decision-making, this integration speeds up processing and encourages sustainability. To help modern farming techniques become more efficient and environmentally friendly, this paper presents a Pliant Decision System (PDS) specifically made for Farming Robots (FRs). The FRs are designed to be time-synchronized and multi-operational, which allows for guided agricultural practices that provide sustainable results. The robotic operations are planned to finish the agricultural processes with individual sustainability verification once they are first classified according to traditional techniques. Two layers of deep recurrent learning are used to evaluate sustainability. Verifying individual completion and sustainability via timely execution and cumulative effect is the primary emphasis of the first layer. In the second layer, the seasonal consistency of production improvements is compared using dual analyses. By training the learning model to synchronize and complete robotic activities promptly, people can execute judgments and synchronize successive agricultural operations throughout the year. The proposed method outwits the existing methods by leveraging sustainable factors by 8.24%, completion by 11.1%, promptness by 7.79%, synchronization by 6.75%, and reducing decision time by 12.07% for the varying process times.

Analyzing anonymous activities using Interrupt-aware Anonymous User-System Detection Method (IAU-S-DM) in IoT

The intrusion detection process is important in various applications to identify unauthorized Internet of Things (IoT) network access. IoT devices are accessed by intermediators while transmitting the information, which causes security issues. Several intrusion detection systems are developed to identify intruders and unauthorized access in different software applications. Existing systems consume high computation time, making it difficult to identify intruders accurately. This research issue is mitigated by applying the Interrupt-aware Anonymous User-System Detection Method (IAU-S-DM). The method uses concealed service sessions to identify the anonymous interrupts. During this process, the system is trained with the help of different parameters such as origin, session access demands, and legitimate and illegitimate users of various sessions. These parameters help to recognize the intruder’s activities with minimum computation time. In addition, the collected data is processed using the deep recurrent learning approach that identifies service failures and breaches, improving the overall intruder detection rate. The created system uses the TON-IoT dataset information that helps to identify the intruder activities while accessing the different data resources. This method’s consistency is verified using the metrics of service failures of 10.65%, detection precision of 14.63%, detection time of 15.54%, and classification ratio of 20.51%.

Recurrent DQN for radio fingerprinting with constrained measurements collection

In this paper, we address the problem of fingerprinting-based radio localization with a particular focus on the measurements collection part. We consider the crucial circumstance where the operator that builds the fingerprinting map by collecting measurements can only travel a limited distance. We propose an iterative formulation that increases the accuracy of the position prediction task by using a recurrent deep reinforcement learning algorithm. Numerical results on a real dataset show the effectiveness of the proposed method, and the comparison with other measurement collection strategies corroborates its value.

SeisRAFT: A recurrent deep learning network for 4D seismic registration and CO2 storage monitoring

The alignment or registration of seismic images is an important and extensively researched aspect in seismic data processing. This process plays a pivotal role in evaluating the evolution of CO2 plumes and assessing the potential for CO2 leakage from its reservoir storage when using 4D or time-lapse seismic methodology. The computer vision community has demonstrated considerable advantages in employing deep learning for such tasks, particularly when confronted with substantial shifts, mismatches, and noises in complex images. In this study, we leverage a cutting-edge recurrent deep learning network known as recurrent all-pairs field transforms (RAFT), incorporating tailored modifications to address seismic matching challenges. We evaluate the network's performance on both synthetic and field 4D CO2 seismic data sets, showcasing its superior accuracy and effectiveness compared to a widely used traditional method. Particularly, in some intricate and ambiguous instances, where even human vision may struggle to identify corresponding events between images, our deep learning method exhibits strong and reliable performance. Validated results confirm that this lightweight multiscale network, trained through self-supervised learning, offers a rapid, parameter-free, and end-to-end solution for 4D image matching in subsurface CO2 storage monitoring with potential applicability to many other seismic registration tasks.

Enhancing stock market prediction accuracy with recurrent deep learning models: A case study on the CAC40 index

This paper explores the application of Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks for stock price prediction over a 10-day horizon. The study aims to compare the predictive performance of these two deep learning architectures within the context of financial forecasting. Utilizing historical stock data from the CAC40 dataset, which represents a capitalization-weighted measure of the 40 most significant stocks on the Euronext Paris, we train and evaluate RNN and LSTM models to forecast future stock prices. Our results demonstrate the superior performance of LSTM networks in capturing the intricate temporal dependencies inherent in stock price data. Compared to standard RNNs, LSTM models exhibit higher accuracy and provide more reliable forecasts over the 10-day prediction period. The specialized memory cells and gating mechanisms in LSTM networks enable them to effectively identify both short-term changes and long-term patterns in stock prices, thus outperforming traditional RNN architectures. This enhanced ability to model the complex dynamics of stock market data underscores the potential of LSTM networks to improve investment decision-making, risk management, and the overall efficiency of financial markets. The insights gained from this study contribute to the growing body of knowledge on the application of deep learning in finance and investment, offering valuable guidance for practitioners and researchers seeking to harness the power of advanced algorithms for stock market prediction and optimization.

Federated semi-supervised representation augmentation with cross-institutional knowledge transfer for healthcare collaboration

In the healthcare field, cross-institutional collaboration can fasten medical research progress. Vertical federated learning (VFL) addresses data heterogeneity across multiple medical institutions while ensuring medical data privacy, thereby enhancing the accuracy of disease diagnoses and treatments. However, traditional VFL only benefits from aligned samples, thereby limiting its applicability due to constrained sample sizes, and a large amount of non-aligned data remains untapped, resulting in wasted data. To exert full leverage on the value of all data obtained from medical institutions, this paper proposes a federated healthcare collaborative framework based on semi-supervised representation augmentation mechanism with cross-institutional knowledge transfer (CrossKT-FRA). Specifically, the developed method comprises three steps. First, the federated representations of shared data (aligned data) among medical institutions are extracted through efficient vertical federated representation learning (FRL) methods. Second, the federated knowledge contained in federated representations and potential labels derived through recurrent learning assist local shared data representations in performing supervised augmented learning. Finally, the federated knowledge is transferred indirectly from the representation augmentation module for shared data to the unsupervised representation augmentation module for local private data (non-aligned data). The experimental results show the effectiveness of the proposed knowledge transfer mechanism, whether applied independently or used to enhance VFL on medical datasets. Our findings contribute to a deeper theoretical understanding of VFL, further facilitating the utilization of high-value medical data. By promoting cross-institutional and cross-disciplinary collaboration in healthcare data sharing, our study enhances the quality efficiency of medical services, thereby accelerating the development of interdisciplinary medical research. Code is available at https://github.com/LieLieLieLieLie/CrossKT-FRA.

Infinite-dimensional reservoir computing

Reservoir computing approximation and generalization bounds are proved for a new concept class of input/output systems that extends the so-called generalized Barron functionals to a dynamic context. This new class is characterized by the readouts with a certain integral representation built on infinite-dimensional state–space systems. It is shown that this class is very rich and possesses useful features and universal approximation properties. The reservoir architectures used for the approximation and estimation of elements in the new class are randomly generated echo state networks with either linear or ReLU activation functions. Their readouts are built using randomly generated neural networks in which only the output layer is trained (extreme learning machines or random feature neural networks). The results in the paper yield a recurrent neural network-based learning algorithm with provable convergence guarantees that do not suffer from the curse of dimensionality when learning input/output systems in the class of generalized Barron functionals and measuring the error in a mean-squared sense.

An efficient scheduling scheme for intelligent driving tasks in a novel vehicle-edge architecture considering mobility and load balancing

With the continuous popularization and evolution of 5G and 6G, mobile edge computing has achieved rapid development. This study explores the New Generation Mobile Edge Computing (NGMEC) architecture, which leverages numerous mobile nodes to provide users with enhanced computing services. Despite its advantages, NGMEC faces challenges such as high node mobility, load balancing difficulties, and incomplete environmental perception by agents, particularly in intelligent driving task offloading scenarios. We address these challenges by introducing novel applications of NGMEC and proposing specialized algorithms for node selection and load balancing. Furthermore, to tackle the issue of environmental perception incompleteness in NGMEC task offloading, we develop the Gated Recurrent Self-Encoding Deep Reinforcement Learning (GRSE-DRL) algorithm. Our research also includes the development of two platforms: the End-Edge-Cloud Simulation Experiment Platform and the Edge Computing Offloading Algorithm Energy Efficiency Test Platform. Experimental results demonstrate that our proposed scheme effectively maintains load balance among nodes, enhances task completion and connection success rates, and optimizes the trade-off between transmission delay and intelligent driving algorithms’ effectiveness, enabling more efficient decision-making.