Recurrent Neural Network Approach Research Articles

Forecasting research influence: a recurrent neural network approach to citation prediction

Forecasting research influence: a recurrent neural network approach to citation prediction

A recurrent neural network approach for magneto‐hydro‐dynamic flow of second‐grade fluid with dissipation effect

A recurrent neural network approach for magneto‐hydro‐dynamic flow of second‐grade fluid with dissipation effect

Image-Based Visual Servoing of Manipulators With Unknown Depth: A Recurrent Neural Network Approach.

The image-based visual servoing (IBVS) of manipulators is important for intelligent manipulation using visual feedbacks. While the traditional IBVS methods for manipulators require the knowledge of the depth information in the interaction matrix, in this article, we propose a novel IBVS method for manipulators without depth estimation by leveraging the property of the associated image Jacobian. Because of a novel transformation, the IBVS problem is converted into a convex optimization problem subject to the kinematic constraint, joint constraints, and other constraints that are not explicitly related to the depth information. The problem is then solved by developing a recurrent neural network of global asymptotic convergence, and a dynamic neural control law without depth estimation emerges for the IBVS of manipulators. The theoretical guarantee and simulation results are provided to show the efficacy of the proposed method.

Localized estimation of event-related neural source activity from simultaneous MEG-EEG with a recurrent neural network

Estimating intracranial current sources underlying the electromagnetic signals observed from extracranial sensors is a perennial challenge in non-invasive neuroimaging. Established solutions to this inverse problem treat time samples independently without considering the temporal dynamics of event-related brain processes.This paper describes current source estimation from simultaneously recorded magneto- and electro-encephalography (MEEG) using a recurrent neural network (RNN) that learns sequential relationships from neural data. The RNN was trained in two phases: (1) pre-training and (2) transfer learning with L1 regularization applied to the source estimation layer. Performance of using scaled labels derived from MEEG, magnetoencephalography (MEG), or electroencephalography (EEG) were compared, as were results from volumetric source space with free dipole orientation and surface source space with fixed dipole orientation. Exact low-resolution electromagnetic tomography (eLORETA) and mixed-norm L1/L2 (MxNE) source estimation methods were also applied to these data for comparison with the RNN method.The RNN approach outperformed other methods in terms of output signal-to-noise ratio, correlation and mean-squared error metrics evaluated against reference event-related field (ERF) and event-related potential (ERP) waveforms. Using MEEG labels with fixed-orientation surface sources produced the most consistent estimates.To estimate sources of ERF and ERP waveforms, the RNN generates temporal dynamics within its internal computational units, driven by sequential structure in neural data used as training labels. It thus provides a data-driven model of computational transformations from psychophysiological events into corresponding event-related neural signals, which is unique among MEEG source reconstruction solutions.

Leveraging hybrid 1D-CNN and RNN approach for classification of brain cancer gene expression

Leveraging deep learning (DL) approaches in genomics data has led to significant advances in cancer prediction. The continuous availability of gene expression datasets over the preceding years has made them one of the most accessible sources of genome-wide data, advancing cancer bioinformatics research and advanced prediction of cancer genomic data. To contribute to this topic, the proposed work is based on DL prediction in both convolutional neural network (CNN) and recurrent neural network (RNN) for five classes in brain cancer using gene expression data obtained from Curated Microarray Database (CuMiDa). This database is used for cancer classification and is publicly accessible on the official CuMiDa website. This paper implemented DL approaches using a One Dimensional-Convolutional Neural Network (1D-CNN) followed by an RNN classifier with and without Bayesian hyperparameter optimization (BO). The accuracy of this hybrid model combination of (BO + 1D-CNN + RNN) produced the highest classification accuracy of 100% instead of the 95% for the ML model in prior work and 90% for the (1D-CNN + RNN) algorithm considered in the paper. Therefore, the classification of brain cancer gene expression according to the hybrid model (BO + 1D-CNN + RNN) provides more accurate and useful assessments for patients with different types of brain cancers. Thus, gene expression data are used to create a DL classification-based- hybrid model that will hold senior promise in the treatment of brain cancer.

Tree-ring based forest model calibrations with a deep learning algorithm

Process-based forest growth models are important tools for forest management and studies for forest growth, carbon sequestration, and forest dynamics. However, their reliable simulations rely not only on the quality of model construction but also on accurate parameters to appropriately depict various physiological and biophysical processes in the models. While explicit physiological measurements are excellent sources for model parameterization, they are not always readily available. It would be ideal to use easy-to-measure tree-ring data as a benchmark for parametrization, but such applications have been rare. Here we present a new approach to reasonably parameterize a stand growth model based on tree-ring data using deep learning algorithms. We integrated a stable carbon isotope (δ13C) version of a simple process-based stand growth model, 3-PG, into a recurrent neural network (RNN). The new 3PG-RNN model trained the RNN network and calibrated 3-PG parameters to minimize the mean squared error between the 3-PG outputs and targeted values. We then tested 3PG-RNN based on two Abies grandis stands located in Northern Idaho, USA. The results showed that the 3PG-RNN model was efficient in calibrating key parameters related to gas-exchange processes, including specifically quantum yield, maximum canopy conductance, and the slope function for stomatal responses to vapor pressure deficit. These calibrated parameters provided reasonable simulations for gas-exchanges similar to those constrained by explicit physiological observations. This was particularly true when both long-term diameter growth (estimated from tree-ring width) and tree-ring δ13C were used for model training. The RNN tool made it possible to use tree ring data as the key benchmark to calibrate the forest model and provide unbiased simulations for gas exchanges and forest growth with the RNN approach, which would greatly facilitate forest studies and management.

Enhancing digital cryptocurrency trading price prediction with an attention-based convolutional and recurrent neural network approach: The case of Ethereum

To predict Ethereum price fluctuations, this study proposes a new two-stage Machine Learning approach using an improved convolutional neural network and a recurrent neural network framework, integrating an attention mechanism-based distribution function algorithm. We construct a dataset and perform model training, fitting, and forecasting. The results indicate that compared with traditional neural networks and time-series models such as GRU and ARIMA, respectively, this approach can effectively use the data information of digital cryptocurrency and improve the prediction accuracy and interpretability of attention-based allocation functions. This study contributes to the literature by offering a new approach for stakeholders.

Wind Speed Prediction Using Deep Recurrent Neural Networks and Farm Platform Features for One-Hour-Ahead Forecast

This paper proposes a deep recurrent neural network (DRNN) approach to model the one-hour-ahead wind speed forecasting by using various meteorological sensory data from the North Wyke farm platform (NWFP). To refine model input, mutual information analysis is applied to eliminate irrelevant sensory data. The DRNN architecture employs three recurrent layers Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and simple Recurrent Neural Network (RNN) to capture temporal relationships. The proposed networks are tested using real-life, one-year data from the NWFP. The results showed a strong correlation between the actual and predicted wind speed for LSTM, GRU, and RNN layers-based DRNN, however, simple RNN slightly outperformed the other two recurrent layers. The distribution of the network errors over the year is also analyzed. Although the observed meteorological data between the years was from different distributions, the proposed network generalized well even though these data were altered due to global warming.

A Recurrent Neural Network Approach for Constrained Distributed Fuzzy Convex Optimization.

This article investigates a class of constrained distributed fuzzy convex optimization problems, where the objective function is the sum of a set of local fuzzy convex objective functions, and the constraints include partial order relation and closed convex set constraints. In undirected connected node communication network, each node only knows its own objective function and constraints, and the local objective function and partial order relation functions may be nonsmooth. To solve this problem, a recurrent neural network approach based on differential inclusion framework is proposed. The network model is constructed with the help of the idea of penalty function, and the estimation of penalty parameters in advance is eliminated. Through theoretical analysis, it is proven that the state solution of the network enters the feasible region in finite time and does not escape again, and finally reaches consensus at an optimal solution of the distributed fuzzy optimization problem. Furthermore, the stability and global convergence of the network do not depend on the selection of the initial state. A numerical example and an intelligent ship output power optimization problem are given to illustrate the feasibility and effectiveness of the proposed approach.

Recurrent neural networks for aerodynamic parameter estimation with Lyapunov stability analysis

ABSTRACT Aircraft parameter estimation is crucial for designing accurate and reliable flight dynamic models, essential for integration into flight simulators and developing effective control laws. These parameters, encompassing aerodynamic characteristics, play a pivotal role in enhancing aircraft performance, safety and overall operational efficiency. This paper explores aircraft parameter estimation using two Recurrent Neural Network (RNN) approaches: (i) the weight-bias method (RNN-WB) and (ii) the gradient-based method (RNN-Gradient). RNN-WB is based on the Hopfield neural network and relies on pre-computation of weight and bias information, applying non-linearity to the states of the network. RNN-Gradient computes direct gradients and applies non-linearity to the error. Simulated flight data is employed in MATLAB implementations to assess estimation results, considering both performance metrics and computation time. The estimated parameters were also compared with estimates obtained from Unscented Kalman Filter (UKF). A novel methodology is introduced to establish the asymptotic stability of RNN-WB and RNN-Gradient methods, utilizing the normalized Lyapunov energy functional. The asymptotic stability ensures the reliability and robustness of these techniques for parameter estimation. The RNN-Gradient shows a 20% performance improvement over RNN-WB. Satisfactory simulation results from both algorithms provide a promising foundation for their potential applications to real flight test data.

Interference Management for a Wireless Communication Network Using a Recurrent Neural Network Approach

Wireless communication technologies have profoundly impacted the interconnectivity of mobile users and terminals. Nevertheless, the exponential increase in the number of users poses significant challenges, particularly in interference management, which is a major concern in wireless communication. Machine learning (ML) approaches have emerged as powerful tools for solving various problems in this domain. However, existing studies have not fully addressed the problem of interference management for wireless communication using ML techniques. In this paper, we explore the application of recurrent neural network (RNN) approaches to address co-channel interference in wireless communication. Specifically, we investigate the effectiveness of long short-term memory (LSTM), bidirectional LSTM (Bi-LSTM), and gated recurrent unit (GRU) network architectures in two different network settings. The first network comprises 10 connected devices, while the second network involves 20 devices. Our experimental results demonstrate that Bi-LSTM outperforms LSTM and GRU in terms of mean squared error, normalized mean squared error, and sum rate. While LSTM and GRU produce similar results, LSTM exhibits a marginal advantage over GRU. In addition, a combined RNN approach is also studied, and it can provide better results in dense networks.

Movie Recommendation System Using Optimized RNN Approach.

This paper proposes a movie recommendation system that utilizes an optimized Recurrent Neural Network (RNN) approach. The proposed system is designed to provide users with personalized movie recommendations based on their previous movie preferences and the sentiments. The system works by taking user input, analyzing their movie preferences using content-based filtering techniques, and generating a list of recommended movies. The RNN architecture used in this system is optimized using a combination of techniques such as dropout regularization, early stopping, and parameter tuning. The proposed optimization techniques aim to reduce overfitting, improve convergence speed, and increase the model's overall accuracy. To evaluate the effectiveness of the proposed approach, we conducted experiments on the Movie Lens dataset. The results indicate that the optimized RNN-based movie recommendation system outperforms other existing recommendation systems such as collaborative filtering, content-based filtering, and standard RNN models. Furthermore, the proposed system achieved a significant improvement in accuracy and provided highly personalized recommendations to users. Overall, the proposed movie recommendation system using optimized RNN approach is a promising solution for providing personalized movie recommendations to users. It can be implemented in various platforms such as movie streaming websites, social media, and other movie-related platforms to improve the user experience and increase engagement

A recurrent neural network approach for nonconvex interval quadratic programming

In the paper, a new approach is presented to tackle an interval optimization problem with nonconvex interval objective function and affine interval equality and inequality constraints utilizing a recurrent neural network. The approach involves converting the nonconvex interval quadratic programming into a cubic optimization via convex combinations and considering its weight as a decision variable. The recurrent neural network is then established to solve the cubic optimization through subgradient techniques and an exterior penalty function method. Moreover, it is proven that there is a unique global solution to the recurrent neural network, and its trajectory can reach the feasible region in a finite time and remain there. Additionally, it is demonstrated that the trajectory of recurrent neural network converges exponentially or in finite time towards a singleton belonging to the set of critical points of the cubic optimization. This approach is beneficial for interval quadratic programming problems where the weight is unknown or challenging to determine in practical applications. Finally, the effectiveness of this approach is verified through two numerical examples.

Short-term wind power forecasting through stacked and bi directional LSTM techniques.

Computational intelligence (CI) based prediction models increase the efficient and effective utilization of resources for wind prediction. However, the traditional recurrent neural networks (RNN) are difficult to train on data having long-term temporal dependencies, thus susceptible to an inherent problem of vanishing gradient. This work proposed a method based on an advanced version of RNN known as long short-term memory (LSTM) architecture, which updates recurrent weights to overcome the vanishing gradient problem. This, in turn, improves training performance. The RNN model is developed based on stack LSTM and bidirectional LSTM. The parameters like mean absolute error (MAE), standard deviation error (SDE), and root mean squared error (RMSE) are utilized as performance measures for comparison with recent state-of-the-art techniques. Results showed that the proposed technique outperformed the existing techniques in terms of RMSE and MAE against all the used wind farm datasets. Whereas, a reduction in SDE is observed for larger wind farm datasets. The proposed RNN approach performed better than the existing models despite fewer parameters. In addition, the approach requires minimum processing power to achieve compatible results.

Model predictive control of nonlinear processes using transfer learning-based recurrent neural networks

Artificial neural networks (ANNs), one of the deep learning techniques that has sparked a lot of attention recently for its exceptional modeling capabilities of nonlinear systems, are an essential candidate for model-based control systems. In particular, recurrent neural networks (RNN) have shown remarkable capacity to forecast the dynamic behavior of nonlinear processes using time-series data. In order to regulate the rate at which chemical species are produced in intricate processes, RNNs have successfully served as the predictive model in model-based controllers. However, the necessity for massive amounts of data to capture complex processes is a drawback of typical RNN models. With the goal of efficient utilization of the available data, a different, transfer learning-based training approach for RNNs is presented in this work. The transfer learning strategy is taken into consideration to overcome the challenges stemming from lack of data for the construction of RNN models. In particular, weight-sharing RNNs are developed using a priori physical knowledge. Next, a comprehensive analysis of a complex chemical process on a large scale is conducted to showcase the benefits of the suggested approach across various data sizes and its effectiveness when combined with MPC in contrast to conventional RNNs.

Automated prediction of diabetes mellitus using infrared thermal foot images: recurrent neural network approach

Millions of people worldwide are affected by Diabetes Mellitus (DM), which is a chronic disease. Evaluation of the DM indicator, namely blood glucose level, requires invasive methods such as glucometer or blood tests, which cause discomfort to the patient. Automated noninvasive monitoring methods are urgently needed to ensure consistency and better treatment. The regular monitoring of DM can prevent or delay the onset of complications. Thermal foot images have been proposed as noninvasive methods for the prediction of DM. Thermograms were acquired at Mittal Eye Hospital, Sangrur, India, from 50 participants in the diabetic (without neuropathic conditions) and non-diabetic groups using a thermal camera (FLIR E-60). This study proposes an automated prediction system for DM using thermal foot images and Recurrent Neural Network (RNN) approach. The proposed system processes the thermal images and extracts relevant features using a CNN (Convolutional Neural Network). The extracted features were then fed to the RNN to predict the presence or absence of the DM. The experimental results demonstrate that the proposed framework attains an accuracy of (97.14 ± 1.5)%, surpassing the predictive capabilities of light-weight convolutional neural network (Lw-CNN), which only achieves an accuracy of (82.9 ± 3) % in predicting DM. This performance outperformed other state-of-the-art methods in the field. Our approach has the potential to be used as prediction tool for DM. Therefore, the proposed system has the potential for prediction of DM and improve patient outcomes by enabling timely intervention. Future work should focus on evaluating the proposed system on a larger dataset and integrating it with clinical decision support systems for personalized care. This study holds the promise of transforming DM screening and diagnosis, leading to enhanced patient outcomes.

Decomposition aided attention-based recurrent neural networks for multistep ahead time-series forecasting of renewable power generation.

Renewable energy plays an increasingly important role in our future. As fossil fuels become more difficult to extract and effectively process, renewables offer a solution to the ever-increasing energy demands of the world. However, the shift toward renewable energy is not without challenges. While fossil fuels offer a more reliable means of energy storage that can be converted into usable energy, renewables are more dependent on external factors used for generation. Efficient storage of renewables is more difficult often relying on batteries that have a limited number of charge cycles. A robust and efficient system for forecasting power generation from renewable sources can help alleviate some of the difficulties associated with the transition toward renewable energy. Therefore, this study proposes an attention-based recurrent neural network approach for forecasting power generated from renewable sources. To help networks make more accurate forecasts, decomposition techniques utilized applied the time series, and a modified metaheuristic is introduced to optimized hyperparameter values of the utilized networks. This approach has been tested on two real-world renewable energy datasets covering both solar and wind farms. The models generated by the introduced metaheuristics were compared with those produced by other state-of-the-art optimizers in terms of standard regression metrics and statistical analysis. Finally, the best-performing model was interpreted using SHapley Additive exPlanations.

Optimizing Remaining Useful Life Predictions for Aircraft Engines: A Dilated Recurrent Neural Network Approach

Predicting the remaining useful life (RUL) plays a crucial rule in the field of prognostics and health management (PHM) for mechanical systems. Specifically within the domain of turbofan engines, predicting RUL plays a vital role in strategically planning maintenance activities. Consequently this aids in optimizing the overall performance of the energy system by reducing downtime and improving sustainability and efficiency. This research endeavors to forecast the RUL of turbofan engines. It employs a Dilated Recurrent Neural Network (D-RNN) Approach, a neural network structure that integrates dilated convolutions into the recurrent layers. The model underwent fine-tuning through a random grid search optimization and was tested using the Commercial Modular Aero Propulsion System Simulation (C-MAPSS) dataset. The results showcase the superior performance of the proposed D-RNN, outperforming the accuracy of other research studies.

Optimal Reconstruction of Probabilistic Boolean Networks.

In gene regulatory networks (GRNs), it is important to model gene regulation based on a priori information and experimental data. As a useful mathematical model, probabilistic Boolean networks (PBNs) have been widely applied in GRNs. This article addresses the optimal reconstruction problem of PBNs based on several priori Boolean functions and sampled data. When all candidate Boolean functions are known in advance, the optimal reconstruction problem is reformulated into an optimization problem. This problem can be well solved by a recurrent neural network approach which decreases the computational cost. When parts of candidate Boolean functions are known in advance, necessary and sufficient conditions are provided for the reconstruction of PBNs. In this case, two types of reconstruction problems are further proposed: one is aimed at minimizing the number of reconstructed Boolean functions, and the other one is aimed at maximizing the selection probability of the main dynamics under noises. At last, examples in GRNs are elaborated to demonstrate the effectiveness of the main results.

Vessel trajectory prediction with recurrent neural networks: An evaluation of datasets, features, and architectures

Maritime situational awareness tasks such as port management, collision avoidance, and search-and-rescue missions rely on accurate knowledge of vessel locations. The availability of historical vessel trajectory data through the Automatic Identification System (AIS) has enabled the development of prediction methods, with a recent focus on trajectory prediction via recurrent neural networks (RNNs) and other deep learning architectures. While these methods have shown promising performance benefits over kinematic and clustering-based models, comparing among RNN-based models remains difficult due to variations in evaluation datasets, region sizes, vessel types, and numerous other design choices. As a result, it is not clear whether recent methods based on highly-sophisticated network architectures are necessary to achieve strong prediction performance. In this work, we present a simple fusion-based RNN approach to vessel trajectory prediction that allows for easy incorporation of exogenous variables. We perform an extensive ablation study to measure the impact of various modeling choices, including preprocessing, loss functions, and the choice of features, as well as the first usage of surface current information in vessel trajectory prediction. We demonstrate that our approach achieves state-of-the-art performance on three large regions off the United States coast, obtaining an improvement of up to 0.88 km over competing methods when predicting three hours into the future. We conclude that our simple architecture can outperform more complicated architectures while incurring a lower memory cost. Further, we show that the choice of loss function and the inclusion of surface current information both have significant impact on prediction performance.