Short-term Memory Recurrent Neural Network Research Articles

Indoor air quality prediction modeling for a naturally ventilated fitness building using RNN-LSTM artificial neural networks

PurposeExposure to indoor air pollutants poses a significant health risk, contributing to various ailments such as respiratory and cardiovascular diseases. These unhealthy consequences are specifically alarming for athletes during exercise due to their higher respiratory rate. Therefore, studying, predicting and curtailing exposure to indoor air contaminants during athletic activities is essential for fitness facilities. The objective of this study is to develop a neural network model designed for predicting optimal (in terms of health) occupancy intervals using monitored indoor air quality (IAQ) data.Design/methodology/approachThis research study presents an innovative approach employing a long short-term memory (LSTM) recurrent neural network (RNN) to determine optimal occupancy intervals for ensuring the safety and well-being of occupants. The dataset was collected over a 3-month monitoring campaign, encompassing 15 meteorological and indoor environmental parameters monitored. All the parameters were monitored in 5-min intervals, resulting in a total of 77,520 data points. The dataset collection parameters included the building’s ventilation methods as well as the level of occupancy. Initial preprocessing involved computing the correlation matrix and identifying highly correlated variables to serve as inputs for the LSTM network model.FindingsThe findings underscore the efficacy of the proposed artificial intelligence model in forecasting indoor conditions, yielding highly specific predicted time slots. Using the training dataset and established threshold values, the model effectively identifies benign periods for occupancy. Validation of the predicted time slots is conducted utilizing features chosen from the correlation matrix and their corresponding standard ranges. Essentially, this process determines the ratio of recommended to non-recommended timing intervals.Originality/valueHumans do not have the capacity to process this data and make such a relevant decision, though the complexity of the parameters of IAQ imposes significant barriers to human decision-making, artificial intelligence and machine learning systems, which are different. Present research utilizing multilayer perceptron (MLP) and LSTM algorithms for evaluating indoor air pollution levels lacks the capability to predict specific time slots. This study aims to fill this gap in evaluation methodologies. Therefore, the utilized LSTM-RNN model can provide a day-ahead prediction of indoor air pollutants, making its competency far beyond the human being’s and regular sensors' capacities.

Elevating sustainability with a multi-renewable hydrogen generation system empowered by machine learning and multi-objective optimization

The global energy landscape is rapidly shifting toward cleaner, lower-carbon electricity generation, necessitating a transition to alternate energy sources. Hydrogen, particularly green hydrogen, looks to be a significant solution for facilitating this transformation, as it is produced by water electrolysis with renewable energy sources such as solar irradiations, wind speed, and biomass residuals. Traditional energy systems are costly and produce energy slowly due to unpredictability in resource supply. To address this challenge, this work provides a novel technique that integrates a multi-renewable energy system using multi objective optimization algorithm to meets the machine learning-based forecasted load model. Several forecasting models, including Autoregressive Integrated Moving Average (ARIMA), Random Forest and Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), are assessed for develop the statistical metrics values such as RMSE, MAE, and MAPE. The selected Non-Sorting Moth Flame Optimization (NSMFO) algorithm demonstrates technological prowess in efficiently achieving global optimization, particularly when handling multiple objective functions. This integrated method shows enormous promise in technological, economic, and environmental terms, emphasizing its ability to promote energy sustainability targets.

LSTM RNN-based excitation force prediction for the real-time control of wave energy converters

Wave energy is a type of abundant and dense renewable energy. Wave force prediction is a critical technology that influences power absorption efficiency in the real-time control of wave energy converter (WEC). Could wave elevation be used to predict wave excitation force directly by training artificial neural network? This method results in rapid and suitable prediction for real-time control. A long short-term memory recurrent neural network (LSTM RNN) algorithm is introduced to identify characteristics of wave excitation forces based on wave elevations. In this method, the wave elevations in front of the structure are measured to obtain sufficient time to actuate the control manipulation. A total of 180 regular wave and 12 irregular wave tests are conducted, and the LSTM RNN model is trained based on the experimental results. The performance of the LSTM algorithm is verified. According to the regular cases in the study, the LSTM prediction can identify high-order harmonic loads, and the anti-noise capability of the LSTM algorithm can filter random noises from the measure signals. In the irregular cases, the LSTM RNN algorithm performs effectively to predict the wave force excited on the structure using wave elevations measured by wave probes. The best combinations of the test setting parameters are determined to guide experimental tests and WEC prototypes.

Decoding Nifty50’s Price Enigma: Machine Learning Powered Approach

Abstract: This study uses machine learning and Long Short-Term Memory (LSTM) networks to decipher the price riddle of Nifty50, a well-known stock market index in India. Using Long Short-Term Memory (LSTM) recurrent neural networks, which are renowned for their capacity to recognise sequential patterns, we aim to forecast and comprehend the intricate dynamics influencing Nifty50's price fluctuations.

Prediction of Diffuse Attenuation Coefficient Based on Informer: A Case Study of Hangzhou Bay and Beibu Gulf

Marine water quality significantly impacts human livelihoods and production such as fisheries, aquaculture, and tourism. Satellite remote sensing facilitates the predictions of large-area marine water quality without the need for frequent field work and sampling. Prediction of diffuse attenuation coefficient (Kd), which describes the speed at which light decays as it travels through water, obtained from satellite-derived ocean color products can reflect the overall water quality trends. However, current models inadequately explore the complex nonlinear features of Kd, and there are difficulties in achieving accurate long-term predictions and optimal computational efficiency. This study innovatively proposes a model called Remote Sensing-Informer-based Kd Prediction (RSIKP). The proposed RSIKP is characterized by a distinctive Multi-head ProbSparse self-attention mechanism and generative decoding structure. It is designed to comprehensively and accurately capture the long-term variation characteristics of Kd in complex water environments while avoiding error accumulation, which has a significant advantage in multi-dataset experiments due to its high efficiency in long-term prediction. A multi-dataset experiment is conducted at different prediction steps, using 70 datasets corresponding to 70 study areas in Hangzhou Bay and Beibu Gulf. The results show that RSIKP outperforms the five prediction models based on Artificial Neural Networks (ANN, Convolutional Neural Networks (CNN), Gated Recurrent Unit (GRU), Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN), and Long Short-Term Memory Networks (LSTM)). RSIKP captures the complex influences on Kd more effectively to achieve higher prediction accuracy compared to other models. It shows a mean improvement of 20.6%, 31.1%, and 22.9% on Mean Absolute Error (MAE), Mean Square Error (MSE), and Mean Absolute Percentage Error (MAPE). Particularly notable is its outstanding performance in the long time-series predictions of 60 days. This study develops a cost-effective and accurate method of marine water quality prediction, providing an effective prediction tool for marine water quality management.

Stock Market Prediction Using Machine Learning Techniques

The prediction of stock market prices holds significant importance within the contemporary economic landscape. Consequently, there has been a notable surge in scholarly interest directed towards exploring novel avenues for enhancing stock market prediction capabilities. Recent research endeavors have illuminated the potential predictive value inherent in various data streams, including historical stock data and user-generated content sourced from platforms such as Twitter and web news. These investigations have revealed a discernible relationship between social mood, as reflected in online discourse, and future stock price movements. However, prior studies have often overlooked the incorporation of such sentiment-derived information, thus presenting an information gap. In the present study, we address this gap by proposing an effective methodology for the integration and analysis of multiple information sources to facilitate more accurate stock price predictions. Leveraging Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) models, we conduct a comprehensive analysis of disparate data sources. Furthermore, we employ an ensemble approach, incorporating Weighted Average and Differential Evolution techniques, to refine the predictive accuracy of our models. Our findings demonstrate the efficacy of the proposed methodology in generating highly accurate stock price predictions across varying future time horizons, including one-day, seven-day, 15-day, and 30-day intervals. These predictions offer valuable insights for investors seeking to make informed decisions regarding their investment strategies and enable companies to gauge their anticipated performance within the stock market landscape.

Energy Expenditure Prediction from Accelerometry Data Using Long Short-Term Memory Recurrent Neural Networks.

The accurate estimation of energy expenditure from simple objective accelerometry measurements provides a valuable method for investigating the effect of physical activity (PA) interventions or population surveillance. Methods have been evaluated previously, but none utilize the temporal aspects of the accelerometry data. In this study, we investigated the energy expenditure prediction from acceleration measured at the subjects' hip, wrist, thigh, and back using recurrent neural networks utilizing temporal elements of the data. The acceleration was measured in children (N = 33) performing a standardized activity protocol in their natural environment. The energy expenditure was modelled using Multiple Linear Regression (MLR), stacked long short-term memory (LSTM) networks, and combined convolutional neural networks (CNN) and LSTM. The correlation and mean absolute percentage error (MAPE) were 0.76 and 19.9% for the MLR, 0.882 and 0.879 and 14.22% for the LSTM, and, with the combined LSTM-CNN, the best performance of 0.883 and 13.9% was achieved. The prediction error for vigorous intensities was significantly different (p < 0.01) from those of the other intensity domains: sedentary, light, and moderate. Utilizing the temporal elements of movement significantly improves energy expenditure prediction accuracy compared to other conventional approaches, but the prediction error for vigorous intensities requires further investigation.

State-of-charge estimation of lithium-ion battery: Joint long short-term memory network and adaptive extended Kalman filter online estimation algorithm

With the widespread use of lithium-ion batteries, estimating the State of Charge (SOC) has been one of the most critical tasks in the battery management system (BMS). In this paper, a joint long short-term memory recurrent neural network (LSTM-RNN) and adaptive extended Kalman filter (AEKF) algorithm is proposed to achieve accurate SOC estimation. The proposed algorithm generates an initial SOC estimation value through a trained LSTM-RNN with a sliding time window signal sequence as input. The initial estimation value is then used as a feedback input to the AEKF. The results show that only a 30-s time series as the input of the LSTM-RNN can achieve satisfactory estimation results. Compared with traditional adaptive extended Kalman filter and unscented Kalman filter (UKF), the proposed algorithm has a better performance of estimation accuracy under various discharging conditions, especially when applied at low temperatures. Finally, the SOC estimation results under four operating conditions and various temperatures show that the maximum error is less than 2.5 %, in particular, the root mean square error (RMSE) and the mean absolute error (MAE) are below 1.01 % and 0.64 %. The proposed algorithm is an effective SOC estimation method with good robustness and high estimation accuracy.

An AI-based approach for modeling the synergy between radiotherapy and immunotherapy

Personalized, ultra-fractionated stereotactic adaptive radiotherapy (PULSAR) is designed to administer tumoricidal doses in a pulsed mode with extended intervals, spanning weeks or months. This approach leverages longer intervals to adapt the treatment plan based on tumor changes and enhance immune-modulated effects. In this investigation, we seek to elucidate the potential synergy between combined PULSAR and PD-L1 blockade immunotherapy using experimental data from a Lewis Lung Carcinoma (LLC) syngeneic murine cancer model. Employing a long short-term memory (LSTM) recurrent neural network (RNN) model, we simulated the treatment response by treating irradiation and anti-PD-L1 as external stimuli occurring in a temporal sequence. Our findings demonstrate that: (1) The model can simulate tumor growth by integrating various parameters such as timing and dose, and (2) The model provides mechanistic interpretations of a “causal relationship” in combined treatment, offering a completely novel perspective. The model can be utilized for in-silico modeling, facilitating exploration of innovative treatment combinations to optimize therapeutic outcomes. Advanced modeling techniques, coupled with additional efforts in biomarker identification, may deepen our understanding of the biological mechanisms underlying the combined treatment.

Applying Recurrent Neural Networks and Blocked Cross-Validation to Model Conventional Drinking Water Treatment Processes

The jar test is the current standard method for predicting the performance of a conventional drinking water treatment (DWT) process and optimizing the coagulant dose. This test is time-consuming and requires human intervention, meaning it is infeasible for making continuous process predictions. As a potential alternative, we developed a machine learning (ML) model from historical DWT plant data that can operate continuously using real-time sensor data without human intervention for predicting clarified water turbidity 15 min in advance. We evaluated three types of models: multilayer perceptron (MLP), the long short-term memory (LSTM) recurrent neural network (RNN), and the gated recurrent unit (GRU) RNN. We also employed two training methodologies: the commonly used holdout method and the theoretically correct blocked cross-validation (BCV) method. We found that the RNN with GRU was the best model type overall and achieved a mean absolute error on an independent production set of as low as 0.044 NTU. We further found that models trained using BCV typically achieve errors equal to or lower than their counterparts trained using holdout. These results suggest that RNNs trained using BCV are superior for the development of ML models for DWT processes compared to those reported in earlier literature.

Deep unrolling networks with recurrent momentum acceleration for nonlinear inverse problems

Combining the strengths of model-based iterative algorithms and data-driven deep learning solutions, deep unrolling networks (DuNets) have become a popular tool to solve inverse imaging problems. Although DuNets have been successfully applied to many linear inverse problems, their performance tends to be impaired by nonlinear problems. Inspired by momentum acceleration techniques that are often used in optimization algorithms, we propose a recurrent momentum acceleration (RMA) framework that uses a long short-term memory recurrent neural network (LSTM-RNN) to simulate the momentum acceleration process. The RMA module leverages the ability of the LSTM-RNN to learn and retain knowledge from the previous gradients. We apply RMA to two popular DuNets—the learned proximal gradient descent (LPGD) and the learned primal-dual (LPD) methods, resulting in LPGD-RMA and LPD-RMA, respectively. We provide experimental results on two nonlinear inverse problems: a nonlinear deconvolution problem, and an electrical impedance tomography problem with limited boundary measurements. In the first experiment we have observed that the improvement due to RMA largely increases with respect to the nonlinearity of the problem. The results of the second example further demonstrate that the RMA schemes can significantly improve the performance of DuNets in strongly ill-posed problems.

Triple Generative Self-Supervised Learning Method for Molecular Property Prediction.

Molecular property prediction is an important task in drug discovery, and with help of self-supervised learning methods, the performance of molecular property prediction could be improved by utilizing large-scale unlabeled dataset. In this paper, we propose a triple generative self-supervised learning method for molecular property prediction, called TGSS. Three encoders including a bi-directional long short-term memory recurrent neural network (BiLSTM), a Transformer, and a graph attention network (GAT) are used in pre-training the model using molecular sequence and graph structure data to extract molecular features. The variational auto encoder (VAE) is used for reconstructing features from the three models. In the downstream task, in order to balance the information between different molecular features, a feature fusion module is added to assign different weights to each feature. In addition, to improve the interpretability of the model, atomic similarity heat maps were introduced to demonstrate the effectiveness and rationality of molecular feature extraction. We demonstrate the accuracy of the proposed method on chemical and biological benchmark datasets by comparative experiments.

An intelligent sequential fraud detection model based on deep learning

AbstractFraud detection and prevention has received a lot of attention from the research community due to its high impact on financial institutions’ revenues and reputation. The increased use of the web and the provision of online services open up the pathway for exposing these systems to numerous threats and jeopardizing their effective functioning. Naturally, financial frauds are increased in number and form imposing various requirements for their efficient and immediate detection. These requirements are related to the performance of the adopted models as well as the timely response of the decision-making mechanism. Machine learning and data mining are two research domains that can provide a number of techniques/algorithms for fraud detection and setup the road for mitigation actions. However, these methods still need to be improved with respect to the detection of unknown fraud patterns and the incorporation of big data processing mechanisms. This paper presents our attempt to build a hybrid system, i.e., a sequential scheme for combining two deep learning models and efficiently detecting potential financial frauds. We elaborate on the combination of an autoencoder and a Long Short-Term Memory Recurrent Neural Network trained upon datasets which are processed through the use of an oversampling technique. Oversampling is adopted to handle heavily imbalanced datasets which is the ‘natural’ scenario due to the limited number of frauds compared to the humongous volumes of transactions. The proposed approach tends to capture much more fraud events in comparison with other conventional ML techniques. Our experimental evaluation exposes that our model exhibits a good performance in terms of recall and precision.

Analyzing Cryptocurrency Prices through Artificial Intelligence

Cryptocurrency is reshaping the financial landscape, gaining popularity and acceptance among merchants. Despite the increasing investments in cryptocurrency, its dynamic features, uncertainties, and predictability remain largely unknown, posing significant investment risks. This study aims to understand the factors influencing cryptocurrency value formation. Leveraging advanced artificial intelligence frameworks like fully connected Artificial Neural Network (ANN) and Long Short-Term Memory (LSTM) Recurrent Neural Network, we analyze the price dynamics of Bitcoin, Ethereum, and Ripple. Our findings indicate that ANN relies more on long-term history, while LSTM focuses on short-term dynamics, suggesting LSTM's superior efficiency in utilizing historical information. However, with sufficient historical data, ANN can achieve comparable accuracy to LSTM. This study underscores the predictability of cryptocurrency market prices, though the explanation may vary depending on the machine-learning model employed.

Machine learning and deep learning models for stock price prediction, case study: Google Company

Stock prediction is crucial for investors to manage risk, diversify portfolios, and plan for long-term financial goals. Companies use predictions to allocate capital wisely, attract investors, and make strategic decisions. Stock prediction is to analyze competitive stock (e.g. Google, Apple and Microsoft), involving parameters of revenue, profit, PE ratio, growth rate, and to predict the future price of the stock. Based on these assumptions, this project objective is to forecast the trend in Google stock prices with four predictive models: linear regression, XGBoost, Long Short-Term Memory Network (LSTM) and Recurrent neural network (RNN). The dataset is Google stock prediction dataset, which involves 17,598 stock information. Among all four models, LSTM method obtains the best final result with 0.0012 of MSE and 0.95115 of R square. LSTM has the lowest MSE, and its R square is similar to that of other models. For future plan, this project may expand more on feature engineering, model techniques and fine-tuning methods.

A metaheuristic algorithm for model predictive control of the oil-cooled motor in hybrid electric vehicles

The current energy management methods for hybrid vehicles are primarily focused on matching the power flow between the engine and the motor. In order to further reduce overall energy consumption and extend the vehicle's lifespan, this paper utilizes model predictive control (MPC) in the energy management of hybrid vehicles to control the energy consumption and temperature of the oil-cooled motor, with a focus on studying the algorithm for solving the optimization problem in MPC. The main work includes: Establishing a simplified model for the oil-cooled motor and training the working condition prediction model based on the worldwide harmonized light vehicles test cycle (WLTC) conditions utilizing long short-term memory (LSTM) recurrent neural network technology. Furthermore, proposing a novel metaheuristic algorithm based on the temporal nature of the problem, multiple hierarchical clustering algorithm (MHCA). In comparison with typical metaheuristic algorithms, the MPC optimized using MHCA exhibits significantly reduced computation time, with an average time of 0.1967s, indicating that using MHCA can theoretically achieve real-time control and eliminate lag in actual control processes. Furthermore, compared to other optimization algorithms, the MHCA algorithm demonstrates the lowest correlation between computation time and operational conditions, highlighting its high robustness in ensuring the safety of practical control.

Research on interference compensation methods for color image sensors based on iterative learning

PurposeFixed mode noise and random mode noise always exist in the image sensor, which affects the imaging quality of the image sensor. The charge diffusion and color mixing between pixels in the photoelectric conversion process belong to fixed mode noise. This study aims to improve the image sensor imaging quality by processing the fixed mode noise.Design/methodology/approachThrough an iterative training of an ergoable long- and short-term memory recurrent neural network model, the authors obtain a neural network model able to compensate for image noise crosstalk. To overcome the lack of differences in the same color pixels on each template of the image sensor under flat-field light, the data before and after compensation were used as a new data set to further train the neural network iteratively.FindingsThe comparison of the images compensated by the two sets of neural network models shows that the gray value distribution is more concentrated and uniform. The middle and high frequency components in the spatial spectrum are all increased, indicating that the compensated image edges change faster and are more detailed (Hinton and Salakhutdinov, 2006; LeCun et al., 1998; Mohanty et al., 2016; Zang et al., 2023).Originality/valueIn this paper, the authors use the iterative learning color image pixel crosstalk compensation method to effectively alleviate the incomplete color mixing problem caused by the insufficient filter rate and the electric crosstalk problem caused by the lateral diffusion of the optical charge caused by the adjacent pixel potential trap.

CRISPR-M: Predicting sgRNA off-target effect using a multi-view deep learning network.

Using the CRISPR-Cas9 system to perform base substitutions at the target site is a typical technique for genome editing with the potential for applications in gene therapy and agricultural productivity. When the CRISPR-Cas9 system uses guide RNA to direct the Cas9 endonuclease to the target site, it may misdirect it to a potential off-target site, resulting in an unintended genome editing. Although several computational methods have been proposed to predict off-target effects, there is still room for improvement in the off-target effect prediction capability. In this paper, we present an effective approach called CRISPR-M with a new encoding scheme and a novel multi-view deep learning model to predict the sgRNA off-target effects for target sites containing indels and mismatches. CRISPR-M takes advantage of convolutional neural networks and bidirectional long short-term memory recurrent neural networks to construct a three-branch network towards multi-views. Compared with existing methods, CRISPR-M demonstrates significant performance advantages running on real-world datasets. Furthermore, experimental analysis of CRISPR-M under multiple metrics reveals its capability to extract features and validates its superiority on sgRNA off-target effect predictions.

Long short-term memory prediction of user’s locomotion in virtual reality

Nowadays, there is still a challenge in virtual reality to obtain an accurate displacement prediction of the user. This could be a future key element to apply in the so-called redirected walking methods. Meanwhile, deep learning provides us with new tools to reach greater achievements in this type of prediction. Specifically, long short-term memory recurrent neural networks obtained promising results recently. This gives us clues to continue researching in this line to predict virtual reality user’s displacement. This manuscript focuses on the collection of positional data and a subsequent new way to train a deep learning model to obtain more accurate predictions. The data were collected with 44 participants and it has been analyzed with different existing prediction algorithms. The best results were obtained with a new idea, the use of rotation quaternions and the three dimensions to train the previously existing models. The authors strongly believe that there is still much room for improvement in this research area by means of the usage of new deep learning models.

Underwater Acoustic Nonlinear Blind Ship Noise Separation Using Recurrent Attention Neural Networks

Ship-radiated noise is the main basis for ship detection in underwater acoustic environments. Due to the increasing human activity in the ocean, the captured ship noise is usually mixed with or covered by other signals or noise. On the other hand, due to the softening effect of bubbles in the water generated by ships, ship noise undergoes non-negligible nonlinear distortion. To mitigate the nonlinear distortion and separate the target ship noise, blind source separation (BSS) becomes a promising solution. However, underwater acoustic nonlinear models are seldom used in research for nonlinear BSS. This paper is based on the hypothesis that the recovery and separation accuracy can be improved by considering this nonlinear effect in the underwater environment. The purpose of this research is to explore and discover a method with the above advantages. In this paper, a model is used in underwater BSS to describe the nonlinear impact of the softening effect of bubbles on ship noise. To separate the target ship-radiated noise from the nonlinear mixtures, an end-to-end network combining an attention mechanism and bidirectional long short-term memory (Bi-LSTM) recurrent neural network is proposed. Ship noise from the database ShipsEar and line spectrum signals are used in the simulation. The simulation results show that, compared with several recent neural networks used for linear and nonlinear BSS, the proposed scheme has an advantage in terms of the mean square error, correlation coefficient and signal-to-distortion ratio.