Long Short-term Memory Recurrent Neural Network Research Articles

Empowering Digital Civility with an NLP Approach for Detecting Twitter Cyberbullying through Boosted Ensembles

As the number of social networking sites grows, so do cyber dangers. Cyberbullying is harmful behaviour that uses technology to intimidate, harass or harm someone, often on social media platforms like Twitter. Machine learning is the optimal approach for cyberbullying detection on Twitter to process large amounts of data, identify patterns of offensive behaviour, and automate the detection process for corpus of tweets. To identify cyber threats using a trained model, the Boosted Ensemble (BE) technique is assessed with various machine learning algorithms such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Naive Bayes (NB), Decision Tree (DT), Support Vector Machine (SVM), Bidirectional LSTM (BILSTM), Recurrent Neural Network LSTM (RNN-LSTM), Multi-Modal Cyberbullying Detection (MMCD) and Random Forest (RF). These classifiers are trained on the vectorized data to classify the tweets to identify cyberbullying threats. The proposed framework can detect cyberbullying cases precisely on tweets. The significance of the work lies in detecting and mitigating cyber threats in real-time and it impacts in enhancing the safety and well-being of social media users by reducing instances of cyberbullying and other cyber threat. The comparative analysis is done using metrices like accuracy, precision, recall and F1-score and the comparison results shows that boosted ensemble technique outperforms other compared algorithms with its overall performance. The accuracy rate of CNN, LSTM, NB, DT, SVM, RF, BILSTM and BE (92.5%, 93.5%, 84.6%, 88%, 89.3%, 92%, 93.75% and 96%), precision rate of CNN, LSTM, NB, DT, SVM, RF, RNN-LSTM and BE (90.2%, 91.3%, 88%, 85%, 86%, 91.6%, 92.1% and 94%), recall rate of CNN, LSTM, NB, DT, SVM, RF, BILSTM and BE (89.8%, 90.7%, 90%, 82%, 88.67%, 89%, 91.04% and 93.7%) and F1 score of CNN, LSTM, NB, DT, SVM, RF, MMCD and BE (90.6%, 91.8%, 85%, 84.56% 87.2%, 90%, 84.6% and 94.89%).

Predicting storm surge extremes on the Southeast Brazilian Coast: Long-term projections with neural networks

Extreme sea level events, resulting from the confluence of tides and storm surges, pose a significant threat to coastal populations and economies. The escalating risks associated with these events are exacerbated by climate change, manifesting in heightened storm intensity, increased frequency, and rising sea levels. Precise estimation of the probability of extreme storm surges is crucial for effective coastal management and adaptation. However, utilizing historical storm data is challenging due to data scarcity and the imperative to consider potential non-stationarity induced by climate change in predicting such events. This study addresses these challenges by introducing two neural network-based machine learning systems: a Multilayer Perceptron (MLP) and a Long Short-Term Memory (LSTM). Leveraging local and remote atmospheric and oceanic conditions, these systems project storm surges until 2060, incorporating climate projections. Trained and evaluated using sea level data from Imbetiba Port in Macaé, Rio de Janeiro, Brazil, the models utilize dynamic regionalization data from the RegCM4 and WW3 models, forced by HadGEM2-ES and MPI climate models. Both neural network models exhibited similar performance patterns, demonstrating high agreement in predicting storm surge heights with a 100-year return value, based on Imbetiba Port data. Projections utilized peaks-over-threshold (POT) methods, and extremes were calculated using a generalized Pareto distribution (GPD). Long-term projections indicated a 28 % increase (MLP ANN) and a substantial 70 % increase (LSTM RNN) in estimating extreme values, surpassing the observed storm surge of 0.67 m. Projected mean values were 0.86 m for the MLP network and 1.15 m for the LSTM network, providing valuable insights into the potential amplification of extreme sea level events in the studied region.

Improving Discharge Predictions in Ungauged Basins: Harnessing the Power of Disaggregated Data Modeling and Machine Learning

AbstractCurrent machine learning methods for discharge prediction often employ aggregated basin‐wide hydrometeorological data (lumped modeling) for parametric and non‐parametric training. This approach may overlook the spatial heterogeneity of river systems and their impact on discharge patterns. We hypothesize that integrating spatiotemporal hydrologic knowledge into the data modeling process (distributed/disaggregated modeling) can improve the performance of discharge prediction models. To test this hypothesis, we designed experiments comparing the performance of identical Long Short‐Term Memory Recurrent Neural Network (LSTM‐RNN) models forced with either lumped or distributed features. We gather meteorological forcing and static attributes for the Mackenzie basin in Canada‐ a large and unique basin. Importantly, discharge performance is assessed out‐of‐sample with k‐fold replication across gauges. Training LSTMs with disaggregated data significantly improved model accuracy. Specifically, there was a 9.6% increase in the mean Nash‐Sutcliffe Efficiency and a 4.6% increase in the mean Kling‐Gupta Efficiency, indicating a better agreement between predicted and actual observations in terms of mean, variability, and correlation. These experiments and results demonstrate the importance of integrating topologically guided geomorphologic and hydrologic information (distributed modeling) in data‐driven discharge predictions.

Harnessding the Power of Machine Learning to Superchange CAPM Forecasts

This paper investigates the improvement of the Capital Asset Pricing Model (CAPM) by incorporating machine learning techniques. The objective is to address the model's conventional constraints in accurately predicting stock returns. The conventional Capital Asset Pricing Model (CAPM), which heavily depends on the beta coefficient to explain returns, frequently proves inadequate in emerging markets and neglects to consider market irregularities such as size and value effects. Our methodology enhances the estimation of beta by refining the Capital Asset Pricing Model (CAPM) to exclude the intercept term and incorporating a dynamic rolling regression method. This approach captures the fluctuations of beta more effectively across different periods, resulting in a more precise estimation. This study utilises ten years of weekly closing price data from companies listed on the NSE Nifty 50 index. It employs rolling regression and two-stage regression analysis to improve the prediction of excess returns above the risk-free rate. The revised CAPM model, which omits the intercept, exhibits a superior level of accuracy, as evidenced by higher adjusted R-squared values and improved F-statistics. In addition, the paper analyses the Fama-French models within the Chinese stock market and explores the potential of advanced machine learning models, such as Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN), to improve forecasting methods in unpredictable markets. Integrating LSTM-RNN models presents a promising approach to capturing intricate patterns in financial time series data, demonstrating substantial enhancement in forecasting accuracy compared to conventional methods such as OLS.

Explainable deep recurrent neural networks for the batch analysis of a pharmaceutical tableting process in the spirit of Pharma 4.0

Due to the continuously increasing Cost of Goods Sold, the pharmaceutical industry has faced several challenges, and the Right First-Time principle with data-driven decision-making has become more pressing to sustain competitiveness. Thus, in this work, three different types of artificial neural network (ANN) models were developed, compared, and interpreted by analyzing an open-access dataset from a real pharmaceutical tableting production process. First, the multilayer perceptron (MLP) model was used to describe the total waste based on 20 raw material properties and 25 statistical descriptors of the time series data collected throughout the tableting (e.g., tableting speed and compression force). Then using 10 process time series data in addition to the raw material properties, the cumulative waste, during manufacturing was also predicted by long short-term memory (LSTM) and bidirectional LSTM (biLSTM) recurrent neural networks (RNN). The LSTM network was used to forecast the waste production profile to allow preventive actions. The results showed that RNNs were able to predict the waste trajectory, the best model resulting in 1096 and 2174 tablets training and testing root mean squared errors, respectively. For a better understanding of the process, and the models and to help the decision-support systems and control strategies, interpretation methods were implemented for all ANNs, which increased the process understanding by identifying the most influential material attributes and process parameters. The presented methodology is applicable to various critical quality attributes in several fields of pharmaceutics and therefore is a useful tool for realizing the Pharma 4.0 concept.

An Artificial Neural Network Predicts Gender Differences of Motor and Non-Motor Symptoms of Patients with Advanced Parkinson's Disease under Levodopa-Carbidopa Intestinal Gel.

Background and Objectives: Currently, no tool exists to predict clinical outcomes in patients with advanced Parkinson's disease (PD) under levodopa-carbidopa intestinal gel (LCIG) treatment. The aim of this study was to develop a novel deep neural network model to predict the clinical outcomes of patients with advanced PD after two years of LCIG therapy. Materials and Methods: This was a longitudinal, 24-month observational study of 59 patients with advanced PD in a multicenter registry under LCIG treatment from September 2019 to September 2021, including 43 movement disorder centers. The data set includes 649 measurements of patients, which make an irregular time series, and they are turned into regular time series during the preprocessing phase. Motor status was assessed with the Unified Parkinson's Disease Rating Scale (UPDRS) Parts III (off) and IV. The NMS was assessed by the NMS Questionnaire (NMSQ) and the Geriatric Depression Scale (GDS), the quality of life by PDQ-39, and severity by Hoehn and Yahr (HY). Multivariate linear regression, ARIMA, SARIMA, and Long Short-Term Memory-Recurrent NeuralNetwork (LSTM-RNN) models were used. Results: LCIG significantly improved dyskinesia duration and quality of life, with men experiencing a 19% and women a 10% greater improvement, respectively. Multivariate linear regression models showed that UPDRS-III decreased by 1.5 and 4.39 units per one-unit increase in the PDQ-39 and UPDRS-IV indexes, respectively. Although the ARIMA-(2,0,2) model is the best one with AIC criterion 101.8 and validation criteria MAE = 0.25, RMSE = 0.59, and RS = 0.49, it failed to predict PD patients' features over a long period of time. Among all the time series models, the LSTM-RNN model predicts these clinical characteristics with the highest accuracy (MAE = 0.057, RMSE = 0.079, RS = 0.0053, mean square error = 0.0069). Conclusions: The LSTM-RNN model predicts, with the highest accuracy, gender-dependent clinical outcomes in patients with advanced PD after two years of LCIG therapy.

Application of RNN-LSTM in Predicting Drought Patterns in Pakistan: A Pathway to Sustainable Water Resource Management

Water is a fundamental and crucial natural resource for human survival. However, the global demand for water is increasing, leading to a subsequent decrease in water availability. This study addresses the critical need for improved water resource forecasting models amidst global water scarcity concerns exacerbated by climate change. This study uses the best weather and water resource forecasting model for sustainable development. Employing a Recurrent Neural Network–Long Short-Term Memory (RNN-LSTM) approach, the research enhances drought prediction capabilities by integrating secondary data of the rainfall, temperature, and ground and surface water supplies. The primary objective is to forecast water resources under changing climatic conditions, facilitating the development of early warning systems for vulnerable regions. The results from the LSTM model show an increased trend in temperature and rainfall patterns. However, a relatively unstable decrease in rainfall is observed. The best statistical analysis result was observed with the LSTM model; the model’s accuracy was 99%, showing that it was quite good at presenting the obtained precipitation, temperature, and water data. Meanwhile, the value of the root mean squared error (RMSE) was about 13, 15, and 20, respectively. Therefore, the study’s results highlight that the LSTM model was the most suitable among the artificial neural networks for forecasting the weather, rainfall, and water resources. This study will help weather forecasting, agriculture, and meteorological departments be effective for water resource forecasting.

Development and Evaluation of Vibration Canceling System Utilizing Macro-Fiber Composites (MFCs) and Long Short-Term Memory (LSTM) Vibration Prediction AI Algorithms for Road Driving Vibrations.

This study developed an innovative active vibration canceling (AVC) system designed to mitigate non-periodic vibrations during road driving to enhance passenger comfort. The macro-fiber composite (MFC) used in the system is a smart material that is flexible, soft, lightweight, and applicable in many fields as a dual-purpose sensor and actuator. The target vibrations are road vibration data that were collected while driving on standard urban (Seoul) and highway roads at 40 km/s. To predict and cancel the target vibration accurately before passing it, we modeled the vibration prediction algorithm using a long short-term memory recurrent neural network (LSTM RNN). We regenerated vibrations on Seoul and highway roads at 40 km/s using MFCs and measured the displacements of the measured, predicted, and AVC vibrations of each road condition. To evaluate the vibration, we computed the root mean squared error (RMSE) and compared standard deviation (SD) values. The accuracies of LSTM RNN vibration prediction algorithms are 97.27% and 96.36% on Seoul roads and highway roads, respectively, at 40 km/s. Although the vibration ratio compared with the AVC results are different, there was no difference between the values of the AVC vibrations. According to a previous study and the principle of the AVC system, the target vibrations decrease by canceling the inverse vibration of the MFC actuator.

Adaptive QoE‐based video delivery by optimizing the bitrate of the video course using network bandwidth and user bandwidth

SummaryWith the increase in internet connectivity worldwide, people have resorted to various video streaming applications for entertainment, education, healthcare, and so on. However, this has resulted in huge internet traffic, which is predominantly occurring due to the increased transmission of videos over wireless networks to mobile devices, Transmission of ultra‐high‐definition videos can be effortlessly carried out, although the video quality perceived by the users is normally lesser than the anticipated quality. In video streaming applications, it is a pre‐requisite to provide users with high quality of experience (QoE) but this is generally affected by the dynamic variations in the network conditions. This work presents a novel QoE‐based video delivery system in a collaborative e‐learning platform. Here, a novel deep learning structure recurrent neural network‐long short term memory (RNN‐LSTM) is developed for adaptive bit rate (ABR) selection, thereby providing users with videos of high QoE. Various network, time, and buffer‐based parameters are considered while selecting the bit rate. Additionally, the proposed RNN‐LSTM is assessed for its superiority in providing high QoE videos based on measures, like QoE, buffer size, and bit rate, and is observed to attain values of 0.975, 45.654, and 90.766 b/s, respectively.

Elevating sustainability with a multi-renewable hydrogen generation system empowered by machine learning and multi-objective optimization

The global energy landscape is rapidly shifting toward cleaner, lower-carbon electricity generation, necessitating a transition to alternate energy sources. Hydrogen, particularly green hydrogen, looks to be a significant solution for facilitating this transformation, as it is produced by water electrolysis with renewable energy sources such as solar irradiations, wind speed, and biomass residuals. Traditional energy systems are costly and produce energy slowly due to unpredictability in resource supply. To address this challenge, this work provides a novel technique that integrates a multi-renewable energy system using multi objective optimization algorithm to meets the machine learning-based forecasted load model. Several forecasting models, including Autoregressive Integrated Moving Average (ARIMA), Random Forest and Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), are assessed for develop the statistical metrics values such as RMSE, MAE, and MAPE. The selected Non-Sorting Moth Flame Optimization (NSMFO) algorithm demonstrates technological prowess in efficiently achieving global optimization, particularly when handling multiple objective functions. This integrated method shows enormous promise in technological, economic, and environmental terms, emphasizing its ability to promote energy sustainability targets.

Prediction of Diffuse Attenuation Coefficient Based on Informer: A Case Study of Hangzhou Bay and Beibu Gulf

Marine water quality significantly impacts human livelihoods and production such as fisheries, aquaculture, and tourism. Satellite remote sensing facilitates the predictions of large-area marine water quality without the need for frequent field work and sampling. Prediction of diffuse attenuation coefficient (Kd), which describes the speed at which light decays as it travels through water, obtained from satellite-derived ocean color products can reflect the overall water quality trends. However, current models inadequately explore the complex nonlinear features of Kd, and there are difficulties in achieving accurate long-term predictions and optimal computational efficiency. This study innovatively proposes a model called Remote Sensing-Informer-based Kd Prediction (RSIKP). The proposed RSIKP is characterized by a distinctive Multi-head ProbSparse self-attention mechanism and generative decoding structure. It is designed to comprehensively and accurately capture the long-term variation characteristics of Kd in complex water environments while avoiding error accumulation, which has a significant advantage in multi-dataset experiments due to its high efficiency in long-term prediction. A multi-dataset experiment is conducted at different prediction steps, using 70 datasets corresponding to 70 study areas in Hangzhou Bay and Beibu Gulf. The results show that RSIKP outperforms the five prediction models based on Artificial Neural Networks (ANN, Convolutional Neural Networks (CNN), Gated Recurrent Unit (GRU), Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN), and Long Short-Term Memory Networks (LSTM)). RSIKP captures the complex influences on Kd more effectively to achieve higher prediction accuracy compared to other models. It shows a mean improvement of 20.6%, 31.1%, and 22.9% on Mean Absolute Error (MAE), Mean Square Error (MSE), and Mean Absolute Percentage Error (MAPE). Particularly notable is its outstanding performance in the long time-series predictions of 60 days. This study develops a cost-effective and accurate method of marine water quality prediction, providing an effective prediction tool for marine water quality management.

Daily scale air quality index forecasting using bidirectional recurrent neural networks: Case study of Delhi, India

This research was established to accurately forecast daily scale air quality index (AQI) which is an essential environmental index for decision-making. Researchers have projected different types of models and methodologies for AQI forecasting, such as statistical techniques, machine learning (ML), and most recently deep learning (DL) models. The modelling development was adopted for Delhi city, India which is a major city with air pollution issues simialir to entire urban cities of India especially during winter seasons. This research was predicted AQI using different versions of DL models including Long-Short Term Memory (LSTM), Bidirectional LSTM (Bi-LSTM) and Bidirectional Recurrent Neural Networks (Bi-RNN) in addition to Kernel Ridge Regression (KRR). Results indicated that Bi-RNN model consistently outperformed the other models in both training and testing phases, while the KRR model consistently displayed the weakest performance. The outstanding performance of the models development displayed the requirement of adequate data to train the models. The outcomes of the models showed that LSTM, BI-LSTM, KRR had lower performance compared with Bi-RNN models. Statistically, Bi-RNN model attained maximum cofficient of determination (R2 = 0.954) and minimum root mean square error (RMSE = 25.755). The proposed model in this research revealed the robust predictable to provide a valuable base for decision-making in the expansion of combined air pollution anticipation and control policies targeted at addressing composite air pollution problems in the Delhi city.

Using an interpretable deep learning model for the prediction of riverine suspended sediment load.

The prediction of suspended sediment load (SSL) within riverine systems is critical to understanding the watershed's hydrology. Therefore, the novelty of our research is developing an interpretable (explainable) model based on deep learning (DL) and Shapley Additive ExPlanations (SHAP) interpretation technique for prediction of SSL in the riverine systems. This paper investigates the abilities of four DL models, including dense deep neural networks (DDNN), long short-term memory (LSTM), gated recurrent unit (GRU), and simple recurrent neural network (RNN) models for the prediction of daily SSL using river discharge and rainfall data at a daily time scale in the Taleghan River watershed, northwestern Tehran, Iran. The performance of models was evaluated by using several quantitative and graphical criteria. The effect of parameter settings on the performance of deep models on SSL prediction was also investigated. The optimal optimization algorithms, maximum iteration (MI), and batch size (BC) were obtained for modeling daily SSL, and structure of the model impact on prediction remarkably. The comparison of prediction accuracy of the models illustrated that DDNN (with R2 = 0.96, RMSE = 333.46) outperformed LSTM (R2 = 0.75, RMSE = 786.20), GRU (R2 = 0.73, RMSE = 825.67), and simple RNN (R2 = 0.78, RMSE = 741.45). Furthermore, the Taylor diagram confirmed that DDNN has the highest performance among other models. Interpretation techniques can address the black-box nature of models, and here, SHAP was applied to develop an interpretable DL model to interpret of DL model's output. The results of SHAP showed that river discharge has the strongest impact on the model's output in estimating SSL. Overall, we conclude that DL models have great potential in watersheds to predict SSL. Therefore, different interpretation techniques as tools to interpret DL model's output (DL model is as black-box model) are recommended in future research.

State-of-charge estimation of lithium-ion battery: Joint long short-term memory network and adaptive extended Kalman filter online estimation algorithm

With the widespread use of lithium-ion batteries, estimating the State of Charge (SOC) has been one of the most critical tasks in the battery management system (BMS). In this paper, a joint long short-term memory recurrent neural network (LSTM-RNN) and adaptive extended Kalman filter (AEKF) algorithm is proposed to achieve accurate SOC estimation. The proposed algorithm generates an initial SOC estimation value through a trained LSTM-RNN with a sliding time window signal sequence as input. The initial estimation value is then used as a feedback input to the AEKF. The results show that only a 30-s time series as the input of the LSTM-RNN can achieve satisfactory estimation results. Compared with traditional adaptive extended Kalman filter and unscented Kalman filter (UKF), the proposed algorithm has a better performance of estimation accuracy under various discharging conditions, especially when applied at low temperatures. Finally, the SOC estimation results under four operating conditions and various temperatures show that the maximum error is less than 2.5 %, in particular, the root mean square error (RMSE) and the mean absolute error (MAE) are below 1.01 % and 0.64 %. The proposed algorithm is an effective SOC estimation method with good robustness and high estimation accuracy.

The Use of AI to Analyze Social Media Attacks for Predictive Analytics

Social engineering, on the other hand, presents weaknesses that are difficult to directly quantify in penetration testing. The majority of expert social engineers utilize phishing and adware tactics to convince victims to provide information voluntarily. Social Engineering (SE) in social media has a similar structural layout to regular postings but has a malevolent intrinsic purpose. Recurrent Neural Network-Long Short-Term Memory (RNN-LSTM) was used to train a novel SE model to recognize covert SE threats in communications on social networks. The dataset includes a variety of posts, including text, images, and videos. It was compiled over a period of several months and was carefully curated to ensure that it is representative of the types of content that is typically posted on social media. First, by using domain heuristics, the social engineering assaults detection (SEAD) pipeline is intended to weed out social posts with malevolent intent. After tokenizing each social media post into sentences, each post is examined using a sentiment analyzer to determine whether it is a training data normal or an abnormality. Subsequently, an RNN-LSTM model is trained to detect five categories of social engineering assaults, some of which may involve information-gathering signals. Comparing the experimental findings to the ground truth labeled by network experts, the SEA model achieved 0.82 classification precision and 0.79 recall.

Deep unrolling networks with recurrent momentum acceleration for nonlinear inverse problems

Combining the strengths of model-based iterative algorithms and data-driven deep learning solutions, deep unrolling networks (DuNets) have become a popular tool to solve inverse imaging problems. Although DuNets have been successfully applied to many linear inverse problems, their performance tends to be impaired by nonlinear problems. Inspired by momentum acceleration techniques that are often used in optimization algorithms, we propose a recurrent momentum acceleration (RMA) framework that uses a long short-term memory recurrent neural network (LSTM-RNN) to simulate the momentum acceleration process. The RMA module leverages the ability of the LSTM-RNN to learn and retain knowledge from the previous gradients. We apply RMA to two popular DuNets—the learned proximal gradient descent (LPGD) and the learned primal-dual (LPD) methods, resulting in LPGD-RMA and LPD-RMA, respectively. We provide experimental results on two nonlinear inverse problems: a nonlinear deconvolution problem, and an electrical impedance tomography problem with limited boundary measurements. In the first experiment we have observed that the improvement due to RMA largely increases with respect to the nonlinearity of the problem. The results of the second example further demonstrate that the RMA schemes can significantly improve the performance of DuNets in strongly ill-posed problems.

Enhancing high-speed digital systems: MVL circuit design with CNTFET and RRAM

Multiple Value Logic (MVL) circuits in high-speed digital systems can represent significantly more information than traditional binary logic circuits. The increasing demand for enhanced data storage capacity has driven the adoption of MVL circuits. However, designing MVL circuits presents challenges, particularly in achieving improved performance aligned with the desired threshold voltage. To address this challenge, harnessing carbon nanotube field-effect transistors (CNTFETs) has been instrumental in designing MVL circuits. CNTFETs offer distinct advantages due to their large resistors. Furthermore, substituting these resistors with Resistive Random Access Memory (RRAM) to implement Ternary logic circuits enhances storage capacity by enabling multiple resistance states within a single cell. This paper presents a novel MVL circuit design that leverages the unique characteristics of CNTFETs and RRAM. Additionally, gate implementation is explored using a Recurrent Neural Network − Long-Short Term Memory (RNN-LSTM) based Tasmanian Devil Optimization algorithm. The practical implementation uses Synopsis HSPICE software with CNTFET technology at 32 nm. A comprehensive performance analysis compares this approach with state-of-the-art solutions regarding delay, area, and power consumption. Significant reductions in area, power consumption, and transistor count are revealed with the proposed approach.

Forecasting the Future Value of a Colombian Investment Fund with LSTM Recurrent Neural Networks (LSTM)

Forecasting the Future Value of a Colombian Investment Fund with LSTM Recurrent Neural Networks (LSTM)

A hybrid self-adaptive DWT-WaveNet-LSTM deep learning architecture for karst spring forecasting

Karst spring discharge plays a vital role in understanding karst systems and managing karst groundwater resources. Due to its inherent heterogeneity and complexity, hydrological factors in the karst system often exhibit highly nonlinear and nonstationary characteristics. To overcome this challenge, a novel self-adaptive deep learning architecture is proposed to forecast daily karst spring runoff with precipitation in this study. The newly proposed model effectively integrates discrete wavelet transform (DWT), WaveNet, and long short-term memory (LSTM), enhanced by the attention mechanism and residual connections to learn underlying patterns, mitigate the overfitting risk, focus on the most relevant time steps, and improve the prediction accuracy. The framework is structured into two parts: one part is dedicated to predicting the karst spring discharge, whereas another part primarily predicts the residuals arising from the main predictions. By intergrading the two-part approach, the innovative model can adaptively correct the systematic errors and refine its understanding of some aspects that it might initially struggle with. The newly proposed model and three existing deep learning models including LSTM, GRU and simple RNN models are applied to predict karst spring discharge at Barton Springs, Texas, USA. The results imply that the newly developed DWT-WaveNet-LSTM model can leverage the advantages of various models, adapt to various aspects of the hydrometeorological data, and better capture intrinsic patterns from nonlinear and nonstationary input features. It outperforms other models at all time steps, evidenced by lower RMSE and higher NSE values. The residual analysis is conducted by comparing the predicted and observed spring discharge values. The self-adaptive DWT-WaveNet-LSTM model exhibits residuals that are closely centered around zero with a symmetrical, bell-shaped distribution, indicating superior performance and robust predictions in the karst system due to its effective capture of complex and nonlinear patterns.

Machine learning to predict grains futures prices

AbstractAccurate commodity price forecasts are crucial for stakeholders in agricultural supply chains. They support informed marketing decisions, risk management, and investment strategies. Machine learning methods have significant potential to provide accurate forecasts by maximizing out‐of‐sample accuracy. However, their inherent complexity makes it challenging to understand the appropriate data pre‐processing steps to ensure proper functionality. This study compares the forecasting performance of Long Short‐Term Memory Recurrent Neural Networks (LSTM‐RNNs) with classical econometric time series models for corn futures prices. The study considers various combinations of data pre‐processing techniques, variable clusters, and forecast horizons. Our results indicate that LSTM‐RNNs consistently outperform classical methods, particularly for longer forecast horizons. In particular, our findings demonstrate that LSTM‐RNNs are capable of automatically handling structural breaks, resulting in more accurate forecasts when trained on datasets that include such shocks. However, in our setting, LSTM‐RNNs struggle to deal with seasonality and trend components, necessitating specific data pre‐processing procedures for their removal.