Long Short-term Memory Output Research Articles

AI-Based System Automates Textual Classification of Daily Drilling Reports

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper OTC 32978, “Development and Implementation of an AI-Based System To Automate Textual Classification on Daily Drilling Reports,” by Stephan Perrout, Aliel F. Riente, and Guilherme S.F. Vanni, SPE, Petrobras, et al. The paper has not been peer reviewed. Copyright 2023 Offshore Technology Conference. Reproduced by permission. _ Structured daily drilling reports (DDRs) are a rich source of information that allows better planning, more-accurate risk analysis, and improved key performance indicators and contracts. However, such information is originally stored in a free-text and unstructured format, which becomes difficult for efficient data mining. With the advance of artificial intelligence (AI) technologies, particularly AI language models, applying such techniques over unstructured data has become critical to digital transformation. The complete paper presents an approach for automatic DDR classification that incorporates new techniques of AI. Introduction This work addresses the complex task of automatic classification of DDRs according to a newly proposed ontology. The ontology follows a hierarchical model that classifies actions into three or four levels depending on the intervention, considering drilling, completion, and abandonment. Each event has an ontology built and reviewed by experts in oil and gas. Classifying DDR constitutes a demanding task, and effectively exploiting AI-based models represents a promising solution. This work bridges the gap by proposing a classifier based on transformers along with recurrent neural networks (RNNs) to classify reported events described in unstructured text related to drilling, completion, and abandonment interventions. A large number of DDRs was used for training and validation of the proposed classifier, yielding promising results for key processes in the company. Neural-Network Techniques Bidirectional Long Short-Term Memory. Early neural-network models are characterized by inputs of fixed length. This is a drawback when working with texts, however, because sentences vary in their number of words. To overcome such an issue and to process data sequentially, RNNs were proposed. The RNNs are characterized by a set of parameters inherent from the early models plus an internal memory (a hidden or internal state) responsible for storing the context of the sequence being processed. Long short-term memory (LSTM) is a variation of RNN proposed to mitigate two problems: Information can be easily lost when processing very long sequences, and the gradient can become quite low because of the high number of mathematical operations performed during the processing while remaining far from reaching the threshold. LSTM consists of a set of parameters called the input gate, forget gate, and output gate that control information flow through the network. This set of additional parameters helps to maintain only what is important for the internal state of the network besides controlling the output. BiLSTM is a variant of LSTM that comprises two LSTMs. One processes texts from left to right, and the second one processes texts from right to left. This feature allows “future” elements to be part of the model’s decision process for “past” elements. The final classification is the combination of the output of both LSTMs.

Enhancing Solar Radiation Forecasting in Diverse Moroccan Climate Zones: A Comparative Study of Machine Learning Models with Sugeno Integral Aggregation

Hourly solar radiation (SR) forecasting is a vital stage in the efficient deployment of solar energy management systems. Single and hybrid machine learning (ML) models have been predominantly applied for precise hourly SR predictions based on the pattern recognition of historical heterogeneous weather data. However, the integration of ML models has not been fully investigated in terms of overcoming irregularities in weather data that may degrade the forecasting accuracy. This study investigated a strategy that highlights interactions that may exist between aggregated prediction values. In the first investigation stage, a comparative analysis was conducted utilizing three different ML models including support vector machine (SVM) regression, long short-term memory (LSTM), and multilayer artificial neural networks (MLANN) to provide insights into their relative strengths and weaknesses for SR forecasting. The comparison showed the proposed LSTM model had the greatest contribution to the overall prediction of six different SR profiles from numerous sites in Morocco. To validate the stability of the proposed LSTM, Taylor diagrams, violin plots, and Kruskal–Wallis (KW) tests were also utilized to determine the robustness of the model’s performance. Secondly, the analysis found coupling the models outputs with aggregation techniques can significantly improve the forecasting accuracy. Accordingly, a novel aggerated model that integrates the forecasting outputs of LSTM, SVM, MLANN with Sugeno λ-measure and Sugeno integral named (SLSM) was proposed. The proposed SLSM provides spatially and temporary interactions of information that are characterized by uncertainty, emphasizing the importance of the aggregation function in mitigating irregularities associated with SR data and achieving an hourly time scale forecasting accuracy with improvement of 11.7 W/m2.

Short-term wind power forecasting by bidirectional attention mechanism LSTM and its probability interval prediction by sliding-window KDE

Deterministic point wind power forecasting (DP-WPF) and its probability interval prediction (PIP) are indispensable to short-term peak alleviation and frequency regulation in power systems with large-scale wind power injection. To improve short-term DP-WPF by long short-term memory (LSTM), a horizontal/vertical bidirectional feature attention (BFA) based LSTM model is proposed. More specifically, the BFA-LSTM model has three parts: first, multivariate time series are fed into LSTM to extract long-short-term temporal features; second, the LSTM outputs are processed horizontally as well as vertically for retrieving step-wise/multistep-wise temporal features, respectively, namely, in the bidirectional attention sense; third, both the horizontal and vertical attention weights are adaptively adjusted according to the feature importance in DP-WPF. Cases comparison shows that the suggested modeling is stably superior to most common counterparts. To address PIP by kernel density estimation (KDE), sliding-window KDE is leveraged for probability analysis. More precisely, probability density functions (PDF) and probability intervals are estimated with sliding-window samples, which are non-parametric operations and involve finitely many local samples. Superior performances of PIP by sliding-window KDE at different confidence levels indicate that the sliding-window PDF approach is highly effective in contrast to those with all samples.

Short-term power load forecasting for combined heat and power using CNN-LSTM enhanced by attention mechanism

This study proposes a new approach for short-term power load forecasting using a combination of convolutional neural networks (CNN), long short-term memory (LSTM), and attention mechanisms to address the issue of information loss due to excessively long input time series data. The objective is to enhance the accuracy of short-term power load prediction, which is crucial for efficient energy management. The study analyzes the relationship between the target load and the collected parameters, identifying the most influential factors using Pearson correlation coefficient analysis. A one-dimensional CNN layer is utilized to extract high-dimensional features from the input data, followed by an LSTM layer that captures temporal correlations within the historical sequences. Finally, an attention mechanism is introduced to optimize the weight of the LSTM output, enhance the influence of key information, and optimize the overall prediction model. The performance of the proposed model is evaluated using two benchmark models based on mean absolute percentage error (MAPE), root mean square error (RMSE), and mean absolute error (MAE) metrics. The results show that the CNN-LSTM-A model outperforms the traditional LSTM model regarding power load prediction accuracy for two thermal power units, with an improvement of 7.3% and 5.7%, respectively, indicating superior performance. Therefore, this study demonstrates the effectiveness of the proposed CNN-LSTM-A model for short-term power load forecasting, which has potential applications in the energy industry. In conclusion, the proposed approach can improve the accuracy of power load forecasting, leading to more efficient energy management and cost savings. Additionally, the study highlights the importance of incorporating attention mechanisms into traditional LSTM models for power load forecasting, as it helps to optimize the weight of the LSTM output and improve the accuracy of the predictions. The proposed CNN-LSTM-A model can be potentially useful for energy companies and policymakers in making informed decisions regarding energy production and consumption. Overall, this study provides a valuable contribution to power load forecasting, and the proposed approach could be extended to other areas of time-series forecasting in the future.

Digital Twin-Driven Remaining Useful Life Prediction for Rolling Element Bearing

Traditional methods for predicting remaining useful life (RUL) ignore the correlation between physical world data and virtual world data, leading to the low prediction accuracy of RUL and affecting the normal working of rolling element bearing (REB). To solve the above problem, we propose a hybrid method based on digital twin (DT) and long short-term memory (LSTM). The hybrid method combines the high simulation capabilities of DT and the strong data processing capabilities of LSTM. Firstly, we develop a DT system for the life characteristics analysis of an REB. When the DT system is implemented, we can obtain the theoretical value of RUL. Then, the experimental data is used to train the LSTM model. The output of LSTM is the actual value of RUL. Finally, the particle swarm optimization (PSO) algorithm fuses the theoretical values of DT with the actual values of LSTM. The case study demonstrates that the prediction accuracy of the hybrid method is greater than 97.5%, which improves the prediction performance and robustness of RUL. Therefore, the hybrid method is an important technology of REB prediction and health management (PHM). It realizes the early intervention and maintenance of mechanical equipment and ensures the safety of enterprises’ production.

An LSTM and ANN Fusion Dynamic Model of a Proton Exchange Membrane Fuel Cell

Proton exchange membrane fuel cell (PEMFC) has great application prospects due to its low emission and high efficiency. An accurate model to predict the dynamic output voltage is essential for the optimal control of PEMFC for the applications on vehicles and power stations. In this paper, a novel deep learning framework with the Long short-term memory (LSTM) and ANN Fusion is proposed to develop the PEMFC dynamic model by extracting both the historical and current information. The LSTM extracts the temporal information from the past PEMFC states with its order determined with autocorrelation function (ACF) and partial autocorrelation (PACF), while the influence of the system current inputs is learnt by the ANN. Then the outputs of LSTM and ANN are concatenated with the multiple information fused to predict the PEMFC dynamic output voltage. After validated by the operating data from a lab-scale PEMFC system, the LSTM and ANN Fusion model is compared with existing models, such as SVR, ANN and LSTM methods. The comparison results show that the proposed LSTM and ANN Fusion model can provide the best prediction performance with the lowest Mean Square Error (MSE) of 1.303. The proposed LSTM and ANN Fusion model can be helpful to develop the optimal control strategy of PEMFC.

A hybrid model for metro passengers flow prediction

In this paper, a novel ensemble learning model named EWT-EnsemLSTM-SSA, which assembles long short-term memory (LSTM), support vector regression (SVR), and sparrow search algorithm (SSA), is a proposed to deal with long term metro passenger flow volume prediction, which is an essential content of traffic flow prediction problems. Firstly, the empirical wavelet transform (EWT) method is introduced to decompose the original dataset into five wavelet time-sequence data for further prediction. Then, a cluster of LSTMs with diverse hidden layers and neuron counts are employed to explore and exploit the implicit information of the EWT-decomposed signals. Next, the output of LSTMs is aggregated into a nonlinear regression method SVR. Lastly, SSA is utilized to optimize the SVR automatically. The proposed EWT-EnsemLSTM-SSA model is applied in three case studies, using the data collected from the passengers’ amount in the Minneapolis, America metro, divided into one hour in one day. Experiment results, which compare the proposed EnsemLSTM-SSA model with five conventional time series forecasting models, show that the proposed model can achieve a better performance than the traditional prediction algorithms.

Projection of climate variables by general circulation and deep learning model for Lahore, Pakistan

Climate change significantly impacts the hydrological cycle and environment. The key parameters driving climate change for densely populated city of Lahore in Pakistan were studied. The projections of these parameters were evaluated using General Circulation Model (GCM) named Community Climate System Model (CCSM4) under two Representative Concentration Pathways (RCP 4.5 and 8.5) scenarios. The outputs of CCSM4 model were bias corrected using quantile mapping using historical data. Additionally, a deep learning model named long short-term memory (LSTM) was developed applying machine learning applications to forecast the climate parameters for the future. LSTM model with two LSTM layers including one fully connected layer was modeled for the projection of climate variables in the region. Total number of parameters were 9888, and the input and forecasted output length was kept as 24 sequential months without overlapping. The conventional projection methods of GCM were compared with LSTM outputs for bridging the gap. The LSTM model was found to be more effective and dependable in forecasting the climate with significant improvement in the statistical parameters for the region. The LSTM model can be applied for projections of climate in comparison to GCM with sufficient precision.

Lipoprotein detection: Hybrid deep classification model with improved feature set

Patients with chronic liver diseases typically experience lipid profile problems, and mortality from cirrhosis complicated by portal vein thrombosis (PVT) is very significant. A lipoprotein (Lp) is a bio-chemical assemblage with the main job of moving fat molecules in water that are hydrophobic. Lipoproteins are present in all eubacterial walls. Lipoproteins are of tremendous interest in the study of spirochaetes’ pathogenic mechanisms. Since spirochaete lipobox sequences are more malleable than other bacteria, it’s proven difficult to apply current prediction methods to new sequence data. The major goal is to present a Lipoprotein detection model in which correlation features, enhanced log energy entropy, raw features, and semantic similarity features are extracted. These extracted characteristics are put through a hybrid model that combines a Gated Recurrent Unit (GRU) and a Long Short-Term Memory (LSTM). Then, the outputs of GRU and LSTM are averaged to obtain the output. Here, GRU weights are optimized via the Selfish combined Henry Gas Solubility Optimization with cubic map initialization (SHGSO) model.

Ionospheric TEC Forecasting over an Indian Low Latitude Location Using Long Short-Term Memory (LSTM) Deep Learning Network

The forecasting of ionospheric electron density has been of great interest to the research scientists and engineers’ community as it significantly influences satellite-based navigation, positioning, and communication applications under the influence of space weather. Hence, the present paper adopts a long short-term memory (LSTM) deep learning network model to forecast the ionospheric total electron content (TEC) by exploiting global positioning system (GPS) observables, at a low latitude Indian location in Bangalore (IISC; Geographic 13.03° N and 77.57° E), during the 24th solar cycle. The proposed model uses about eight years of GPS-TEC data (from 2009 to 2017) for training and validation, whereas the data for 2018 was used for independent testing and forecasting of TEC. Apart from the input TEC parameters, the model considers sequential data of solar and geophysical indices to realize the effects. The performance of the model is evaluated by comparing the forecasted TEC values with the observed and global empirical ionosphere model (international reference ionosphere; IRI-2016) through a set of validation metrics. The analysis of the results during the test period showed that LSTM output closely followed the observed GPS-TEC data with a relatively minimal root mean square error (RMSE) of 1.6149 and the highest correlation coefficient (CC) of 0.992, as compared to IRI-2016. Furthermore, the day-to-day performance of LSTM was validated during the year 2018, inferring that the proposed model outcomes are significantly better than IRI-2016 at the considered location. Implementation of the model at other latitudinal locations of the region is suggested for an efficient regional forecast of TEC across the Indian region. The present work complements efforts towards establishing an efficient regional forecasting system for indices of ionospheric delays and irregularities, which are responsible for degrading static, as well as dynamic, space-based navigation system performances.

Word recognition and automated epenthesis removal for Indonesian sign system sentence gestures

This research <span>focuses on building a system to translate continuous Indonesian sign system (SIBI) gestures into text. In a continuous gesture, a signer will add an epenthesis (transitional) gesture, which is hand movement with no meaning but needed to connect the hand movement of one word with the next word in a continuous gesture. Reducing the number of irrelevant inputs to the model through automated epenthesis removal can improve the system's ability to recognize the words in continuous gestures. We implemented threshold conditional random fields (TCRF) to identify epenthesis gestures. The dataset consists of 2,255 videos representing 28 common sentences in SIBI. The translation system consists of MobileNetV2 as a feature extraction technique, removing epenthesis gestures found by the TCRF, and a long short-term memory (LSTM) for the classifier. With the MobileNetV2-TCRF-bidirectional LSTM model, the best word error rate (WER) and sentence accuracy (SAcc) were 33.4% and 16.2%, respectively. Intermediate-stage processing steps consisting of sandwiched majority voting of the TCRF and the removal of word labels whose number of frames is less than two frames, along with LSTM output grouping, were able to reduce WER from 33.4% to 3.4% and increase SAcc from 16.2% to 80.2%.</span>

Research on Impulse Power Load Forecasting Based on Improved Recurrent Neural Networks

Deep learning is good at extracting the required feature quantity from the massive input information through multiple hidden layers and completing the learning through training to achieve the task of load forecasting. The impulse power load data contain a lot of noise, burrs, and strong randomness. As an improved recurrent neural networks, the output of long short-term memory (LSTM) network is not only related to the current input, but also closely related to the historical information, which can effectively predict the impact power load. An impulse power load forecasting model based on improved recurrent neural networks is proposed. To solve the training difficulties caused by deep networks, database is divided into training data set and test data set. To accelerate running speed and improve accuracy and reliability, parameter setting in deep learning neural network is analyzed. The proposed load forecasting model is verified by simulation and compared with the existing methods. Taking the average relative error as the standard, the effectiveness of the proposed model for the forecasting of impulse power load connected to the bus is verified.

Deep Learning for seasonality modelling in Inflation-Indexed Swap pricing

An Inflation-Indexed Swap (IIS) is a derivative in which, at every payment date, the counterparties swap an inflation rate with a fixed rate. For the calculation of the Inflation Leg cash flows it is necessary to build a mathematical model suitable for the Consumer Price Index (CPI) projection. For this purpose, quants typically start by using market quotes for the Zero-Coupon swaps in order to derive the future trend of the inflation index, together with a seasonality model for capturing the typical periodical effects. In this study, we propose a forecasting model for inflation seasonality based on a Long Short Term Memory (LSTM) network: a deep learning methodology particularly useful for forecasting purposes. The CPI predictions are conducted using a FinTech paradigm, but in respect of the traditional quantitative finance theory developed in this research field. The paper is structured according to the following sections: the first two parts illustrate the pricing methodologies for the most popular IIS: the Zero Coupon Inflation-Indexed Swap (ZCIIS) and the Year-on-Year Inflation-Indexed Swap (YYIIS); section 3 deals with the traditional standard method for the forecast of CPI values (trend + seasonality), while section 4 describes the LSTM architecture, and section 5 focuses on CPI projections, also called inflation bootstrap. Then section 6 describes a robust check, implementing a traditional SARIMA model in order to improve the interpretation of the LSTM outputs; finally, section 7 concludes with a real market case, where the two methodologies are used for computing the fair-value for a YYIIS and the model risk is quantified.

Management of environmental streaming data to optimize Arctic shipping routes

Dynamic accurate predictions of Arctic sea ice, ocean, atmosphere, and ecosystem are necessary for safe and efficient Arctic maritime transportation; however a related technical roadmap has not yet been established. In this paper, we propose a management system for trans-Arctic maritime transportation supported by near real-time streaming data from air-space-ground-sea integrated monitoring networks and high spatio-temporal sea ice modeling. As the core algorithm of integrated monitoring networks, a long short-term memory (LSTM) neural network is embedded to improve Arctic sea ice mapping algorithms. Since the LSTM is localized in time and space, it can make full use of streaming data characteristics. The sea ice–related parameters from satellite remote sensing raw data are used as the input of the LSTM, while streaming data from shipborne radar networks and/or buoy measurements are used as training datasets to enhance the accuracy and resolution of environmental streaming data from outputs of LSTM. Due to large size of streaming data, the proposed management system of trans-Arctic shipping should be built on a cloud distribution platform using existing wireless communications networks among vessels and ports. Our management system will be used by the ongoing European Commission Horizon 2020 Programme “ePIcenter.”

Feature Mapping and Deep Long Short Term Memory Network-Based Efficient Approach for Parkinson’s Disease Diagnosis

In this paper, a novel approach was developed for Parkinson’s disease (PD) diagnosis based on speech disorders. When the literature about the speech disorders-based PD diagnosis was reviewed, it was seen that the most of approaches were concentrated on the feature selection as the datasets contained a huge number of features. In contrast, in the proposed approach, instead of eliminating some of the features by using any feature selection method, all features were initially used for forming a mapping procedure where the input feature vectors were converted to the input images. Then, a deep Long Short Term Memory (LSTM) network was employed for PD detection where the obtained images were used. The deep LSTM network carried out both feature extraction and classification processes and its training was carried out in an end-to-end fashion. The activations in the convolutional layer were converted to sequence data through the sequence-folding and sequence-unfolding layers. The activations in the LSTM output with learning parameters were conveyed to the Softmax layer for the classification process. A publically available PD dataset was used in the experimental works and classification accuracy, sensitivity, specificity, precision, and F-score metrics were used for performance evaluation. The obtained accuracy, sensitivity, specificity, precision and F-score values were 94.27%, 0.960, 0.960, 0.910 and 0.930, respectively. The obtained results were also compared with some of the published results and it had seen that most of the achievements of the proposed method are better than the compared methods.

Adaptive Attention-based High-level Semantic Introduction for Image Caption

There have been several attempts to integrate a spatial visual attention mechanism into an image caption model and introduce semantic concepts as the guidance of image caption generation. High-level semantic information consists of the abstractedness and generality indication of an image, which is beneficial to improve the model performance. However, the high-level information is always static representation without considering the salient elements. Therefore, a semantic attention mechanism is used for the high-level information instead of conventional of static representation in this article. The salient high-level semantic information can be considered as redundant semantic information for image caption generation. Additionally, the generation of visual words and non-visual words can be separated, and an adaptive attention mechanism is employed to realize the guidance information of image caption generation switching between new fusion information (fusion of image feature and high-level semantics) and a language model. Therefore, visual words can be generated according to the image features and high-level semantic information, and non-visual words can be predicted by the language model. The semantics attention, adaptive attention, and previous generated words are fused to construct a special attention module for the input and output of long short-term memory. An image caption can be generated as a concise sentence on the basis of accurately grasping the rich content of the image. The experimental results show that the performance of the proposed model is promising for the evaluation metrics, and the captions can achieve logical and rich descriptions.

Glaucoma Detection from Raw SD-OCT Volumes: A Novel Approach Focused on Spatial Dependencies

Background and objective:Glaucoma is the leading cause of blindness worldwide. Many studies based on fundus image and optical coherence tomography (OCT) imaging have been developed in the literature to help ophthalmologists through artificial-intelligence techniques. Currently, 3D spectral-domain optical coherence tomography (SD-OCT) samples have become more important since they could enclose promising information for glaucoma detection.To analyse the hidden knowledge of the 3D scans for glaucoma detection, we have proposed, for the first time, a deep-learning methodology based on leveraging the spatial dependencies of the features extracted from the B-scans.Methods:The experiments were performed on a database composed of 176 healthy and 144 glaucomatous SD-OCT volumes centred on the optic nerve head (ONH). The proposed methodology consists of two well-differentiated training stages: a slide-level feature extractor and a volume-based predictive model. The slide-level discriminator is characterised by two new, residual and attention, convolutional modules which are combined via skip-connections with other fine-tuned architectures. Regarding the second stage, we first carried out a data-volume conditioning before extracting the features from the slides of the SD-OCT volumes. Then, Long Short-Term Memory (LSTM) networks were used to combine the recurrent dependencies embedded in the latent space to provide a holistic feature vector, which was generated by the proposed sequential-weighting module (SWM).Results:The feature extractor reports AUC values higher than 0.93 both in the primary and external test sets. Otherwise, the proposed end-to-end system based on a combination of CNN and LSTM networks achieves an AUC of 0.8847 in the prediction stage, which outperforms other state-of-the-art approaches intended for glaucoma detection. Additionally, Class Activation Maps (CAMs) were computed to highlight the most interesting regions per B-scan when discerning between healthy and glaucomatous eyes from raw SD-OCT volumes.Conclusions:The proposed model is able to extract the features from the B-scans of the volumes and combine the information of the latent space to perform a volume-level glaucoma prediction. Our model, which combines residual and attention blocks with a sequential weighting module to refine the LSTM outputs, surpass the results achieved from current state-of-the-art methods focused on 3D deep-learning architectures.

Exploring a rich spatial–temporal dependent relational model for skeleton-based action recognition by bidirectional LSTM-CNN

With the fast development of effective and low-cost human skeleton capture systems, skeleton-based action recognition has attracted much attention recently. Most existing methods using Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM) have achieved promising performance for skeleton-based action recognition. However, these approaches are limited in the ability to explore the rich spatial–temporal relational information. In this paper, we propose a new spatial–temporal model with an end-to-end bidirectional LSTM-CNN (BiLSTM-CNN). First, a hierarchical spatial–temporal dependent relational model is used to explore rich spatial–temporal information in the skeleton data. Then a new framework is proposed to fuse CNN and LSTM. In this framework, the skeleton data are built by the dependent relational model and serve as the input of the proposed network. Then LSTM is used to extract the temporal features, and followed by a standard CNN to explore the spatial information from the output of LSTM. Finally, the experimental results demonstrate the effectiveness of the proposed model on the NTU RGB+D, SBU Interaction and UTD-MHAD dataset.

A novel long short-term memory neural-network-based self-excited force model of limit cycle oscillations of nonlinear flutter for various aerodynamic configurations

Due to the strong nonlinear properties of the entire flutter process [including growth stages, decay stages and steady limit cycle oscillations (LCOs)], the generalization of the self-excited force model for the entire flutter process of bluff bodies is a very critical issue. This paper proposes a self-excited force model based on a novel long short-term memory (LSTM) neural network to simulate the entire flutter process for various leading-edge configurations. To obtain the dataset, the nonlinear flutter with the growth stages, decay stages and steady LCOs for different leading-edge aerodynamic configurations is investigated by a series of two-degrees-of-freedom spring-suspended sectional model tests. Based on the measured flutter responses and self-excited forces in the tests, a data-driven self-excited force model is proposed on the basis of a novel LSTM neural network. Considering that the nonlinear dynamic system is very sensitive to the accuracy of the model parameters, the Newmark-$$\beta $$ method is incorporated into the LSTM networks and forms a closed-loop process to restrain the error amplification effects and improve the model robustness, that is, the response calculated from the output of LSTM (the self-excited force) by the Newmark-$$\beta $$ method is set as the input of LSTM at the next step rather than the measured response. To validate the performance of the proposed model, the flutter critical wind speed, time history of oscillation and self-excited forces, steady-state oscillation amplitude and flutter development process predicted by the model are compared with the test results. The comparison indicates that the predicted results agree well with the test results, meaning that the proposed model has high accuracy, generalization and robustness in describing the nonlinear characteristics of the flutter for various aerodynamic configurations.

A CNN–LSTM Method for the Morphology Evolution Prediction of Beach Mega-Nourishment

Sand nourishment is widely adopted as an effective soft approach to provide long-term coastal safety, protect the ecology environment, and promote tourism and recreation. With the increase in frequency and expenses in beach nourishment worldwide, an adequate prediction of morphology evolution is greatly desired for coastline management. Based on detailed monitoring data of the mega-nourishment Sand Engine, this article integrates a convolutional neural network (CNN) and long short-term memory (LSTM) to predict the nourishment morphology evolution. The historical surveyed data are transformed into sequence grids, which are input into the CNN to obtain the spatial features of beach nourishment. The CNN is constructed by performing the convolutional and pooling operation on the historical data, which can extract actual spatial features and reduce network complexity. The output of the CNN is input to LSTM to learn the temporal relationship to predict future nourishment terrain using past time-series features. Finally, the LSTM output is decoded by the fully connected layer to obtain the prediction result. The complex spatiotemporal correlations among the input data are identified through effective training of the proposed model. The major contribution of this article is to propose a data-driven model that combines CNN and LSTM for the morphology evolution prediction of beach nourishment, and validate the effectiveness of the proposed model by comparing with the performances of other popular methods in predicting the nourishment changes.