Federated LSTM-based deep learning model for privacy-preserving predictions of human trajectories across multiple data providers

Joint prediction of next location and travel time from urban vehicle trajectories using long short-term memory neural networks

Water inrush and mud burst disasters pose severe challenges to the safe and efficient construction of underground engineering. Water inflow prediction is closely related to drainage design, disaster prevention and control, and the safety of the surrounding ecological environment. Thus, assessing the water inflow accurately is of importance. This study proposes a Bayesian Optimization-Long Short-Term Memory (BOA-LSTM) recurrent neural network for predicting tunnel water inflow. The model is based on four input parameters, namely tunnel depth (H), groundwater level (h), rock quality designation (RQD), and water-richness (W), with water inflow (WI) as the single-output variable. The model first processes and analyzes the data, quantitatively characterizing the correlations between input parameters. The tunnel water inflow is predicted using the long short-term memory (LSTM) recurrent neural network, and the Bayesian optimization algorithm (BOA) is employed to select the hyperparameters of the LSTM, primarily including the number of hidden layer units, initial learning rate, and L2 regularization coefficient. The modeling process incorporates a five-fold cross-validation strategy for dataset partitioning, which effectively mitigates overfitting risks and enhances the model’s generalization capability. After a comprehensive comparison among a series of machine learning models, including a long short-term memory recurrent neural network (LSTM), random forest (RF), back propagation neural network (BP), extreme learning machine (ELM), radial basis function neural network (RBFNN), least squares support vector machine (LIBSVM), and convolutional neural network (CNN), BOA-LSTM performed excellently. The proposed BOA-LSTM model substantially surpasses the standard LSTM and other comparative models in tunnel water inflow prediction, demonstrating superior performance in both accuracy and generalization. Hence, it provides a reference basis for tunnel engineering water inflow prediction.

The safety of tunneling with shield tunnel boring machines largely depends on the tunnel face pressure, which is currently decided by human operators empirically. Face pressure control is vulnerable to human misjudgment and human errors can cause severe consequences, especially in difficult ground conditions. From a practical perspective, it is therefore beneficial to have a model capable of predicting the tunnel face pressure given operation and the changing geology. In this paper, we propose such a model based on deep learning. More specifically, a long short-term memory (LSTM) recurrent neural network is employed for tunnel face pressure prediction. To correlate with PLC data, linear interpolation is employed to transform the borehole geological data into sequential geological data according to the shield machine position. The slurry pressure in the excavation chamber (SPE) is taken as the output in the case study of Nanning Metro, which is confronted with the clogging problem due to the mixed ground of mudstone and round gravel. The LSTM-based SPE prediction model achieved an overall MAPE and RMSE of 3.83% and 10.3 kPa, respectively, in mudstone rich ground conditions. Factors that influence the model, including different kinds and length of input data and comparison with the traditional machine learning-based model, are also discussed.

This article explores the suitability of a long short-term memory recurrent neural network (LSTM-RNN) and artificial intelligence (AI) method for low-flow time series forecasting. The long short-term memory works on the sequential framework which considers all of the predecessor data. This forecasting method used daily discharged data collected from the Basantapur gauging station located on the Mahanadi River basin, India. Different metrics [root-mean-square error (RMSE), Nash–Sutcliffe efficiency (ENS), correlation coefficient (R) and mean absolute error] were selected to assess the performance of the model. Additionally, recurrent neural network (RNN) model is also used to compare the adaptability of LSTM-RNN over RNN and naive method. The results conclude that the LSTM-RNN model (R = 0.943, ENS = 0.878, RMSE = 0.487) outperformed RNN model (R = 0.935, ENS = 0.843, RMSE = 0.516) and naive method (R = 0.866, ENS = 0.704, RMSE = 0.793). The finding of this research concludes that LSTM-RNN can be used as new reliable AI technique for low-flow forecasting.

Recurrent neural networks (RNN) are at the core of modern automatic speech recognition (ASR) systems. In particular, long short-term memory (LSTM) recurrent neural networks have achieved state-of-the-art results in many speech recognition tasks, due to their efficient representation of long and short term dependencies in sequences of inter-dependent features. Nonetheless, internal dependencies within the element composing multidimensional features are weakly considered by traditional real-valued representations. We propose a novel quaternion long short-term memory (QL-STM) recurrent neural network that takes into account both the external relations between the features composing a sequence, and these internal latent structural dependencies with the quaternion algebra. QLSTMs are compared to LSTMs during a memory copy-task and a realistic application of speech recognition on the Wall Street Journal (WSJ) dataset. QLSTM reaches better performances during the two experiments with up to 2.8 times less learning parameters, leading to a more expressive representation of the information.

With the development of China's Belt and Road Initiative (BRI), the port plays a significant role and its operation management faces some pressure. In this regard, prediction of daily container volumes will provide the manager with data support for better plan of a storage yard. In this work, by deep learning the historical dataset, the long short-term memory (LSTM) recurrent neural network (RNN) is trained and used to predict daily volumes of containers which will enter the storage yard. The raw dataset of a certain port from 2013 to 2016 is chosen as the training set and the dataset of 2017 is used as the test set to evaluate the performance of the proposed prediction model. Then the LSTM model is established with Python and Tensorflow framework. The structure parameters are adjusted to find the optimal LSTM network, so as to improve the prediction accuracy. It appears that the LSTM model with two hidden layers and 30 hidden layer units has less prediction error between the real data and predicted data of 2017. The prediction error of daily container volumes between predicted value and real data of 2017 is about 12.39%, which is less than the people-predicted error. It is promising that the proposed LSTM RNN model can be applied to predict the daily volumes of containers and have higher prediction accuracy.

Grapheme-to-phoneme (G2P) models are key components in speech recognition and text-to-speech systems as they describe how words are pronounced. We propose a G2P model based on a Long Short-Term Memory (LSTM) recurrent neural network (RNN). In contrast to traditional joint-sequence based G2P approaches, LSTMs have the flexibility of taking into consideration the full context of graphemes and transform the problem from a series of grapheme-to-phoneme conversions to a word-to-pronunciation conversion. Training joint-sequence based G2P require explicit grapheme-to-phoneme alignments which are not straightforward since graphemes and phonemes don't correspond one-to-one. The LSTM based approach forgoes the need for such explicit alignments. We experiment with unidirectional LSTM (ULSTM) with different kinds of output delays and deep bidirectional LSTM (DBLSTM) with a connectionist temporal classification (CTC) layer. The DBLSTM-CTC model achieves a word error rate (WER) of 25.8% on the public CMU dataset for US English. Combining the DBLSTM-CTC model with a joint n-gram model results in a WER of 21.3%, which is a 9% relative improvement compared to the previous best WER of 23.4% from a hybrid system.

In the conventional frame feature based music genre classification methods, the audio data is represented by independent frames and the sequential nature of audio is totally ignored. If the sequential knowledge is well modeled and combined, the classification performance can be significantly improved. The long short-term memory(LSTM) recurrent neural network (RNN) which uses a set of special memory cells to model for long-range feature sequence, has been successfully used for many sequence labeling and sequence prediction tasks. In this paper, we propose the LSTM RNN based segment features for music genre classification. The LSTM RNN is used to learn the representation of LSTM frame feature. The segment features are the statistics of frame features in each segment. Furthermore, the LSTM segment feature is combined with the segment representation of initial frame feature to obtain the fusional segment feature. The evaluation on ISMIR database show that the LSTM segment feature performs better than the frame feature. Overall, the fusional segment feature achieves 89.71% classification accuracy, about 4.19% improvement over the baseline model using deep neural network (DNN). This significant improvement show the effectiveness of the proposed segment feature.

The effective forecast of container volumes can provide decision support for port scheduling and operating. In this work, by deep learning the historical dataset, the long short-term memory (LSTM) recurrent neural network (RNN) is used to predict daily volumes of containers which will enter the storage yard. The raw dataset of daily container volumes in a certain port is chosen as the training set and preprocessed with box plot. Then the LSTM model is established with Python and Tensorflow framework. The comparison between LSTM and other prediction methods like ARIMA model and BP neural network is also provided in this study, and the prediction gap of LSTM is lower than other methods. It is promising that the proposed LSTM is helpful to predict the daily volumes of containers.

Automatic language identification (LID) belongs to the automatic process whereby the identity of the language spoken in a speech sample can be distinguished. In recent decades, LID has made significant advancement in spoken language identification which received an advantage from technological achievements in related areas, such as signal processing, pattern recognition, machine learning and neural networks. This work investigates the employment of Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) for automatic language identification. The main reason of applying LSTM RNNs to the current task is their reasonable capacity in handling sequences. This study shows that LSTM RNNs can efficiently take advantage of temporal dependencies in acoustic data in order to learn relevant features for language recognition tasks. In this paper, we show results for conducted language identification experiments for Kazakh and Russian languages and the presented LSTM RNN model can deal with short utterances (2s). The model was trained using open-source high-level neural networks API Keras on limited computational resources.

We claim that modelling network traffic as a time series with a supervised learning approach, using known genuine and malicious behaviour, improves intrusion detection. To substantiate this, we trained long short-term memory (LSTM) recurrent neural networks with the training data provided by the DARPA / KDD Cup ’99 challenge. To identify suitable LSTM-RNN network parameters and structure we experimented with various network topologies. We found networks with four memory blocks containing two cells each offer a good compromise between computational cost and detection performance. We applied forget gates and shortcut connections respectively. A learning rate of 0.1 and up to 1,000 epochs showed good results. We tested the performance on all features and on extracted minimal feature sets respectively. We evaluated different feature sets for the detection of all attacks within one network and also to train networks specialised on individual attack classes. Our results show that the LSTM classifier provides superior performance in comparison to results previously published results of strong static classifiers. With 93.82% accuracy and 22.13 cost, LSTM outperforms the winning entries of the KDD Cup ’99 challenge by far. This is due to the fact that LSTM learns to look back in time and correlate consecutive connection records. For the first time ever, we have demonstrated the usefulness of LSTM networks to intrusion detection.

This work focuses on the use of electroencephalogram (EEG) signals to classify four human emotions, i.e., amused, disgust, sad, and scared that are elicited by custom-made video clips. The proposed model uses the independent component analysis (ICA) for artifact removal, band power and Hjorth parameters for feature extraction, and neighborhood component analysis (NCA) and minimum redundancy maximum relevance (mRMR) for feature selection. These computational techniques are combined because when individually used, they tend to give better accuracy results. However, they are not jointly used in many EEG-based emotion studies. A comparison has been made on the results obtained from six machine learning models, namely, decision trees, support vector machines, k-nearest neighbors, naive Bayes, random forest, and long short-term memory (LSTM) recurrent neural network (RNN). The highest accuracy attained in this study is 99.1% that used long short-term memory recurrent neural network as a machine learning model, a combined NCA and mRMR for feature selection, and a combined band power and Hjorth parameters for feature extraction.

This work examines the use of ant colony optimization (ACO) to improve long short-term memory (LSTM) recurrent neural networks (RNNs) by refining their cellular structure. The evolved networks were trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. Results were obtained using MPI (Message Passing Interface) on a high performance computing (HPC) cluster, which evolved 1000 different LSTM cell structures using 208 cores over 5 days. The new evolved LSTM cells showed an improvement in prediction accuracy of 1.37%, reducing the mean prediction error from 6.38% to 5.01% when predicting excessive engine vibrations 10 seconds in the future, while at the same time dramatically reducing the number of trainable weights from 21,170 to 11,650. The ACO optimized LSTM also performed significantly better than traditional Nonlinear Output Error (NOE), Nonlinear AutoRegression with eXogenous (NARX) inputs, and Nonlinear Box-Jenkins (NBJ) models, which only reached error rates of 11.45%, 8.47% and 9.77%, respectively. The ACO algorithm employed could be utilized to optimize LSTM RNNs for any time series data prediction task.

Improved LSTM-based deep learning model for COVID-19 prediction using optimized approach

Today, fake information has become a significant problem, exacerbated by the acceleration of access to information. The spread of fake information has a dangerous impact, especially regarding global health issues, for example COVID-19. People can access various resources to obtain information, including online sites and social media. One of the methods to control the spread of false information is detecting hoaxes. Many methods have been developed to identify hoaxes; most previous studies have focused on developing hoax detection methods using data from a single source in English. The present study is carried out to detect fake news in Indonesian language using multiple data sources, including traditional and social media in the context of COVID-19. The study uses Long Short-Term Memory (LSTM) and the Robustly Optimised Bidirectional Encoder Representations from Transformers Pre-Training Approach (RoBERTa). The LSTM approach is used to develop four different architectures that varied based on: (1) the use of text-only versus the use of both title and text; (2) the number of LSTM and dense layers; and (3) the activation function. The LSTM model with text-only data, a single LSTM layer and two dense layers, outperformed other LSTM architectures, achieving the highest accuracy of 92.17%. The LSTM models require a considerably short training time of 23–27 minutes for 3,847 articles and has a detection time of 3.8–4.1 ms per article. The RoBERTa classifiers outperformed all LSTM models with an accuracy of over 97% and a significantly better training time, with a margin of more than 50% compared to LSTM classifiers, although it had a slightly longer test time. Both LSTM and RoBERTa models outperformed the Naïve Bayes and SVM benchmark methods in terms of accuracy, precision, and recall. Therefore, this study shows that both LSTM and RoBERTa methods are reliable and can be reasonably implemented for real-time fake news detection.