Single-layer Long Short-Term Memory Research Articles

Implementation of deep learning models in predicting ESG index volatility

The consideration of environmental, social, and governance (ESG) aspects has become an integral part of investment decisions for individual and institutional investors. Most recently, corporate leaders recognized the core value of the ESG framework in fulfilling their environmental and social responsibility efforts. While stock market prediction is a complex and challenging task, several factors associated with developing an ESG framework further increase the complexity and volatility of ESG portfolios compared with broad market indices. To address this challenge, we propose an integrated computational framework to implement deep learning model architectures, specifically long short-term memory (LSTM), gated recurrent unit, and convolutional neural network, to predict the volatility of the ESG index in an identical environment. A comprehensive analysis was performed to identify a balanced combination of input features from fundamental data, technical indicators, and macroeconomic factors to delineate the cone of uncertainty in market volatility prediction. The performance of the constructed models was evaluated using standard assessment metrics. Rigorous hyperparameter tuning and model-selection strategies were implemented to identify the best model. Furthermore, a series of statistical analyses was conducted to validate the robustness and reliability of the model. Experimental results showed that a single-layer LSTM model with a relatively small number of neurons provides a superior fit with high prediction accuracy relative to more complex models.

Forecasting Indian Trade Trends through LSTM- based Predictive Modeling

The efficacy of Long Short-Term Memory (LSTM) neural networks and attention-based models in predicting next-day closing prices of the MSFT 500index is meticulously examined. A comprehensive suite of nine carefully chosen predictors spanning fundamental market data, macroeconomic indicators, and technical metrics is amalgamated, fostering a holistic comprehension of market behavior. Through rigorous analysis, the research evaluates single-layer and multilayer LSTM architectures alongside attention- based LSTM variants, juxtaposed against traditional ARIMA models. Surprisingly, the single-layer LSTM consistently outperforms its multilayer counterpart, demonstrating superior accuracy and model fit. The integration of corporate accounting statistics augments predictive capabilities, enriching the models' efficacy. Notably, attention-based LSTM models, particularly the Attention-LSTM variant, exhibit markedly lower prediction errors and higherreturns in trading strategies compared to other methodologies. However, the heightened complexity of stacked-LSTM structures fails to surpass the predictive acumen of simpler LSTM architectures.This inquiry underscores the paramount importance of leveraging advanced AI techniques and comprehensive datasets in navigating the intricate nuances of modern financialmarkets, offering invaluable insights for both researchers and practitioners engaged in stock priceforecasting endeavors.

Optimizing Hyperparameters for Enhanced LSTM-Based Prediction System Performance

This research paper explores the application of deep learning and supervised machine learning algorithms, specifically Long Short-Term Memory (LSTM), for stock market prediction. The study focuses on the closing prices of three companies - Tata Steel, Apple, and Powergrid - using a dataset sourced from Yahoo Finance. Performance evaluation of the LSTM model employed RMSE, MAPE, and accuracy metrics, along with hyperparameter calibration to determine the optimal model parameters. The findings indicate that a single-layer LSTM model outperformed a multilayer LSTM model across all companies and evaluation metrics. Furthermore, a comparison with existing research demonstrated the superiority of the proposed model. The study emphasizes the effectiveness of LSTM models for stock price prediction, underscores the significance of proper hyperparameter tuning for optimal performance, and concludes that a single-layer LSTM model can yield superior results compared to a multilayer model.

Lightweight Football Motion Recognition and Intensity Analysis Using Low-Cost Wearable Sensors.

In recent years, machine learning has been utilized in health informatics and sports science. There is a great demand and development potential for combining the Internet of Things (IoT) and artificial intelligence (AI) to be applied to football sports. The conventional teaching and training methods of football sports have limited collection and mining of real raw data using wearable devices, and lack human motion capture and gesture recognition based on sports science theories. In this study, a low-cost AI + IoT system framework is designed to recognize football motion and analyze motion intensity. To reduce the communication delay and the computational resource consumption caused by data operations, a multitask learning model is designed to achieve motion recognition and intensity estimation. The model can perform classification and regression tasks in parallel and output the results simultaneously. A feature extraction scheme is designed in the initial data processing, and feature data augmentation is performed to solve the small sample data problem. To evaluate the performance of the designed football motion recognition algorithm, this paper proposes a data extraction experimental scheme to complete the data collection of different motions. Model validation is performed using three publicly available datasets, and the features learning strategies are analyzed. Finally, experiments are conducted on the collected football motion datasets and the experimental results show that the designed multitask model can perform two tasks simultaneously and can achieve high computational efficiency. The multitasking single-layer long short-term memory (LSTM) network with 32 neural units can achieve the accuracy of 0.8372, F1 score of 0.8172, mean average precision (mAP) of 0.7627, and mean absolute error (MAE) of 0.6117, while the multitasking single-layer LSTM network with 64 neural units can achieve the accuracy of 0.8407, F1 score of 0.8132, mAP of 0.7728, and MAE of 0.5966.

Energy Usage Forecasting Model Based on Long Short-Term Memory (LSTM) and eXplainable Artificial Intelligence (XAI)

The accurate forecasting of energy consumption is essential for companies, primarily for planning energy procurement. An overestimated or underestimated forecasting value may lead to inefficient energy usage. Inefficient energy usage could also lead to financial consequences for the company, since it will generate a high cost of energy production. Therefore, in this study, we proposed an energy usage forecasting model and parameter analysis using long short-term memory (LSTM) and explainable artificial intelligence (XAI), respectively. A public energy usage dataset from a steel company was used in this study to evaluate our models and compare them with previous study results. The results showed that our models achieved the lowest root mean squared error (RMSE) scores by up to 0.08, 0.07, and 0.07 for the single-layer LSTM, double-layer LSTM, and bi-directional LSTM, respectively. In addition, the interpretability analysis using XAI revealed that two parameters, namely the leading current reactive power and the number of seconds from midnight, had a strong influence on the model output. Finally, it is expected that our study could be useful for industry practitioners, providing LSTM models for accurate energy forecasting and offering insight for policymakers and industry leaders so that they can make more informed decisions about resource allocation and investment, develop more effective strategies for reducing energy consumption, and support the transition toward sustainable development.

Incorporating Sentiment and Temporal Information for Bitcoin Price Prediction

Recent years have witnessed the rapid development of bitcoin as the first digital currency. Considering the advantages of bitcoin for both individuals and society, the price prediction of bitcoin is a hot topic. However, there remain two main challenges to be addressed in this task. Firstly, because bitcoin is vulnerable to the attitudes of the investors, incorporating the sentiment semantics from the social media into the prediction is challenging. Secondly, it is intractable to predict extreme volatility of the bitcoin price. To tackle the above challenges, this paper proposes to incorporate sentiment and temporal information simultaneously. For the first challenge, this paper employs external unsupervised corpus to conduct the domain-specific post-pretraining on the off-the-shelf language model. And the sentiment analysis on the tweets is done to obtain the scores. For the second one, Long Short Term Memory (LSTM) network is leveraged to joint model the temporal price data and the sentiment scores, thus deriving the final predictions. Experiments on the real data show that compared with the single-layer LSTM model, the model in this paper works better, which provides help for investors to specify trading strategies and also provides implications for government agencies that are developing digital currencies.

Predicting stock market index using LSTM

The rapid advancement in artificial intelligence and machine learning techniques, availability of large-scale data, and increased computational capabilities of the machine opens the door to develop sophisticated methods in predicting stock price. In the meantime, easy access to investment opportunities has made the stock market more complex and volatile than ever. The world is looking for an accurate and reliable predictive model which can capture the market’s highly volatile and nonlinear behavior in a holistic framework. This study uses a long short-term memory (LSTM), a particular neural network architecture, to predict the next-day closing price of the S&P 500 index. A well-balanced combination of nine predictors is carefully constructed under the umbrella of the fundamental market data, macroeconomic data, and technical indicators to capture the behavior of the stock market in a broader sense. Single layer and multilayer LSTM models are developed using the chosen input variables, and their performances are compared using standard assessment metrics–Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Correlation Coefficient (R). The experimental results show that the single layer LSTM model provides a superior fit and high prediction accuracy compared to multilayer LSTM models.

A high resistivity anomaly identification method for marine controlled-source electromagnetic data

The magnitude versus offset (MVO) curve, a type of frequency domain marine controlled-source electromagnetic data, is the most common way to identify electromagnetic anomalies in oil and gas reservoirs. However, in actual exploration, it can be difficult to identify the boundary of the high resistance anomaly when there are response signals of multiple emission frequencies. Also, the noise would reduce the accuracy of manually detecting electromagnetic anomalies. The robustness of the bidirectional long short-term memory (LSTM) network is relatively strong, and the LSTM neural network would get the most out of the sequence information of the data for feature extraction purposes and to achieve automatic classification and identification. Therefore, this paper proposes a method of using bidirectional LSTM to solve the problem of anomaly identification in marine controlled-source electromagnetic data. The LSTM unit was applied to establish anomaly identification models of single-layer LSTM, two-layer LSTM, and bidirectional LSTM, respectively. In this paper, theoretical data were calculated by a one-dimensional uniform layered medium model, and the synthetic noise data were constructed by adding random noise with different signal-to-noise ratios. The three types of models were trained, verified, and tested, respectively, to compare the accuracy of electromagnetic anomaly identification. According to the comparison, a conclusion can be drawn that the bidirectional LSTM model suggests the best manifestation of learning the characteristics of the sample. Its electromagnetic anomaly identification accuracy reached 100% in the theoretical dataset and 79.58% in the synthetic noise dataset.

Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker

In recent years, the advancement of pattern recognition algorithms, specifically the deep learning-related techniques, have propelled a tremendous amount of researches in fall event recognition systems. It is important to detect a fall incident as early as possible, whereby a slight delay in providing immediate assistance can cause severe unrecoverable injuries. One of the main challenges in fall event recognition is the imbalanced training data between fall and no-fall events, where a real-life fall incident is a sporadic event that occurs infrequently. Most of the recent techniques produce a lot of false alarms, as it is hard to train them to cover a wide range of fall situations. Hence, this paper aims to detect the exact fall frame in a video sequence, as such it will not be dependent on the whole clip of the video sequence. Our proposed approach consists of a two-stage module where the first stage employs a compact convolutional neural network tracker to generate the object trajectory information. Features of interest will be sampled from the generated trajectory paths, which will be fed as the input to the second stage. The next stage network then models the temporal dependencies of the trajectory information using symmetrical Long Short-Term Memory (LSTM) architecture. This two-stage module is a novel approach as most of the techniques rely on the detection module rather than the tracking module. The simulation experiments were tested using Fall Detection Dataset (FDD). The proposed approach obtains an expected average overlap of 0.167, which is the best performance compared to Multi-Domain Network (MDNET) and Tree-structured Convolutional Neural Network (TCNN) trackers. Furthermore, the proposed 3-layers of stacked LSTM architecture also performs the best compared to the vanilla recurrent neural network and single-layer LSTM. This approach can be further improved if the tracker model is firstly pre-tuned in offline mode with respect to a specific type of object of interest, rather than a general object.

A method of straightening armoured face conveyor based on space kinematics of reserve pushing mechanism

The straightness control of armoured face conveyor is one of the key problems in longwall mining. Aiming at the problems of less sensing information and low accuracy of straightening of armoured face conveyor, a method of straightening based on the movement laws of reverse floating connection mechanism and deep learning trajectory is proposed in VR (Virtual Reality) environment. Firstly, the spatial kinematic model of the reverse floating connection mechanism of roof support and armoured face conveyor are established with knowledge of the industrial robot, the movement laws are obtained. Secondly, SL-LSTM (Single Layer Long Short Term Memory) prediction model is established by sensitivity analysis method after super parameters are selected, and by considering the influence of sensor error and fluctuation of coal floor, the establishment method of date sets and thought of trajectory prediction are put forward. Thirdly, according to requirements for straightness of fully mechanized working face and data availability, the correction model of prediction trajectory and the trajectory-attitude transformation model are established. Finally, based on the obtained armoured face conveyor trajectory, the relative position between a roof support and the corresponding middle trough is determined after timely advancing of roof support, and then the straightening process of armoured face conveyor is realized after precise pushing of the roof support. The straightness error of the straightening method proposed in this paper is within ±0.14 cm and the straightening effect is better by experiment verification.

Multi-distributed Speech Emotion Recognition Based on Mel Frequency Cepstogram and Parameter Transfer

Speech emotion recognition (SER) is the use of speech signals to estimate the state of emotion. At present, machine learning is one of the main research methods of SER, the test and training dataS of traditional machine learning all have the same distribution and feature space, but the data of speech is accessed from different environments and devices, with different distribution characteristics in real life. Thus, the traditional machine learning method is applied to the poor performance of SER. This paper proposes a multi-distributed SER method based on Mel frequency cepstogram (MFCC) and parameter transfer. The method is based on single-layer Long short-term memory (LSTM), pre-trained inception-v3 network and multi-distribution corpus. The speech pre-processed MFCC is taken as the input of single-layer LSTM, and input to the pre-trained inception-v3 network. the features are extracted through the pre-trained inception-v3 model. Then the features are sent to the newly defined the fully connected layer and classification layer, let the parameters of the fully connected layer be fine-tuned, finally get the classification result. The experiment proves that the method can effectively complete the classification of multi-distribution speech emotions and is more effective than the traditional machine learning framework of SER.

Training and Testing of a Single-Layer LSTM Network for Near-Future Solar Forecasting

Increasing integration of renewable energy sources, like solar photovoltaic (PV), necessitates the development of power forecasting tools to predict power fluctuations caused by weather. With trustworthy and accurate solar power forecasting models, grid operators could easily determine when other dispatchable sources of backup power may be needed to account for fluctuations in PV power plants. Additionally, PV customers and designers would feel secure knowing how much energy to expect from their PV systems on an hourly, daily, monthly, or yearly basis. The PROGNOSIS project, based at the Cyprus University of Technology, is developing a tool for intra-hour solar irradiance forecasting. This article presents the design, training, and testing of a single-layer long-short-term-memory (LSTM) artificial neural network for intra-hour power forecasting of a single PV system in Cyprus. Four years of PV data were used for training and testing the model (80% for training and 20% for testing). With a normalized root mean squared error (nRMSE) of 10.7%, the single-layer network performed similarly to a more complex 5-layer LSTM network trained and tested using the same data. Overall, these results suggest that simple LSTM networks can be just as effective as more complicated ones.

Traffic index prediction based on sequence to sequence learning

The traffic index prediction plays an important role in many intelligent transportation applications. The challenge mainly comes from the strong nonlinearities in the changing of traffic index, which is caused by transitions of various traffic states including smooth traffic, congestion, breakdown and recovery. Deep learning performs well in capturing and describing the nonlinear relationship among variables. This paper uses a Sequence to Sequence (Seq2Seq) deep learning framework to predict the traffic index. In the proposed method, we use Long Short Term Memory (LSTM) as the basic circulating unit. And the LSTM units are stacked to extract the time changing characteristic and the periodic characteristic of the input traffic index sequence. Besides, the extracted feature vectors are decoded by another single layer LSTM to predict the traffic index of each time after inputting sequence. Experiments are conducted on Beijing traffic index datasets. The proposed method outperformed Autoregressive Integrated Moving Average Model (ARIMA) and LSTM under some commonly used evaluation metrics.

Condition Monitoring of Equipment in Oil Wells using Deep Learning

In this paper, a fault prediction method for oil well equipment based on the analysis of time series data obtained from multiple sensors is proposed. The proposed method is based on deep learning (DL). For this purpose, comparative analysis of single-layer long short-term memory (LSTM) with the convolutional neural network (CNN) and stacked LSTM methods is provided. To demonstrate the efficacy of the proposed method, some experiments are conducted on the real data set obtained from eight sensors installed in oil wells. In this paper, compared to the single-layer LSTM model, the CNN and stacked LSTM predicted the faulty time series with a minimal loss.

A Hierarchical Recurrent Neural Network for Symbolic Melody Generation.

In recent years, neural networks have been used to generate symbolic melodies. However, the long-term structure in the melody has posed great difficulty to design a good model. In this article, we present a hierarchical recurrent neural network (HRNN) for melody generation, which consists of three long-short-term-memory (LSTM) subnetworks working in a coarse-to-fine manner along time. Specifically, the three subnetworks generate bar profiles, beat profiles, and notes, in turn, and the output of the high-level subnetworks are fed into the low-level subnetworks, serving as guidance to generate the finer time-scale melody components in the low-level subnetworks. Two human behavior experiments demonstrate the advantage of this structure over the single-layer LSTM which attempts to learn all hidden structures in melodies. Compared with the recently proposed models MidiNet and MusicVAE, the HRNN produces better melodies evaluated by humans.

Bispectrum and Recurrent Neural Networks: Improved Classification of Interictal and Preictal States

This work proposes a novel approach for the classification of interictal and preictal brain states based on bispectrum analysis and recurrent Long Short-Term Memory (LSTM) neural networks. Two features were first extracted from bilateral intracranial electroencephalography (iEEG) recordings of dogs with naturally occurring focal epilepsy. Single-layer LSTM networks were trained to classify 5-min long feature vectors as preictal or interictal. Classification performances were compared to previous work involving multilayer perceptron networks and higher-order spectral (HOS) features on the same dataset. The proposed LSTM network proved superior to the multilayer perceptron network and achieved an average classification accuracy of 86.29% on held-out data. Results imply the possibility of forecasting epileptic seizures using recurrent neural networks, with minimal feature extraction.

Payload-Based Traffic Classification Using Multi-Layer LSTM in Software Defined Networks

Recently, with the advent of various Internet of Things (IoT) applications, a massive amount of network traffic is being generated. A network operator must provide different quality of service, according to the service provided by each application. Toward this end, many studies have investigated how to classify various types of application network traffic accurately. Especially, since many applications use temporary or dynamic IP or Port numbers in the IoT environment, only payload-based network traffic classification technology is more suitable than the classification using the packet header information as well as payload. Furthermore, to automatically respond to various applications, it is necessary to classify traffic using deep learning without the network operator intervention. In this study, we propose a traffic classification scheme using a deep learning model in software defined networks. We generate flow-based payload datasets through our own network traffic pre-processing, and train two deep learning models: 1) the multi-layer long short-term memory (LSTM) model and 2) the combination of convolutional neural network and single-layer LSTM models, to perform network traffic classification. We also execute a model tuning procedure to find the optimal hyper-parameters of the two deep learning models. Lastly, we analyze the network traffic classification performance on the basis of the F1-score for the two deep learning models, and show the superiority of the multi-layer LSTM model for network packet classification.

Single Layer & Multi-layer Long Short-Term Memory (LSTM) Model with Intermediate Variables for Weather Forecasting

Weather forecasting has gained attention many researchers from various research communities due to its effect to the global human life. The emerging deep learning techniques in the last decade coupled and the wide availability of massive weather observation data have motivated many researches to explore hidden hierarchical pattern in the large volume of weather dataset for weather forecasting.The purposes of this research are to build a robust and adaptive statistical model for forecasting univariate weather variable in Indonesian airport area and to explore the effect of intermediate weather variable related to accuracy prediction using single layer Long Short Memory Model (LSTM) model and multi layers LSTM model. The proposed forecasting model is an extension of LSTM model by adding intermediate variable signal into LSTM memory block. The premise is that two highly related patterns in input dataset will rectify the input patterns so make it easier for the model to learn and recognize the pattern from the training dataset. In an effort to achieve a robust model for learning and recognizing weather pattern, this research will also explore various architectures such as single layer LSTM and Multiple Layer LSTM (4 layers LSTM). The dataset is weather variable data collected by Weather Underground at Hang Nadim Indonesia airport. This research used visibility as predicted data and temperature, pressure, humidity, dew point as intermediates data. The best model of LSTM in this experiment is multiple layers LSTM and the best intermediate data is pressure variable. Using the pressure variable this model has gained the validation accuracy 0.8060 and RMSE 0.0775.

Long Short-Term Memory Projection Recurrent Neural Network Architectures for Piano’s Continuous Note Recognition

Long Short-Term Memory (LSTM) is a kind of Recurrent Neural Networks (RNN) relating to time series, which has achieved good performance in speech recogniton and image recognition. Long Short-Term Memory Projection (LSTMP) is a variant of LSTM to further optimize speed and performance of LSTM by adding a projection layer. As LSTM and LSTMP have performed well in pattern recognition, in this paper, we combine them with Connectionist Temporal Classification (CTC) to study piano’s continuous note recognition for robotics. Based on the Beijing Forestry University music library, we conduct experiments to show recognition rates and numbers of iterations of LSTM with a single layer, LSTMP with a single layer, and Deep LSTM (DLSTM, LSTM with multilayers). As a result, the single layer LSTMP proves performing much better than the single layer LSTM in both time and the recognition rate; that is, LSTMP has fewer parameters and therefore reduces the training time, and, moreover, benefiting from the projection layer, LSTMP has better performance, too. The best recognition rate of LSTMP is 99.8%. As for DLSTM, the recognition rate can reach 100% because of the effectiveness of the deep structure, but compared with the single layer LSTMP, DLSTM needs more training time.