Multilayer Long Short-term Memory Research Articles

Indoor air quality prediction modeling for a naturally ventilated fitness building using RNN-LSTM artificial neural networks

PurposeExposure to indoor air pollutants poses a significant health risk, contributing to various ailments such as respiratory and cardiovascular diseases. These unhealthy consequences are specifically alarming for athletes during exercise due to their higher respiratory rate. Therefore, studying, predicting and curtailing exposure to indoor air contaminants during athletic activities is essential for fitness facilities. The objective of this study is to develop a neural network model designed for predicting optimal (in terms of health) occupancy intervals using monitored indoor air quality (IAQ) data.Design/methodology/approachThis research study presents an innovative approach employing a long short-term memory (LSTM) recurrent neural network (RNN) to determine optimal occupancy intervals for ensuring the safety and well-being of occupants. The dataset was collected over a 3-month monitoring campaign, encompassing 15 meteorological and indoor environmental parameters monitored. All the parameters were monitored in 5-min intervals, resulting in a total of 77,520 data points. The dataset collection parameters included the building’s ventilation methods as well as the level of occupancy. Initial preprocessing involved computing the correlation matrix and identifying highly correlated variables to serve as inputs for the LSTM network model.FindingsThe findings underscore the efficacy of the proposed artificial intelligence model in forecasting indoor conditions, yielding highly specific predicted time slots. Using the training dataset and established threshold values, the model effectively identifies benign periods for occupancy. Validation of the predicted time slots is conducted utilizing features chosen from the correlation matrix and their corresponding standard ranges. Essentially, this process determines the ratio of recommended to non-recommended timing intervals.Originality/valueHumans do not have the capacity to process this data and make such a relevant decision, though the complexity of the parameters of IAQ imposes significant barriers to human decision-making, artificial intelligence and machine learning systems, which are different. Present research utilizing multilayer perceptron (MLP) and LSTM algorithms for evaluating indoor air pollution levels lacks the capability to predict specific time slots. This study aims to fill this gap in evaluation methodologies. Therefore, the utilized LSTM-RNN model can provide a day-ahead prediction of indoor air pollutants, making its competency far beyond the human being’s and regular sensors' capacities.

Forecasting Indian Trade Trends through LSTM- based Predictive Modeling

The efficacy of Long Short-Term Memory (LSTM) neural networks and attention-based models in predicting next-day closing prices of the MSFT 500index is meticulously examined. A comprehensive suite of nine carefully chosen predictors spanning fundamental market data, macroeconomic indicators, and technical metrics is amalgamated, fostering a holistic comprehension of market behavior. Through rigorous analysis, the research evaluates single-layer and multilayer LSTM architectures alongside attention- based LSTM variants, juxtaposed against traditional ARIMA models. Surprisingly, the single-layer LSTM consistently outperforms its multilayer counterpart, demonstrating superior accuracy and model fit. The integration of corporate accounting statistics augments predictive capabilities, enriching the models' efficacy. Notably, attention-based LSTM models, particularly the Attention-LSTM variant, exhibit markedly lower prediction errors and higherreturns in trading strategies compared to other methodologies. However, the heightened complexity of stacked-LSTM structures fails to surpass the predictive acumen of simpler LSTM architectures.This inquiry underscores the paramount importance of leveraging advanced AI techniques and comprehensive datasets in navigating the intricate nuances of modern financialmarkets, offering invaluable insights for both researchers and practitioners engaged in stock priceforecasting endeavors.

Mixed Learning- and Model-Based Mass Estimation of Heavy Vehicles

This research utilized long short-term memory (LSTM) to oversee an RLS-based mass estimator based on longitudinal vehicle dynamics for heavy-duty vehicles (HDVs) instead of using the predefined rules. A multilayer LSTM network that analyzed parameters such as vehicle speed, longitudinal acceleration, engine torque, engine speed, and estimated mass from the RLS mass estimator was employed as the supervision method. The supervisory LSTM network was trained offline to recognize when the vehicle was operated so that the RLS estimator gave an estimate with the desired accuracy and the network was used as a reliability flag. High-fidelity simulation software was employed to collect data used to train and test the network. A threshold on the error percentage of the RLS mass estimator was used by the network to check the reliability of the algorithm. The preliminary findings indicate that the reliability of the RLS mass estimator could be predicted by using the LSTM network.

A sequence to sequence prediction model for remaining useful life of lithium-ion batteries with Bayesian optimisation process visualization

Accurate battery remaining useful life (RUL) prediction plays an important role in ensuring reliable operation of electric vehicles. In this paper, a hybrid model based on Bayesian optimization of deep convolutional neural network and long short-term memory neural network (BO-DCNN-LSTM) is proposed for battery RUL prediction. Feature extraction of raw charging characteristic curves is performed by the multilayer CNN and preliminary capacity prediction is performed by the multilayer LSTM. The model performance is explored with different training, validation and testing strategies and different prediction starting points. Validation using NASA battery aging data shows that the mean absolute error (MAE) and root mean square error (RMSE) of the RUL prediction are 0.0139 Ah and 0.0195 Ah, respectively, when the prediction starting point is the 50th cycle. In addition, this paper visualizes the process of how the Bayesian optimization (BO) algorithm searches for the global optimal combinations in the high-dimensional hyperparameter space and discusses the impact of these hyperparameters on the prediction, filling the gap in this part of the research.

Cryptocurrency returns prediction using candlestick patterns analysis and multi-layer deep LSTM neural networks

Cryptocurrency returns prediction using candlestick patterns analysis and multi-layer deep LSTM neural networks

Speech Inpainting Based on Multi-Layer Long Short-Term Memory Networks

Audio inpainting plays an important role in addressing incomplete, damaged, or missing audio signals, contributing to improved quality of service and overall user experience in multimedia communications over the Internet and mobile networks. This paper presents an innovative solution for speech inpainting using Long Short-Term Memory (LSTM) networks, i.e., a restoring task where the missing parts of speech signals are recovered from the previous information in the time domain. The lost or corrupted speech signals are also referred to as gaps. We regard the speech inpainting task as a time-series prediction problem in this research work. To address this problem, we designed multi-layer LSTM networks and trained them on different speech datasets. Our study aims to investigate the inpainting performance of the proposed models on different datasets and with varying LSTM layers and explore the effect of multi-layer LSTM networks on the prediction of speech samples in terms of perceived audio quality. The inpainted speech quality is evaluated through the Mean Opinion Score (MOS) and a frequency analysis of the spectrogram. Our proposed multi-layer LSTM models are able to restore up to 1 s of gaps with high perceptual audio quality using the features captured from the time domain only. Specifically, for gap lengths under 500 ms, the MOS can reach up to 3~4, and for gap lengths ranging between 500 ms and 1 s, the MOS can reach up to 2~3. In the time domain, the proposed models can proficiently restore the envelope and trend of lost speech signals. In the frequency domain, the proposed models can restore spectrogram blocks with higher similarity to the original signals at frequencies less than 2.0 kHz and comparatively lower similarity at frequencies in the range of 2.0 kHz~8.0 kHz.

A novel stock trading utilizing long short term memory prediction and evolutionary operating-weights strategy

In the realm of quantitative investing, predicting stock prices and developing effective trading strategies are paramount. The inherent randomness of the stock market, however, limits the precision of stock price predictions, thus undermining investment performance when these predictions are the sole basis for strategy. Nonetheless, forecasting future prices can offer invaluable insights for strategic models. When predictions are paired with well-designed strategies, they can augment investment returns. This paper proposes a novel approach to enhance investment returns by integrating Long Short Term Memory (LSTM) predictions with the Evolutionary Operating-weights (EOW) algorithmic strategy. The proposed method employs a multi-layer LSTM to forecast future stock prices, incorporating the predictions with real market data, subsequently deriving an operational strategy using the EOW algorithm. The results of this research indicate that this proposed methodology outperforms existing approaches.

Research on the artistic aesthetic value of violin performance under the integration of information technology

Abstract This study utilizes the ERP model of aesthetic cognition to reveal the neural basis of aesthetic experience, pointing out the complex cognitive processes including emotional escalation. Advanced techniques are used to pre-process violin sound signals and multiple algorithms are applied to extract critical features for accurate emotion recognition. The developed DBM-LSTM model combines the advantages of multi-layer stacked RBM and LSTM, which substantially improves emotion recognition accuracy with a success rate of more than 0.85. Finally, through a questionnaire survey, it is found that the public’s aesthetic preference of violin performance is significantly correlated with emotion perception, indicating that the more the performance conforms to the public’s aesthetics, the more it can stimulate positive emotions. This study not only provides theoretical and practical guidance on the combination of violin performance art and information technology, but also opens up a new way for the emotional communication of artistic performances and enhancing public aesthetic experience.

Engine remaining useful life prediction based on PSO optimized multi-layer long short-term memory and multi-source information fusion

Engine as the core component of mechanical equipment, its operating state directly affects whether the equipment can operate normally. Predicting the engine remaining useful life (RUL) can monitor the health of the engine in real time and formulate a timely and reasonable maintenance plan. Aiming at the engine monitoring data with various and long time span, we propose a direct prediction method of engine RUL based on particle swarm optimization (PSO) optimized multi-layer Long Short-Term Memory (LSTM) in this paper. Firstly, the monitoring data that can well reflect the engine degradation trend is screened out, and the samples are constructed through a sliding time window. Then, a multi-layer LSTM model is constructed to mine the deep-seated features of the samples for predicting the engine RUL. Finally, the hyperparameters of the multi-layer LSTM model are optimized automatically by the PSO algorithm to optimize the performance of the model. The effectiveness of this method is verified by NASA data set. RMSE, MAE and the scoring function are used as evaluation indexes. RMSE and score of the prediction results are 12.35 and 284.1, respectively. It has higher prediction accuracy compared with traditional deep learning and machine learning methods.

Research on the Reform and Practice of Informatization Mode of Adult Higher and Continuing Education Academic Records Management in the Context of Three-Whole Parenting

Abstract Information technology reform is an important way for schools to realize efficient management and information technology should be introduced into the management of student registration files to accelerate the process of informationization of registration file management. This paper builds an informatization database of adult higher continuing education student registration file management based on informatization technology and carries out the design of the main business process of student registration files. The multi-layer stacked LSTM network in the ELMo model is utilized to learn the preprocessing of the text of the school records data, and the CART decision tree algorithm is improved based on the reinforcement learning mechanism in order to improve the classification prediction accuracy of the decision tree in the dataset. In order to verify the feasibility of the school registration file informationization database constructed in this paper, a test analysis is conducted for the application of data mining technology and informationization database. The results show that the check accuracy of the CART decision tree algorithm improved by the reinforcement learning mechanism is around 80%, and the throughput of this paper’s database reaches 4827ops/s when the data writing volume is 1*105GB. When the number of threads reaches 128, the throughput of this paper’s informative database is around 5486ops/s, and when the data volume reaches 12*105 entries, the database in this paper the query response time is around 602ms. Under the background of three-round education, it is necessary to fully integrate information technology with the management of school registration files, which will lead to the innovation of the informationization mode of the management of school registration files in adult higher education.

Multilayer LSTM Model for Wind Power Estimation in the Scada System

Wind energy is clean energy that does not pollute the environment. However, the complex and variable operating environment of a wind turbine often makes it difficult to predict the instantaneous active power generated. In this study, a wind turbine active power estimation system based on a short-term memory network (LSTM) using time series analysis is proposed. The data obtained from the wind turbine SCADA system is used as input variables. In the proposed method, a multilayer LSTM architecture is designed to train the model. The first LSTM network consists of 64 units, and the second one consists of 32 units. This is followed by a dense layer consisting of 16 neurons. In the last layer, the architecture is finalized by using a linear activation function for the prediction process. The proposed deep learning (DL)-based LSTM prediction model takes into account environmental factors such as wind speed and wind direction for active power forecasting. The results show that the LSTM-based time series analysis method is capable of effectively capturing time series features among the data. Thus, the proposed architecture can realize high-accuracy active power forecasting.

LSTM time series NDVI prediction method incorporating climate elements: A case study of Yellow River Basin, China

Accurate prediction of the trend of Normalized Difference Vegetation Index (NDVI) time series in the Yellow River Basin (YRB) is crucial for the assessment of the hydrological and ecological environment in this region. Currently, the NDVI time series prediction model is primarily based on traditional models and single-variable neural network models. Nevertheless, these models present challenges in considering the limitations of multiple factors, causing the NDVI time series prediction results to lack reliability. To predict NDVI time-series in the YRB of China, this study constructed a multilayer multivariate Long-Short Term Memory (LSTM) neural network model including climatic components. The initial important climatic elements in this region were identified using GeoDetector. Then, the relationship between NDVI and climatic factors in the YRB of China is established. Finally, numerical scale data are used to train and predict a multilayer multivariate LSTM model with climatic components. According to the results, the three-layer multivariate LSTM neural network NDVI time series prediction model developed in this study has the best performance among the evaluated indices. When compared to existing time series prediction models, the proposed model in this study takes into account the common constraint effect of various climate factors on NDVI. This leads to a significantly improved prediction accuracy, presenting new opportunities for enhancing the prediction model. By analyzing the NDVI time series prediction outcomes for the YRB, it has been determined that the ecological environment of the area will continuously improve in the future. This study offers significant technological and theoretical backing for assessing the hydrological and ecological environment of the YRB and comparable ecologically vulnerable regions in China.

Mutual generation in neuronal activity across the brain via deep neural approach, and its network interpretation

In the brain, many regions work in a network-like association, yet it is not known how durable these associations are in terms of activity and could survive without structural connections. To assess the association or similarity between brain regions with a generating approach, this study evaluated the similarity of activities of neurons within each region after disconnecting between regions. The “generation” approach here refers to using a multi-layer LSTM (Long Short-Term Memory) model to learn the rules of activity generation in one region and then apply that knowledge to generate activity in other regions. Surprisingly, the results revealed that activity generation from one region to disconnected regions was possible with similar accuracy to generation between the same regions in many cases. Notably, firing rates and synchronization of firing between neuron pairs, often used as neuronal representations, could be reproduced with precision. Additionally, accuracies were associated with the relative angle between brain regions and the strength of the structural connections that initially connected them. This outcome enables us to look into trends governing non-uniformity of the cortex based on the potential to generate informative data and reduces the need for animal experiments.

Improved LSTM based state of health estimation using random segments of the charging curves for lithium-ion batteries

Battery State of Health (SOH) estimation is crucial for providing valuable information for optimizing battery usage and improving battery efficiency. Considering the uncertainties in battery charging behavior during practical usage, this paper proposes an ensemble model based on an improved long short-term memory (LSTM) neural network. The model takes random segments of charging curves as input to estimate the SOH of lithium-ion batteries. In this paper, a multi-layer LSTM network with attention mechanism is proposed as the base learner. Then, multiple base learners are trained using different parts of the charging curve segments. The BP neural network is used to integrate the SOH estimation value of each base learner to obtain the final SOH estimation value. The accuracy and robustness of the proposed method are validated using the Oxford lithium-ion battery dataset and NASA battery degradation dataset. Additionally, this paper investigates the influence of charging segment length and the number of sampling points on the estimation results.

A Hybrid DNN Multilayered LSTM Model for Energy Consumption Prediction

The ability to predict energy consumption in a world in which energy needs are ever-increasing is important for future growth and development. In recent years, deep learning models have made significant advancements in energy forecasting. In this study, a hybrid deep neural network (DNN) multilayered long short-term memory (LSTM) model was used to predict energy consumption in households. When evaluating the model, the individual household electric power consumption dataset was used to train, validate, and test the model. Preprocessing was applied to the data to minimize any prediction errors. Afterward, the DNN algorithm extracted the spatial features, and the multilayered LSTM model was used for sequential learning. The model showed a highly accurate predictive performance, as the actual consumption trends matched the predictive trends. The coefficient of determination, root-mean-square error, mean absolute error, and mean absolute percentage error were found to be 0.99911, 0.02410, 0.01565, and 0.01826, respectively. A DNN model and LSTM model were also trained to study how much improvement the proposed model would provide. The proposed model showed better performance than the DNN and LSTM models. Moreover, similar to other deep learning models, the proposed model’s performance was superior and provided accurate and reliable energy consumption predictions.

Optimizing Hyperparameters for Enhanced LSTM-Based Prediction System Performance

This research paper explores the application of deep learning and supervised machine learning algorithms, specifically Long Short-Term Memory (LSTM), for stock market prediction. The study focuses on the closing prices of three companies - Tata Steel, Apple, and Powergrid - using a dataset sourced from Yahoo Finance. Performance evaluation of the LSTM model employed RMSE, MAPE, and accuracy metrics, along with hyperparameter calibration to determine the optimal model parameters. The findings indicate that a single-layer LSTM model outperformed a multilayer LSTM model across all companies and evaluation metrics. Furthermore, a comparison with existing research demonstrated the superiority of the proposed model. The study emphasizes the effectiveness of LSTM models for stock price prediction, underscores the significance of proper hyperparameter tuning for optimal performance, and concludes that a single-layer LSTM model can yield superior results compared to a multilayer model.

Review on positional significance of LSTM and CNN in the multilayer deep neural architecture for efficient sentiment classification

The deep learning revolution in the current decade has transformed the artificial intelligence industry. Eventually, deep learning techniques have become essential for many computational modeling tasks. Nevertheless, deep neural models provide a high degree of automation for natural language processing (NLP) applications. Deep neural models are extensively used to decode public reviews subjective to specific products, services, and other social activities. Further, to improve sentiment classification accuracy, several neural architectures have been developed. Convolutional neural networks (CNN) and Long-short term memory (LSTM) are the popular deep models employed in ensemble architectures for sentiment classification tasks. This review article extensively compares the competence of CNN and LSTM-based ensemble models to improve the sentiment accuracy for online review datasets. Further, this article also provides an empirical study on various ensemble models concerning the position of LSTM and CNN for efficient sentiment classification. This empirical study provides deep learning researchers with insights into building effective multilayer LSTM and CNN models for many sentiment analysis tasks.

Research on Medium and Long Term Generation Side Deviation Prediction of New Power Market Based on Multi-Layer LSTM

Background: With the large-scale grid connection operation of new or renewable energy and the access of active loads such as electric vehicles and air conditioners, the electric energy trading business in the power market faces problems such as the rapid expansion of the number of market settlement subjects, explosive growth, various subjects responsible for deviation assessment, various electric energy trading methods and so on. Objective: This paper focuses on the medium and long-term generation side power trading in the new power market. Through cause analysis, induction and summary, algorithm design and case analysis, the problem of generation side deviation prediction is solved and power waste is reduced. Methods: This paper puts forward the reasons for the imbalance of medium and long-term power trading in the new power market dominated by new energy, as well as the deviation prediction algorithm based on multi-layer LSTM, which brings the total historical deviation, total planned deviation, total measurement deviation, new energy consumption and other data into the M-LSTM deep learning network for testing in each provincial power market center. Results: We use the neural network prediction algorithm. Compared with a single LSTM, the multi-layer LSTM can better maintain the characteristics of the sample time series and reduce the prediction error. Compared with BPNN、M-BPNN and Cooperative game theory, LSTM has a better memory effect. Conclusion: The experiment shows that the more accurate prediction deviation of this method can better arrange the generation plan, reduce the loss caused by excessive deviation, reduce the "price trampling" of the power market, and ensure the fair and efficient development of the power market.

Energy Consumption Forecasting in Commercial Buildings during the COVID-19 Pandemic: A Multivariate Multilayered Long-Short Term Memory Time-Series Model with Knowledge Injection

The COVID-19 pandemic and the subsequent implementation of lockdown measures have significantly impacted global electricity consumption, necessitating accurate energy consumption forecasts for optimal energy generation and distribution during a pandemic. In this paper, we propose a new forecasting model called the multivariate multilayered long short-term memory (LSTM) with COVID-19 case injection (mv−M−LSTM−CI) for improved energy forecast during the next occurrence of a similar pandemic. We utilized data from commercial buildings in Melbourne, Australia, during the COVID-19 pandemic to predict energy consumption and evaluate the model’s performance against commonly used methods such as LSTM, bidirectional LSTM, linear regression, support vector machine, and multilayered LSTM (M-LSTM). The proposed forecasting model was analyzed using the following metrics: mean percent absolute error (MPAE), normalized root mean square error (NRMSE), and R2 score values. The model mv−M−LSTM−CI demonstrated superior performance, achieving the lowest mean percentage absolute error values of 0.061, 0.093, and 0.158 for DatasetS1, DatasetS2, and DatasetS3, respectively. Our results highlight the improved precision and accuracy of the model, providing valuable information for energy management and decision making during the challenges posed by the occurrence of a pandemic like COVID-19 in the future.

A Multilayer Approach for Intrusion Detection with Lightweight Multilayer Perceptron and LSTM Deep Learning Models

Intrusion detection is essential in the field of cybersecurity for protecting networks and computer systems from nefarious activity. We suggest a novel multilayer strategy that combines the strength of the Lightweight Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM) deep learning models in order to improve the precision and effectiveness of intrusion detection.The initial layer for extraction of features and representation is the Lightweight MLP. Its streamlined architecture allows for quick network data processing while still maintaining competitive performance. The LSTM deep learning model, which is excellent at identifying temporal correlations and patterns in sequential data, receives the extracted features after that.Our multilayer technique successfully manages the highly dimensional and dynamic nature of data from networks by merging these two models. We undertake extensive tests on benchmark datasets, and the outcomes show that our strategy performs better than conventional single-model intrusion detection techniques.The suggested multilayer method also demonstrates outstanding efficiency, which makes it particularly ideal for real-time intrusion detection in expansive network environments. Our multilayer approach offers a strong and dependable solution for identifying and reducing intrusions, strengthening the security position of computer systems and networks as cyber threats continue to advance.