Forecasting the Required Quantity of Cement Manufacturing Materials Using Time Series and Q-Network Techniques

In this paper, the primary focus is of Slot Tagging of Gujarat Dialogue, which enables the Gujarati language communication between human and machine, allowing machines to perform given task and provide desired output. The accuracy of tagging entirely depends on bifurcation of slots and word embedding. It is also very challenging for a researcher to do proper slot tagging as dialogue and speech differs from human to human, which makes the slot tagging methodology more complex. Various deep learning models are available for slot tagging for the researchers, however, in the instant paper it mainly focuses on Long Short-Term Memory (LSTM), Convolutional Neural Network - Long Short-Term Memory (CNN-LSTM) and Long Short-Term Memory – Conditional Random Field (LSTM-CRF), Bidirectional Long Short-Term Memory (BiLSTM), Convolutional Neural Network - Bidirectional Long Short-Term Memory (CNN-BiLSTM) and Bidirectional Long Short-Term Memory – Conditional Random Field (BiLSTM-CRF). While comparing the above models with each other, it is observed that BiLSTM models performs better than LSTM models by a variation ~2% of its F1-measure, as it contains an additional layer which formulates the word string to traverse from backward to forward. Within BiLSTM models, BiLSTM-CRF has outperformed other two Bi-LSTM models. Its F1-measure is better than CNN-BiLSTM by 1.2% and BiLSTM by 2.4%.KeywordsSpoken Language Understanding (SLU)Long Short-Term Memory (LSTM)Slot taggingBidirectional Long Short-Term Memory (BiLSTM)Convolutional Neural Network - Bidirectional Long Short-Term Memory (CNN-BiLSTM)Bidirectional Long Short-Term Memory (BiLSTM-CRF)

Bi-LSTM based vertical total electron content prediction at low-latitude equatorial ionization anomaly region of South India

The Synthetic Aperture Radar (SAR) time series allows describing the rice phenological cycle by the backscattering time signature. Therefore, the advent of the Copernicus Sentinel-1 program expands studies of radar data (C-band) for rice monitoring at regional scales, due to the high temporal resolution and free data distribution. Recurrent Neural Network (RNN) model has reached state-of-the-art in the pattern recognition of time-sequenced data, obtaining a significant advantage at crop classification on the remote sensing images. One of the most used approaches in the RNN model is the Long Short-Term Memory (LSTM) model and its improvements, such as Bidirectional LSTM (Bi-LSTM). Bi-LSTM models are more effective as their output depends on the previous and the next segment, in contrast to the unidirectional LSTM models. The present research aims to map rice crops from Sentinel-1 time series (band C) using LSTM and Bi-LSTM models in West Rio Grande do Sul (Brazil). We compared the results with traditional Machine Learning techniques: Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (k-NN), and Normal Bayes (NB). The developed methodology can be subdivided into the following steps: (a) acquisition of the Sentinel time series over two years; (b) data pre-processing and minimizing noise from 3D spatial-temporal filters and smoothing with Savitzky-Golay filter; (c) time series classification procedures; (d) accuracy analysis and comparison among the methods. The results show high overall accuracy and Kappa (>97% for all methods and metrics). Bi-LSTM was the best model, presenting statistical differences in the McNemar test with a significance of 0.05. However, LSTM and Traditional Machine Learning models also achieved high accuracy values. The study establishes an adequate methodology for mapping the rice crops in West Rio Grande do Sul.

Objectives: Predicting the amount of rainfall is difficult due to its complexity and non-linearity. The objective of this study is to predict the average rainfall one month ahead using the all-India monthly average rainfall dataset from 1871 to 2016. Methods: This study proposed a Bidirectional Long Short-Term Memory (LSTM) model to predict the average monthly rainfall in India. The parameters of the models are determined using the grid search method. This study utilized the average monthly rainfall as an input, and the dataset consists of 1752 months of rainfall data prepared from thirty (30) meteorological sub-divisions in India. The model was compiled using the Mean Square Error (MSE) loss function and Adam optimizer. The models' performances were evaluated using statistical metrics such as Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). Findings: This study discovered that the proposed Bidirectional LSTM model achieved an RMSE of 240.79 and outperformed an existing Recurrent Neural Network (RNN), Vanilla LSTM and Stacked LSTM by 8%, 4% and 2% respectively. The study also finds that increasing the input time step and increasing the number of cells in the hidden layer enhanced the prediction performance of the proposed model, and the Bidirectional LSTM converges at a lower epoch compared to RNN and LSTM models. Novelty: This study applied the Bidirectional LSTM for the first time in predicting all-India monthly average rainfall and provides a new benchmark for this dataset. Keywords: Deep Learning, LSTM, Rainfall prediction, Stacked LSTM, Bidirectional LSTM

Machine and deep learning-based algorithms are the emerging approaches in addressing prediction problems in time series. These techniques have been shown to produce more accurate results than conventional regression-based modeling. It has been reported that artificial Recurrent Neural Networks (RNN) with memory, such as Long Short-Term Memory (LSTM), are superior compared to Autoregressive Integrated Moving Average (ARIMA) with a large margin. The LSTM-based models incorporate additional “gates” for the purpose of memorizing longer sequences of input data. The major question is that whether the gates incorporated in the LSTM architecture already offers a good prediction and whether additional training of data would be necessary to further improve the prediction. Bidirectional LSTMs (BiLSTMs) enable additional training by traversing the input data twice (i.e., 1) left-to-right, and 2) right-to-left). The research question of interest is then whether BiLSTM, with additional training capability, outperforms regular unidirectional LSTM. This paper reports a behavioral analysis and comparison of BiLSTM and LSTM models. The objective is to explore to what extend additional layers of training of data would be beneficial to tune the involved parameters. The results show that additional training of data and thus BiLSTM-based modeling offers better predictions than regular LSTM-based models. More specifically, it was observed that BiLSTM models provide better predictions compared to ARIMA and LSTM models. It was also observed that BiLSTM models reach the equilibrium much slower than LSTM-based models.

The objective of this research is to develop a Deep Learning model to forecast the stock price, by using the variant of Long Short-Term Memory. This model predicts the close price of the stock for the future selected date, choosing as inputs the following data: open, high, low, adj close and close prices. This model shows a comparative analysis between three different LSTM networks: Long Short-Term Memory (LSTM), Stacked Long Short-Term Memory (Stacked LSTM), and Stacked Bi-directional Long Short-Term Memory (Stacked Bidirectional LSTM) concluding which one is the best and implementing the model using that variant. We have used the historical stock prices data from yahoo’s financial website over 5 years, by choosing multiple datasets: Apple, Amazon, Google, Meta, Microsoft and Tesla (daily values). In order to get effective in the forecasting model, we have tested the network with different iterations and epochs. The model represents Multiple Graphs for data visualization in different comparisons. We have estimated the effectiveness of our proposed model by using the following performance indicators: the Mean Square Error (MSE), the Root Mean Square Error (RMSE), and the R-Squared of the model. The experimental results clearly show that our Stacked Bi-LSTM model has the highest accuracy values when comparing with the LSTM and Stacked LSTM Models. Hence, we can conclude that our Stacked Bi-LSTM Model is suitable for accurate prediction of the stock market time series. Key Words: stock price prediction, Machine Learning, stacked LSTM, Bi-directional LSTM, Deep Learning, Data pre-processing techniques, Data normalization, Data Visualization, Training and Testing Set, Financial Time Series, Future prediction

Predicting rainfall is a challenging and critical task due to its significant impact on society. Timely and accurate predictions are essential for minimizing human and financial losses. The dependence of approximately 60% of agricultural land in India on monsoon rainfall implies the crucial nature of accurate rainfall prediction. Precise rainfall forecasts can facilitate early preparedness for disasters associated with heavy rains, enabling the public and government to take necessary precautions. In the North-Western Himalayas, where meteorological data are limited, the need for improved accuracy in traditional modeling methods for rainfall forecasting is pressing. To address this, our study proposes the application of advanced machine learning (ML) algorithms, including random forest (RF), support vector regression (SVR), artificial neural network (ANN), and k-nearest neighbour (KNN) along with various deep learning (DL) algorithms such as long short-term memory (LSTM), bi-directional LSTM, deep LSTM, gated recurrent unit (GRU), and simple recurrent neural network (RNN). These advanced techniques hold the potential to significantly improve the accuracy of rainfall prediction, offering hope for more reliable forecasts. Additionally, time series techniques, including autoregressive integrated moving average (ARIMA) and trigonometric, Box-Cox transform, arma errors, trend, and seasonal components (TBATS), are proposed for predicting rainfall across the altitudinal gradients of India’s North-Western Himalayas. This approach can potentially revolutionise how we approach rainfall forecasting, ushering in a new era of accuracy and reliability. The effectiveness and accuracy of the proposed algorithms were assessed using meteorological data obtained from six weather stations at different elevations spanning from 1980 to 2021. The results indicate that DL methods exhibit the highest accuracy in predicting rainfall, as measured by the root mean squared error (RMSE) and mean absolute error (MAE), followed by ML algorithms and time series techniques. Among the DL algorithms, the accuracy order was bi-directional LSTM, LSTM, RNN, deep LSTM, and GRU. For the ML algorithms, the accuracy order was ANN, KNN, SVR, and RF. These findings suggest that altitude significantly affects the accuracy of the models, highlighting the need for additional weather stations in this mountainous region to enhance the precision of rainfall prediction.

In big data scenarios, traditional LSTM (Long Short-Term Memory) only focuses on one-way sequences and cannot use future information, resulting in weak understanding of contextual information and limited ability to handle non-stationarity. When faced with massive amounts of data, the unidirectional structure of the traditional LSTM can lose some of the potential features of the time series information, making it difficult to fully capture the long-term relationships in the data. The BiLSTM (Bidirectional Long Short-Term Memory) model introduced in this paper has a bidirectional structure and stronger context understanding ability, can better identify nonlinear and non-stationary patterns, and can more effectively capture long-term dependencies. The BiLSTM model is used to predict the time series of college student satisfaction. A sliding window is designed to generate time series training samples, and feature selection and standardization are combined to improve data quality. By setting multiple hyperparameters, the model is configured with a learning rate of 0.001, a hidden layer size of 100, and a sliding window size of 10, achieving training and stable prediction effects for colleges and universities. The experimental results show that using the optimally configured BiLSTM model, the loss of the training set is reduced to 0.245, and the loss of the validation set is reduced to 0.312. The MSE is 0.1975, the MAE is 0.290, the R 2 is 0.89, and the time series correlation coefficient is 0.94, indicating that the model has an advantage in dealing with nonlinear, non-stationary data and long-term dependency problems. The BiLSTM model optimizes model configuration through multi-hyperparameter tuning and cross-validation, achieving efficient training and stable prediction results, providing university administrators with more accurate student satisfaction trend analysis and decision support.

A wind speed forecasting technique, using deep learning architectures based on long short-term memory (LSTM) model and bidirectional long short-term memory (BiLSTM) model is presented in this work. The coastal belts of the Indian peninsula are vulnerable to natural disasters like storm surges and inundations due to cyclones each year. The wind speed is a major parameter for analyzing extreme weather events. Prediction using numerical models is not efficient enough due to the irregular patterns in the data and, thus, deep neural network models involving many layers have been tested. The shallow feed-forward model has also been considered along with deep learning models to estimate future values from past data. The present work employs a comparison study of different models to forecast wind speed time series at two locations in the Bay of Bengal and the Arabian Sea, respectively, having different dynamics and randomness. For training the models, daily wind speed data are considered for the period 2006–2017 and an independent validation set is chosen comprising 2018 wind speed data to check the accuracy. To evaluate forecast efficiency among different network models fitted to given time series, mean square error (MSE) and root mean square error (RMSE) have been computed. Multiple experiments are conducted with different hidden unit values and epoch values to obtain the minimum error. Regression equations generated may be used for forecasting future time series. The BiLSTM model connecting hidden states of opposite directions proved to be most efficient for the wind speed forecasting in different regions.

Sign language is the primary communication tool used by the deaf community and people with speaking difficulties, especially during emergencies. Numerous deep learning models have been proposed to solve the sign language recognition problem. Recently. Bidirectional LSTM (BLSTM) has been proposed and used in replacement of Long Short-Term Memory (LSTM) as it may improve learning long-team dependencies as well as increase the accuracy of the model. However, there needs to be more comparison for the performance of LSTM and BLSTM in LRCN model architecture in sign language interpretation applications. Therefore, this study focused on the dense analysis of the LRCN model, including 1) training the CNN from scratch and 2) modeling with pre-trained CNN, VGG-19, and ResNet50. Other than that, the ConvLSTM model, a special variant of LSTM designed for video input, has also been modeled and compared with the LRCN in representing emergency sign language recognition. Within LRCN variants, the performance of a small CNN network was compared with pre-trained VGG-19 and ResNet50V2. A dataset of emergency Indian Sign Language with eight classes is used to train the models. The model with the best performance is the VGG-19 + LSTM model, with a testing accuracy of 96.39%. Small LRCN networks, which are 5 CNN subunits + LSTM and 4 CNN subunits + BLSTM, have 95.18% testing accuracy. This performance is on par with our best-proposed model, VGG + LSTM. By incorporating bidirectional LSTM (BLSTM) into deep learning models, the ability to understand long-term dependencies can be improved. This can enhance accuracy in reading sign language, leading to more effective communication during emergencies.

With the development of cities, the total number of trucks has increased year by year. Traffic flow forecasting has become an indispensable part of the cargo transportation industry and directly affects the development of the transportation industry. In the field of traffic flow prediction, Long Short-Term Memory(LSTM) model has advantages in processing time series, but it cannot extract the periodicity in time series. Therefore, in this experiment, a Bidirectional Long Short-Term Memory (BLSTM) model was constructed to predict traffic flow in the road network. It is worth mentioning that this article considers the non-parametric model autoregressive integrated moving average model (ARIMA) and the parametric model recurrent neural network (RNN) to compare and analyze with LSTM. Data from Guangwu Toll Station, Zhengzhou city, China were used to calibrate and evaluate the models. The experimental results show that the performance of RNN based on deep learning such as BLSTM and LSTM model is better than that of ARIMA. In order to better illustrate the advantages of BLSTM model, we comprehensively considered the performance effects of four models under morning peak, evening peak and flat peak. Experiments have proved that BLSTM has good nonlinear fitting ability and anti-noise ability, and the average prediction accuracy reaches 92.873%.

Protein Secondary structure prediction is an emerging topic in bioinformatics to understand briefly the functions of protein and their role in drug invention, medicine and biology. In our research we have applied two recurrent neural network based approach Bi-LSTM (Bidirectional Long Short-Term Memory) and LSTM (Long Short-Term Memory). Our research was focused on primary structure up to 134 in length of amino acids. Initially our proposed model produced a ‘Indexed Lexicon of corpus’ using tri-gram conversion for primary structure strings. Each primary structure tri-gram transformed snippets is substituted with its associated index mentioned in ‘Indexed corpus’. The indexed parameter vector inputted into our proposed Bi-LSTM and LSTM model. We got best accuracy when we have used two Bi-LSTM and three LSTM layers respectively in Bi-LSTM and LSTM models. To prevent biasness and minimize overfitting problem we have utilized two dropout layers for each of Bi-LSTM and LSTM model. We have operated our model on ccPDB 2.0 benchmark dataset. There is total eight states protein secondary structure in this dataset. For this sst8 secondary structure we have achieved 83.24% accuracy for our proposed LSTM model and 89.10% accuracy for our Bi-LSTM model. We have configured our model to run for 50 epochs with batch size 64. For compilation of our models we have utilized ‘adam’ optimizer and the ‘categorical crossentropy’ loss function. To make dataset balanced to our model we have also employed 5-fold cross validation.

In recent times, there is an increasing interest in employing technology to process natural language with the aim of providing information that can benefit society. Language identification refers to the process of detecting which speech a speaker appears to be using. This paper presents an audio-based Ethio-semitic language identification system using Recurrent Neural Network. Identifying the features that can accurately differentiate between various languages is a difficult task because of the very high similarity between characters of each language. Recurrent Neural Network (RNN) was used in this paper in relation to the Mel-frequency cepstral coefficients (MFCCs) features to bring out the key features which helps provide good results. The primary goal of this research is to find the best model for the identification of Ethio-semitic languages such as Amharic, Geez, Guragigna, and Tigrigna. The models were tested using an 8-h collection of audio recording. Experiments were carried out using our unique dataset with an extended version of RNN, Long Short Term Memory (LSTM) and Bidirectional Long Short Term Memory (BLSTM), for 5 and 10 s, respectively. According to the results, Bidirectional Long Short Term Memory (BLSTM) with a 5 s delay outperformed Long Short Term Memory (LSTM). The BLSTM model achieved average results of 98.1, 92.9, and 89.9% for training, validation, and testing accuracy, respectively. As a result, we can infer that the best performing method for the selected Ethio-Semitic language dataset was the BLSTM algorithm with MFCCs feature running for 5 s.

<abstract> <p>The adjusted precision of economic values is essential in the global economy. In recent years, researchers have increased their interest in making accurate predictions in this type of time series; one of the reasons is that the characteristics of this type of time series makes predicting a complicated task due to its non-linear nature. The evolution of artificial neural network models enables us to research the suitability of models generated for other purposes, applying their potential to time series prediction with promising results. Specifically, in this field, the application of transformer models is assuming an innovative approach with great results. To improve the performance of this type of networks, in this work, the empirical model decomposition (EMD) methodology was used as data preprocessing for prediction with a transformer type network. The results confirmed a better performance of this approach compared to networks widely used in this field, the bidirectional long short term memory (BiLSTM), and long short term memory (LSTM) networks using and without EMD preprocessing, as well as the comparison of a Transformer network without applying EMD to the data, with a lower error in all the error metrics used: The root mean square error (RMSE), the root mean square error (MSE), the mean absolute percentage error (MAPE), and the R-square (R<sup>2</sup>). Finding a model that provides results that improve the literature allows for a greater adjustment in the predictions with minimal preprocessing.</p> </abstract>

BackgroundPredicting the bed occupancy rate (BOR) is essential for efficient hospital resource management, long-term budget planning, and patient care planning. Although macro-level BOR prediction for the entire hospital is crucial, predicting occupancy at a detailed level, such as specific wards and rooms, is more practical and useful for hospital scheduling.ObjectiveThe aim of this study was to develop a web-based support tool that allows hospital administrators to grasp the BOR for each ward and room according to different time periods.MethodsWe trained time-series models based on long short-term memory (LSTM) using individual bed data aggregated hourly each day to predict the BOR for each ward and room in the hospital. Ward training involved 2 models with 7- and 30-day time windows, and room training involved models with 3- and 7-day time windows for shorter-term planning. To further improve prediction performance, we added 2 models trained by concatenating dynamic data with static data representing room-specific details.ResultsWe confirmed the results of a total of 12 models using bidirectional long short-term memory (Bi-LSTM) and LSTM, and the model based on Bi-LSTM showed better performance. The ward-level prediction model had a mean absolute error (MAE) of 0.067, mean square error (MSE) of 0.009, root mean square error (RMSE) of 0.094, and R2 score of 0.544. Among the room-level prediction models, the model that combined static data exhibited superior performance, with a MAE of 0.129, MSE of 0.050, RMSE of 0.227, and R2 score of 0.600. Model results can be displayed on an electronic dashboard for easy access via the web.ConclusionsWe have proposed predictive BOR models for individual wards and rooms that demonstrate high performance. The results can be visualized through a web-based dashboard, aiding hospital administrators in bed operation planning. This contributes to resource optimization and the reduction of hospital resource use.