Like Long Short-Term Memory Research Articles

Modeling the Time Evolution of Compact Binary Systems with Machine Learning

This work introduces advanced computational techniques for modeling the time evolution of compact binary systems using machine learning. The dynamics of compact binary systems, such as black holes and neutron stars, present significant nonlinear challenges due to the strong gravitational interactions and the requirement for precise numerical simulations. Traditional methods, like the post-Newtonian approximation, often require significant computational resources and face challenges in accuracy and efficiency. Here, we employed machine learning algorithms, including deep learning models like long short-term memory (LSTM) and temporal convolutional network (TCN), to predict the future evolution of these systems based on extensive simulation data. Our results demonstrate that employing both LSTM and TCN even as black-box predictors for sequence prediction can also significantly improve the prediction accuracy without physics-informed neural networks (as partial differential equation solvers with prior knowledge or inductive bias. By employing LSTM and TCN, we obtained R 2 values of 99.74% and 99.19% for the evolutionary orbits of the compact binaries data set, respectively. Our models demonstrate the ability to effectively capture the dynamics of the binaries, achieving high prediction performance with significantly reduced computational overhead by a factor of 40, compared to conventional numerical methods. This study paves the way for more effective and computationally scalable approaches to the understanding of gravitational phenomena and predictive modeling in gravitational-wave astronomy.

Reservoir Operation based Machine Learning Models: Comprehensive Review for Limitations, Research Gap, and Possible Future Research Direction

The operation of dams and reservoirs is critical for water resource management, including flood control, irrigation, hydropower generation, and environmental conservation. Traditional optimization techniques like Dynamic Programming (DP), Linear Programming (LP), and Nonlinear Programming (NLP) have been foundational in managing these operations. However, they often fall short in addressing the complexities of modern water management challenges posed by climate variability and increasing water demands. Machine learning (ML) techniques have emerged as powerful tools to enhance the efficiency and accuracy of dam and reservoir operations. This paper provides a comprehensive review of various ML models, including Neural Networks, Genetic Algorithms, Decision Trees, and Ensemble Methods, highlighting their applications in predicting reservoir inflows, optimizing water release schedules, and improving flood risk management. Notably, ML models like Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNNs) have shown significant improvements in forecasting accuracy and operational decision-making. Despite these advancements, several limitations and research gaps persist, including the need for real-time data integration, adaptive learning mechanisms, and models that consider socio-economic and climatic factors. This review underscores the importance of addressing these gaps to develop more robust and generalizable ML models. Future research directions are suggested to focus on hybrid models combining ML with traditional optimization techniques, comprehensive validation across diverse conditions, and the integration of ecological and economic considerations. By systematically identifying and addressing these limitations, this research aims to pave the way for more effective and sustainable dam and reservoir management practices. Leading towards suggesting that enhancing real-time data integration and developing adaptive learning mechanisms are in order to improve model responsiveness.

Impact of Effective Word Vectors on Deep Learning Based Subjective Classification of Online Reviews

Sentiment Analysis tasks are made considerably simpler by extracting subjective statements from online reviews, thereby reducing the overhead of the classifiers. The review dataset encompasses both subjective and objective sentences, where subjective writing expresses the author's opinions, and objective text presents factual information. Assessing the subjectivity of review statements involves categorizing them as objective or subjective. The effectiveness of word vectors plays a crucial role in this process, as they capture the semantics and contextual cues of a subjective language. This study investigates the significance of employing sophisticated word vector representations to enhance the detection of subjective reviews. Several methodologies for generating word vectors have been investigated, encompassing both conventional approaches, such as Word2Vec and Global Vectors for word representation, and recent innovations, such as like Bidirectional Encoder Representations from Transformers (BERT), ALBERT, and Embeddings from Language Models. These neural word embeddings were applied using Keras and Scikit-Learn. The analysis focuses on Cornell subjectivity review data within the restaurant domain, and metrics evaluating performance, such as accuracy, F1-score, recall, and precision, are assessed on a dataset containing subjective reviews. A wide range of conventional vector models and deep learning-based word embeddings are utilized for subjective review classification, frequently in combination with deep learning architectures like Long Short-Term Memory (LSTM). Notably, pre-trained BERT-base word embeddings exhibited exceptional accuracy of 96.4%, surpassing the performance of all other models considered in this study. It has been observed that BERT-base is expensive because of its larger structure.

Exploring Different Dynamics of Recurrent Neural Network Methods for Stock Market Prediction - A Comparative Study

ABSTRACT The intricate and unpredictable nature of stock markets underscores the importance of precise forecasting for timely detection of downturns and subsequent rebounds. Various factors, including news, rumors surrounding events or companies, market sentiments, and governmental policies, can significantly impact stock prices. Nevertheless, the precision of current methods remained insufficient until the adoption of artificial neural network architectures like long short-term memory (LSTM). The aim of this study is to create a precise AI-driven platform tailored for both the Indian and international stock markets. This platform is designed to assist retail investors in navigating digital environments by employing various LSTM algorithms. Its primary goals include predicting stock price fluctuations, pinpointing potential investment prospects, and refining trading strategies. The application aims to leverage advanced LSTM algorithms to analyze historical market data, recognize patterns, and provide real-time insights. It will take past price and process it through LSTM algorithms to take a logical decision. In the quest to broaden retail participation in the capital markets, the effort is to develop an application for novice investors who either have no time in research or are the victims of financial mis-selling and enable them to leverage the technology to their advantage.

Evaluation of Data Sufficiency for Interannual Knowledge Transfer of Crop Type Classification Models

We present a study on the effectiveness of using varying data sizes to transfer crop type classification models from one year to the next, emphasizing the balance between data sufficiency and model accuracy. The significance of crop detection through satellite imaging lies in its potential to enhance agricultural productivity and resource management. Machine learning, particularly techniques like long short-term memory (LSTM) models, has become instrumental in interpreting these satellite data due to its predictive accuracy and adaptability. However, the direct application of models trained in one year to subsequent years poses challenges due to variations in environmental conditions and agricultural practices. Fine-tuning pre-existing models is a prevalent strategy to overcome these temporal discrepancies, though it necessitates a careful evaluation of the quantity and relevance of new data. This study explores the cost–benefit of fine-tuning existing models versus developing new ones based on the quantity of new data, utilizing LSTM models for their transferability and practicality in agricultural applications. Experiments conducted using satellite data from farms in southern Alberta reveal that smaller datasets, with fewer than 25 fields per class, can effectively fine-tune models for accurate interannual classification, while larger datasets are more conducive to training new models. This poses a key challenge in optimizing data usage for crop classification, straddling the line between data sufficiency and computational efficiency. The findings offer valuable insights for optimizing data use in crop classification, benefiting both academic research and practical agricultural applications.

Unraveling Financial Markets: Deep Neural Network-Based Models for Stock Price Prediction

This paper delves into the potential and challenges of leveraging deep neural networks (DNNs) in stock price forecasting. Traditional econometric models often grapple with the complexities of financial time series data, leading to the exploration of DNNs, especially architectures like Long Short-Term Memory (LSTM) networks, to capture intricate patterns in such data. While these networks present promising results, challenges such as model interpretability, non-stationarity of data, overfitting, and computational demands remain. The financial sector's increasing digitization and influx of alternative data offer a unique opportunity for DNNs, emphasizing their capability for automatic feature extraction. However, the integration of DNNs necessitates a multi-disciplinary approach, involving financial experts, data scientists, and computational specialists. The paper concludes with an optimistic outlook for the synergy between deep learning and traditional financial theories, aiming for a harmonious blend of modern computational techniques with foundational financial principles.

Probing the limit of hydrologic predictability with the Transformer network

For a number of years since their introduction to hydrology, recurrent neural networks like long short-term memory (LSTM) networks have proven remarkably difficult to surpass in terms of daily hydrograph metrics on community-shared benchmarks. Outside of hydrology, Transformers have now become the model of choice for sequential prediction tasks, making it a curious architecture to investigate for application to hydrology. Here, we first show that a vanilla (basic) Transformer architecture is not competitive against LSTM on the widely benchmarked CAMELS streamflow dataset, and lagged especially prominently for the high-flow metrics, perhaps due to the lack of memory mechanisms. However, a recurrence-free variant of the Transformer model obtained mixed comparisons with LSTM, producing very slightly higher Kling-Gupta efficiency coefficients (KGE), along with other metrics. The lack of advantages for the vanilla Transformer network is linked to the nature of hydrologic processes. Additionally, similar to LSTM, the Transformer can also merge multiple meteorological forcing datasets to improve model performance. Therefore, the modified Transformer represents a rare competitive architecture to LSTM in rigorous benchmarks. Valuable lessons were learned: (1) the basic Transformer architecture is not suitable for hydrologic modeling; (2) the recurrence-free modification is beneficial, so future work should continue to test such modifications; and (3) the performance of state-of-the-art models may be close to the prediction limits of the dataset. As a non-recurrent model, the Transformer may bear scale advantages for learning from bigger datasets and storing knowledge. This work lays the groundwork for future explorations into pretraining models, serving as a foundational benchmark that underscores the potential benefits in hydrology.

Machine Learning Algorithms for Predicting Energy Consumption in Educational Buildings

In the past few years, there has been a notable interest in the application of machine learning methods to enhance energy efficiency in the smart building industry. The paper discusses the use of machine learning in smart buildings to improve energy efficiency by analyzing data on energy usage, occupancy patterns, and environmental conditions. The study focuses on implementing and evaluating energy consumption prediction models using algorithms like long short-term memory (LSTM), random forest, and gradient boosting regressor. Real-life case studies on educational buildings are conducted to assess the practical applicability of these models. The data is rigorously analyzed and preprocessed, and performance metrics such as root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are used to compare the effectiveness of the algorithms. The results highlight the importance of tailoring predictive models to the specific characteristics of each building’s energy consumption.

Admission Prediction Using Time Series Analysis

Efficient resource allocation and strategic planning in educational institutions heavily rely on accurate predictions of student admissions. This paper presents a detailed investigation into the application of time series analysis techniques for admission prediction. We explore the utilization of historical admission data, spanning multiple academic cycles, to train and evaluate several time series models.The study encompasses a comparative analysis of traditional statistical models such as autoregressive integrated moving average (ARIMA) and seasonal autoregressive integrated moving average (SARIMA) alongside more advanced deep learning techniques like long short-term memory (LSTM) networks. By evaluating the performance metrics, including accuracy, robustness, and computational efficiency, we aim to identify the most suitable model for admission prediction tasks. Moreover, the research delves into the impact of various external factors such as changes in the academic calendar, socio-economic indicators, and demographic shifts on admission patterns. Understanding these factors is crucial for enhancing the predictive capabilities of the models and enabling institutions to adapt their strategies accordingly. The experimental results and comparative analysis provide valuable insights for educational institutions, enabling them to make data-driven decisions regarding enrollment management strategies and resource allocation. Ultimately, this research contributes to the advancement of admission prediction methodologies, facilitating more efficient and informed decision-making processes in the educational domain.

Intelligence model for Alzheimer’s disease detection with optimal trained deep hybrid model

Alzheimer’s disease (AD), a neurodegenerative disorder, is the most common cause of dementia and continuing cognitive deficits. Since there are more cases each year, AD has grown to be a serious social and public health issue. Early detection of the diagnosis of Alzheimer’s and dementia disease is crucial, as is giving them the right care. The importance of early AD diagnosis has recently received a lot of attention. The patient cannot receive a timely diagnosis since the present methods of diagnosing AD take so long and are so expensive. That’s why we created a brand-new AD detection method that has four steps of operation: pre-processing, feature extraction, feature selection, and AD detection. During the pre-processing stage, the input data is pre-processed using an improved data normalization method. Following the pre-processing, these pre-processed data will go through a feature extraction procedure where features including statistical, enhanced entropy-based and mutual information-based features will be extracted. The appropriate features will be chosen from these extracted characteristics using the enhanced Chi-square technique. Based on the selected features, a hybrid model will be used in this study to detect AD. This hybrid model combines classifiers like Long Short Term Memory (LSTM) and Deep Maxout neural networks, and the weight parameters of LSTM and Deep Maxout will be optimized by the Self Updated Shuffled Shepherd Optimization Algorithm (SUSSOA). Our Proposed SUSSOA-based method’s statistical analysis of best values such as 57%, 53%, 28%, 25%, and 21% is higher than the other models like SSO, BMO, HGS, BRO, BES, and ISSO respectively.

Prediction of Load Demand in Home Area Network Using LSTM

Load forecasting is one of the most important tools for the energy management system in the modern world. Forecasting power consumption and load demand calculation became an interesting topic for the stakeholders in the electricity market. Decision-making in purchasing and generating electric power, load switching, and demand side management are dependent on load forecasting. This research work focuses on the prediction of power consumption in a Home Area Network (HAN) using time series forecasting methods like Long short-term memory (LSTM) neural Networks. Our goal was to design a model that could precisely forecast the electrical load required in a Home Area Network. Utility companies and homeowners can utilize this model to better plan and control their power usage. It can assist in lowering the likelihood of power outages and enhancing the effectiveness of the electrical system. The Root Mean Squared Error (RMSE) is used as the performance measure in our work. The dataset considered is from the UCI repository with 350400 rows and 4 parameters such as active power, and 3 submetering values, to achieve realistic data for training the model used the encoder-decoder LSTM model for load prediction and achieved an RMSE value of 23 and an MAE value of 18.6

Masked Statement Classification in Conversational Text

Sarcasm can be really tricky for computer programs that analyze text to understand people’s feelings and opinions. This is because sarcasm involves saying something with the opposite meaning of what you actually feel, often to make fun of a situation. When computers analyze text, they focus on the words used rather than the intended meaning, which can lead to mistakes in how the text is categorized. Sarcasm also creates problems when trying to figure out the author’s true emotions in tasks like sentiment analysis. So, there is a need of a way to identify sarcasm in text and correct the sentiment it conveys. Recent studies have employed deep learning and machine learning methods to identify sarcasm. Deep learning is quite powerful, but traditional machine learning methods are still widely used and can work as well if trained properly. This study aims to compare different algorithms like Long Short Term Memory (LSTM), K-Nearest Neighbors (KNN), Computational Neural Network(CNN), and Artificial Neural Network(ANN) etc to see how well they work in sarcasm detection. The survey examined several systems and algorithms for detecting sarcasm, as well as any faults or drawbacks associated with these approaches. It also discusses various important challenges and future directions for sarcasm detection research in order to provide useful insights in this field

Empowering IoT Healthcare Systems with Deep Learning: From Sensor Data Fusion to Predictive Modeling and Intervention

Adding Internet of Things (IoT) technology to healthcare systems has changed the way patients are cared for by letting them be monitored and data collected in real time. This essay looks at how deep learning can be used to improve IoT-enabled healthcare systems, with a focus on combining sensor data, making predictions, and coming up with ways to help.Sensor data fusion is a key part of putting together data from different sources, like medical equipment, smart tech, and electronic health records. Deep learning algorithms, especially CNN and RNN are very good at handling different types of data streams. This makes it possible to get a full picture of a patient's health. Healthcare professionals can get a full picture of a patient's health by combining information from many sources, such as bodily signs, exercise levels, and outdoor factors. Based on past data, predictive modeling uses the power of deep learning to guess what will happen with people's health in the future. IoT healthcare systems can predict how a disease will get worse, find risk factors, and suggest early treatment using methods like long short-term memory (LSTM) networks and attention mechanisms. These prediction models allow for quick treatments, methods for preventive care, and the best use of resources, which improves patient results and lowers healthcare costs in the long run.Deep learning also makes it easier to come up with smart management methods that are specific to each patient's needs. Machine learning algorithms can make personalized treatment suggestions and adaptable care plans by looking at real-time monitor data along with old patient records. These treatments could include changes to medications or lifestyles, or tips for medical workers. These give patients and healthcare staff more information to help them make better choices and better handle chronic conditions. When IoT technology and deep learning are combined, they have the ability to completely change the way healthcare is provided. IoT-enabled healthcare systems can improve patient tracking, analysis, and treatment by using advanced algorithms for sensor data fusion, predictive models, and smart actions. This leads to better quality of care and better health results.

LiteTransNet: An interpretable approach for landslide displacement prediction using transformer model with attention mechanism

Accurate landslide displacement prediction is crucial for effective early warning systems to mitigate hazards. The importance of historical information varies with time during prediction due to the underlying landslide deformation mechanism. Despite advances in dynamic machine learning models like Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), they struggle to fully capture and interpret the varying importance of historical information during prediction, resulting in decreased accuracy and limited understanding of landslide physical mechanisms. Additionally, these approaches rely on manual feature selection, which is independent of the learning process and therefore is prone to estimation errors. To address these challenges, we propose LiteTransNet, a Lightweighted Transformer Network tailored for landslide displacement prediction. Built on the revolutionary Transformer model for sequential data, LiteTransNet leverages localized self-attention to selectively focus on relevant timestamps and provides interpretable results through its attention heatmap. Furthermore, LiteTransNet performs end-to-end time series modeling without extensive feature engineering. We validate LiteTransNet on two landslides in China's Three Gorges Reservoir Area and demonstrate its improved accuracy over recurrent baselines utilizing feature selection methods. Notably, the attention heatmap generated by LiteTransNet provides interpretable insights into the model's temporal dependency learning. It uncovers the varying importance of historical information at different timestamps, showcasing how LiteTransNet's attention aligns with specific periods of the external environment that cause significant disruptions in the landslide system. Our findings reveal that these disruptions continuously impact displacement prediction until their effects fade or are replaced by subsequent intense disturbances, which traditional recurrent methods are unable to capture. Further experiment shows that LiteTransNet enhances efficiency through its streamlined architecture and its parallelization capability. Overall, LiteTransNet innovates landslide prediction by providing accuracy, interpretability, and efficiency, enhancing understanding of deformation mechanisms to facilitate effective early warning systems.

Hybrid KNN-SVM machine learning approach for solar power forecasting

Predictions about solar power will have a significant impact on large-scale renewable energy plants. Photovoltaic (PV) power generation forecasting is particularly sensitive to measuring the uncertainty in weather conditions. Although several conventional techniques like long short-term memory (LSTM), support vector machine (SVM), etc. are available, but due to some restrictions, their application is limited. To enhance the precision of forecasting solar power from solar farms, a hybrid machine learning model that includes blends of the K-Nearest Neighbor (KNN) machine learning technique with the SVM to increase reliability for power system operators is proposed in this investigation. The conventional LSTM technique is also implemented to compare the performance of the proposed hybrid technique. The suggested hybrid model is improved by the use of structural diversity and data diversity in KNN and SVM, respectively. For the solar power predictions, the suggested method was tested on the Jodhpur real-time series dataset obtained from the data centers of weather stations using Meteonorm. The data set includes metrics such as Hourly Average Temperature (HAT), Hourly Total Sunlight Duration (HTSD), Hourly Total Global Solar Radiation (HTGSR), and Hourly Total Photovoltaic Energy Generation (HTPEG). The collated data has been segmented into training data, validation data, and testing data. Furthermore, the proposed technique performed better when evaluated on the three performance indices, viz., accuracy, sensitivity, and specificity. Compared with the conventional LSTM technique, the hybrid technique improved the prediction with 98 % accuracy.

Investigating Performance of ESN’s in Forecasting Financial Metrics When Compared To Traditional RNN Types

This research investigates the performance of Echo State Networks (ESN) in forecasting financial metrics and compares their effectiveness against traditional recurrent neural network (RNN) architectures like Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRU), as well as Generalized Auto Regressive Conditional Heteroskedasticity (GARCH) models. By analyzing datasets sourced from Yahoo Finance for various financial indices, exchange-traded funds and stocks over five years, this study examines the accuracy, and structural simplicity of ESNs in predicting close prices, daily volatility, and log returns. Results indicate that ESNs, with their reservoir computing capabilities, outperform traditional RNNs by achieving lower mean absolute error (MAE) and mean squared error (MSE) overall, highlighting their potential as efficient and robust tools for financial time-series forecasting.

CANBLWO: A Novel Hybrid Approach for Semantic Text Generation

Semantic text generation is critical in Natural Language Processing (NLP) as it faces challenges such as maintenance of coherence among texts, contextual relevance, and quality output. Traditional language models often produce grammatically inconsistent text. To address these issues, we introduce Convolutional Attention Bi-LSTM with Whale Optimization (CANBLWO), a novel hybrid model that integrates a Convolutional Attention Network (CAN), Bidirectional Long Short-Term Memory (Bi-LSTM), and Whale Optimization Algorithm (WOA). CANBLWO aims to generate semantically rich and coherent text and outperforms the traditional models like Long Short-Term Memory (LSTM), Recurrent Neural Networks (RNN), Bi-LSTM, and Bi-LSTM with attention, Bidirectional Encoder Representations from Transformers (BERT), and Generative Pre-trained Transformer 2 (GPT-2). Our model achieved 0.79, 0.78, 0.76, and 0.82 scores in Metric for Evaluation of Translation with Explicit Ordering (METEOR), Bi-Lingual Evaluation Understudy (BLEU), Consensus-based Image Description Evaluation (Ciders), and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics, respectively. The proposed model also demonstrates 97% and 96% accuracy on Wiki-Bio and Code/Natural Language Challenge (CoNaLa) datasets, highlighting its effectiveness against Large Language Models (LLMs). This study underscores the potential capability of hybrid approaches in enhancing semantic text generation

Tri-model classifiers for EEG based mental task classification: hybrid optimization assisted framework

The commercial adoption of BCI technologies for both clinical and non-clinical applications is drawing scientists to the creation of wearable devices for daily living. Emotions are essential to human existence and have a significant impact on thinking. Emotion is frequently linked to rational decision-making, perception, interpersonal interaction, and even basic human intellect. The requirement for trustworthy and implementable methods for the detection of individual emotional responses is needed with rising attention of the scientific community towards the establishment of some significant emotional connections among people and computers. This work introduces EEG recognition model, where the input signal is pre-processed using band pass filter. Then, the features like discrete wavelet transform (DWT), band power, spectral flatness, and improved Entropy are extracted. Further, for recognition, tri-classifiers like long short term memory (LSTM), improved deep belief network (DBN) and recurrent neural network (RNN) are used. Also to enhance tri-model classifier performance, the weights of LSTM, improved DBN, and RNN are tuned by model named as shark smell updated BES optimization (SSU-BES). Finally, the perfection of SSU-BES is demonstrated over diverse metrics.

Scene description with context information using dense-LSTM

Generating natural language description for visual content is a technique for describing the content available in the image(s). It requires knowledge of both the domains of computer vision and natural language processing. For this, various models with different approaches are suggested. One of them is encoder-decoder-based description generation. Existing papers used only objects for descriptions, but the relationship between them is equally essential, requiring context information. Which required techniques like Long Short-Term Memory (LSTM). This paper proposes an encoder-decoder-based methodology to generate human-like textual descriptions. Dense-LSTM is presented for better description as a decoder with a modified VGG19 encoder to capture information to describe the scene. Standard datasets Flickr8K and Flickr30k are used for testing and training purposes. BLEU (Bilingual Evaluation Understudy) score is used to evaluate the generated text. For the proposed model, a GUI (Graphical User Interface) is developed, which produces the audio description of the output received and provides an interface for searching the related visual content and query-based search.

Algorithms for Hyperparameter Tuning of LSTMs for Time Series Forecasting

The rapid growth in the use of Solar Energy for sustaining energy demand around the world requires accurate forecasts of Solar Irradiance to estimate the contribution of solar power to the power grid. Accurate forecasts for higher time horizons help to balance the power grid effectively and efficiently. Traditional forecasting techniques rely on physical weather parameters and complex mathematical models. However, these techniques are time-consuming and produce accurate results only for short forecast horizons. Deep Learning Techniques like Long Short Term Memory (LSTM) networks are employed to learn and predict complex varying time series data. However, LSTM networks are susceptible to poor performance due to improper configuration of hyperparameters. This work introduces two new algorithms for hyperparameter tuning of LSTM networks and a Fast Fourier Transform (FFT) based data decomposition technique. This work also proposes an optimised workflow for training LSTM networks based on the above techniques. The results show a significant fitness increase from 81.20% to 95.23% and a 53.42% reduction in RMSE for 90 min ahead forecast after using the optimised training workflow. The results were compared to several other techniques for forecasting solar energy for multiple forecast horizons.