Short-term Memory Network Research Articles

A novel hybrid deep learning IChOA-CNN-LSTM model for modality-enriched and multilingual emotion recognition in social media.

In the rapidly evolving field of artificial intelligence, the importance of multimodal sentiment analysis has never been more evident, especially amid the ongoing COVID-19 pandemic. Our research addresses the critical need to understand public sentiment across various dimensions of this crisis by integrating data from multiple modalities, such as text, images, audio, and videos sourced from platforms like Twitter. Conventional methods, which primarily focus on text analysis, often fall short in capturing the nuanced intricacies of emotional states, necessitating a more comprehensive approach. To tackle this challenge, our proposed framework introduces a novel hybrid model, IChOA-CNN-LSTM, which leverages Convolutional Neural Networks (CNNs) for precise image feature extraction, Long Short-Term Memory (LSTM) networks for sequential data analysis, and an Improved Chimp Optimization Algorithm for effective feature fusion. Remarkably, our model achieves an impressive accuracy rate of 97.8%, outperforming existing approaches in the field. Additionally, by integrating the GeoCoV19 dataset, we facilitate a comprehensive analysis that spans linguistic and geographical boundaries, enriching our understanding of global pandemic discourse and providing critical insights for informed decision-making in public health crises. Through this holistic approach and innovative techniques, our research significantly advances multimodal sentiment analysis, offering a robust framework for deciphering the complex interplay of emotions during unprecedented global challenges like the COVID-19 pandemic.

A Combined Long Short-Term Memory and Temporal Convolutional Network Model for Crop Yield Prediction

A Combined Long Short-Term Memory and Temporal Convolutional Network Model for Crop Yield Prediction

Generative adversarial networks for stack voltage degradation and RUL estimation in PEMFCs under static and dynamic loads

Accurately predicting the degradation and remaining useful life (RUL) of Proton Exchange Membrane Fuel Cells (PEMFCs) is crucial for enhancing their reliability, particularly in automotive and energy sectors. Traditional models often fail to capture the complex, non-linear degradation patterns of PEMFCs, resulting in suboptimal predictions. This study addresses these technological gaps by introducing a novel approach, the Self-Attention Temporal Dual Discriminator Generative Adversarial Network (SAT-DD-GAN). This model is designed to overcome the limitations of existing methods by integrating advanced Long Short-Term Memory (LSTM) networks for time-series analysis, a self-attention mechanism to focus on critical degradation signals, and dual discriminators to improve prediction accuracy and robustness. By addressing the gap in handling long-term dependencies and complex degradation patterns, the proposed SAT-DD-GAN bridges the disconnect between traditional models and the real-world degradation behavior of PEMFCs. The SAT-DD-GAN achieved groundbreaking predictive performance, with the lowest RMSE (0.983E-3 for static load, 1.012E-3 for dynamic load), MAPE (0.036E-1 and 0.253E-1), MAE (0.776E-3 and 0.806E-3), and the highest R2-score (0.9998 and 0.9983), alongside zero relative error for both datasets. These results demonstrate the model's ability to capture the intricate degradation dynamics more effectively than existing state-of-the-art methods. By filling the technology gap of accurately modeling PEMFC degradation under varying load conditions, this study establishes a new standard for PEMFC prognostics and paves the way for future innovations in optimizing fuel cell performance and longevity.

Three-dimensional cooperative guidance with impact angle constraints via value-policy decomposed multi-agent reinforcement learning

For a three-dimensional impact angle-constrained time-coordination guidance problem, a value-policy decomposed multi-agent twin delayed deep deterministic (VPD-MATD3) policy gradient algorithm is proposed in this paper, and a temporal-spatial cooperative guidance law is designed consequently. The derived cooperative engagement kinematics is formulated as a value-policy decomposed partially observable Markov decision process. Value decomposition and policy decomposition are correspondingly proposed to address the two bottlenecks of reward coupling and policy coupling in 3D guidance, thus improving the training process. The training environment with multiple uncertainties, such as observation noise, maneuverability perturbation, autopilot time lag, switching communication topology, and random initialization, is subsequently constructed. Time-energy type reward functions are designed, incorporating energy consumption as a direct optimization indicator via a penalty term. A long short-term memory network layer is inserted inside the policy networks to suppress the negative effect of observation noise. The policy training curves verify the significant effect of the proposed algorithm in improving training convergence. The policy testing results illustrate the robustness and generalization of the VPD-MATD3 cooperative guidance law and its ability to cope with randomly switched communication topologies. Moreover, the comparative testing further reveals its time-energy suboptimal property.

DEEP LEARNING MODEL FOR DETECTING TERROR FINANCING PATTERNS IN FINANCIAL TRANSACTIONS

Terror financing remains a critical threat to global security, with illicit actors continually adapting their methods to evade detection. Traditional financial monitoring systems often struggle to identify the complex and covert patterns associated with terror-related transactions due to their reliance on predefined rules and statistical thresholds. This study introduces advanced deep learning models designed to detect terror financing patterns within vast datasets of financial transactions. We developed and evaluated several neural network architectures, including Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs), to capture both spatial and temporal transaction features. The models were trained and tested on a dataset comprising anonymized financial transactions labeled for suspicious activities related to terror financing. Our deep learning models demonstrated superior performance over conventional machine learning approaches, achieving higher accuracy, precision, and recall in identifying suspicious transactions. Notably, the LSTM-based model excelled in detecting sequential transaction patterns indicative of layering and integration stages commonly used in terror financing. The results underscore the potential of deep learning techniques in enhancing the capabilities of financial institutions and regulatory bodies to combat terror financing. Implementing such models can lead to more proactive and effective monitoring systems that adapt to evolving illicit financing strategies.

Parameter selection for long short-term memory networks with multi-criteria decision-making tools: an application for G7 countries stock market forecasting

Parameter selection for long short-term memory networks with multi-criteria decision-making tools: an application for G7 countries stock market forecasting

A Highly Accurate Adverse Drug Reactions (ADR) Detection from Medical Forum Comments Using Long Short-Term Memory Networks

Purpose: Adverse Drug Reactions (ADR) classification is useful in modern medical diagnostics and related applications. ADR is an example of how medical information is frequently accessible on social media platforms for healthcare, where people can share their experiences with treatments on desktop computers and mobile devices. Many researchers are interested in gathering valuable medical data from social media for the ADR system training and classification process. Materials and Methods: This research explores the effects of three aspects on recognizing ADR mentions in social media for the medical field and proposes a deep neural network of Long Short-Term Memory (LSTM) neural networks to do so. The comments are collected from various social media platforms to implement the ADR system with proper training and testing processes. The texts from the dataset are initially preprocessed by using a data filtering and clustering process to remove the input data's redundant information to increase the training process's quality. Characteristic features, such as semantic features and text statistics, are extracted from the input text using the American Standard Code for Information Interchange (ASCII) array. Further, the features are converted and fed to LSTM networks for training and validation. Results and Conclusion: This work is evaluated using two datasets, CODEC, and ADR Corpus datasets are used to evaluate the performance of the proposed ADR technique via multiple angles. Via extensive experiments, this work achieved 99.79 accuracy, 98.37 sensitivity, 97.63 specificity, 99.72 precision, 98.39 recall, 97.62 F1-score for the CODEC dataset, 98.16 for accuracy, 99.19 for sensitivity, 98.49 for specificity, 99.49 for precision, 96.72 for recall, and 93.16 for F1-score for ADR corpus, respectively.

Deep learning for blood glucose level prediction: How well do models generalize across different data sets?

Deep learning-based models for predicting blood glucose levels in diabetic patients can facilitate proactive measures to prevent critical events and are essential for closed-loop control therapy systems. However, selecting appropriate models from the literature may not always yield conclusive results, as the choice could be influenced by biases or misleading evaluations stemming from different methodologies, datasets, and preprocessing techniques. This study aims to compare and comprehensively analyze the performance of various deep learning models across diverse datasets to assess their applicability and generalizability across a broader spectrum of scenarios. Commonly used deep learning models for blood glucose level forecasting, such as feed-forward neural network, convolutional neural network, long short-term memory network (LSTM), temporal convolutional neural network, and self-attention network (SAN), are considered in this study. To evaluate the generalization capabilities of each model, four datasets of varying sizes, encompassing samples from different age groups and conditions, are utilized. Performance metrics include Root Mean Square Error (RMSE), Mean Absolute Difference (MAD), and Coefficient of Determination (CoD) for analytical asssessment, Clarke Error Grid (CEG) for clinical assessments, Kolmogorov-Smirnov (KS) test for statistical analysis, and generalization ability evaluations to obtain both coarse and granular insights. The experimental findings indicate that the LSTM model demonstrates superior performance with the lowest root mean square error and highest generalization capability among all other models, closely followed by SAN. The ability of LSTM and SAN to capture long-term dependencies in blood glucose data and their correlations with various influencing factors and events contribute to their enhanced performance. Despite the lower predictive performance, the FFN was able to capture patterns and trends in the data, suggesting its applicability in forecasting future direction. Moreover, this study helps in identifying the optimal model based on specific objectives, whether prioritizing generalization or accuracy.

Advanced Data Augmentation Techniques for Enhanced Fault Diagnosis in Industrial Centrifugal Pumps

This study presents an advanced data augmentation framework to enhance fault diagnostics in industrial centrifugal pumps using vibration data. The proposed framework addresses the challenge of insufficient defect data in industrial settings by integrating traditional augmentation techniques, such as Gaussian noise (GN) and signal stretching (SS), with advanced models, including Long Short-Term Memory (LSTM) networks, Autoencoders (AE), and Generative Adversarial Networks (GANs). Our approach significantly improves the robustness and accuracy of machine learning (ML) models for fault detection and classification. Key findings demonstrate a marked reduction in false positives and a substantial increase in fault detection rates, particularly in complex operational scenarios where traditional statistical methods may fall short. The experimental results underscore the effectiveness of combining these augmentation techniques, achieving up to a 30% improvement in fault detection accuracy and a 25% reduction in false positives compared to baseline models. These improvements highlight the practical value of the proposed framework in ensuring reliable operation and the predictive maintenance of centrifugal pumps in diverse industrial environments.

A Deep Learning Automatic Speech Recognition Model for Shona Language

This study presented the development of a deep learning-based Automatic Speech Recognition (ASR) system for Shona, a low-resource language characterized by unique tonal and grammatical complexities. The research aimed to address the challenges posed by limited training data, a lack of labelled data, and the intricate tonal nuances present in Shona speech, with the objective of achieving significant improvements in recognition accuracy compared to traditional statistical models. Motivated by the limitations of existing approaches, the research addressed three key questions. Firstly, it explored the feasibility of using deep learning to develop an accurate ASR system for Shona. Secondly, it investigated the specific challenges involved in designing and implementing deep learning architectures for Shona speech recognition and proposed strategies to mitigate these challenges. Lastly, it compared the performance of the deep learning-based model with existing statistical models in terms of accuracy. The developed ASR system utilized a hybrid architecture consisting of a Convolutional Neural Network (CNN) for acoustic modelling and a Long Short-Term Memory (LSTM) network for language modelling. To overcome the scarcity of data, the research employed data augmentation techniques and transfer learning. Attention mechanisms were also incorporated to accommodate the tonal nature of Shona speech. The resulting ASR system achieved impressive results, with a Word Error Rate (WER) of 29%, Phoneme Error Rate (PER) of 12%, and an overall accuracy of 74%. These metrics indicated a significant improvement over existing statistical models, highlighting the potential of deep learning to enhance ASR accuracy for under-resourced languages like Shona. This research contributed to the advancement of ASR technology for under-resourced languages like Shona, ultimately fostering improved accessibility and communication for Shona speakers worldwide.

Abnormal phenotypic defects detection of jujube using explainable machine learning enhanced computer vision.

Jujube is susceptible to biotic and abiotic adversity stresses resulting in abnormal phenotypic defects. Therefore, abnormal phenotype fruits should be removed during postharvest sorting to increase added value. An improved maximum horizontal diameter linear regression (MHD-LR) method for size grading of jujube prior to detection of abnormal phenotypic defects was developed. The accuracy of the MHD-LR model is 95%, with an error of only 0.95mm. In addition, a method for detecting abnormal phenotypic defects in jujube was established. It can effectively and accurately classify seven kinds of jujube phenotypes (regular, irregular, wrinkled, moldy, hole-broken, skin-broken, and scarred). The data augmentation method based on linear interpolation can effectively expand the dataset with a variance of only 0.0006. Support vector machine-decision tree (SVMDT), logistic regression, back propagation neural network, and long short-term memory network models were established to classify jujube samples with different phenotypes, with accuracies of 99.57%, 99.00%, 99.14%, and 99.29%, respectively. The results showed that the SVMDT model had higher accuracy and explainability. This research is expected to provide a new method to improve the precise classification of abnormal phenotypic defects in postharvest jujube.

Fault Diagnosis for Motor Bearings via an Intelligent Strategy Combined with Signal Reconstruction and Deep Learning

To overcome the incomplete decomposition of vibration signals in traditional motor-bearing fault diagnosis algorithms and improve the ability to characterize fault characteristics and anti-interference, a diagnostic strategy combining dual signal reconstruction and deep learning architecture is proposed. In this study, an improved complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and variational mode decomposition (VMD)-based signal reconstruction method is first introduced to extract features representing motor bearing faults. A feature matrix construction method based on improved information entropy is then proposed to quantify these fault features. Finally, a fault diagnosis algorithm architecture integrating a multi-scale convolutional neural network (MSCNN) with attention mechanisms and a bidirectional long short-term memory network (BiLSTM) is developed. The experimental results for four fault states show that this model can effectively extract fault features from original vibration signals and, compared to other fault diagnosis models, offer high diagnostic accuracy and strong generalization, maintaining high accuracy even under varying speeds and noise interference.

MmSTCT: spatial–temporal convolution transformer network considering driving intention for multimodal vehicle trajectory prediction of highway

Due to the dynamic nature of traffic flow and the uncertainty of drivers' maneuvers, multimodal trajectory prediction remains challenging. In this paper, a multimodal vehicle trajectory prediction model is proposed based on the encoder-decoder structure, utilizing a spatial-temporal dilated causal convolutional transformer network for the hierarchical extraction of driving intention and spatial interaction features. Specifically, temporal information is extracted from the initial motion states by utilizing a bi-directional long short-term memory network. Subsequently, the social interaction module and spatial-temporal dependency module are introduced to model the interactive influences among surrounding vehicles and the target vehicle at the same timestamp as well as at different timestamps. Then adaptive fusion of driving intention is performed to generate multiple queries for accommodating the output of multimodal trajectory prediction module in a non-autoregressive manner. The effectiveness of the proposed model is demonstrated through experimental evaluations on the NGSIM, HighD and CitySim datasets.

Vessel Trajectory Prediction Based on Automatic Identification System Data: Multi-Gated Attention Encoder Decoder Network

Utilizing time-series data from ship trajectories to forecast their subsequent movement is crucial for enhancing the safety within maritime traffic environments. The application of deep learning techniques, leveraging Automatic Identification System (AIS) data, has emerged as a pivotal area in maritime traffic studies. Within this domain, the precise forecasting of ship trajectories stands as a central challenge. In this study, we propose the multi-gated attention encoder decoder (MGAED) network, a model based on an encoder–decoder structure specialized for predicting ship trajectories in canals. The model employs a long short-term memory network (LSTM) as an encoder, combined with multiple Gated Recurrent Units (GRUs) and an attention mechanism for the decoder. Long-term dependencies in time-series data are captured through GRUs, while the attention mechanism is used to strengthen the model’s ability to capture key information, and a soft threshold residual structure is introduced to handle sparse features, thus enhancing the model’s generalization ability and robustness. The efficacy of our model is substantiated by an extensive evaluation against current deep learning benchmarks. Through comprehensive comparison experiments with existing deep learning methods, our model shows significant improvements in prediction accuracy, with an at least 9.63% reduction in the mean error (MAE) and an at least 20.0% reduction in the mean square error (MSE), providing a new solution to improve the accuracy and efficiency of ship trajectory prediction.

Research and Application of Traffic Flow Prediction in Chengdu City Based on Neural Network Algorithms

As urban traffic systems increase in complexity, accurately predicting traffic flow has become a critical task in traffic management and planning. This paper employs neural network technologies, specifically Graph Convolutional Networks (GCN) and Long Short-Term Memory networks (LSTM), to develop a traffic flow prediction model for Chengdu that incorporates spatio-temporal characteristics. Through empirical analysis, the model demonstrates high accuracy and good generalization capability, effectively predicting and analyzing traffic flow at different times and regions. Additionally, this study explores the application of the prediction model in traffic law regulation and policy formulation, providing data support and scientific basis for advancing smarter traffic systems.

Deep reinforcement learning for solving car resequencing with selectivity banks in automotive assembly shops

In the automobile manufacturing process, the complexity of the production environment and the different objectives of the upstream and downstream shops necessitate the resequencing of the car sequences using buffers. This paper addresses a car resequencing problem in automotive assembly shops by utilising selectivity banks to reshuffle the given upstream sequences to minimise the number of violations of sequencing rules in the downstream sequence. A new resequencing approach combining heuristics and deep reinforcement learning is proposed to solve the problem. Heuristics solve the automobile storage problem, while a deep reinforcement learning method based on proximal policy optimisation addresses the automobile release problem. Three state matrices, agent actions, and the reward function are designed based on the characteristics of the objective and selectivity banks. Additionally, a novel network structure combining convolutional neural networks and long short-term memory networks is proposed to train the agent. The results demonstrate that the proposed method outperforms two efficient deep reinforcement learning algorithms and two heuristics and obtains better solutions in handling dynamic events.

Accuracy of COVID-19 Prediction Modeling Techniques

The unprecedented impact of the COVID-19 pandemic has revealed that forecasting capability is critically needed in making strategic decisions and formulating reasonable countermeasures. This study aimed to assess the predictive accuracy in forecasting the numbers of COVID-19 cases using Thailand’s national COVID-19 surveillance database from January 2020– June 2021 based on three analytical models: a susceptible-exposed-infectious-recovery compartmental model, an auto-regressive integrated moving average model, and a long short-term memory (LSTM) network model. All forecasting methods had model parameters adjusted weekly according to the most recent situation and predictive accuracy measures, including the mean absolute percent error (MAPE). We found that the MAPE values ranged from 19.65%–22.54%, 28.95%–32.35%, 47.78%–53.55%, and 75.03–84.91% for forecasting one, two, four, and eight weeks ahead, respectively. Among the three models, the LSTM model had slightly higher accuracy than the other two models within the same forecasting range. These prediction models can be used for short-range forecasts in other similar settings while long-range forecasting requires monitoring and updating periodically.

Investigations into the flow dynamics of mixed biomass particles in a fluidized bed through Hilbert-Huang transformation and data-driven modelling

Flow dynamics of binary particles are investigated to realize the monitoring and optimization of fluidized beds. It is a challenge to accurately classify the mass fraction of mixed biomass, considering the limitations of existing techniques. The data collected from an electrostatic sensor array is analyzed. Cross correlation, empirical mode decomposition (EMD), Hilbert-Huang transform (HHT) are applied to process the signals. Under a higher mass fraction of the wood sawdust, the segregation behavior occurs, and the high energy region of HHT spectrum increases. Furthermore, two data-driven models are trained based on a hybrid wavelet scattering transform and bidirectional long short-term memory (ST-BiLSTM) network and a EMD and BiLSTM (EMD-BiLSTM) network to identify the mass fractions of the mixed biomass, with accuracies of 92% and 99%. The electrostatic sensing combined with the EMD-BiLSTM model is effective to classify the mass fraction of the mixed biomass.

Tone classification of online medical services based on 1DCNN-BiLSTM

In order to improve the recognition rate of the tone classification of doctors in online medical services scenarios, we propose a model that integrates a one-dimensional convolutional neural network (1DCNN) with a bidirectional long short-term memory network (BiLSTM). Firstly, significant tone types within online medical services scenarios were identified through a survey questionnaire. Secondly, 68 features in both the time and frequency domains of doctors’ tone were extracted using Librosa, serving as the initial input for the model. We utilize the 1DCNN branch to extract local features in the time and frequency domains, while the BiLSTM branch captures the global sequential features of the audio, and a feature-level fusion is performed to enhance tone classification effectiveness. When applied in online medical services scenarios, experimental results show that the model achieved an average recognition rate of 84.4% and an F1 score of 84.4%, significantly outperforming other models and effectively improving the efficiency of doctor-patient communication. Additionally, a series of ablation experiments were conducted to validate the effectiveness of the 1DCNN and BiLSTM modules and the parameter settings.

MLSNet: a deep learning model for predicting transcription factor binding sites.

Accurate prediction of transcription factor binding sites (TFBSs) is essential for understanding gene regulation mechanisms and the etiology of diseases. Despite numerous advances in deep learning for predicting TFBSs, their performance can still be enhanced. In this study, we propose MLSNet, a novel deep learning architecture designed specifically to predict TFBSs. MLSNet innovatively integrates multisize convolutional fusion with long short-term memory (LSTM) networks to effectively capture DNA-sparse higher-order sequence features. Further, MLSNet incorporates super token attention and Bi-LSTM to systematically extract and integrate higher-order DNA shape features. Experimental results on 165 ChIP-seq (chromatin immunoprecipitation followed by sequencing) datasets indicate that MLSNet consistently outperforms several state-of-the-art algorithms in the prediction of TFBSs. Specifically, MLSNet reports average metrics: 0.8306 for ACC, 0.8992 for AUROC, and 0.9035 for AUPRC, surpassing the second-best methods by 1.82%, 1.68%, and 1.54%, respectively. This research delineates the effectiveness of combining multi-size convolutional layers with LSTM and DNA shape-based features in enhancing predictive accuracy. Moreover, this study comprehensively assesses the variability in model performance across different cell lines and transcription factors. The source code of MLSNet is available at https://github.com/minghaidea/MLSNet.