Term Memory Research Articles

Analysing weighted disease symptoms to recommend an expert through an integrated healthcare model

The assessment and adaptation of remote symptom analysis remains a significant challenge for researchers. Qualitative evaluation in real-time clinical settings is crucial for the formative adaptation of remote symptom analysis. Remote patient monitoring and cloud integration unlock opportunities for affordability, personalization, and transparency. Despite facing barriers, the authors skilfully showcase how the integration of Natural language processing (NLP) and Deep learning (DL) can lead to impactful interventions. The study identifies Bi-directional long short term memory (Bi-LSTM) and contextual embedding techniques as pivotal in improving model accuracy. By combining patient symptom sequences with specialist recommendations, the research aims to facilitate the smooth transition of patients from the hospital to home, ensuring they receive the necessary attention. The 96% accuracy rate discovered by the researchers is a compelling indicator of the potential to drive the development of future remote palliative care units. This paper emphasizes the importance of capturing patient symptom notes and highlights the extensive pilot studies conducted by the researcher before to developing this innovative adaptation.

Research on Gold Price Prediction Based on LSTM Modeling

Gold plays a pivotal role in asset allocation, and the construction of gold price prediction models represents a complex yet rewarding task within the field of finance. The problem of international gold price is addressed in this paper forecasting by proposing a standard Long-Short Term Memory (LSTM) model and introducing bi-directional LSTM (Bi-LSTM) networks and multivariate analysis to compare the forecasting accuracy of the relevant models. This comparison is based on the daily gold price of the London Bullion Market Association (LBMA) for 2013-2022. This methodology takes into account various factors that influence the gold market and incorporates them as input sources into the Multi-factor LSTM model, thereby enhancing the interpretability of the LSTM model. Correlation analysis and the Granger test are employed to analyze these influencing factors. However, the results of this study suggest that while the Multi-factor LSTM model may be more accurate in predicting outliers, it tends to increase the number of small errors. On the other hand, the inclusion of Bi-LSTM networks can improve the overall accuracy of gold price prediction.

Long-Short Term Memory decision model for device argumentation in ambient assisted living

Long-Short Term Memory decision model for device argumentation in ambient assisted living

ECG signal analysis and detection system based on deep learning

In view of the problems of large workload and easy error in traditional manual recognition of ECG signals, and the existing ECG monitoring equipment still has few ECG signal recognition types, low diagnostic accuracy, and excessive reliance on network services, in order to improve the performance of ECG monitoring equipment, an ECG (Electrocardiogram Signals) analysis and detection system is designed based on deep learning technology. The SENet-LSTM (Squeeze-and-Excitation Networks Long Short Term Memory) network model is built to realize automatic diagnosis of 7 categories of ECG signals. The model is built on an intelligent hardware platform that uses ADS1292R as ECG acquisition module, STM32F103 as data processing module, and Raspberry Pi as central processing module. The system uses the integrated high- performance microcomputer Raspberry Pi for calculation and analysis, providing users with offline artificial intelligence (AI: Artificial Intelligence) services. At the same time, the accuracy and precision of the model are and respectively 98.44%, 90.00%thereby realizing real-time detection and accurate classification of ECG, providing patients with accurate disease diagnosis.

Hybrid Intelligent Anomaly Detection System Using Attention based Deep Learning Approach for Cyber Attacks Prevention

Objectives: Network Intrusion Detection System (NIDS) plays an important role in finding and preventing cyber-attacks, which helps to improve the entire security posture of an organization’s network infrastructure. The development of Deep Learning (DL) techniques possess the ability of IDS to detect attacks without delay and protects from intrusions even in real-time environment. Methods: The present study proposes an improved IDS framework called Enhanced Gated Recurrent Unit Hyper-Model combined Attention Bidirectional Long-Short Term Memory (EGHAB) approach, to effectively address the detection of attacks with maximum accuracy and minimal error rate. The proposed model is enhanced by using methods like numericalization and normalisation for pre-processing the input data, Genetic Algorithm (GA) for extracting intricate features from the input data, and the EGHAB classifier, which works on the principles of both Gated Recurrent Unit (GRU) and Bidirectional Long-Short Term Memory (BiLSTM) models along with attention mechanism. Findings: The EGHAB model efficiently learns the abnormal behaviour of network and classified the attack and non-attack data using the publicly available NSL-KDD dataset and a generated real time data set, scapy. And it achieved 99.94% accuracy over NSL KDD dataset during multiclass classification and 93.3% on real time data set with reduced Error rate. Additionally, to examine the efficacy, the proposed approach is compared with other existing methods and proved its improved performance. Novelty: A combined ensemble classifier with GRU, BiLSTM and attention mechanism is designed to predict the attacks in earlier stage rather than due over. The model achieved better accuracy and reduced false assumption using anomaly prediction mechanism. Keywords: Cyber Attack, Anomaly Detection, Deep learning, Gated Recurrent Unit, Bi­ Directional Long­-Short Term Memory, Attention mechanism

Novel hybrid data-driven models for enhanced renewable energy prediction

Global power grid management depends on accurate solar energy estimation, yet present prediction techniques frequently suffer from unreliability as a result of abnormalities in solar energy data. Solar radiation projections are affected by variables such as anticipated horizon length, meteorological classification, and power measuring techniques. Therefore, a Solar Wind Energy Prediction System (SWEPS) is proposed as a solution to these problems. It improves renewable energy projections by taking sun trajectories and atmospheric characteristics into account. In addition to using a variety of optimization methods and pre-processing techniques, such as Principal Component Analysis (PCA), Recursive Feature Elimination (RFE), Least Absolute Shrinkage Selection Operator (LASSO), and recursive feature addition processes (RFA), complemented by a genetic algorithm for feature selection (GAFS). The SWEPS also makes use of sophisticated machine learning algorithms and Statistical Correlation Analysis (SCA) to find important connections. Neural Network Algorithms (NNA) and other metaheuristic techniques like Cuckoo Search Optimization (CSO), Social Spider Optimization (SSO), and Particle Swarm Optimization (PSO) are adopted in this work to increase the predictability and accuracy of models. Utilizing the strengths of machine learning and deep learning techniques (Artificial Neural Networks (ANN), Decision Trees, Support Vector Machine (SVM), Recurrent Neural Networks (RNN), and Long Short Term Memory (LSTM)) for robust forecasting, as well as meta-heuristic optimization techniques to fine-tune hyper-parameters and achieve near-optimal values and significantly improve model performance, are some of this work contributions to the development of a comprehensive prediction system.

A real-time network traffic classification system using cost-sensitive deep neural networks

Due to the continuous growth of Internet and online applications, network traffic classification is not only becoming one of the most crucial disciplines in network management but is also becoming quintessential for providing advanced tasks such as Quality of Service and network security. Moreover, even though many studies have been undertaken in recent years, real-time encrypted traffic classification continues to be an important challenge in the field of network traffic classification. Therefore, in this paper a real-time network traffic classification system is proposed together with five new models. The real-time classification system classified each incoming real-time packets into appropriate classes of interest and the five new models make use of a cost-sensitive learning strategy to deal with the unbalanced data problem during the training phase. The proposed models, which are called Cost-Sensitive Long-Short Term Memory (CSLSTM), Cost-Sensitive Gated Recurrent Unit (CSGRU), Cost-Sensitive Convolution Neural Network (CSCNN), CSNN with LSTM and CSCNN with GRU, can handle both traffic categorization and application identification. These proposed models were compared with prominent methods in this field and the proposed CSCNN was observed to outperform the researched deep learning models by at least 4% to 16% in correctly classifying packets from the ISCX VPN-nonVPN dataset.

Drone motion prediction from flight data: a nonlinear time series approach

In this paper, we explore the application of data-driven predictive systems in enhancing unmanned aerial vehicle (UAV) control capabilities. We introduce a new model for predicting the motion of individual drones by utilizing fundamental flight control data. The model aims to improve the autonomy of individual drones and circumvent the complexity of traditional flight control systems, thus eliminating intricate nested controls. The proposed model lays the foundation for studying collective behaviours within a cluster of drones, thereby advancing the research into swarm behaviour exhibited by drones. The research findings demonstrate the potential of data-driven methods in the construction of UAV control systems. In particular, we here show a comparison of the prediction performances between two neural network architectures using real drone flight data involved in various kinds of motions. We explore the utility of using long short term memory (LSTM) and nonlinear autoregressive with exogenous inputs (NARX) family of nonlinear time series models in developing a virtual drone model using real experimental data.

A deep learning approach for non-invasive Alzheimer’s monitoring using microwave radar data

Over 50 million people globally suffer from Alzheimer’s disease (AD), emphasizing the need for efficient, early diagnostic tools. Traditional methods like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) scans are expensive, bulky, and slow. Microwave-based techniques offer a cost-effective, non-invasive, and portable solution, diverging from conventional neuroimaging practices. This article introduces a deep learning approach for monitoring AD , using realistic numerical brain phantoms to simulate scattered signals via the CST Studio Suite. The obtained data is preprocessed using normalization, standardization, and outlier removal to ensure data integrity. Furthermore, we propose a novel data augmentation technique to enrich the dataset across various AD stages. Our deep learning approach combines Recursive Feature Elimination (RFE) with Principal Component Analysis (PCA) and Autoencoders (AE) for optimal feature selection. Convolution Neural Network (CNN) is combined with Gated Recurrent Unit (GRU), Bidirectional Long Short Term Memory (Bidirectional-LSTM), and Long Short-Term Memory (LSTM) to improve classification performance. The integration of RFE-PCA-AE significantly elevates performance, with the CNN+GRU model achieving an 87% accuracy rate, thus outperforming existing studies.

Automatic estimation of the wind turbine noise with recurrent neural networks

There is growing interest in the development of renewable energies, particularly wind power. However, wind turbines generate noise that can affect the sound environment of nearby residents. This study focuses on the isolation of wind farm noise (WFN) from the surrounding ambient noise. Our method is based on a Recurrent Neural Network (RNN) Architecture that captures temporal dependencies in the acoustic signal. This proposal is compared to Non-Negative Matrix Factorization (NMF) that has shown first promising results on a previous study on simulated sound scenes. Our approach relies on Long Short Term Memory (LSTM) RNN, conducted using an end-to-end trained model and a Gated Recurrent Unit (GRU). The training and testing dataset is constructed by superimposing measured background noise with synthesized wind turbine noise, effectively simulating realistic environmental conditions. This study aims to reduce the acoustic impact of wind turbines on communities and attempt to control turbine operation using advanced machine learning techniques.

Intelligent Handwritten Identification Using Novel Hybrid Convolutional Neural Networks – Long-short-term Memory Architecture

Handwritten character identification finds broad applications in document analysis, digital forensics, and human-computer interaction. Conventional methods encounter challenges in accurately deciphering a range of handwriting styles and variations. Consequently, investigating intelligent system for handwriting identification becomes crucial to enhance accuracy and efficiency. This paper introduces an innovative hybrid deep learning architecture, seamlessly integrating the strengths of convolutional neural networks (CNN) and long-short term memory (LSTM) within a one single framework. The combination of these structures enables the model to effectively capture both spatial features and temporal dependencies inherent in handwritten strokes, resulting in improved recognition performance. The proposed 2DCNN-LSTM algorithm has been tested on MNIST dataset. The proposed hybrid CNN-LSTM structure has been compared to conventional intelligent machine learning methods, and the results demonstrate the superior performance of the hybrid CNN-LSTM, showcasing heightened accuracy, sensitivity, specificity, and other evaluation metrics.

Grammar and Expectation in Active Dependency Resolution: Experimental and Modeling Evidence From Norwegian.

Filler-gap dependency resolution is often characterized as an active process. We probed the mechanisms that determine where and why comprehenders posit gaps during incremental processing using Norwegian as our test language. First, we investigated why active filler-gap dependency resolution is suspended inside island domains like embedded questions in some languages. Processing-based accounts hold that resource limitations prevent gap-filling in embedded questions across languages, while grammar-based accounts predict that active gap-filling is only blocked in languages where embedded questions are grammatical islands. In a self-paced reading study, we find that Norwegian participants exhibit filled-gap effects inside embedded questions, which are not islands in the language. The findings are consistent with grammar-based, but not processing, accounts. Second, we asked if active filler-gap processing can be understood as a special case of probabilistic ambiguity resolution within an expectation-based framework. To do so, we tested whether word-by-word surprisal values from a neural language model could predict the location and magnitude of filled-gap effects in our behavioral data. We find that surprisal accurately tracks the location of filled-gap effects but severely underestimates their magnitude. This suggests either that mechanisms above and beyond probabilistic ambiguity resolution are required to fully explain active gap-filling behavior or that surprisal values derived from long-short term memory are not good proxies for humans' incremental expectations during filler-gapresolution.

Prediction of spatiotemporal dynamics using deep learning: Coupled neural networks of long short-terms memory, auto-encoder and physics-informed neural networks

Several classic reaction-diffusion models using partial differential equations (PDEs) have been established to elucidate the formation mechanism of vegetation patterns. However, predictive modeling of complex spatiotemporal dynamics using traditional numerical methods can be significantly challenging in many practical scenarios. Physics-Informed Neural Networks (PINNs) provide a new approach to predict the solution of PDEs. However, the generalization of PINNs is not satisfactory when pretrained PINNs is directly used in non-trained space (defined as explorations). This may be attributed to the lack of training in the time dimension. Therefore, a framework (LA-PINNs) is proposed to predict the evolutionary solution of the non-dimensional vegetation-sand model. The framework couples neural networks of Long-Short Terms Memory, Auto-Encoder and Physics-Informed Neural Networks. The predictions of LA-PINNs are much better than those of PINNs. Then we studied the effects of hyperparameters on the accuracy of predictions. Based on training in time dimension by LSTM module and pretraining for quick-training strategy, LA-PINNs can improve the accuracy of explorations.

Multi-temperature capable enhanced bidirectional long short term memory-multilayer perceptron hybrid model for lithium-ion battery SOC estimation

In this study, we propose a novel hybrid modeling framework for State of Charge (SOC) estimation across a broad temperature spectrum. First, we build a hybrid model to optimize stacked layers of stacked bidirectional long short term memory networks by introducing dropout mechanisms. At the same time, we also optimize the traditional multi-layer perceptron model to ResMLP, which is improved by introducing residual linkage, and then integrate the two optimization models. Finally, the synergistic effect and attention mechanism of genetic algorithm and particle swarm optimization are used to optimize its parameters. We then rigorously tested the model on nine datasets, including HPPC, DST and BBDST, at different temperatures of 5 °C, 15 °C and 35 °C. Using MAE, RMSE and MAXE benchmarks, our research results show that the proposed hybrid model outperforms the benchmark algorithm, achieving significantly enhanced performance and higher accuracy, and the maximum SOC estimation error is kept below 4.53 %. In addition, experimental evaluation at different temperatures shows the robustness and adaptability of the proposed algorithm.

Human behavior recognition method based on wearable devices

Abstract Human behavior recognition remains a prevalent subject of inquiry in contemporary scientific studies. Behavior recognition technology has penetrated into every aspect of our lives, mainly in video surveillance, health monitoring, and smart homes. Aiming at some people who need behavioral monitoring to prevent danger, a human behavior recognition method based on wearable devices is proposed, which first acquires the three-axis acceleration data of behavioral activities of such people through wearable devices and then performs sliding window segmentation and feature extraction on the acquired data and finally inputs the obtained features into the dual-directional long and short term memory framework (BiLSTM) model to complete the identification of human behavior. To affirm the method’s reliability, we performed activity detection tests on a dataset from UCI featuring a non-powered wearable sensor used by the senior population. The outcomes indicate that the method is proficient in identifying the routine daily activities of this demographic, demonstrating its practical utility.

Innovative Multi-Temporal Evapotranspiration Forecasting Using Empirical Fourier Decomposition and Bidirectional Long Short-Term Memory

Reference evapotranspiration (ETo) is an essential environmental variable that is intimately significant to agriculture. Managing water and crop planning relies heavily on precise forecasting of ETo. This research used a novel time series decomposition technique, Empirical Fourier Decomposition (EFD), to forecast ETo accurately. Four machine learning techniques were used to forecast ETo using decomposed lagged ETo values. The input data source was from Prince Edward Island (PEI) weather stations (Harrington and St Peters Stations). First, autocorrelation analysis was performed to determine effective lags. Then, ETo data were decomposed using EFD, and lagged data was created based on EFD results. The Kbest feature selection algorithm was used to choose effective inputs, reducing the training time. The accuracy of models was evaluated using different statistical metrics such as correlation coefficient (R) and root mean square error (RMSE). The results showed that using EFD decomposition can significantly improve forecast accuracy. The comparison between different machine learning models showed that the deep learning-based model (Bidirectional LSTM (Long Short Term Memory)) (R=0.956, RMSE= 0.451 mm/day for Harrington station and R=0.956, RMSE= 0.451 mm/day for St Peters station) performed better than the Generalized Regression Neural Network (GRNN), K-nearest neighbor (KNN), and Random Forest (RF) models. Finally, the best model (EFD-Bidirectional LSTM) was used to forecast multitemporal ETo at both stations. Results showed that the developed model can forecast ETo for up to 28 days with reasonable accuracy. However, the accuracy of multi-step ahead forecasting decreases when evapotranspiration values are high, as the models tend to underestimate these values. The findings of this study can assist in accurately calculating crop water requirements and help farmers optimize their irrigation schedules.

Mpox outbreak: Time series analysis with multifractal and deep learning network.

This article presents an overview of an mpox epidemiological situation in the most affected regions-Africa, Americas, and Europe-tailoring fractal interpolation for pre-processing the mpox cases. This keen analysis has highlighted the irregular and fractal patterns in the trend of mpox transmission. During the current scenario of public health emergency of international concern due to an mpox outbreak, an additional significance of this article is the interpretation of mpox spread in light of multifractality. The self-similar measure, namely, the multifractal measure, is utilized to explore the heterogeneity in the mpox cases. Moreover, a bidirectional long-short term memory neural network has been employed to forecast the future mpox spread to alert the outbreak as it seems to be a silent symptom for global epidemic.

Character-Level Embedding using FastText and LSTM for Biomedical Named Entity Recognition

Extracting biomedical entities has caught many researchers’ attention in which the recent technique of word embedding is employed for such a task. Yet, the traditional word embedding architectures of Word2vec or Glove are still suffering from the ‘out-of-vocabulary’ (OOV) problem. This problem occurs when an unseen term might be encountered during the testing which leads to absence of embedding vector. Hence, this study aims to propose a character-level embedding through FastText architecture. In fact, handling the character-level seems a promising solution for the OOV problem. To this end, the proposed FastText architecture has been used to generate embedding vectors for the possible N-gram combinations of each word. Consequentially, these vectors have been fed to a Long Short Term Memory (LSTM) architecture for classifying the words into its biomedical classes. Using two benchmark datasets of BioCreative-II and NCBI, the proposed method was able to produce an f-measure of 0.912 and 0.918 respectively. Comparing these results with the baseline studies demonstrates the superiority of the proposed character-level embedding of FastText in terms of Biomedical Named Entity Recognition (BNER) task.

Modeling continental US stream water quality using long-short term memory and weighted regressions on time, discharge, and season

The temporal dynamics of solute export from catchments are challenging to quantify and model due to confounding hydrological and biogeochemical processes and sparse measurements. Conventionally, the concentration-discharge relationship (C-Q) and statistical approaches to describe it, such as the Weighted Regressions on Time, Discharge and Seasons (WRTDS), have been widely used. Recently, deep learning (DL) approaches, especially Long-Short-Term-Memory (LSTM) models, have shown predictive capability for discharge, temperature, and dissolved oxygen. However, it is not clear if such advances can be expanded to water quality variables driven by complex subsurface biogeochemical processes. This work evaluates the performance of LSTM and WRTDS for 20 water quality variables across ~500 catchments in the continental US. We find that LSTM does not markedly outperform WRTDS in our dataset, potentially limited by the current measurement capabilities of water quality across CONUS. Both models present similar performance patterns across water quality variables, with the LSTM displaying better performance for nutrients compared to weathering-derived solutes. Additionally, the LSTM does not benefit from flexibility in the inputs. For example, incorporation of climate data that constrains streamflow generation, does not significantly improve the LSTM performance. We also find that data availability is not a straightforward predictor of LSTM model performance, although higher availability tends to stabilize performance. To fully assess the potential of the LSTM model, it may be necessary to use a higher frequency dataset across the CONUS, which does not exist today. To evaluate the dynamics of C-Q patterns relative to model performance, we introduce a “simplicity index” considering both the seasonality in the concentration pattern and the linearity in the C-Q relationship, or the C-Q-t pattern. The simplicity index is strongly correlated with model performance and differentiates the underlying controls on water quality dynamics. Further DL experiments and model-intercomparison highlight the strengths and deficiencies of existing frameworks, pointing to the need for further hydrogeochemical theories that are amenable to complex basins and solutes.

A Long Short Term Memory Model for Character-based Analysis of DNS Tunneling Detection

DNS tunneling is the attempt to create a hidden tunnel through a domain name service. Such a tunnel would jeopardize the targeted network and open the door for illegal access, control, and data exfiltration. The information security research community showed the variety of techniques that have been proposed to detect the tunnel. The majority of these efforts were relying on machine learning techniques where features of tunneling are considered such as length of DNS query, size, and entropy of the query. However, an additional analysis of the lexical information of the DNS query has been depicted recently and showed remarkable performance. This paper aims to examine the role of Long Short Term Memory (LSTM) model in terms of DNS lexical analysis. Two benchmark datasets related to DNS have been used. In addition, a character mapping mechanism has been used to replace every possible character with an integer number. Consequentially, the mapped representation has been fed into an LSTM model for DNS tunneling detection. Results showed that the proposed method was able to obtain a weighted average F1-score of 98% for both datasets respectively. Such results are competitive in the context of the state of the art and demonstrate the efficacy of the lexical analysis within the DNS tunneling detection task.