Memory Network Research Articles

Face Manipulation Detection

The rise of digital manipulation techniques has led to the proliferation of deepfakes and other manipulated facial images, posing significant challenges to online trust and security. This paper proposes a novel deep learning model aimed at efficiently and effectively detecting face manipulations. The architecture combines the strengths of Efficient Net, a convolutional neural network (CNN) renowned for its accuracy and efficiency, with Long Short-Term Memory (LSTM) networks. Efficient Net is utilized for extracting high-level features from facial images, enabling the model to capture subtle inconsistencies that may indicate manipulation. These features serve as crucial inputs to the subsequent analysis performed by LSTM networks. LSTMs excel at capturing temporal dependencies within sequences of data, making them particularly well-suited for detecting manipulations in video sequences. By leveraging the power of CNNs for feature extraction and the sequential learning capabilities of LSTMs, the proposed hybrid approach aims to achieve superior performance in face manipulation detection. This combination allows the model to effectively analyse both spatial and temporal aspects of facial images, enhancing its ability to detect various forms of manipulation accurately

Heart Signal Analysis Using Multistage Classification Denoising Model

Cardiovascular disease is a major cause of death worldwide, and the COVID-19 pandemic has only made the situation worse. The purpose of this work is to explore various time-frequency analysis methods that can be used to classify heart sound signals and identify multiple abnormalities in the heart, such as aortic stenosis, mitral stenosis, and mitral valve prolapse. The signal has been modified using three techniques—tunable quality wavelet transform (TQWT), discrete wavelet transform (DWT), and empirical mode decomposition—to detect heart signal abnormality. The proposed model detects heart signal abnormality at two stages, the user end and the clinical end. At the user end, binary classification of signals is performed, and if signals are abnormal then further classification is done at the clinic. The approach starts with signal preprocessing and uses the discrete wavelet transform (DWT) coefficients to train the hybrid model, which consists of one long short-term memory (LSTM) network layer and three convolutional neural network (CNN) layers. This method produced comparable results, with a 98.9% classification accuracy for signals, through the utilization of the CNN and LSTM model. Combining the CNN’s skill in feature extraction with the LSTM’s capacity to record time-dependent features improves the efficacy of the model. Identifying issues early and initiating appropriate medication can alleviate the burden associated with heart valve diseases.

A feature extractor for temporal data of electronic nose based on parallel long short-term memory network in flavor discrimination of Chinese vinegars

Volatile flavor is a key indicator of food quality which can directly affect consumer preference and purchase intention. Electronic nose is considered as a promising intelligent sensory analysis tool for food flavor assessment, however, extracting effective features from the gas sensor array is still a major challenge, which largely determines the performance of subsequent classifiers. Here, a parallel long short-term memory (LSTM) network is proposed as a feature extractor for automatically extracting features from the whole time series of sensor responses in flavor discrimination of five Chinese vinegars. The network was trained by the temporal data from the sensor array and yielded different feature patterns corresponding to different vinegars, which were then fed to other conventional classifiers for pattern recognition. We also evaluated the influence of the extracted feature dimension that is related to the dimension of the hidden state of the LSTM layer on the classification performance. The results indicate that a larger dimension of extracted feature is unnecessary for promoting classification accuracy, instead, the optimum dimension 4 of the hidden state gives the highest accuracy of 95.8% in this application under the softmax evaluator. Moreover, much higher accuracies were obtained when combined with other sophisticated classifiers such as support vector machine. The results demonstrate that the proposed network is competent to extract features directly and automatically from the temporal data of the electronic nose.

Tool Wear Prediction Based on LSTM and Deep Residual Network

To improve the accuracy and efficiency of tool wear predictions, this study proposes a tool wear prediction model called LSTM_ResNet which is based on the long short-term memory (LSTM) network and the Residual Network (ResNet). The model utilizes LSTM layers for processing, where the first block and loop blocks serve as the core modules of the deep residual network. The model employs a series of methods including convolution, batch normalization (BN), and Rectified Linear Unit (ReLU) to enhance the model’s expression and prediction capabilities. The performance of the LSTM_ResNet model was evaluated using experimental data from the PHM2010 datasets and two different depths (64 and 128 layers), training both LSTM_ResNet models for 200 epochs. The 64-layer model’s root mean square error (RMSE) values are 3.36, 4.35, and 3.59, and the mean absolute error (MAE) values are 2.42, 2.85, and 2.21; using 128 layers, the RMSE values are 3.66, 3.99, and 3.77, and the MAE values are 2.49, 2.73, and 3.01. The results indicate that the 64-layer LSTM has smaller average errors, suggesting that compared to other common network structures, the LSTM_ResNet network has a higher performance. This research provides an effective solution for tool wear prediction and helps to improve the technical level of tool wear prediction in China.

An Efficient Water Quality Prediction and Assessment Method Based on the Improved Deep Belief Network—Long Short-Term Memory Model

The accuracy of water quality prediction and assessment has always been the focus of environmental departments. However, due to the high complexity of water systems, existing methods struggle to capture the future internal dynamic changes in water quality based on current data. In view of this, this paper proposes a data-driven approach to combine an improved deep belief network (DBN) and long short-term memory (LSTM) network model for water quality prediction and assessment, avoiding the complexity of constructing a model of the internal mechanism of water quality. Firstly, using Gaussian Restricted Boltzmann Machines (GRBMs) to construct a DBN, the model has a better ability to extract continuous data features compared to classical DBN. Secondly, the extracted time-series data features are input into the LSTM network to improve predicting accuracy. Finally, due to prediction errors, noise that randomly follows the Gaussian distribution is added to the assessment results based on the predicted values, and the probability of being at the current water quality level in the future is calculated through multiple evolutionary computations to complete the water quality assessment. Numerical experiments have shown that our proposed algorithm has a greater accuracy compared to classical algorithms in challenging scenarios.

A thermodynamically consistent physics-informed deep learning material model for short fiber/polymer nanocomposites

This work proposes a physics-informed deep learning (PIDL)-based constitutive model for investigating the viscoelastic–viscoplastic behavior of short fiber-reinforced nanoparticle-filled epoxies under various ambient conditions. The deep-learning model is trained to enforce thermodynamic principles, leading to a thermodynamically consistent constitutive model. To accomplish this, a long short-term memory network is combined with a feed-forward neural network to predict internal variables required for characterizing the internal dissipation of the nanocomposite materials. In addition, another feed-forward neural network is used to indicate the free-energy function, which enables defining the thermodynamic state of the entire system. The PIDL model is initially developed for the three-dimensional case by generating synthetic data from a classical constitutive model. The model is then trained by extracting the data directly from cyclic loading–unloading experimental tests. Numerical examples show that the PIDL model can accurately predict the mechanical behavior of epoxy-based nanocomposites for different volume fractions of fibers and nanoparticles under various hygrothermal conditions.

Time Series Classification Based on Forward Echo State Convolution Network

The Echo state network (ESN) is an efficient recurrent neural network that has achieved good results in time series prediction tasks. Still, its application in time series classification tasks has yet to develop fully. In this study, we work on the time series classification problem based on echo state networks. We propose a new framework called forward echo state convolutional network (FESCN). It consists of two parts, the encoder and the decoder, where the encoder part is composed of a forward topology echo state network (FT-ESN), and the decoder part mainly consists of a convolutional layer and a max-pooling layer. We apply the proposed network framework to the univariate time series dataset UCR and compare it with six traditional methods and four neural network models. The experimental findings demonstrate that FESCN outperforms other methods in terms of overall classification accuracy. Additionally, we investigated the impact of reservoir size on network performance and observed that the optimal classification results were obtained when the reservoir size was set to 32. Finally, we investigated the performance of the network under noise interference, and the results show that FESCN has a more stable network performance compared to EMN (echo memory network).

MulTFBS: A Spatial-Temporal Network with Multichannels for Predicting Transcription Factor Binding Sites.

Revealing the mechanisms that influence transcription factor binding specificity is the key to understanding gene regulation. In previous studies, DNA double helix structure and one-hot embedding have been used successfully to design computational methods for predicting transcription factor binding sites (TFBSs). However, DNA sequence as a kind of biological language, the method of word embedding representation in natural language processing, has not been considered properly in TFBS prediction models. In our work, we integrate different types of features of DNA sequence to design a multichanneled deep learning framework, namely MulTFBS, in which independent one-hot encoding, word embedding encoding, which can incorporate contextual information and extract the global features of the sequences, and double helix three-dimensional structural features have been trained in different channels. To extract sequence high-level information effectively, in our deep learning framework, we select the spatial-temporal network by combining convolutional neural networks and bidirectional long short-term memory networks with attention mechanism. Compared with six state-of-the-art methods on 66 universal protein-binding microarray data sets of different transcription factors, MulTFBS performs best on all data sets in the regression tasks, with the average R2 of 0.698 and the average PCC of 0.833, which are 5.4% and 3.2% higher, respectively, than the suboptimal method CRPTS. In addition, we evaluate the classification performance of MulTFBS for distinguishing bound or unbound regions on TF ChIP-seq data. The results show that our framework also performs well in the TFBS classification tasks.

A multi-scale fuel cell degradation prediction method based on isometric convolution block and long short-term memory networks

Performance degradation prediction is considered an effective method to enhance the durability of Proton Exchange Membrane Fuel Cells (PEMFCs). However, accurate prediction under dynamic conditions remains a challenge. The MSD-ICB-LSTM model is proposed to improve the accuracy of the performance degradation prediction, which incorporates the information from both long-term and short-term modes within the PEMFC history operating data. Firstly, redundant variables are removed from the data, and the noise is eliminated using locally weighted scatterplot smoothing (LOWESS) to raise the quality of the input samples. Secondly, the Multi-Scale Decomposition (MSD) method is used to decompose the pre-processed data into long-term and short-term modes. Thirdly, a combined module containing encoding, internal decomposition and a double-layer Isometric Convolution Block (ICB) is used to deeply explore the nonlinear degradation characteristics in the short-term mode, while a Long-Short-Term Memory Network (LSTM) is utilized to estimate the long-term mode. The final result of prediction is obtained by combining the predictions of two modes. The experimental results show that the prediction accuracy of the proposed MSD-ICB-LSTM model is 99.74% in R2，and the Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) are 0.0034 and 0.77%, respectively. Thus, the method can efficiently explore the multi-scale features and enhance the accuracy of PEMFC performance degradation prediction.

UNet-BiLSTM: A Deep Learning Method for Reconstructing Electrocardiography from Photoplethysmography

Electrocardiography (ECG) is generally used in clinical practice for cardiovascular diagnosis and for monitoring cardiovascular status. It is considered to be the gold standard for diagnosing cardiovascular diseases and assessing cardiovascular status. However, it is not always easy to obtain. Unlike ECG devices, photoplethysmography (PPG) devices can be placed on body parts such as the earlobes, fingertips, and wrists, making them more comfortable and easier to obtain. Several methods for reconstructing ECG signals using PPG signals have been proposed, but some of these methods are subject-specific models. These models cannot be applied to multiple subjects and have limitations. This study proposes a neural network model based on UNet and bidirectional long short-term memory (BiLSTM) networks as a group model for reconstructing ECG from PPG. The model was verified using 125 records from the MIMIC III matched subset. The experimental results demonstrated that the proposed model was, on average, able to achieve a Pearson‘s correlation coefficient, root mean square error, percentage root mean square difference, and Fréchet distance of 0.861, 0.077, 5.302, and 0.278, respectively. This research can use the correlation between PPG and ECG to reconstruct a better ECG signal from PPG, which is crucial for diagnosing cardiovascular diseases.

Fluvial bedload transport modelling: advanced ensemble tree-based models or optimized deep learning algorithms?

The potential of advanced tree-based models and optimized deep learning algorithms to predict fluvial bedload transport was explored, identifying the most flexible and accurate algorithm, and the optimum set of readily available and reliable inputs. Using 926 datasets for 20 rivers, the performance of three groups of models was tested: (1) standalone tree-based models Alternating Model Tree (AMT) and Dual Perturb and Combine Tree (DPCT); (2) ensemble tree-based models Iterative Absolute Error Regression (IAER), ensembled with AMT and DPCT; and (3) optimized deep learning models Long Short-Term Memory (LSTM) and Recurrent Neural Network (RNN) ensembled with Grey Wolf Optimizer. Comparison of the predictive performance of the models with that of commonly used empirical equations and sensitivity analysis of the driving variables revealed that: (i) the coarse grain-size percentile D 90 was the most effective variable in bedload transport prediction (where D x is the xth percentile of the bed surface grain size distribution), followed by D 84, D 50, flow discharge, D 16, and channel slope and width; (ii) all tree-based models and optimized deep learning algorithms displayed ‘very good’ or ‘good’ performance, outperforming empirical equations; and (iii) all algorithms performed best when all input parameters were used. Thus, a range of different input variable combinations must be considered in the optimization of these models. Overall, ensemble algorithms provided more accurate predictions of bedload transport than their standalone counterpart. In particular, the ensemble tree-based model IAER-AMT performed most strongly, displaying great potential to produce robust predictions of bedload transport in coarse-grained rivers based on a few readily available flow and channel variables.

Operational Forecasting of Global Ionospheric TEC Maps 1-, 2-, and 3-Day in Advance by ConvLSTM Model

In this paper, we propose a global ionospheric total electron content (TEC) maps (GIM) prediction model based on deep learning methods that is both straightforward and practical, meeting the requirements of various applications. The proposed model utilizes an encoder-decoder structure with a Convolution Long Short-Term Memory (ConvLSTM) network and has a spatial resolution of 5° longitude and 2.5° latitude, with a time resolution of 1 h. We utilized the Center for Orbit Determination in Europe (CODE) GIM dataset for 18 years from 2002 to 2019, without requiring any other external input parameters, to train the ConvLSTM models for forecasting GIM 1, 2, and 3 days in advance. Using the CODE GIM data from 1 January 2020 to 31 December 2023 as the test dataset, the performance evaluation results show that the average root mean square errors (RMSE) for 1, 2 and 3 days of forecasts are 2.81 TECU, 3.16 TECU, and 3.41 TECU, respectively. These results show improved performance compared to the IRI-Plas model and CODE’s 1-day forecast product c1pg, and comparable to CODE’s 2-day forecast c2pg. The model’s predictions get worse as the intensity of the storm increases, and the prediction error of the model increases with the lead time.

Optimal integration and planning of PV and wind renewable energy sources into distribution networks using the hybrid model of analytical techniques and metaheuristic algorithms: A deep learning-based approach

The optimal integration of Renewable Energy Sources (RESs) into distribution networks is crucial for sustainable energy generation. However, existing approaches utilized for optimal RES integration, including metaheuristic and analytical methods, exhibit limitations. Analytical methods are simple to implement, computationally efficient, and able to model power system mechanisms, yet they face scalability issues, making them inefficient for large-scale distribution systems. On the other hand, metaheuristic approaches, while flexible for complex problems, suffer from premature convergence problems, limiting their application in real distribution networks. To address these issues, this paper introduces a novel hybrid model that combines the strengths of analytical and metaheuristic techniques. The proposed model employs a deep learning-based method called bi-directional long short-term memory (B-LSTM) network is utilized to handle the uncertainties associated with the Photovoltaic (PV) and Wind Turbine (WT) generation and load demand. Subsequently, a distinctive analytical multi-objective index-based procedure is developed to minimize the real and reactive energy loss and voltage deviation by analyzing the branch current change caused by the RES penetration in distribution networks. The optimization problem is then solved using the binary particle swarm optimization (BPSO) technique. Importantly, the presented method considers the long-term nature of optimal RES integration and reflects the seasonal impacts of the load and RES output powers. The proposed approach is applied to the IEEE 33-bus test system. The simulation results show that the introduced hybrid model, by addressing the limitations of existing approaches, offers a promising solution for efficient and effective RES integration in distribution systems while requiring low computational effort, thereby advancing the transition towards sustainable energy infrastructure.

Machine learning based data governance methods for demand response databases

With the increasing level of grid intelligence and the related demand response database expanding, it is important to study a compound problem data governance method for demand response, while the traditional data governance methods have problems such as not considering data temporality and ignoring the impact of noise and duplicate data on data repair. As a result, this project will develop an anomaly data extraction and repair model based on two-way long and short memory networks, and repair the anomaly data by respective noise smoothing, missing data filling, and duplicate data cleaning. The paper also provides an adaptive moment estimation approach for optimisation to raise the model’s accuracy. The outcomes demonstrated that the study model’s precision for anomalous data extraction was 100% and its recall rate was 80%, which was a significant improvement over the previous state. In terms of anomalous data repair, the research model had the root mean square error value and lowest mean absolute percentage error value when compared with related models, at 0.0049 MPa and 1.375% respectively. Both the abnormal data extraction and repair performance of the research model are greatly improved over the related models, and have important value in the abnormal data governance of demand response databases.

AEMNet: Unsupervised Video Anomaly Detection Method Based on Attention-Enhanced Memory Networks

AEMNet: Unsupervised Video Anomaly Detection Method Based on Attention-Enhanced Memory Networks

An intelligent assessment method of criminal psychological attribution based on unbalance data

Criminal cases often exhibit imbalance and cannot be extended by data augmentation when classified into attribution types. To solve the problem of unbalance data in offenders’ attribution classification, this paper proposes a criminal psychological attribution assessment model by an improved Balanced TF-Distinguishing IDF method (B-TF-dIDF) and constructed a hybrid network with attention method to fuse numerical and text features for improving the accuracy. First, as a statistical method, B-TF-dIDF is presented to reduce the impact of class-imbalance for extraction of numerical features, in which a balanced element is added to reduce the effects of incorrect type keywords on classification, and a distinguishing element is added to discriminate the types of keywords. Then, an improved hybrid network model composed of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) is constructed to balance the influence of different lengths of text samples for extracting the semantic features of criminal texts. For evaluating different feature weights by their importance, Spatial Attention is used to improve CNN in the feature maps. Moreover, the self-attention is also performed to re-evaluate the mixed features. Finally, the softmax classifier provides a scientific basis for developing a hierarchical treatment mechanism further. Additionally, we build a criminal data set with labels from real cases for testing. The experiment proved that the proposed model is better than other related methods in various evaluation indicators, including the micro and macro scopes. Moreover, the F1 of minority samples has increased by 6%–8%, indicating that the proposed method can reduce the impact of class-imbalance.

The application of artificial neural networks to forecast financial time series

Abstract The amount of information that is produced on a daily basis in the financial markets is vast and complex; consequently, the development of systems that simplify decision-making is an essential endeavor. In this article, several intelligent systems are proposed and tested to predict the closing price of the IBEX 35 index using more than ten years of historical data and five distinct architectures for neural networks. A multi-layer perceptron was the first step, followed by a simple recurrent neural network, a gated recurrent unit network and two long-short-term memory (LSTM) networks. The results of the analyses performed on these models have demonstrated a powerful capacity for prediction. Additionally, the findings of this research point to the fact that the application of intelligent systems can simplify the decision-making process in financial markets, which is a substantial advantage. Furthermore, by comparing the predicted outcome errors between the models, the LSTM presents the lowest error with a higher computational time in the training phase. The LSTM was able to accurately forecast the closing price of the day as well as the price for the following one and two days in advance. In conclusion, the empirical results demonstrated that these models could accurately predict financial data for trading purposes and that the application of intelligent systems, such as the LSTM network, represents a promising advancement in financial technology.

MMG-Based Knee Dynamic Extension Force Estimation Using Cross-Talk and IGWO-LSTM

Mechanomyography (MMG) is an important muscle physiological activity signal that can reflect the amount of motor units recruited as well as the contraction frequency. As a result, MMG can be utilized to estimate the force produced by skeletal muscle. However, cross-talk and time-series correlation severely affect MMG signal recognition in the real world. These restrict the accuracy of dynamic muscle force estimation and their interaction ability in wearable devices. To address these issues, a hypothesis that the accuracy of knee dynamic extension force estimation can be improved by using MMG signals from a single muscle with less cross-talk is first proposed. The hypothesis is then confirmed using the estimation results from different muscle signal feature combinations. Finally, a novel model (improved grey wolf optimizer optimized long short-term memory networks, i.e., IGWO-LSTM) is proposed for further improving the performance of knee dynamic extension force estimation. The experimental results demonstrate that MMG signals from a single muscle with less cross-talk have a superior ability to estimate dynamic knee extension force. In addition, the proposed IGWO-LSTM provides the best performance metrics in comparison to other state-of-the-art models. Our research is expected to not only improve the understanding of the mechanisms of quadriceps contraction but also enhance the flexibility and interaction capabilities of future rehabilitation and assistive devices.

A review on evaluating mental stress by deep learning using EEG signals

AbstractMental stress is a common problem that affects individuals all over the world. Stress reduces human functionality during routine work and may lead to severe health defects. Early detection of stress is important for preventing diseases and other negative health-related consequences of stress. Several neuroimaging techniques have been utilized to assess mental stress, however, due to its ease of use, robustness, and non-invasiveness, electroencephalography (EEG) is commonly used. This paper aims to fill a knowledge gap by reviewing the different EEG-related deep learning algorithms with a focus on Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs) for the evaluation of mental stress. The review focuses on data representation, individual deep neural network model architectures, hybrid models, and results amongst others. The contributions of the paper address important issues such as data representation and model architectures. Out of all reviewed papers, 67% used CNN, 9% LSTM, and 24% hybrid models. Based on the reviewed literature, we found that dataset size and different representations contributed to the performance of the proposed networks. Raw EEG data produced classification accuracy around 62% while using spectral and topographical representation produced up to 88%. Nevertheless, the roles of generalizability across different deep learning models and individual differences remain key areas of inquiry. The review encourages the exploration of innovative avenues, such as EEG data image representations concurrently with graph convolutional neural networks (GCN), to mitigate the impact of inter-subject variability. This novel approach not only allows us to harmonize structural nuances within the data but also facilitates the integration of temporal dynamics, thereby enabling a more comprehensive assessment of mental stress levels.

Resource-efficient, sensor-based human activity recognition with lightweight deep models boosted with attention

With their automatic feature extraction capabilities, deep learning models have become more widespread in sensor-based human activity recognition, particularly on larger datasets. However, their direct use on mobile and wearable devices is challenging due to the extensive resource requirements. Concurrently, attention-based models are emerging to improve recognition performance by dynamically emphasizing relevant parts of features and disregarding the irrelevant ones, particularly in the computer vision domain. This study introduces a novel application of attention mechanisms to smaller deep architectures, investigating whether smaller models can achieve comparable recognition performance to larger models in sensor-based human activity recognition systems while keeping resource usage at lower levels. For this purpose, we integrate the convolutional block attention module into a hybrid model, deep convolutional and long short-term memory network. Experiments are conducted using five public datasets in three model sizes: lightweight, moderate and original. The results show that applying attention to the lightweight model enables achieving similar recognition performances to the moderate-size model, and the lightweight model requires approximately 2–13 times fewer parameters and 3.5 times fewer flops. We also conduct experiments with sensor data at lower sampling rates and from fewer sensors attached to different body parts. The results show that attention improves recognition performance under lower sampling rates, as well as under higher sampling rates when model sizes are smaller, and mitigates the impact of missing data from one or more body parts, making the model more suitable for real-world sensor-based applications.