Short-term Memory Mechanism Research Articles

Research on a Fault Diagnosis Method of an A-Class Thermal Insulation Panel Production Line Based on Multi-Sensor Data Fusion

To detect the running state of an A-class thermal insulation board production line in real time, conveniently and accurately, a fault diagnosis method based on multi-sensor data fusion was proposed. The proposed algorithm integrates the ideas of Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) and Attention Mechanism, and combines a Dilated Convolution Module (DCM) with LSTM to recognize complex signals of multiple sensors. By introducing an attention mechanism, the recognition performance of the network was improved. Finally, the real-time status information of the production line was obtained by integrating attention weight. Experimental results show that for the custom multi-sensor dataset of A-class insulation board production line, the proposed CNN-LSTM fault diagnosis method achieved 98.97% accuracy. Compared with other popular algorithms, the performance of the proposed CNN-LSTM model performed excellently in each evaluation index is better.

Global and local feature fusion via long and short-term memory mechanism for dance emotion recognition in robot.

In recent years, there are more and more intelligent machines in people's life, such as intelligent wristbands, sweeping robots, intelligent learning machines and so on, which can simply complete a single execution task. We want robots to be as emotional as humans. In this way, human-computer interaction can be more natural, smooth and intelligent. Therefore, emotion research has become a hot topic that researchers pay close attention to. In this paper, we propose a new dance emotion recognition based on global and local feature fusion method. If the single feature of audio is extracted, the global information of dance cannot be reflected. And the dimension of data features is very high. In this paper, an improved long and short-term memory (LSTM) method is used to extract global dance information. Linear prediction coefficient is used to extract local information. Considering the complementarity of different features, a global and local feature fusion method based on discriminant multi-canonical correlation analysis is proposed in this paper. Experimental results on public data sets show that the proposed method can effectively identify dance emotion compared with other state-of-the-art emotion recognition methods.

Research on the Filtering and Classification Method of Interactive Music Education Resources Based on Neural Network.

This work intends to classify and integrate music genres and emotions to improve the quality of music education. This work proposes a web image education resource retrieval method based on semantic network and interactive image filtering for a music education environment. It makes a judgment on these music source data and then uses these extracted feature sequences as the emotions expressed in the model of the combination of Long Short-Term Memory (LSTM) and Attention Mechanism (AM), thus judging the emotion category of music. The emotion recognition accuracy has increased after improving LSTM-AM into the BiGR-AM model. The greater the difference between emotion genres is, the easier it is to analyze the feature sequence containing emotion features, and the higher the recognition accuracy is. The classification accuracy of the excited, relieved, relaxed, and sad emotions can reach 76.5%, 71.3%, 80.8%, and 73.4%, respectively. The proposed interactive filtering method based on a Convolutional Recurrent Neural Network can effectively classify and integrate music resources to improve the quality of music education.

DeepPAD: Detection of Peripheral Arterial Disease Using Deep Learning

Peripheral arterial disease (PAD) is a common circulatory disease caused by the deposition of fatty plaque on the arterial wall. Early detection and treatment of PAD are essential in preventing further cardiovascular and health complications. It was recently shown that PAD can be detected using an oscillometric device by characterizing the peripheral arterial system at different externally applied cuff pressures. However, the extraction of the complex relationship between the pattern of the oscillometric waveforms and the presence of PAD remained challenging. This study proposes a novel deep learning approach for the detection of PAD by capturing the peripheral arterial system behavior at different cuff pressures. The periodic pattern of the oscillometric pulses and their variations as a function of cuff pressure were modeled using a deep recurrent neural network based on the bidirectional long short-term memory and attention mechanism. The proposed model was evaluated by analyzing the raw oscillometric pulses as well as statistical features on data collected from 33 individuals (14 PAD and 19 normal). The results show a high accuracy of up to 94.8%, a sensitivity of up to 90.0%, and a specificity of up to 97.4% in detecting PAD. The proposed method provides new opportunities for noninvasive cardiovascular screening and early detection of PAD using the oscillometric principle.

Alzheimer’s Disease Prediction Using Attention Mechanism with Dual-Phase 18F-Florbetaben Images

IntroductionAmyloid-beta (Aβ) imaging test plays an important role in the early diagnosis and research of biomarkers of Alzheimer’s disease (AD) but a single test may produce Aβ-negative AD or Aβ-positive cognitively normal (CN). In this study, we aimed to distinguish AD from CN with dual-phase 18F-Florbetaben (FBB) via a deep learning–based attention method and evaluate the AD positivity scores compared to late-phase FBB which is currently adopted for AD diagnosis.Materials and MethodsA total of 264 patients (74 CN and 190 AD), who underwent FBB imaging test and neuropsychological tests, were retrospectively analyzed. Early- and delay-phase FBB images were spatially normalized with an in-house FBB template. The regional standard uptake value ratios were calculated with the cerebellar region as a reference region and used as independent variables that predict the diagnostic label assigned to the raw image.ResultsAD positivity scores estimated from dual-phase FBB showed better accuracy (ACC) and area under the receiver operating characteristic curve (AUROC) for AD detection (ACC: 0.858, AUROC: 0.831) than those from delay phase FBB imaging (ACC: 0.821, AUROC: 0.794). AD positivity score estimated by dual-phase FBB (R: −0.5412) shows a higher correlation with psychological test compared to only dFBB (R: −0.2975). In the relevance analysis, we observed that LSTM uses different time and regions of early-phase FBB for each disease group for AD detection.ConclusionsThese results show that the aggregated model with dual-phase FBB with long short-term memory and attention mechanism can be used to provide a more accurate AD positivity score, which shows a closer association with AD, than the prediction with only a single phase FBB.

Riemannian dynamic generalized space quantization learning

Many existing works represent signals by covariance matrices and then develop learning methods on the Riemannian symmetric positive-definite (SPD) manifold to deal with such data. However, they summarize each instance with a single covariance matrix, omitting some potential important information, such as the time evolution of the correlation in signals. In this paper, we represent each instance by a sequence of covariance matrices and develop a novel dynamic generalized learning Riemannian space quantization (DGLRSQ) method to deal with such data representations. The proposed DGLRSQ method incorporates short-term memory mechanism in generalized learning Riemannian space quantization (GLRSQ), which is an extension of Euclidean generalized learning vector quantization to deal with SPD matrix-valued data. The proposed method can capture the temporal evolution of the correlation in signals and thus provides better performance to its the counterpart – GLRSQ, which treats each instance as a signal covariance matrix. Empirical investigations on synthetic data and motor imagery EEG data show the superior performance of the proposed method.

An improved residual network using deep fusion for identifying RNA 5-methylcytosine sites.

5-Methylcytosine (m5C) is a crucial post-transcriptional modification. With the development of technology, it is widely found in various RNAs. Numerous studies have indicated that m5C plays an essential role in various activities of organisms, such as tRNA recognition, stabilization of RNA structure, RNA metabolism and so on. Traditional identification is costly and time-consuming by wet biological experiments. Therefore, computational models are commonly used to identify the m5C sites. Due to the vast computing advantages of deep learning, it is feasible to construct the predictive model through deep learning algorithms. In this study, we construct a model to identify m5C based on a deep fusion approach with an improved residual network. First, sequence features are extracted from the RNA sequences using Kmer, K-tuple nucleotide frequency component (KNFC), Pseudo dinucleotide composition (PseDNC) and Physical and chemical property (PCP). Kmer and KNFC extract information from a statistical point of view. PseDNC and PCP extract information from the physicochemical properties of RNA sequences. Then, two parts of information are fused with new features using bidirectional long- and short-term memory and attention mechanisms, respectively. Immediately after, the fused features are fed into the improved residual network for classification. Finally, 10-fold cross-validation and independent set testing are used to verify the credibility of the model. The results show that the accuracy reaches 91.87%, 95.55%, 92.27% and 95.60% on the training sets and independent test sets of Arabidopsis thaliana and M.musculus, respectively. This is a considerable improvement compared to previous studies and demonstrates the robust performance of our model. The data and code related to the study are available at https://github.com/alivelxj/m5c-DFRESG.

Estimating urban functional distributions with semantics preserved POI embedding

We present a novel approach for estimating the proportional distributions of function types (i.e. functional distributions) in an urban area through learning semantics preserved embeddings of points-of-interest (POIs). Specifically, we represent POIs as low-dimensional vectors to capture (1) the spatial co-occurrence patterns of POIs and (2) the semantics conveyed by the POI hierarchical categories (i.e. categorical semantics). The proposed approach utilizes spatially explicit random walks in a POI network to learn spatial co-occurrence patterns, and a manifold learning algorithm to capture categorical semantics. The learned POI vector embeddings are then aggregated to generate regional embeddings with long short-term memory (LSTM) and attention mechanisms, to take account of the different levels of importance among the POIs in a region. Finally, a multilayer perceptron (MLP) maps regional embeddings to functional distributions. A case study in Xiamen Island, China implements and evaluates the proposed approach. The results indicate that our approach outperforms several competitive baseline models in all evaluation measures, and yields a relatively high consistency between the estimation and ground truth. In addition, a comprehensive error analysis unveils several intrinsic limitations of POI data for this task, e.g. ambiguous linkage between POIs and functions.

Event temporal relation extraction with attention mechanism and graph neural network

Event temporal relation extraction is an important part of natural language processing. Many models are being used in this task with the development of deep learning. However, most of the existing methods cannot accurately obtain the degree of association between different tokens and events, and event-related information cannot be effectively integrated. In this paper, we propose an event information integration model that integrates event information through multilayer bidirectional long short-term memory (Bi-LSTM) and attention mechanism. Although the above scheme can improve the extraction performance, it can still be further optimized. To further improve the performance of the previous scheme, we propose a novel relational graph attention network that incorporates edge attributes. In this approach, we first build a semantic dependency graph through dependency parsing, model a semantic graph that considers the edges' attributes by using top-k attention mechanisms to learn hidden semantic contextual representations, and finally predict event temporal relations. We evaluate proposed models on the TimeBank-Dense dataset. Compared to previous baselines, the Micro-F1 scores obtained by our models improve by 3.9% and 14.5%, respectively.

Rejection-Free Monte Carlo Simulation of QUBO and Lechner–Hauke–Zoller Optimization Problems

Many studies have attempted to develop simple and efficient methods for solving global optimization problems. Simulated annealing (SA) has been recognized as a powerful tool for performing this task. In this study, we implemented a rejection-free Monte Carlo (RFMC) algorithm, which is a kernel of rejection-free SA (RFSA). We showed its validity and advantage in obtaining globally optimal solution for quadratic unconstrained binary optimization (QUBO) and spin-glass problem embedded in the Lechner–Hauke–Zoller (LHZ) architecture. Landscapes of success probabilities of finding the globally optimal solution as well as feasible solutions were evaluated as a function of a set of hyper-parameters used to characterize the weights of cost and penalty functions. We demonstrated that the efficiency of RFMC was greatly enhanced compared to that of standard MC. Furthermore, we found that the landscapes for QUBO and those for LHZ problem show considerably different views. When we focus on a specific class of problems, the smaller the problem size, the more scattered is the success probability. We also show that the success probabilities can be further improved by avoiding inefficiencies due to cycling of state transitions using short-term memory mechanisms. We suggest a reasonable strategy to tune hyper-parameters and find the correct global solution using RFSA.

Emotion recognition model based on CLSTM and channel attention mechanism

In this paper, we propose an emotion recognition model based on convolutional neural network (CNN), long short term memory (LSTM) and channel attention mechanism, aiming at the low classification accuracy of machine learning methods and the uneven spatial distribution of electroencephalogram (EEG) electrodes. This model can effectively integrate the frequency, space and time information of EEG signals, and improve the accuracy of emotion recognition by adding channel attention mechanism after the last convolutional layer of the model. Firstly, construct a 4-dimensional structure representing EEG signals. Then, a CLSTM model structure combining CNN and LSTM is designed. CNN is used to extract frequency and spatial information from 4-dimensional input, and LSTM is used to extract time information. Finally, the channel attention module is added after the last convolutional layer of CLSTM model structure to allocate the weight of different electrodes. In this paper, an emotion recognition model based on CLSTM and channel attention mechanism was proposed from the perspective of integrating the frequency, space and time 3-dimensional information of EEG signals. The average classification accuracy of the model on SEED public data set reached 93.36%, which was significantly improved over the existing CNN and LSTM emotion recognition models.

Implementation of unsupervised clustering based on population coding of magnetic tunnel junctions

Developing suitable algorithms that utilize the natural advantages of the corresponding devices is a key issue in the hardware research of brain-inspired computing. Population coding is one of the computational schemes in biological neural systems and it contains the mechanisms for noise reduction, short-term memory and implementation of complex nonlinear functions. Here we show the controllable stochastic dynamical behaviors for the technically mature spintronic device, magnetic tunnel junctions, which can be used as the basis of population coding. As an example, we construct a two-layer spiking neural network, in which groups of magnetic tunnel junctions are used to code input data. After unsupervised learning, this spiking neural network successfully classifies the iris data set. Numerical simulation demonstrates that the population coding is robust enough against the nonuniform dispersion in devices, which is inevitable in fabrication and integration of hardware devices.

A Relation-Augmented Embedded Graph Attention Network for Remote Sensing Object Detection

Multiclass geospatial object detection in high spatial resolution remote sensing imagery (HSRI) is still a challenging task. The main reason is that the objects in HRSI are location-variable and semantic-confusable, which results in the difficulties in differentiating the complicated spatial patterns and deriving the implicitly semantic labels among different categories of objects. In this article, we propose a relation-augmented embedded graph attention network (EGAT), which enables the full exploitation of the underlying spatial and semantic relations among objects for improving the detection performance. Specifically, we first construct two sets of spatial and semantic graphs of objects–objects for object relations modeling. Second, a Siamese architecture-based embedding spatial and semantic graph attention network is designed for relations reasoning, which is implemented by introducing the long short-term memory (LSTM) mechanism into the EGAT, for learning the relations among different categories of intraobjects and interobjects. Driven by the spatial and semantic LSTM, the EGAT-LSTM can adaptively focus on the critical information of reason graphs for spatial–semantic correlation discrimination in the embedding non-Euclidean feature space. By this way, the EGAT-LSTM can effectively capture the global and local spatial–semantic relationships of objects–objects, and then produce relations-augmented features for improving the performance of object detection. We conduct comprehensive experiments on three public datasets for multiclass geospatial object detection. Our method achieves state-of-the-art performance, which demonstrates the superiority and effectiveness of the proposed method.

Deep Learning-Based Speech Emotion Recognition Using Multi-Level Fusion of Concurrent Features

The detection and classification of emotional states in speech involves the analysis of audio signals and text transcriptions. There are complex relationships between the extracted features at different time intervals which ought to be analyzed to infer the emotions in speech. These relationships can be represented as spatial, temporal and semantic tendency features. In addition to emotional features that exist in each modality, the text modality consists of semantic and grammatical tendencies in the uttered sentences. Spatial and temporal features have been extracted sequentially in deep learning-based models using convolutional neural networks (CNN) followed by recurrent neural networks (RNN) which may not only be weak at the detection of the separate spatial-temporal feature representations but also the semantic tendencies in speech. In this paper, we propose a deep learning-based model named concurrent spatial-temporal and grammatical (CoSTGA) model that concurrently learns spatial, temporal and semantic representations in the local feature learning block (LFLB) which are fused as a latent vector to form an input to the global feature learning block (GFLB). We also investigate the performance of multi-level feature fusion compared to single-level fusion using the multi-level transformer encoder model (MLTED) that we also propose in this paper. The proposed CoSTGA model uses multi-level fusion first at the LFLB level where similar features (spatial or temporal) are separately extracted from a modality and secondly at the GFLB level where the spatial-temporal features are fused with the semantic tendency features. The proposed CoSTGA model uses a combination of dilated causal convolutions (DCC), bidirectional long short-term memory (BiLSTM), transformer encoders (TE), multi-head and self-attention mechanisms. Acoustic and lexical features were extracted from the interactive emotional dyadic motion capture (IEMOCAP) dataset. The proposed model achieves 75.50% and 75.82% of weighted and unweighted accuracy, 75.32% and 75.57% of recall and F1 score respectively. These results imply that concurrently learned spatial-temporal features with semantic tendencies learned in a multi-level approach improve the model’s effectiveness and robustness.

Deep Learning-based Wireless Signal Classification in the IoT Environment

With the development of the Internet of Things (IoT), diverse wireless devices are increasing rapidly. Those devices have different wireless interfaces that generate incompatible wireless signals. Each signal has its own physical characteristics with signal modulation and demodulation scheme. When there exist different wireless devices, they can suffer from severe Cross-Technology Interferences (CTI). To reduce the communication overhead due to the CTI in the real IoT environment, a central coordinator can be able to detect and identify wireless signals existing in the same communication areas. This paper investigates how to classify various radio signals using Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) and attention mechanism. CNN can reduce the amount of computation by reducing weights by using convolution, and LSTM belonging to RNN models can alleviate the long-term dependence problem. Furthermore, attention mechanism can reduce the short-term memory problem of RNNs by re-examining the data output from the decoder and the entire data entered into the encoder at every point in time. To accurately classify radio signals according to their weights, we design a model based on CNN, LSTM, and attention mechanism. As a result, we propose a model CLARINet that can classify original data by minimizing the loss and detects changes in sequences. In a case of the real IoT environment with Wi-Fi, Bluetooth and ZigBee devices, we can normally obtain wireless signals from 10 to 20 dB. The accuracy of CLARINet's radio signal classification with CNN-LSTM and attention mechanism can be seen that signal-to-noise ratio (SNR) exhibits high accuracy at 16 dB to about 92.03%.

Human Motion Tracking Using 3D Image Features with a Long Short-Term Memory Mechanism Model-An Example of Forward Reaching.

Human motion tracking is widely applied to rehabilitation tasks, and inertial measurement unit (IMU) sensors are a well-known approach for recording motion behavior. IMU sensors can provide accurate information regarding three-dimensional (3D) human motion. However, IMU sensors must be attached to the body, which can be inconvenient or uncomfortable for users. To alleviate this issue, a visual-based tracking system from two-dimensional (2D) RGB images has been studied extensively in recent years and proven to have a suitable performance for human motion tracking. However, the 2D image system has its limitations. Specifically, human motion consists of spatial changes, and the 3D motion features predicted from the 2D images have limitations. In this study, we propose a deep learning (DL) human motion tracking technology using 3D image features with a deep bidirectional long short-term memory (DBLSTM) mechanism model. The experimental results show that, compared with the traditional 2D image system, the proposed system provides improved human motion tracking ability with RMSE in acceleration less than 0.5 (m/s2) X, Y, and Z directions. These findings suggest that the proposed model is a viable approach for future human motion tracking applications.

IDS-attention: an efficient algorithm for intrusion detection systems using attention mechanism

Network attacks are illegal activities on digital resources within an organizational network with the express intention of compromising systems. A cyber attack can be directed by individuals, communities, states or even from an anonymous source. Hackers commonly conduct network attacks to alter, damage, or steal private data. Intrusion detection systems (IDS) are the best and most effective techniques when it comes to tackle these threats. An IDS is a software application or hardware device that monitors traffic to search for malevolent activity or policy breaches. Moreover, IDSs are designed to be deployed in different environments, and they can either be host-based or network-based. A host-based intrusion detection system is installed on the client computer, while a network-based intrusion detection system is located on the network. IDSs based on deep learning have been used in the past few years and proved their effectiveness. However, these approaches produce a big false negative rate, which impacts the performance and potency of network security. In this paper, a detection model based on long short-term memory (LSTM) and Attention mechanism is proposed. Furthermore, we used four reduction algorithms, namely: Chi-Square, UMAP, Principal Components Analysis (PCA), and Mutual information. In addition, we evaluated the proposed approaches on the NSL-KDD dataset. The experimental results demonstrate that using Attention with all features and using PCA with 03 components had the best performance, reaching an accuracy of 99.09% and 98.49% for binary and multiclass classification, respectively.

A decision-making algorithm combining the aspect-based sentiment analysis and intuitionistic fuzzy-VIKOR for online hotel reservation.

In the process of hotel reservation on online traveling platforms, online reviews, as a fundamental source where the actual information of a product can be had access to, have been attached with high importance by customers when they have difficulty making a decision on which hotel to pick. However, with enormous amount of online reviews distributed in diverse online traveling platforms, customers tend to have few patience or time to manually read all these reviews and get the exact information they want. Inspired by the widespread application of aspect-based sentiment analysis in the field of data mining, a bidirectional long short-term memory (Bi-LSTM) and attention mechanism based model to predict multiple attributes of a product from online review texts is proposed. Experimental result shows that such Bi-LSTM with attention mechanism model apparently improves the accuracy of the prediction, compared with single LSTM model. Meanwhile, based on the output of the prediction, we analyze and transfer it into a statistical matrix. With an intuitionistic fuzzy compromise decision-making method VIKOR applied, an overall ranking, according to multiple product attributes can be made, in which way to help customers make decisions. To prove the rationality of the algorithm, online hotel reviews from three stream online travelling platforms are crawled as a case.

Disrupting Short-Term Memory Maintenance in Premotor Cortex Affects Serial Dependence in Visuomotor Integration.

Human behavior is biased by past experience. For example, when intercepting a moving target, the speed of previous targets will bias responses in future trials. Neural mechanisms underlying this so-called serial dependence are still under debate. Here, we tested the hypothesis that the previous trial leaves a neural trace in brain regions associated with encoding task-relevant information in visual and/or motor regions. We reasoned that injecting noise by means of transcranial magnetic stimulation (TMS) over premotor and visual areas would degrade such memory traces and hence reduce serial dependence. To test this hypothesis, we applied bursts of TMS pulses to right visual motion processing region hV5/MT+ and to left dorsal premotor cortex (PMd) during intertrial intervals of a coincident timing task performed by twenty healthy human participants (15 female). Without TMS, participants presented a bias toward the speed of the previous trial when intercepting moving targets. TMS over PMd decreased serial dependence in comparison to the control Vertex stimulation, whereas TMS applied over hV5/MT+ did not. In addition, TMS seems to have specifically affected the memory trace that leads to serial dependence, as we found no evidence that participants' behavior worsened after applying TMS. These results provide causal evidence that an implicit short-term memory mechanism in premotor cortex keeps information from one trial to the next, and that this information is blended with current trial information so that it biases behavior in a visuomotor integration task with moving objects.SIGNIFICANCE STATEMENT Human perception and action are biased by the recent past. The origin of such serial bias is still not fully understood, but a few components seem to be fundamental for its emergence: the brain needs to keep previous trial information in short-term memory and blend it with incoming information. Here, we present evidence that a premotor area has a potential role in storing previous trial information in short-term memory in a visuomotor task and that this information is responsible for biasing ongoing behavior. These results corroborate the perspective that areas associated with processing information of a stimulus or task also participate in maintaining that information in short-term memory even when this information is no longer relevant for current behavior.

Short-term memory by transient oscillatory dynamics in recurrent neural networks

Despite the significance of short-term memory in cognitive function, the process of encoding and sustaining the input information in neural activity dynamics remains elusive. Herein, we unveiled the significance of transient neural dynamics to short-term memory. By training recurrent neural networks to short-term memory tasks and analyzing the dynamics, the characteristics of the short-term memory mechanism were obtained in which the input information was encoded in the amplitude of transient oscillations, rather than the stationary neural activities. This transient trajectory was attracted to a slow manifold, which permitted the discarding of irrelevant information. Additionally, we investigated the process by which the dynamics acquire robustness to noise. In this transient oscillation, the robustness to noise was obtained by a strong contraction of the neural states after perturbation onto the manifold. This mechanism works for several neural network models and tasks, which implies its relevance to neural information processing in general.