Extract Temporal Features Research Articles

A Multi-Scaling Reinforcement Learning Trading System Based on Multi-Scaling Convolutional Neural Networks

Advancements in machine learning have led to an increased interest in applying deep reinforcement learning techniques to investment decision-making problems. Despite this, existing approaches often rely solely on single-scaling daily data, neglecting the importance of multi-scaling information, such as weekly or monthly data, in decision-making processes. To address this limitation, a multi-scaling convolutional neural network for reinforcement learning-based stock trading, termed multi-scaling convolutional neural network SARSA (state, action, reward, state, action), is proposed. Our method utilizes a multi-scaling convolutional neural network to obtain multi-scaling features of daily and weekly financial data automatically. This involves using a convolutional neural network with several filter sizes to perform a multi-scaling extraction of temporal features. Multiple-scaling feature mining allows agents to operate over longer time scaling, identifying low stock positions on the weekly line and avoiding daily fluctuations during continuous declines. This mimics the human approach of considering information at varying temporal and spatial scaling during stock trading. We further enhance the network’s robustness by adding an average pooling layer to the backbone convolutional neural network, reducing overfitting. State, action, reward, state, action, as an on-policy reinforcement learning method, generates dynamic trading strategies that combine multi-scaling information across different time scaling, while avoiding dangerous strategies. We evaluate the effectiveness of our proposed method on four real-world datasets (Dow Jones, NASDAQ, General Electric, and AAPLE) spanning from 1 January 2007 to 31 December 2020, and demonstrate its superior profits compared to several baseline methods. In addition, we perform various comparative and ablation tests in order to demonstrate the superiority of the proposed network architecture. Through these experiments, our proposed multi-scaling module yields better results compared to the single-scaling module.

CGAN-rIRN: a data-augmented deep learning approach to accurate classification of mental tasks for a fNIRS-based brain-computer interface.

Functional near-infrared spectroscopy (fNIRS) is increasingly used to investigate different mental tasks for brain-computer interface (BCI) control due to its excellent environmental and motion robustness. Feature extraction and classification strategy for fNIRS signal are essential to enhance the classification accuracy of voluntarily controlled BCI systems. The limitation of traditional machine learning classifiers (MLCs) lies in manual feature engineering, which is considered as one of the drawbacks that reduce accuracy. Since the fNIRS signal is a typical multivariate time series with multi-dimensionality and complexity, it makes the deep learning classifier (DLC) ideal for classifying neural activation patterns. However, the inherent bottleneck of DLCs is the requirement of substantial-scale, high-quality labeled training data and expensive computational resources to train deep networks. The existing DLCs for classifying mental tasks do not fully consider the temporal and spatial properties of fNIRS signals. Therefore, a specifically-designed DLC is desired to classify multi-tasks with high accuracy in fNIRS-BCI. To this end, we herein propose a novel data-augmented DLC to accurately classify mental tasks, which employs a convolution-based conditional generative adversarial network (CGAN) for data augmentation and a revised Inception-ResNet (rIRN) based DLC. The CGAN is utilized to generate class-specific synthetic fNIRS signals to augment the training dataset. The network architecture of rIRN is elaborately designed in accordance with the characteristics of the fNIRS signal, with serial multiple spatial and temporal feature extraction modules (FEMs), where each FEM performs deep and multi-scale feature extraction and fusion. The results of the paradigm experiments show that the proposed CGAN-rIRN approach improves the single-trial accuracy for mental arithmetic and mental singing tasks in both the data augmentation and classifier, as compared to the traditional MLCs and the commonly used DLCs. The proposed fully data-driven hybrid deep learning approach paves a promising way to improve the classification performance of volitional control fNIRS-BCI.

STGNN-FAM: A Traffic Flow Prediction Model for Spatiotemporal Graph Networks Based on Fusion of Attention Mechanisms

Network traffic state prediction has been constantly challenged by complex spatiotemporal features of traffic information as well as imperfection in streaming data. This paper proposes a traffic flow prediction model for spatiotemporal graph networks based on fusion of attention mechanisms (STGNN-FAM) to simultaneously tackle these challenges. This model contains a spatial feature extraction layer, a bidirectional temporal feature extraction layer, and an attention fusion layer, which not only fully considers the temporal and spatial features of the traffic flow problem but also uses the attention mechanism to enhance the critical temporal and spatial features to achieve more accurate and robust predictions. Experimental results on a network traffic speed dataset PeMSD7 show that the proposed STGNN-FAM outperforms several important benchmarks in prediction accuracy and the ability to withstand interference in the data stream, especially for mid- and long-term prediction of 30 minutes and 45 minutes.

Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach.

The quality of videos varies due to the different capabilities of sensors. Video super-resolution (VSR) is a technology that improves the quality of captured video. However, the development of a VSR model is very costly. In this paper, we present a novel approach for adapting single-image super-resolution (SISR) models to the VSR task. To achieve this, we first summarize a common architecture of SISR models and perform a formal analysis of adaptation. Then, we propose an adaptation method that incorporates a plug-and-play temporal feature extraction module into existing SISR models. The proposed temporal feature extraction module consists of three submodules: offset estimation, spatial aggregation, and temporal aggregation. In the spatial aggregation submodule, the features obtained from the SISR model are aligned to the center frame based on the offset estimation results. The aligned features are fused in the temporal aggregation submodule. Finally, the fused temporal feature is fed to the SISR model for reconstruction. To evaluate the effectiveness of our method, we adapt five representative SISR models and evaluate these models on two popular benchmarks. The experiment results show the proposed method is effective on different SISR models. In particular, on the Vid4 benchmark, the VSR-adapted models achieve at least 1.26 dB and 0.067 improvement over the original SISR models in terms of PSNR and SSIM metrics, respectively. Additionally, these VSR-adapted models achieve better performance than the state-of-the-art VSR models.

A temporal segmentation approach for dendrometers signal-to-noise discrimination

Automatic analysis of point dendrometer time series (DTS) registering radial stem variations of trees is of relevant interest to study tree water use and growth. Unfortunately, data from such sensors are often characterized by a large amount of noise that needs to be distinguished by sensors responses induced by biological processes (signal) and irregular fluctuations due to electrical disturbances, malfunctions, or external inputs (noise) such as stem flow, perturbation induced by animals, stem temperature and humidity variations. Although some algorithms have been developed to adjust jumps and correct peaks in DTS, it is nowadays challenging to extract biological signals after a large number of corrections and artifacts introduced during denoising processes. In this study, we present an alternative methodology consisting of the first attempt to automatically identify days in which the dendrometers are registering information related to the activity of the tree and relevant for a specific study (days-of-signal). Through (i) per-day temporal segmentation of different stem behaviors, (ii) daily temporal features extraction, and (iii) automatic days-of-signal and days-of-noise discrimination, we automatically analyzed 19 million DTS records acquired during three years by 12 dendrometers installed on xylem and bark at different stem heights from the collar, at the bottom and top-level, of Pinus sylvestris trees. To train and assess the performance of the model, we constructed a reference dataset by labelling 600 daily DTS into days-of-signal or days-of-noise. As a result of our model application, we detected 3,534 days-of-signal among the altogether 13,152 measurement days with a per sensor overall accuracy, calculated using the reference dataset, ranging between 100% and 82%. Finally, we showed the trend of stem shrinkage and swelling over the three years study period. The large accuracies obtained over the different sensors suggest that our method is versatile and generalizable.

TSFN: A Novel Malicious Traffic Classification Method Using BERT and LSTM.

Traffic classification is the first step in network anomaly detection and is essential to network security. However, existing malicious traffic classification methods have several limitations; for example, statistical-based methods are vulnerable to hand-designed features, and deep learning-based methods are vulnerable to the balance and adequacy of data sets. In addition, the existing BERT-based malicious traffic classification methods only focus on the global features of traffic and ignore the time-series features of traffic. To address these problems, we propose a BERT-based Time-Series Feature Network (TSFN) model in this paper. The first is a Packet encoder module built by the BERT model, which completes the capture of global features of the traffic using the attention mechanism. The second is a temporal feature extraction module built by the LSTM model, which captures the time-series features of the traffic. Then, the global and time-series features of the malicious traffic are incorporated together as the final feature representation, which can better represent the malicious traffic. The experimental results show that the proposed approach can effectively improve the accuracy of malicious traffic classification on the publicly available USTC-TFC dataset, reaching an F1 value of 99.50%. This shows that the time-series features in malicious traffic can help improve the accuracy of malicious traffic classification.

Emotion Recognition via Multiscale Feature Fusion Network and Attention Mechanism

Traditional manual feature-based machine learning methods and deep learning networks have been used for Electroencephalography (EEG)-based emotion recognition in recent years. However, some existing studies ignore the low signal-to-noise ratio and the fact that each subject has unique EEG traits, which suffer from low recognition accuracy and poor robustness. To solve these problems, we propose a novel attention mechanism-based multi-scale feature fusion network (AM-MSFFN) that considers high-level features at different scales to improve the generalization of the model for different subjects. Specifically, we first utilize a spatial-temporal convolutional block to extract temporal and spatial features of EEG signals sequentially. Subsequently, considering the sampling rate of EEG signals, the multi-scale separable convolutions are designed for capturing emotional state-related information, to better combine and output feature mapping relationships. Convolutional module attention mechanism (CBAM) is applied after point-wise convolution, to better handle EEG variations of different subjects and the key information which facilitates classification. In addition, we adopt a preprocessing module based on data augmentation and data alignment to improve the quality of the training samples. Moreover, ablation studies show that the proposed attention mechanism and multiscale separable convolution contribute significant and consistent gain to the performance of our AM-MSFFN model. To verify the effectiveness of the proposed algorithm, we conducted extensive experiments on the DEAP dataset and SEED. The average accuracies achieve 99.479% and 99.297% for arousal and valence, respectively. The results demonstrated the feasibility of the proposed method.

Integrating Spatial and Temporal Information for Violent Activity Detection from Video Using Deep Spiking Neural Networks.

Increasing violence in workplaces such as hospitals seriously challenges public safety. However, it is time- and labor-consuming to visually monitor masses of video data in real time. Therefore, automatic and timely violent activity detection from videos is vital, especially for small monitoring systems. This paper proposes a two-stream deep learning architecture for video violent activity detection named SpikeConvFlowNet. First, RGB frames and their optical flow data are used as inputs for each stream to extract the spatiotemporal features of videos. After that, the spatiotemporal features from the two streams are concatenated and fed to the classifier for the final decision. Each stream utilizes a supervised neural network consisting of multiple convolutional spiking and pooling layers. Convolutional layers are used to extract high-quality spatial features within frames, and spiking neurons can efficiently extract temporal features across frames by remembering historical information. The spiking neuron-based optical flow can strengthen the capability of extracting critical motion information. This method combines their advantages to enhance the performance and efficiency for recognizing violent actions. The experimental results on public datasets demonstrate that, compared with the latest methods, this approach greatly reduces parameters and achieves higher inference efficiency with limited accuracy loss. It is a potential solution for applications in embedded devices that provide low computing power but require fast processing speeds.

An adaptive multiscale fully convolutional network for bearing fault diagnosis under noisy environments

Intelligent algorithms based on convolutional neural network (CNN) has demonstrated remarkable potential in diagnosing bearing faults. However, Accurate and robust fault diagnosis using CNNs in noisy environments remains a challenge. In this paper, we propose an end-to-end adaptive multiscale fully convolutional network (AMFCN) for intelligent bearing fault diagnosis under various noise environments. Firstly, we enhance noise adaptability by retrofitting raw signals with random sampling. Then the convolution with huge kernel is innovatively applied for wide-area temporal feature extraction and high-frequency noise suppression. By stacked residual adaptive multiscale convolution (Res_AM) blocks, the AMFCN can adaptively adjust the feature weights from different convolution scales and selectively focus on important features. The AMFCN outstrips five advanced baseline models across two bearing fault datasets with various noise environments. The proposed AMFCN significantly enhances the feature extraction ability, noise immunity, and robustness, surpassing conventional CNNs and other advanced multi-scale CNNs.

Spatial-Temporal EEG Fusion Based on Neural Network for Major Depressive Disorder Detection.

In view of the major depressive disorder characteristics such as high mortality as well as high recurrence, it is important to explore an objective and effective detection method for major depressive disorder. Considering the advantages complementary of different machine learning algorithms in information mining process, as well as the fusion complementary of different information, in this study, the spatial-temporal electroencephalography fusion framework using neural network is proposed for major depressive disorder detection. Since electroencephalography is a typical time series signal, we introduce recurrent neural network embedded in long short-term memory unit for extract temporal domain features to solve the problem of long-distance information dependence. To reduce the volume conductor effect, the temporal electroencephalography data are mapping into a spatial brain functional network using phase lag index, then the spatial domain features were extracted from brain functional network using 2D convolutional neural networks. Considering the complementarity between different types of features, the spatial-temporal electroencephalography features are fused to achieve data diversity. The experimental results show that spatial-temporal features fusion can improve the detection accuracy of major depressive disorder with a highest of 96.33%. In addition, our research also found that theta, alpha, and full frequency band in brain regions of left frontal, left central, right temporal are closely related to MDD detection, especially theta frequency band in left frontal region. Only using single-dimension EEG data as decision basis, it is difficult to fully explore the valuable information hidden in the data, which affects the overall detection performance of MDD.Meanwhile, different algorithms have their own advantages for different application scenarios. Ideally, different algorithms should use their respective advantages to jointly address complex problems in engineering fields.To this end, we propose a computer-aided MDD detection framework based on spatial-temporal EEG fusion using neural network, as shown in Fig.1. The simplified process is as follows: (1) Raw EEG data acquisition and preprocessing. (2) The time series EEG data of each channel are input as recurrent neural network (RNN), and RNN is used to process and extract temporal domain (TD) features. (3) The BFN among different EEG channels is constructed, and CNN is used to process and extract the spatial domain (SD) features of the BFN. (4) Based on the theory of information complementarity, the spatial-temporal information is fused to realize efficient MDD detection. Fig. 1 MDD detection framework based on spatial-temporal EEG fusion.

ECG-NETS – A novel integration of capsule networks and extreme gated recurrent neural network for IoT based human activity recognition

Human Activity Recognition (HAR) has reached its new dimension with the support of Internet of Things (IoT) and Artificial Intelligence (AI). To observe human activities, motion sensors like accelerometer or gyroscope can be integrated with microcontrollers to collect all the inputs and send to the cloud with the help of IoT transceivers. These inputs give the characteristics such as, angular velocity of movements, acceleration and apply them for an effective HAR. But reaching high recognition rate with less complicated computational overhead still represents a problem in the research. To solve this aforementioned issue, this work proposes a novel ensembling of Capsule Networks (CN) and modified Gated Recurrent Units (MGRU) with Extreme Learning Machine (ELM) for an effective HAR classification based on data collected using IoT systems called Ensemble Capsule Gated (ECG)-Networks (NETS). The proposed system uses Capsule networks for spatio-feature extraction and modified (Gated Recurrent Unit) GRU for temporal feature extraction. The powerful feed forward training networks are then employed to train these features for human activity recognition. The proposed model is validated on real time IoT data and WISDM datasets. Experimental results demonstrates that proposed model achieves better results comparatively higher than existing (Deep Learning) DL models.

A Decomposition Dynamic graph convolutional recurrent network for traffic forecasting

Our daily lives are greatly impacted by traffic conditions, making it essential to have accurate predictions of traffic flow within a road network. Traffic signals used for forecasting are usually generated by sensors along roads, which can be represented as nodes on a graph. These sensors typically produce normal signals representing normal traffic flows and abnormal signals indicating unknown traffic disruptions. Graph convolution networks are widely used for traffic prediction due to their ability to capture correlations between network nodes. However, existing approaches use a predefined or adaptive adjacency matrix that does not accurately reflect real-world relationships between signals. To address this issue, we propose a decomposition dynamic graph convolutional recurrent network (DDGCRN) for traffic forecasting. DDGCRN combines a dynamic graph convolution recurrent network with an RNN-based model that generates dynamic graphs based on time-varying traffic signals, allowing for the extraction of both spatial and temporal features. Additionally, DDGCRN separates abnormal signals from normal traffic signals and models them using a data-driven approach to further improve predictions. Results from our analysis of six real-world datasets demonstrate the superiority of DDGCRN compared to the current state-of-the-art. The source codes are available at: https://github.com/wengwenchao123/DDGCRN.

Enhanced LSTM Model for Daily Runoff Prediction in the Upper Huai River Basin, China

Runoff prediction is of great significance to flood defense. However, due to the complexity and randomness of the runoff process, it is hard to predict daily runoff accurately, especially for peak runoff. To address this issue, this study proposes an enhanced long short-term memory (LSTM) model for runoff prediction, where novel loss functions are introduced and feature extractors are integrated. Two loss functions (peak error tanh (PET), peak error swish (PES)) are designed to strengthen the importance of the peak runoff's prediction while weakening the weight of the normal runoff's prediction. The feature extractor consisting of three LSTM networks is established for each meteorological station, aiming to extract temporal features of the input data at each station. Taking the upper Huai River Basin in China as a case study, daily runoff from 1960–2016 is predicted using the enhanced LSTM model. Results indicate that the enhanced LSTM model performed well, achieving Nash–Sutcliffe efficiency (NSE) coefficient ranging from 0.917–0.924 during the validation period (November 2005–December 2016), outperforming the widely used lumped hydrological models (Australian Water Balance Model (AWBM), Sacramento, SimHyd and Tank Model) and the data-driven models (artificial neural network (ANN), support vector regression (SVR), and gated recurrent units (GRU)). The enhanced LSTM with PES as loss function performed best on extreme runoff prediction with a mean NSE for floods of 0.873. In addition, precipitation at a meteorological station with a higher altitude contributes more runoff prediction than the closest stations. This study provides an effective tool for daily runoff prediction, which will benefit the basin's flood defense and water security management.

Intrusion Detection Method for Internet of Vehicles Based on Parallel Analysis of Spatio-Temporal Features.

The problems with network security that the Internet of Vehicles (IoV) faces are becoming more noticeable as it continues to evolve. Deep learning-based intrusion detection techniques can assist the IoV in preventing network threats. However, previous methods usually employ a single deep learning model to extract temporal or spatial features, or extract spatial features first and then temporal features in a serial manner. These methods usually have the problem of insufficient extraction of spatio-temporal features of the IoV, which affects the performance of intrusion detection and leads to a high false-positive rate. To solve the above problems, this paper proposes an intrusion detection method for IoV based on parallel analysis of spatio-temporal features (PA-STF). First, we built an optimal subset of features based on feature correlations of IoV traffic. Then, we used the temporal convolutional network (TCN) and long short-term memory (LSTM) to extract spatio-temporal features in the IoV traffic in a parallel manner. Finally, we fused the spatio-temporal features extracted in parallel based on the self-attention mechanism and used a multilayer perceptron to detect attacks in the Internet of Vehicles. The experimental results show that the PA-STF method reduces the false-positive rate by 1.95% and 1.57% on the NSL-KDD and UNSW-NB15 datasets, respectively, with the accuracy and F1 score also being superior.

A hybrid prediction model of air quality for sparse station based on spatio-temporal feature extraction

Accurate prediction of air quality is helpful for effective prevention and control of air pollution. However, there are relatively few studies on the air-quality prediction for sparse station. Through the extraction of spatio-temporal features, this research aims to achieve the hybrid prediction of air quality for sparse station. The proposed method is comprised of three parts. The first part is the extraction of multi-scale temporal features. Specifically, the multi-scale time lags of meteorological factors are extracted to cope with sudden changes, and the key features from multi-source data are extracted to avoid the interference of redundant features. The second part is to extract spatial features of related stations based on spatial hierarchy division. The third part is the hybrid model prediction based on spatio-temporal feature groups. The following results are obtained. (1) For sudden-change prediction, the processing method proposed in this paper is more effective. (2) Compared with the baselines and models popular in dense station, the proposed model has superior performance in the air-quality prediction of sparse stations. (3) Compared with the common spatial-related station selection method, the proposed method is more suitable for sparse stations. (4) For multi-step prediction, the proposed model has significant advantages in long-term prediction.

Graph convolutional network – Long short term memory neural network- multi layer perceptron- Gaussian progress regression model: A new deep learning model for predicting ozone concertation

Ozone is one of the most important air pollutants. The high ozone concertation (OZC) affects the environment and public health. Since OZC depends on the number of different variables, it is difficult and complex to predict ozone concertation. A multilayer perceptron model is robust for predicting ozone concertation, although it has some limitations. The multi-layer perceptron (MLP) model may be unable to extract spatial and temporal features from time series data. Additionally, modelers need to developed models for handling unstructured and structured data. The purpose of this study is to combine deep learning models with MLP models to extract spatial and temporal features and improve the accuracy of predictions. We coupled deep learning models with the MLP model to overcome the limitations of the MLP model. We propose two new hybrid deep models for predicting ozone concertation. Graph convolutional network (GCN)-Long short term memory neural network (LSTM)-multi layer perceptron model (MLP)- Gaussian progress models (GPR) and convolutional neural network (CNN)-LSTM-MLP-GPR are introduced to predict hourly ozone concentration. Novelties of the papers include the development of a new model for spatial feature extraction, the introduction of a new model for temporal feature extraction, and the quantification of uncertainty. Spatial features can be extracted using CNN and GCN models. The LSTM model is a robust model for extracting temporal features. The GPR model quantifies uncertainty values. We use the models to predict ozone concentrations in three stations of Malaysia and one basin in Iran (Sefidrood basin). For the first case study (Malaysian stations), air pollution data and meteorological data were used. GCN-LSTM-MLP-GPR decreased the training MAE of CNN-LSTM-MLP-GPR, CNN-MLP-GPR, GNC-LSTM, CNN-LSTM, GCN-MLP, CNN-MLP, GCN-GPR, CNN-GPR, LSTM-GPR, MLP-GPR, GCNN, CNN, LSTM, MLP, and GPR by 1.7%, 9.2%, 14%, 15%, 18%, 21%, 22%, 27%, 30%, 32%, 34%, 35%, 37%, 42%, and 50% respectively at station 1. At station 2, the PBIAS of the CNN-LSTM-MLP-GPR and GCN-LSTM-LSTM-GPR was 2 and 4%, respectively. The testing MAE of the GCN-LSTM, CNN-LSTM, GCN-MLP, and CNN-MLP was 0.130, 0.131, 0.133, and 0.141 at station 3. In the Sefidrrod basin, the GCN-LSTM-MLP-GPR decreased the MAE of the GCN, CNN, LSTM, MLP, and GPR models by 53%, 55%, 56%, 57%, and 59%, respectively. The uncertainty analysis indicated that the uncertainty of the GCN-LSTM-MLP-GPR model was lower than that of the CNN-LSTM-MLP-GPR, CNN-MLP-GPR, GNC-LSTM, CNN-LSTM, GCN-MLP, CNN-MLP, GCN-GPR, CNN-GPR, LSTM-GPR, MLP-GPR, GCNN, CNN, LSTM, MLP, and GPR model. The GCN-LSTM-MLP-GPR model was a reliable tool for extracting data and predicting ozone concentration. The CGN-LSTM-MLP-GPR is a robust model for spatial and temporal predictions. It can be used as an early warning system to monitor air quality.

Fast Adaptive Task Offloading and Resource Allocation via Multiagent Reinforcement Learning in Heterogeneous Vehicular Fog Computing

In vehicular fog computing, task offloading enables mobile vehicles (MVs) to offer ultralow latency services for computation-intensive tasks. Nevertheless, the edge server (ES) may have a high load when a large number of MVs offload their tasks to it, causing many tasks either experience long processing times or being dropped, particularly for latency-sensitive tasks. Moreover, most existing methods are largely limited to training a model from scratch for new environments. This is because they focus more on model structures with fixed input and output sizes, impeding the transfer of trained models across different environments. To solve these problems, we propose a decentralized task offloading method based on transformer and policy decoupling-based multiagent actor–critic (TPDMAAC). We first introduce a transformer-based long sequence forecasting network (TLSFN) for predicting the current and future queuing delay of ESs to solve uncertain load. Second, we redesign the actor-network using transformer-based temporal feature extraction network (TTFEN) and policy decoupling network (PDN). TTFEN can adapt to various input sizes through a transformer that accepts different tokens we build from the raw input. PDN provides a mapping between the transformer-based embedding features and offloading policies utilizing self-attention mechanism to address various output dimensions. Finally, the experiments on two real-world data sets show that TPDMAAC can quickly adapt to a new environment. And compared to existing algorithms, TPDMAAC reduces the system cost by 11.01%–12.03% as well as improves task completion rates by 10.45%–13.56%.

A Spatio-Temporal Siamese Neural Network for Multimodal Handwriting Abnormality Screening of Parkinson’s Disease

Currently, hand motion recognition of single-modality data has been extensively explored for the analysis of various contact and noncontact sensors, and it is recognized that all the existing technologies have both strengths and limitations. As a significant motor symptom, hand tremor is usually utilized for the diagnosis and evaluation of Parkinson’s disease; furthermore, a multimodal analysis of the handwriting pattern of the patient has made up for the one-sided way of learning the hand movement in a single measurement dimension. Especially, considering a variety of measurement resources, it shows promising performance in recognizing handwriting patterns of Parkinson’s disease. In this work, a novel Spatio-temporal Siamese neural network (ST-SiamNN) is proposed to learn the handwriting differences between healthy individuals and patients with Parkinson’s disease, process data onto multiple sensors, and enhance the characteristics of handwriting in Parkinson’s disease. Uniquely, it is a discriminative model of multilabel and multinetwork constructed by a Siamese network, which consists of four modules: a preprocessor for handwritten data enhancement, a Siamese bidirectional memory neural network (SiamBiMNN) for temporal and texture feature extraction and difference enhancement, a Siamese octave convolutional neural network (SiamOctCNN) for spatial feature extraction and difference enhancement, and a decision-making layer to rejudge the output features of the Siamese networks to obtain more accurate auxiliary diagnosis results. The framework proposed in this article is verified on two handwritten datasets of multiple modalities, i.e., images, smart pen signals, and graphics tablet signals, which are compared with several state-of-the-art studies.

GDNet-EEG: An attention-aware deep neural network based on group depth-wise convolution for SSVEP stimulation frequency recognition.

Steady state visually evoked potentials (SSVEPs) based early glaucoma diagnosis requires effective data processing (e.g., deep learning) to provide accurate stimulation frequency recognition. Thus, we propose a group depth-wise convolutional neural network (GDNet-EEG), a novel electroencephalography (EEG)-oriented deep learning model tailored to learn regional characteristics and network characteristics of EEG-based brain activity to perform SSVEPs-based stimulation frequency recognition. Group depth-wise convolution is proposed to extract temporal and spectral features from the EEG signal of each brain region and represent regional characteristics as diverse as possible. Furthermore, EEG attention consisting of EEG channel-wise attention and specialized network-wise attention is designed to identify essential brain regions and form significant feature maps as specialized brain functional networks. Two publicly SSVEPs datasets (large-scale benchmark and BETA dataset) and their combined dataset are utilized to validate the classification performance of our model. Based on the input sample with a signal length of 1 s, the GDNet-EEG model achieves the average classification accuracies of 84.11, 85.93, and 93.35% on the benchmark, BETA, and combination datasets, respectively. Compared with the average classification accuracies achieved by comparison baselines, the average classification accuracies of the GDNet-EEG trained on a combination dataset increased from 1.96 to 18.2%. Our approach can be potentially suitable for providing accurate SSVEP stimulation frequency recognition and being used in early glaucoma diagnosis.

State of Health prediction of lithium-ion batteries based on temporal degeneration feature extraction with Deep Cycle Attention Network

The State of Health (SOH) forecasting is essential for applying lithium-ion batteries in energy storage systems. The streaming sensor data collected by battery management systems (BMS) contain information about the battery SOH. Therefore, the battery degradation can be treated as the attention-based similarity in the battery cycles from the data point of view. The paper develops the Deep Cycle Attention Network (DCAN) with the attention mechanism to measure the battery state similarity in different cycles with the temporal patterns extracted from the streaming sensor data. The DCAN model can efficiently predict battery SOH under various operational working conditions with the attention mechanism. Essentially, the attention-based similarity regarding the first cycle can describe the battery SOH development in the charge and discharge cycles by extracting the degeneration-related temporal patterns from the streaming sensor data. Based on the MIT-Stanford dataset and NASA PCoE dataset, the prediction performance of the DCAN model is tested under constant current load profiles. Furthermore, the Oxford dataset is used to test the prediction performance with the characterization load profiles under the conditions of driving cycles. As a result, the proposed DCAN model demonstrates high accuracy and strong robustness for the two kinds of load profiles.