Temporal Network Research Articles

WVETT-Net: A Novel Hybrid Prediction Model for Wireless Network Traffic Based on Variational Mode Decomposition

Precise prediction of wireless communication network traffic is indispensable in the operational deployment of base station resources and improvement of the user experience. Cellular wireless network traffic has both spatial and temporal characteristics. The existing modeling algorithms have achieved good results in extracting the spatial features, but there are still deficiencies in the extraction models for the time dependencies. To resolve these problems, this paper proposes a novel hybrid neural network prediction model, called WVETT-Net. Firstly, variational mode decomposition (VMD) is used to preprocess network traffic, and the whale optimization algorithm (WOA) is used to select the optimal parameters for VMD. Secondly, the local and global features are extracted from each subsequence by a temporal convolutional network (TCN) and an improved Transformer network with a multi-head ProbSparse self-attention mechanism (Pe-Transformer), respectively. Finally, the extracted feature representation is enhanced by using an efficient channel attention (ECA) mechanism to achieve accurate wireless network traffic predictions. Experimental results on two wireless network traffic datasets show that the proposed model (WVETT-Net) outperforms the traditional single or combined models in wireless network traffic prediction.

Dual temporal attention mechanism-based convolutional LSTM model for industrial dynamic soft sensor

Abstract Deep learning is an appropriate methodology for modeling complex industrial data in the field of soft sensors, owing to its powerful feature representation capability. Given the nonlinear and dynamic nature of the process industry, the key challenge for soft sensor technology is to effectively mine dynamic information from long sequences and accurately extract features of relevance to quality. A dual temporal attention mechanism-based convolutional long short-term memory network (DTA-ConvLSTM) under an encoder-decoder framework is proposed as a soft sensor model to acquire quality-relevant dynamic features from serial data. Considering different influences of process variables for prediction at multiple time steps and various locations, ConvLSTM and temporal self-attention mechanism are utilized as the encoder to adaptively fuse spatiotemporal features and capture long-term dynamic properties of process in order to capture the trends of industrial variables. Furthermore, a quality-driven temporal attention mechanism is employed throughout the decoding process to dynamically select relevant features to more accurately track quality changes. The encoder-decoder model meticulously analyses the interactions between process and quality variables by incorporating dual-sequence dynamic information to improve the prediction performance. The validity and superiority of the DTA-ConvLSTM model was validated on two industrial case studies of the debutanizer column and sulfur recovery unit. Compared to the traditional LSTM model, the proposed model demonstrated a substantial improvement with the accuracy R2 up to 97.3% and 94.9% and the root mean square error reducing to 0.122 and 0.022.

A Stock Market Decision-Making Framework Based on CMR-DQN

In the dynamic and uncertain stock market, precise forecasting and decision-making are crucial for profitability. Traditional deep neural networks (DNN) often struggle with capturing long-term dependencies and multi-scale features in complex financial time series data. To address these challenges, we introduce CMR-DQN, an innovative framework that integrates discrete wavelet transform (DWT) for multi-scale data analysis, temporal convolutional network (TCN) for extracting deep temporal features, and a GRU–LSTM–Attention mechanism to enhance the model’s focus and memory. Additionally, CMR-DQN employs the Rainbow DQN reinforcement learning strategy to learn optimal trading strategies in a simulated environment. CMR-DQN significantly improved the total return rate on six selected stocks, with increases ranging from 20.37% to 55.32%. It also demonstrated substantial improvements over the baseline model in terms of Sharpe ratio and maximum drawdown, indicating increased excess returns per unit of total risk and reduced investment risk. These results underscore the efficiency and effectiveness of CMR-DQN in handling multi-scale time series data and optimizing stock market decisions.

A data-driven approach to detect upper limb functional use during daily life in breast cancer survivors using wrist-worn sensors

To gain insights into the impact of upper limb (UL) dysfunctions after breast cancer treatment, this study aimed to develop a temporal convolutional neural network (TCN) to detect functional daily UL use in breast cancer survivors using data from a wrist-worn accelerometer. A pre-existing dataset of 10 breast cancer survivors was used that contained raw 3-axis acceleration data and simultaneously recorded video data, captured during four daily life activities. The input of our TCN consists of a 3-axis acceleration sequence with a receptive field of 243 samples. The 4 ResNet TCN blocks perform dilated temporal convolutions with a kernel of size 3 and a dilation rate that increases by a factor of 3 after each iteration. Outcomes of interest were functional UL use (minutes) and percentage UL use. We found strong agreement between the video and predicted data for functional UL use (ICC = 0.975) and moderately strong agreement for %UL use (ICC = 0.794). The TCN model overestimated the functional UL use by 0.71 min and 3.06%. Model performance showed good accuracy, f1, and AUPRC scores (0.875, 0.909, 0.954, respectively). In conclusion, using wrist-worn accelerometer data, the TCN model effectively identified functional UL use in daily life among breast cancer survivors.

Decentralized Adaptive temporal-difference learning over time-varying networks and its finite-time analysis

In reinforcement learning, centralized temporal-difference (TD) learning is commonly used to solve the policy evaluation problem. However, the decentralized adaptive variant of the TD learning algorithm has rarely been investigated in multi-agent reinforcement learning. To fill this gap, based on linear function approximation, we propose a decentralized adaptive TD learning algorithm over time-varying networks, referred to as D-AdaTD. We rigorously analyze the convergence performance of D-AdaTD, i.e., the explicit finite-time analysis is established under different step-sizes. Specifically, under constant step-sizes, the average estimated value function can converge to a neighborhood of the optimal value at rate O(1/(k+1)) and the estimated parameter of each agent can converge to a neighborhood of the optimal parameter at rate O(ξk), where k is the number of iterations and ξ∈(0,1). Under diminishing step-sizes, the average estimated value function can converge to the optimal value and the average estimated parameter can converge to the optimal parameter at rate O((1+log(k+1))/k) and O((1+log(k+1))/(k+1)), respectively. In addition, we evaluate the performance of D-AdaTD via simulation experiments, which are commonly insufficient in the existing decentralized temporal-difference learning. The experimental results also validate the effectiveness of D-AdaTD.

A comprehensive fault detection and diagnosis method for chemical processes

Deep learning methods based on supervised learning have various applications in chemical process fault detection and diagnosis (FDD). However, these methods as black boxes are unable to provide guidance for process recovery, and the labels and fault samples available in actual chemical processes are insufficient for ideal supervised learning. In this study, eMixDTFCN containing FDD backbone network, semi-supervised learning framework, and explainable method is proposed for the first time to address the above challenges simultaneously. Dense temporal feature convolutional network (DTFCN) combining process variable embedding and dense temporal convolution is taken as the backbone network, where the sparse features from the high-dimensional embedded space are learned efficiently through the dense temporal convolution. MixMatch based on consistent regularization and entropy minimization enables DTFCN with semi-supervised learning and imbalanced learning. SHAP method trains an explainer to provide fault-related variables for process recovery. The high effectiveness and comprehensiveness of eMixDTFCN are validated in two typical chemical processes.

Deep learning for Bitcoin price direction prediction: models and trading strategies empirically compared

This paper applies deep learning models to predict Bitcoin price directions and the subsequent profitability of trading strategies based on these predictions. The study compares the performance of the convolutional neural network–long short-term memory (CNN–LSTM), long- and short-term time-series network, temporal convolutional network, and ARIMA (benchmark) models for predicting Bitcoin prices using on-chain data. Feature-selection methods—i.e., Boruta, genetic algorithm, and light gradient boosting machine—are applied to address the curse of dimensionality that could result from a large feature set. Results indicate that combining Boruta feature selection with the CNN–LSTM model consistently outperforms other combinations, achieving an accuracy of 82.44%. Three trading strategies and three investment positions are examined through backtesting. The long-and-short buy-and-sell investment approach generated an extraordinary annual return of 6654% when informed by higher-accuracy price-direction predictions. This study provides evidence of the potential profitability of predictive models in Bitcoin trading.

EEG-VTTCNet: A loss joint training model based on the vision transformer and the temporal convolution network for EEG-based motor imagery classification

Brain-computer interface (BCI) is a technology that directly connects signals between the human brain and a computer or other external device. Motor imagery electroencephalographic (MI-EEG) signals are considered a promising paradigm for BCI systems, with a wide range of potential applications in medical rehabilitation, human–computer interaction, and virtual reality. Accurate decoding of MI-EEG signals poses a significant challenge due to issues related to the quality of the collected EEG data and subject variability. Therefore, developing an efficient MI-EEG decoding network is crucial and warrants research. This paper proposes a loss joint training model based on the vision transformer (VIT) and the temporal convolutional network (EEG-VTTCNet) to classify MI-EEG signals. To take advantage of multiple modules together, the EEG-VTTCNet adopts a shared convolution strategy and a dual-branching strategy. The dual-branching modules perform complementary learning and jointly train shared convolutional modules with better performance. We conducted experiments on the BCI Competition IV-2a and IV-2b datasets, and the proposed network outperformed the current state-of-the-art techniques with an accuracy of 84.58% and 90.94%, respectively, for the subject-dependent mode. In addition, we used t-SNE to visualize the features extracted by the proposed network, further demonstrating the effectiveness of the feature extraction framework. We also conducted extensive ablation and hyperparameter tuning experiments to construct a robust network architecture that can be well generalized.

Spatial and temporal attention embedded spatial temporal graph convolutional networks for skeleton based gait recognition with multiple IMUs

Gait recognition is one of the key technologies for exoskeleton robot control, while the current IMU-based gait recognition methods only use inertial data and do not fully consider the interconnections of human spatial structure and human joints. In this regard, a skeleton-based gait recognition approach with inertial measurement units using spatial temporal graph convolutional networks with spatial and temporal attention is proposed. A human forward kinematics solver module was used for constructing different human skeleton models and a temporal attention module was added for capturing the more important time frames in the gait cycle. Moreover, the two-stream structure was used to construct spatial temporal graph convolutional networks with spatial and temporal attention for gait recognition, and an average accuracy of about 99% was obtained in user experiments, which is the best performance compared to other algorithms, provides certain reference for gait recognition and real-time control of exoskeleton robots.

ANALYSIS OF RESOLVING EFFICIENT DOMINATING SET AND ITS APPLICATION SCHEME IN SOLVING ETLE PROBLEMS

This research focuses on the analysis of Resolving Efficient Dominating Set (REDS) and its application in solving Electronic Traffic Law Enforcement (ETLE) problems using the Spatial Temporal Graph Neural Network (STGNN). Resolving Efficient Dominating Set (REDS) is a concept in graph theory that studies a set of points in a graph that efficiently monitors other points. It involves ensuring that each point v ∈ V (G) - D is dominated by exactly one point in D, with no adjacent points in D, and the representation of point v ∈ V (G) concerning D is not the same, which is termed as a resolving efficient dominating set. In the context of Electronic Traffic Law Enforcement (ETLE), the analysis of REDS has a significant impact. The theorem resulting from the analysis of REDS enables the determination of the number of traffic violation sensors required. Furthermore, by taking simulation data from road points, violation forecasting can be performed. The accurate predictions from this forecasting can assist authorities in anticipating and addressing traffic violation issues more effectively.

Machine-Learning-Based Multi-Modal Force Estimation for Steerable Ablation Catheters.

Catheter-based cardiac ablation is a minimally invasive procedure for treating atrial fibrillation (AF). Electrophysiologists perform the procedure under image guidance during which the contact force between the heart tissue and the catheter tip determines the quality of lesions created. This paper describes a novel multi-modal contact force estimator based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The estimator takes the shape and optical flow of the deflectable distal section as two modalities since frames and motion between frames complement each other to capture the long context in the video frames of the catheter. The angle between the tissue and the catheter tip is considered a complement of the extracted shape. The data acquisition platform measures the two-degrees-of-freedom contact force and video data as the catheter motion is constrained in the imaging plane. The images are captured via a camera that simulates single-view fluoroscopy for experimental purposes. In this sensor-free procedure, the features of the images and optical flow modalities are extracted through transfer learning. Long Short-Term Memory Networks (LSTMs) with a memory fusion network (MFN) are implemented to consider time dependency and hysteresis due to friction. The architecture integrates spatial and temporal networks. Late fusion with the concatenation of LSTMs, transformer decoders, and Gated Recurrent Units (GRUs) are implemented to verify the feasibility of the proposed network-based approach and its superiority over single-modality networks. The resulting mean absolute error, which accounted for only 2.84% of the total magnitude, was obtained by collecting data under more realistic circumstances in contrast to previous research studies. The decrease in error is considerably better than that achieved by individual modalities and late fusion with concatenation. These results emphasize the practicality and relevance of utilizing a multimodal network in real-world scenarios.

Time-dependent personalized PageRank for temporal networks: Discrete and continuous scales.

In this paper, we explore the PageRank of temporal networks (networks that evolve with time) with time-dependent personalization vectors. We consider both continuous and discrete time intervals and show that the PageRank of a continuous-temporal network can be nicely estimated by the PageRanks of the discrete-temporal networks arising after sampling. Additionally, precise boundaries are given for the estimated influence of the personalization vector on the ranking of a particular node. All ingredients in the classic PageRank definition, namely, the normalized matrix collecting the topology of the network, the damping factor, and the personalization vector are allowed, to the best of our knowledge, for the first time in the literature to vary independently with time. The theoretical results are illustrated by means of some real and synthetic examples.

Fault diagnosis of planetary gearboxes under variable operating conditions based on AWM-TCN

In the fault diagnosis of planetary gearboxes under varying operating conditions, fault features are often submerged and difficult to characterize. To address this issue, this paper proposes a fault diagnosis method for planetary gearboxes based on an Attention Mechanism Wide-scale Multi-channel Temporal Convolutional Network (AWM-TCN). This method introduces an attention mechanism into the TCN framework, ensuring the model learns the spatiotemporal characteristics of fault signals while enhancing the importance of fault-related features. The experimental section innovatively employs three different validation methods for multi-angle evaluation of the model, discusses the impact of different structures on the proposed model, and compares it with various classical models. The results indicate that AWM-TCN can achieve effective diagnosis of planetary gearboxes under varying conditions and demonstrates rapid responsiveness.

SEMG-driven Hand Dynamics Estimation with Incremental Online Learning on a Parallel Ultra-Low-Power Microcontroller.

Surface electromyography (sEMG) is a State-of-the-Art (SoA) sensing modality for non-invasive human-machine interfaces for consumer, industrial, and rehabilitation use cases. The main limitation of the current sEMG-driven control policies is the sEMG's inherent variability, especially cross-session due to sensor repositioning; this limits the generalization of the Machine/Deep Learning (ML/DL) in charge of the signal-to-command mapping. The other hot front on the ML/DL side of sEMG-driven control is the shift from the classification of fixed hand positions to the regression of hand kinematics and dynamics, promising a more versatile and fluid control. We present an incremental online-training strategy for sEMG-based estimation of simultaneous multi-finger forces, using a small Temporal Convolutional Network suitable for embedded learning-on-device. We validate our method on the HYSER dataset, cross-day. Our incremental online training reaches a cross-day Mean Absolute Error (MAE) of (9.58 ± 3.89)% of the Maximum Voluntary Contraction on HYSER's RANDOM dataset of improvised, non-predefined force sequences, which is the most challenging and closest to real scenarios. This MAE is on par with an accuracy-oriented, non-embeddable offline training exploiting more epochs. Further, we demonstrate that our online training approach can be deployed on the GAP9 ultra-low power microcontroller, obtaining a latency of 1.49 ms and an energy draw of just 40.4 uJ per forward-backward-update step. These results show that our solution fits the requirements for accurate and real-time incremental training-on-device.

Chondroitin/dermatan sulphate proteoglycan, desmosealin, showing affinity to desmosomes.

Desmosomes are the most prominent interkeratinocyte junctions. The correct barrier function of stratified epithelia such as epidermis depends on their expression. During epidermal differentiation, the molecular composition of desmosomes evolves and so do their physical and chemical properties. Desquamation of corneocytes at the surface of the stratum corneum depends on an orderly degradation of desmosomes by endogenous enzymes. This process may be regulated by glycosylated molecules. We focused on the detection and characterization of potentially implicated players bearing ‘sugar’ characteristics. Using an original monoclonal antibody and biochemical methods, we partially characterized a proteoglycan of the exclusively chondroitin/dermatan sulphate type, secreted into the interkeratinocyte spaces, that is incorporated into the extracellular parts of desmosomes in quantities proportional to the degree of cell differentiation, as visualized with immuno-electron microscopy. This antigen, that we named desmosealin, displays biochemical and immunocytochemical characteristics that clearly differentiate it from known desmosomal elements. Unlike so far described epidermal proteoglycans, which belong to the heparan sulphate family, desmosealin displays chondroitin/dermatan sulphate glycosaminoglycan chains. It can be detected within the extracellular ‘cores’ of desmosomes in the upper viable epidermal layers and in corneodesmosomes from the lowermost part of the stratum corneum. Extensive integration of proteoglycans into the extracellular parts of desmosomes at the late stages of keratinocyte maturation is likely of functional importance. Given its biochemical profile, its pattern of expression in the epidermis and its desmosomal localization, desmosealin may emerge as a key element in the regulation of desmosome processing, epidermal cohesion and formation of a functional epidermal barrier.

Performance degradation prediction of rolling bearing based on temporal graph convolutional neural network

Performance degradation prediction of rolling bearing based on temporal graph convolutional neural network

Semantics-Enhanced Temporal Graph Networks for Content Popularity Prediction

Semantics-Enhanced Temporal Graph Networks for Content Popularity Prediction

A panel network analysis of posttraumatic stress disorder and obsessive-compulsive disorder symptoms across the perinatal period.

The perinatal period is marked by a higher risk of experiencing depressive, anxiety, and/or trauma-related symptoms, a phenomenon that affects millions of individuals each year. Obsessive-compulsive disorder (OCD) and posttraumatic stress disorder (PTSD) symptoms commonly co-occur but have rarely been examined together beyond prevalence estimates in the perinatal period. Our study aimed to explore patterns of associations among OCD and PTSD symptoms to elucidate within- and between-person effects and how these effects may change over time. Participants (N = 270) were recruited during pregnancy from an academic medical center affiliated with a midwestern university. PTSD, OCD, and depressive symptoms were assessed at pregnancy, 4, 8, and 12 weeks postpartum. A panel graphical vector autoregression model was used to estimate networks. The temporal network provided information regarding directed predictive effects between symptoms, and hyperarousal, neutralizing, and ordering were the most stable and predictive symptoms across time. The contemporaneous network, which yields undirected partial correlations between symptoms at a given moment, indicated that there were positive associations between intrusions and avoidance, hyperarousal and negative alterations in cognitions and mood, as well as between hyperarousal and dysphoria. This study identified hyperarousal and neutralizing as the PTSD and OCD symptoms with the strongest stability, predictive power, and association with other symptoms. Clinically, this indicates that screening for hyperarousal and neutralizing symptoms may identify individuals who could maximally benefit from treatment in the perinatal period. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Age-associated proteins explain the role of medial temporal lobe networks in Alzheimer's disease.

The structural connectivity (SC) of the medial temporal lobe and its associated cortical anterior temporal and posterior medial networks (MTL-AT-PM) is linked to pathologies and memory decline in Alzheimer's disease (AD). However, neuroimaging analyses cannot tell us how SC changes occur in AD at the molecular level and do not provide a means of intervening to slow/prevent pathology-related changes in MTL-AT-PM SC. The current study aimed to understand how and where AD-related changes occur within MTL-AT-PM using proteomics. We used a 4-step approach in 101 older adults froma local sample, aiming to understand how proteins and SC in combination at the multivariate level predict AD pathology, and to identify specific proteins related to SC and AD pathology. Separately, we validated the discovered proteins in relation to SC and AD pathology using ADNI sample. We identified 12 latent factors linking proteins and SC; five showed significant relationships with AD pathology and/or episodic memory. Insulin-like growth factor binding proteins and tumor necrosis factor receptors, and hippocampal/parahippocampal edges contributed most to AD-related latent factors. Fast causal inference found protein-protein, protein-SC, and protein-pathology pathways, with seven proteins showing directional links to SC and AD-related neurodegeneration. We validated these results by identifying significant relationships between six available proteins with SC and amyloid-beta and phosphorylated tau in ADNI. We identified multivariate relationships between proteins and MTL-AT-PM networks that add to our understanding of AD pathology and suggest specific non-pathological proteins that warrant further study in relation to brain networks and AD pathology as possible therapeutic targets.

Temporal graph convolutional network soft sensor for molecular weight distribution prediction

In chemical processes with distributed outputs, characteristics of products are influenced by their distributions and significantly correlated with process variables. It is crucial for an accurate distribution characteristic prediction to adequately describe variable relationships and their temporal variations. For this purpose, a temporal graph convolutional network (TGCN) soft sensor is developed to describe the distribution of outputs. First, the variable relationships are represented in a topology subgraph based on prior knowledge. Then, the graph is supplemented based on variable screening results with the maximal information coefficient (MIC) as standard. Finally, the graph convolutional mechanism is used to model variable relationships, the gated recurrent unit to capture temporal dependencies, and GNNexplainer to provide a comprehensive explanation for the prediction. Results suggest that prediction accuracy and explainability is improved by the proposed TGCN soft sensor on the basis of prior knowledge, and verified in the case of molecular weight distribution (MWD) modeling.