Temporal Dependence Research Articles

HGCTNet: Handcrafted Feature-Guided CNN and Transformer Network for Wearable Cuffless Blood Pressure Measurement.

Biosignals collected by wearable devices, such as electrocardiogram and photoplethysmogram, exhibit redundancy and global temporal dependencies, posing a challenge in extracting discriminative features for blood pressure (BP) estimation. To address this challenge, we propose HGCTNet, a handcrafted feature-guided CNN and transformer network for cuffless BP measurement based on wearable devices. By leveraging convolutional operations and self-attention mechanisms, we design a CNN-Transformer hybrid architecture to learn features from biosignals that capture both local information and global temporal dependencies. Then, we introduce a handcrafted feature-guided attention module that utilizes handcrafted features extracted from biosignals as query vectors to eliminate redundant information within the learned features. Finally, we design a feature fusion module that integrates the learned features, handcrafted features, and demographics to enhance model performance. We validate our approach using two large wearable BP datasets: the CAS-BP dataset and the Aurora-BP dataset. Experimental results demonstrate that HGCTNet achieves an estimation error of 0.9 ± 6.5 mmHg for diastolic BP (DBP) and 0.7 ± 8.3 mmHg for systolic BP (SBP) on the CAS-BP dataset. On the Aurora-BP dataset, the corresponding errors are -0.4 ± 7.0 mmHg for DBP and -0.4 ± 8.6 mmHg for SBP. Compared to the current state-of-the-art approaches, HGCTNet reduces the mean absolute error of SBP estimation by 10.68% on the CAS-BP dataset and 9.84% on the Aurora-BP dataset. These results highlight the potential of HGCTNet in improving the performance of wearable cuffless BP measurements.

Dynamic Temporal Dependency Model for Multiple Steps Ahead Short-Term Load Forecasting of Power System

Dynamic Temporal Dependency Model for Multiple Steps Ahead Short-Term Load Forecasting of Power System

Multi-site solar irradiance prediction based on hybrid spatiotemporal graph neural network

Constructing accurate spatiotemporal correlations is a challenging task in joint prediction of multiple photovoltaic sites. Some advanced algorithms for incorporating other surrounding site information have been proposed, such as graph neural network-based methods, which are usually based on static or dynamic graphs to build spatial dependencies between sites. However, the possibility of the simultaneous existence of multiple spatial dependencies is not considered. This paper establishes a spatiotemporal prediction model based on hybrid spatiotemporal graph neural network. In this model, we apply adaptive hybrid graph learning to learn composite spatial correlations among multiple sites. A temporal convolution module with multi-subsequence temporal data input is used to extract local semantic information to better predict future nonlinear temporal dependencies. A spatiotemporal adaptive fusion module is added to address the issue of integrating diverse spatiotemporal trends among multiple sites. To assess the model's predictive performance, nine solar radiation observation stations were selected in two different climatic environments. The average root mean square error (RMSE) of the constructed model was 38.51 and 49.90 W/m2, with average mean absolute error (MAE) of 14.72 and 23.06 W/m2, respectively. Single-site and multi-site prediction models were selected as baseline models. Compared with the baseline models, the RMSE and MAE reduce by 3.1%–20.8% and 8.9%–32.8%, respectively, across all sites. The proposed model demonstrates the effectiveness of improving accuracy in forecasting solar irradiance through multi-site predictions.

Global Environmental Health Assessment: Analyzing Air Quality and Water Pollution Data

Abstract: This study employs machine learning techniques to assess global environmental health, focusing on air quality and water pollution. Through extensive data collection and preprocessing, including feature engineering, insights are extracted from diverse datasets encompassing pollutant concentrations, meteorological conditions, and socio-economic indicators. Machine learning algorithms, including LSTM models, are employed to analyze temporal dependencies and predict pollution levels. Additionally, clustering, regression analysis, and spatial analysis techniques aid in identifying pollution hotspots and trends. The proposed system integrates IoT technology for real-time data collection and Apache Spark for efficient processing. Evaluation metrics such as Mean Absolute Error and Root Mean Square Error assess model performance. The dataset comprises hourly averaged responses from chemical sensors deployed in a polluted area, complemented by ground truth data from a reference analyzer. This research contributes to informed decision-making for environmental management and sustainable development in smart city environments

STFGCN: Spatial–temporal fusion graph convolutional network for traffic prediction

Accurate traffic prediction plays a crucial role in improving traffic conditions and optimizing road utilization. Effectively capturing the multi-scale temporal dependencies and dynamic spatial dependencies is crucial for accurate traffic prediction. These features can effectively reflect complex dynamic spatial–temporal processes, which have not been comprehensively addressed in most existing research work. Motivated by this issue, the primary contribution of this paper lies in proposing a novel Spatial–Temporal Fusion Graph Neural Network (STFGCN) for accurate traffic prediction, achieved by extracting multi-scale temporal dependencies from multiple semantic environments and constructing a dynamic adaptive graph to model spatial dependencies based on temporal characteristics. Specifically, to capture the multi-scale dynamic temporal dependencies effectively, a Multi-Scale Fusion Convolution (MSFC) module is designed, in which the temporal dependencies are extracted from multiple textual environments by utilizing multi-scale convolution. In order to model dynamic spatial dependencies, a Spatial Adaptive Fusion Convolution (SAFC) module is designed by combining the recent coherence and periodicity to infer dynamic graphs, which are then fused to model dynamic spatial dependencies. Extensive experimental results on five real-world datasets demonstrate that the proposed STFGCN has superior performance. Specifically, compared with the state-of-the-art baselines, STFGCN reduced 1.2% to 16.4% in RMSE measure.

Construction of explanations at the local level of decision-making process representation for internal users of the intelligent system

The subject of study in the article is the processes of constructing explanations for the obtained results and the sequence of decision-making actions in an intelligent information system. The goal is to develop an approach to constructing explanations at the local level of representation of the decision-making process of an intelligent system based on building temporal and causal dependencies, which creates conditions for identifying "bottlenecks" of this process when representing the intelligent system as a "gray" box. Tasks: development of an explanation model at the local level of representation for internal users of an intelligent system; development of a method for constructing explanations at the local level of representation of the decision-making process of an intelligent system. The approaches used are: methods of constructing explanations, approaches to constructing temporal knowledge. Conclusions. The scientific novelty of the obtained results is as follows. An explanation model at the local level of representation of the decision-making process for internal users of an intelligent information system has been developed. The model uses a combination of temporal dependencies that determine the linear sequence of process actions and causal rules-constraints that specify mandatory actions performed for all variants of the decision-making process. The proposed model creates conditions for identifying actions that limit the normal execution of the decision-making process and lead to results that do not meet the requirements of external users. A method for constructing explanations at the local level of representation of the decision-making process of an intelligent system has been developed. The method involves identifying temporal dependencies between pairs of consecutive states of the current decision-making process, selecting temporal dependencies that are performed for all variants of the decision-making process, and forming causal dependencies based on them. Using the method makes it possible to present decision-making constraints as an explanation, which creates conditions for identifying bottlenecks in the sequence of actions for obtaining a result in an intelligent system.

Machine Learning in Short-Reach Optical Systems: A Comprehensive Survey

Recently, extensive research has been conducted to explore the utilization of machine learning (ML) algorithms in various direct-detected and (self)-coherent short-reach communication applications. These applications encompass a wide range of tasks, including bandwidth request prediction, signal quality monitoring, fault detection, traffic prediction, and digital signal processing (DSP)-based equalization. As a versatile approach, ML demonstrates the ability to address stochastic phenomena in optical systems networks where deterministic methods may fall short. However, when it comes to DSP equalization algorithms such as feed-forward/decision-feedback equalizers (FFEs/DFEs) and Volterra-based nonlinear equalizers, their performance improvements are often marginal, and their complexity is prohibitively high, especially in cost-sensitive short-reach communications scenarios such as passive optical networks (PONs). Time-series ML models offer distinct advantages over frequency-domain models in specific contexts. They excel in capturing temporal dependencies, handling irregular or nonlinear patterns effectively, and accommodating variable time intervals. Within this survey, we outline the application of ML techniques in short-reach communications, specifically emphasizing their utilization in high-bandwidth demanding PONs. We introduce a novel taxonomy for time-series methods employed in ML signal processing, providing a structured classification framework. Our taxonomy categorizes current time-series methods into four distinct groups: traditional methods, Fourier convolution-based methods, transformer-based models, and time-series convolutional networks. Finally, we highlight prospective research directions within this rapidly evolving field and outline specific solutions to mitigate the complexity associated with hardware implementations. We aim to pave the way for more practical and efficient deployment of ML approaches in short-reach optical communication systems by addressing complexity concerns.

Real-time unsupervised monitoring of earth pressure balance shield-induced sinkholes in mixed-face ground conditions via convolutional variational autoencoders

This study introduces a real-time unsupervised monitoring framework for monitoring sinkhole formation events during earth pressure balance (EPB) shield tunneling operations. A feature extractor (FE) is constructed by coupling variational Autoencoders structure with convolutional neural network layers (VAE-CNN) to manage the complexity of EPB operational data, including non-linearity and temporal dependencies. The monitoring framework consists of two main phases: offline modeling and online monitoring. In the offline modeling phase, an FE model is trained using data-intensive techniques to define a subspace characterizing the behavior of multivariate data without sinkhole formations. The squared prediction error (SPE) statistics and the control limits are computed for detection. During the online monitoring phase, unseen EPB data is propagated to generate SPE values and determine sinkhole events based on whether these values surpass the control limit. Sensor validity index violation counts were used to isolate the most influential variables, while the results demonstrated the superiority of the proposed VAE-CNN method, achieving a 100% detection rate and a 0.9% false alarm rate. The influential variables identified include cutter resolutions per minute, jack speed, screw pressure, torque, and cutter seal components. The monitoring system shows great potential for early warnings during EPB operations to mitigate sinkhole formation risks.

Enhancing Lithium-Ion Battery Health Predictions by Hybrid-Grained Graph Modeling.

Predicting the health status of lithium-ion batteries is crucial for ensuring safety. The prediction process typically requires inputting multiple time series, which exhibit temporal dependencies. Existing methods for health status prediction fail to uncover both coarse-grained and fine-grained temporal dependencies between these series. Coarse-grained analysis often overlooks minor fluctuations in the data, while fine-grained analysis can be overly complex and prone to overfitting, negatively impacting the accuracy of battery health predictions. To address these issues, this study developed a Hybrid-grained Evolving Aware Graph (HEAG) model for enhanced prediction of lithium-ion battery health. In this approach, the Fine-grained Dependency Graph (FDG) helps us model the dependencies between different sequences at individual time points, and the Coarse-grained Dependency Graph (CDG) is used for capturing the patterns and magnitudes of changes across time series. The effectiveness of the proposed method was evaluated using two datasets. Experimental results demonstrate that our approach outperforms all baseline methods, and the efficacy of each component within the HEAG model is validated through the ablation study.

Uncertainty graph convolution recurrent neural network for air quality forecasting

Accurate and reliable air quality forecasts are essential for informed decision-making in urban planning and environmental engineering. However, predicting air quality faces challenges due to significant fluctuations in data, including the impact of extreme weather. To improve the precision of air quality predictions, it is important to account for the long- and short-term temporal dependencies within historical data from stations, as well as the local and global spatial structures among different stations in a specific region. In this paper, we propose a novel uncertainty graph convolution recurrent neural network for air quality forecasting (AirNN). Concretely, we design an uncertainty-guided graph convolution module to enhance the robustness of air quality forecasting. It treats the parameters on each graph convolution layer as probability distributions and employs variational inference to derive optimal estimates for these distributions. In addition, to capture the global correlation of stations in the whole region, we develop a spatial information modulator. It captures the spatial correlation between long-range stations in graph data. Extensive experiments on different real-world datasets demonstrate the proposed AirNN performs favorably against state-of-the-art methods. Especially in the 24-hour prediction, AirNN reduced the root mean square error (RMSE) by 0.5 on two real datasets, indicating that our proposed AirNN exhibits good performance in long-term forecasting.

Graph-Based Electroencephalography Analysis in Tinnitus Therapy.

Tinnitus is the perception of sounds like ringing or buzzing in the ears without any external source, varying in intensity and potentially becoming chronic. This study aims to enhance the understanding and treatment of tinnitus by analyzing a dataset related to tinnitus therapy, focusing on electroencephalography (EEG) signals from patients undergoing treatment. The objectives of the study include applying various preprocessing techniques to ensure data quality, such as noise elimination and standardization of sampling rates, and extracting essential features from EEG signals, including power spectral density and statistical measures. The novelty of this research lies in its innovative approach to representing different channels of EEG signals as new graph network representations without losing any information. This transformation allows for the use of Graph Neural Networks (GNNs), specifically Graph Convolutional Networks (GCNs) combined with Long Short-Term Memory (LSTM) networks, to model intricate relationships and temporal dependencies within the EEG data. This method enables a comprehensive analysis of the complex interactions between EEG channels. The study reports an impressive accuracy rate of 99.41%, demonstrating the potential of this novel approach. By integrating graph representation and deep learning, this research introduces a new methodology for analyzing tinnitus therapy data, aiming to contribute to more effective treatment strategies for tinnitus sufferers.

Variational quantum recurrent neural networks for multi-feature forecasting

Recurrent neural networks are a kind of recursion which takes sequence data as input and carries on the evolution of temporal dependency data. Variational quantum algorithms use classical computers as the quantum optimizer to update the circuit parameters. In this work, we propose a variational quantum algorithm of the recurrent neural network, which we dub VQRNN, to find approximate optima in the time series forecasting. Motivated by the variational quantum algorithms, we train classical activation functions to assist quantum computing. Here, unlike the quantum tensor networks (QTN) algorithm that predicts a single output feature with a single time step, our algorithm can forecast multi-output features by adjusting the recurrent hidden state. Finally, we deploy the QTN and VQRNN algorithms on the Origin Quantum platform with the numerical simulator backends using the Meteorological data set. Experimental results show that the atmospheric pressure prediction accuracy of VQRNN is [Formula: see text] in multi-feature forecasting tasks. In addition, we conclude that the variation-based model has an excellent performance in multi-feature output forecasting.

Sparse dynamic graph learning for district heat load forecasting

Accurate heat load forecasting is crucial for the efficient operation and management of district heating systems. This study introduces a novel Sparse Dynamic Graph Neural Network (SDGNN) framework designed to address the complexities of forecasting heat load in district heating networks. The proposed model represents the district heating network as a dynamic graph, with nodes corresponding to consumers or heat sources and edges denoting temporal dependencies. The SDGNN framework comprises three key components: (1) a sparse graph learning module that identifies the most relevant nodes and edges, (2) a spatio-temporal memory enhancement module that captures both short-term and long-term dependencies, and (3) a temporal fusion module that integrates node representations into a comprehensive global forecast. Evaluated on a real-world district heating dataset from Denmark, the SDGNN model demonstrates superior accuracy and efficiency compared to existing methods. The results indicate that the SDGNN framework effectively captures intricate spatio-temporal patterns in historical heat load data, achieving up to 5.7% improvement in RMSE, 7.4% in MAE, and 5.7% in CVRMSE over baseline models. Additionally, incorporating meteorological factors into the model further enhances its predictive performance. These findings suggest that the SDGNN framework is a robust and scalable solution for district heat load forecasting, with potential applications in other domains involving spatio-temporal graph data.

Time series generative adversarial network for muscle force prognostication using statistical outlier detection

AbstractMachine learning approaches, such as artificial neural networks (ANN), effectively perform various tasks and provide new predictive models for complicated physiological systems. Examples of Robotics applications involving direct human engagement, such as controlling prosthetic arms, athletic training, and investigating muscle physiology. It is now time for automated systems to take over modelling and monitoring tasks. However, there is a problem with the massive amount of time series data collected to build accurate forecasting systems. There may be inconsistencies in forecasting muscle forces due to the enormous amount of data. As a result, anomaly detection techniques play a significant role in detecting anomalous data. Detecting anomalies can help reduce redundancy and free up large storage space for storing relevant time‐series data. This paper employs several anomaly detection techniques, including Isolation Forest (iforest), K‐Nearest Neighbour (KNN), Open Support Vector Machine (OSVM), Histogram, and Local Outlier Factor (LOF). These techniques have been used by Long Short‐Term Memory (LSTM), Auto‐Regressive Integrated Moving Average (ARIMA), and Prophet models. The dataset used in this study contained raw measurements of body movements (kinematics) and the forces generated during walking (kinetics) of 57 healthy people (29 Female, 28 Male) without walking abnormalities or recent leg injuries. To increase the data samples, we used TimeGAN that generates synthetic time series data with temporal dependencies, aiding in training robust predictive models for muscle force prediction. The results are then compared with different evaluation metrics for five different samples. It is found that anomaly detection techniques with LSTM, ARIMA, and Prophet models provided better performance in forecasting muscle forces. The iforest method achieved the best Pearson's Correlation Coefficient (r) of 0.95, which is a competitive score with existing systems that perform between 0.7 and 0.9. The methodology provides a foundation for precision medicine, enhancing prognostic capability over relying solely on population averages.

A deep learning model integrating a wind direction-based dynamic graph network for ozone prediction

Ozone pollution is an important environmental issue in many countries. Accurate forecasting of ozone concentration enables relevant authorities to enact timely policies to mitigate adverse impacts. This study develops a novel hybrid deep learning model, named wind direction-based dynamic spatio-temporal graph network (WDDSTG-Net), for hourly ozone concentration prediction. The model uses a dynamic directed graph structure based on hourly changing wind direction data to capture evolving spatial relationships between air quality monitoring stations. It applied the graph attention mechanism to compute dynamic weights between connected stations, thereby aggregating neighborhood information adaptively. For temporal modeling, it utilized a sequence-to-sequence model with attention mechanism to extract long-range temporal dependencies. Additionally, it integrated meteorological predictions to guide the ozone forecasting. The model achieves a mean absolute error of 6.69 μg/m3 and 18.63 μg/m3 for 1-h prediction and 24-h prediction, outperforming several classic models. The model's IAQI accuracy predictions at all stations are above 75 %, with a maximum of 81.74 %. It also exhibits strong capabilities in predicting severe ozone pollution events, with a 24-h true positive rate of 0.77. Compared to traditional static graph models, WDDSTG-Net demonstrates the importance of incorporating short-term wind fluctuations and transport dynamics for data-driven air quality modeling. In principle, it may serve as an effective data-driven approach for the concentration prediction of other airborne pollutants.

Temporally Dynamic, Cohort-Varying Value-Added Models.

We aim to estimate school value-added dynamically in time. Our principal motivation for doing so is to establish school effectiveness persistence while taking into account the temporal dependence that typically exists in school performance from one year to the next. We propose two methods of incorporating temporal dependence in value-added models. In the first we model the random school effects that are commonly present in value-added models with an auto-regressive process. In the second approach, we incorporate dependence in value-added estimators by modeling the performance of one cohort based on the previous cohort's performance. An identification analysis allows us to make explicit the meaning of the corresponding value-added indicators: based on these meanings, we show that each model is useful for monitoring specific aspects of school persistence. Furthermore, we carefully detail how value-added can be estimated over time. We show through simulations that ignoring temporal dependence when it exists results in diminished efficiency in value-added estimation while incorporating it results in improved estimation (even when temporal dependence is weak). Finally, we illustrate the methodology by considering two cohorts from Chile's national standardized test in mathematics.

Assessment of ammonia-diesel fuel blends on compression ignition engine performance and emissions using machine learning techniques

This study investigates the performance and emission characteristics of a multi-cylinder, naturally aspirated, compression ignition diesel engine using ammonia as a secondary fuel under full load conditions across various engine speeds. Ammonia dispersed with neat diesel at the concentration of 5, 10, and 15 liters per minutes via inlet manifold. Experimental tests assessed the impact of diesel-ammonia fuel mixtures at operating speeds of 1000 rpm to 3000 rpm. The performance metrics such as brake thermal efficiency, brake specific fuel consumption, exhaust gas temperature, and emissions characteristics were determined. Moreover, the study employed predictive models such a simple extraction, linear regression, and Long Short-Term Memory (LSTM) networks to analyze engine behaviour. The results demonstrated that increasing ammonia concentration improved brake thermal efficiency and reduced brake specific fuel consumption and exhaust gas temperature across all tested speeds. Specifically, ammonia additions led to significant decreases in carbon monoxide and carbon dioxide emissions due to reduced carbon content in the fuel mix, while nitrogen of oxides emissions presented a more complex pattern. Inclusion of the ammonia increases the NOx due to the inherent nitrogen content. Particulate matter emissions were also reduced, highlighting ammonia’s potential to mitigate particulate formation. Compared to difference prediction models, LSTM showed superior capability in capturing complex non-linear relationships and temporal dependencies in engine performance data, outperforming traditional linear models.

STAFFormer: Spatio-temporal adaptive fusion transformer for efficient 3D human pose estimation

Existing two-stage methods for 3D Human Pose Estimation often use 2D poses as input, which are then lifted to obtain 3D representations. This typically involves frame-by-frame estimation, whether in 2D or 3D, resulting in high computational demands unsuitable for edge devices. Due to the continuity of human movement, the differences between adjacent frames can be minimal, a question arises: is frame-by-frame estimation necessary? Previous works demonstrated the feasibility of using Transformer-based models to estimate poses with sparse frames, focusing on either temporal or spatial dependencies but neglecting holistic spatio-temporal correlations. To address this, we introduce the Spatio-Temporal Adaptive Fusion Transformer (STAFFormer). First, STAFFormer recovers dense temporal frames from sparsely sampled ones obtained from a 2D pose estimator through the Temporal Dense Frame Recovery (TDFR) module. This significantly reduces the computational complexity. Second, STAFFormer employs an adaptive fusion attention mechanism, enhancing accuracy by attentively navigating both spatial and temporal dimensions through the Spatio-Temporal Adaptive Fusion (STAF) module. Furthermore, We introduce a kinematic coherence loss to adapt to subtle joint movements, improving pose estimation fidelity. Finally, we explore the possibility of integrating different pre-training strategies using extensive marker-based datasets. Experimental results on challenging datasets show our network achieves state-of-the-art performance with low computational complexity.

Dual-branch Cross-scale Feature Interaction for Temporal Action Detection

Temporal Action Detection (TAD) is aim to predict action boundary and category simultaneously. Most existing RGB-based methods model temporal dependency using pyramid-style features without interaction among different scales, which usually result in inaccurate prediction for long-term actions. The reason is that features at different scales involve information with different granularity, which is suitable for either prediction of action boundary or category. In this paper, we present a novel Dual-branch Cross-scale Feature Interaction (DCFI) method that directly exchanges different scale information from both temporal and spatial perspective for TAD. To be specific, in one branch, a cross-scale temporal transformer module is devised to enable both semantic and temporal communications among different scale features with a merge-to-split mechanism. While the other branch designs a cross-scale spatial mixer module to mine the most salient spatial difference between consecutive and long-term frames via a scale-mixer. Benefiting from these two modules, DCFI achieves comprehensive temporal as well as spatial interaction across all feature scales, and thus accurately predicts the boundaries of different time-span action instances. Extensive experiments on two challenging benchmarks, i.e.., THUMOS-14 and ActivityNet-1.3, demonstrate that our DCFI achieves new state-of-the-art performance with only RGB.

TODEAR: Promoting explainable TKG reasoning through temporal offset enhanced dynamic embedding and adaptive reinforcement learning

Recently, Reinforcement Learning (RL) is utilized in Temporal Knowledge Graph (TKG) reasoning to generate analyzable reasoning paths, which achieves explainable reasoning on queries. The process of generating high-quality reasoning paths is facing two major challenges. The first is to construct an efficient entity embedding model from the complex temporal dependencies among entities. The second is to design a fine-grained reward mechanism for the reasoner based on the deep semantics of exploration paths. Motivated by the two challenges, a TKG reasoning framework based on Temporal Offset Enhanced Dynamic Embedding and Adaptive Reinforcement Learning (TODEAR) is proposed in this paper. Firstly, a temporal offset enhanced dynamic embedding model with a distance scoring function is designed to fully exploit the complex temporal dependencies among entities. In order to capture the evolutionary patterns of historical facts, it encodes both the relational structures of the TKG and the temporal offsets between events and queries. Then, a fine-grained adaptive reward mechanism is designed to optimize the reasoner. It generates real-time rewards by analyzing the logic and semantics of exploration paths to mitigate the adverse effects of sparse rewards. Extensive experiments on four benchmark datasets show that TODEAR significantly outperforms the state-of-the-art methods.