Sequence Dependence Research Articles

Adaptive Deep Reinforcement Learning Energy Management for Hybrid Electric Vehicles Considering Driving Condition Recognition

Adaptive energy management strategy (EMS) can adjust the underlying control scheme of hybrid electric vehicles (HEVs) according to different external conditions, thus maintaining high performance consistently. For this reason, this study designs a double-layer EMS for a power-split HEV, which combines deep reinforcement learning (DRL) with driving condition recognition (DCR) to achieve efficient operation of powertrain under different driving conditions. At the upper layer, the long-term dependencies in velocity sequences are captured through the sequential networks to enhance the accuracy of DCR. At the lower layer, multi-objective reward functions are formulated, and the optimal weight parameters are identified for different driving conditions. Furthermore, the deep deterministic policy gradient (DDPG) algorithm is employed to dynamically optimize power flow. The simulation experiments are conducted to verify the feasibility and the energy-saving effects of the proposed strategy. The results indicate that integrating variations in driving conditions into the energy allocation of HEVs can further improve the fuel economy and maintain battery health. Specifically, the method enhances the fuel economy by an average of 10.76%, compared to a number of traditional benchmark strategies.

Application of state of health estimation and remaining useful life prediction for lithium-ion batteries based on AT-CNN-BiLSTM

Ensuring the long-term safe usage of lithium-ion batteries hinges on accurately estimating the State of Health \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(\ extrm{SOH})$$\\end{document} and predicting the Remaining Useful Life (RUL). This study proposes a novel prediction method based on a AT-CNN-BiLSTM architecture. Initially, key parameters such as voltage, current, temperature, and SOH are extracted and averaged for each cycle to ensure the uniformity and reliability of the input data. The CNN is utilized to extract deep features from the data, followed by BiLSTM to analyze the temporal dependencies in the data sequences. Since multidimensional parameter data are used to predict the SOH trend of lithium-ion batteries, an attention mechanism is employed to enhance the weight of highly relevant vectors, improving the model’s analytical capabilities. Experimental results demonstrate that the CNN-BiLSTM-Attention model achieves an absolute error of 0 in RUL prediction, an \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$R^{2}$$\\end{document} value greater than 0.9910 , and a MAPE value less than 0.9003 . Comparative analysis with hybrid neural network algorithms such as LSTM, BiLSTM, and CNN-LSTM confirms the proposed model’s high accuracy and stability in SOH estimation and RUL prediction.

Multi-view Traffic Flow Prediction Model Based on Spatiotemporal Convolution

Predicting traffic flow has always been a significant task in intelligent transportation systems. Due to the substantial temporal and spatial dependencies of traffic flow sequences, accurately predicting traffic flow poses a considerable challenge. Many existing works primarily rely on recurrent neural networks, graph neural networks, and Transformer models to establish traffic flow prediction models. To better extract features and enhance efficiency, a traffic flow prediction model based on multi-view spatiotemporal convolution (MVSC) is proposed. This model learns the representation of sequence data at the input encoding layer and incorporates location and time information. In the spatiotemporal feature representation learning layer, considering the diverse periodic patterns in sequences, several representation learning modules are designed, conducting local spatiotemporal feature exploration through one-dimensional convolution and then accomplishing global spatiotemporal feature mining based on causal convolution. To further enhance the model's utilization of spatiotemporal features, a channel attention mechanism is introduced at the prediction layer. The forecasting method employed in the study is direct multistep, and subsequent experiments conducted on two real datasets demonstrate that the MVSC model exhibits a certain degree of superiority in MAE, RMSE, and MAPE for both short-term and long-term predictions compared to existing models. And through the latest experiments and investigations, it has been found that MVSC has improved MAPE performance by about 1.2% compared to recent models such as RTGCN and STRGCN, achieving the intended outcomes.

TCM: An efficient lightweight MLP-based network with affine transformation for long-term time series forecasting

Time series forecasting (TSF) involves extracting underlying patterns from past information to predict future sequences over a specific period. Extending the prediction length of time series and improving the prediction accuracy have always been challenging tasks. Autoregressive prediction methods based on Markov chains tend to accumulate errors over time. Although Transformer-based methods with various self-attention mechanisms have shown some improvements, they require higher memory and computational resources. In this work, we present an effective MLP-based TSF framework named TCM, which models the sequence and channel dependencies separately using Token MLP and Channel MLP. Additionally, we employ the Affine Transformation to replace layer normalization or batch normalization, leading to substantial enhancements in both accuracy and inference speed. Compared to current state-of-the-art long-term time series forecasting models, TCM achieves 6.0% relative improvement on seven real-world datasets, including electricity, weather, and illness domains. The TCM model, characterized by its efficiency and lightweight architecture, also makes it suitable for scenarios with high real-time requirements.

GRMD: A Two-Stage Design Space Exploration Strategy for Customized RNN Accelerators

Recurrent neural networks (RNNs) have produced significant results in many fields, such as natural language processing and speech recognition. Owing to their computational complexity and sequence dependencies, RNNs need to be deployed on customized hardware accelerators to satisfy performance and energy-efficiency constraints. However, designing hardware accelerators for RNNs is challenged by the vast design space and the reliance on ineffective optimization. An efficient automated design space exploration (DSE) strategy that can balance conflicting objectives is wanted. To address the low efficiency and insufficient universality of the resource allocation process employed for hardware accelerators, we propose an automated two-stage design space exploration (DSE) strategy for customized RNN accelerators. The strategy combines a genetic algorithm (GA) and a reinforcement learning (RL) algorithm, and it utilizes symmetrical exploration and exploitation to find the optimal solutions. In the first stage, the area of the hardware accelerator is taken as the optimization objective, and the GA is used for partial exploration purposes to narrow the design space while maintaining diversity. Then, the latency and power of the hardware accelerator are taken as the optimization objectives, and the RL algorithm is used in the second stage to find the corresponding Pareto solutions. To verify the effectiveness of the developed strategy, it is compared with other algorithms. We use three different network models as benchmarks: a vanilla RNN, LSTM, and a GRU. The results demonstrate that the strategy proposed in this paper can provide better solutions and can achieve latency, power, and area reductions of 9.35%, 5.34%, and 11.95%, respectively. The HV of GRMD is reduced by averages of 6.33%, 6.32%, and 0.67%, and the runtime is reduced by averages of 18.11%, 14.94%, and 10.28%, respectively. Additionally, given different weights, it can make reasonable trade-offs between multiple objectives.

Multi-Scale Adaptive Skeleton Transformer for action recognition

Transformer has demonstrated remarkable performance in various computer vision tasks. However, its potential is not fully explored in skeleton-based action recognition. On one hand, existing methods primarily utilize fixed function or pre-learned matrix to encode position information, while overlooking the sample-specific position information. On the other hand, these approaches focus on single-scale spatial relationships, while neglecting the discriminative fine-grained and coarse-grained spatial features. To address these issues, we propose a Multi-Scale Adaptive Skeleton Transformer (MSAST), including Adaptive Skeleton Position Encoding Module (ASPEM), Multi-Scale Embedding Module (MSEM), and Adaptive Relative Location Module (ARLM). ASPEM decouples spatial–temporal information in the position encoding procedure, which acquires inherent dependencies of skeleton sequences. ASPEM is also designed to be dependent on input tokens, which can learn sample-specific position information. The MSEM employs multi-scale pooling to generate multi-scale tokens that contain multi-grained features. Then, the spatial transformer captures multi-scale relations to address the subtle differences between various actions. Another contribution of this paper is that ARLM is presented to mine suitable location information for better recognition performance. Extensive experiments conducted on three benchmark datasets demonstrate that the proposed model achieves Top-1 accuracy of 94.9%/97.5% on NTU-60 C-Sub/C-View, 88.7%/91.6% on NTU-120 X-Sub/X-Set and 97.4% on NW-UCLA, respectively.

Reconstructing Richtmyer–Meshkov instabilities from noisy radiographs using low dimensional features and attention-based neural networks

We develop an ML-based approach for density reconstruction based on transformer neural networks. This approach is demonstrated in the setting of ICF-like double shell hydrodynamic simulations wherein the parameters related to material properties and initial conditions are varied. The new method can robustly recover the complex topologies given by the Richtmyer-Meshkoff instability (RMI) from a sequence of hydrodynamic features derived from radiographic images corrupted with blur, scatter, and noise. A noise model is developed to characterize errors in extracting features from synthetic radiographs of the simulated density field. The key component of the network is a transformer encoder that acts on a sequence of features extracted from noisy radiographs. This encoder includes numerous self-attention layers that act to learn temporal dependencies in the input sequences and increase the expressiveness of the model. This approach is shown to exhibit an excellent ability to accurately recover the RMI growth rates, despite the gas-metal interface being greatly obscured by radiographic noise. Our approach can be applied in a broad array of fields involving shock physics and material science.

Dynamic inferences of crash risks in freeway merging zones: a spatio-temporal deep learning model

Freeway merging zones are critical for freeway operations and management due to potential crashes arising from complex vehicle merging behaviours. This paper investigates the application of a spatio-temporal deep learning model to infer crash risks in the zones. We first introduce a crash risk index based on Time-To-Collision and vehicle merging patterns. An innovation is that the developed spatio-temporal transformer model can analyze the evolving risk index. This model effectively captures dynamic risk features through a multi-head attention mechanism within its spatio-temporal learning components. Numerical experiments on nine inference tasks with varying spatial resolutions show improved performance of the model with lower resolution. Moreover, the ST-Transformer model is benchmarked against three advanced deep learning models, which consistently demonstrates its superiority in capturing spatio-temporal dependence in risk sequences. This investigation significantly contributes to a richer understanding of proactive traffic safety, providing valuable insights for advanced freeway management and driver assistance systems.

Abstract B053: NucAE: Autoencoder-based enhancement of nucleosome occupancy signals from low-coverage cfDNA sequencing

Abstract Introduction: Cell-free DNA (cfDNA) is emerging as a valuable cancer biomarker, enabling both the detection and epigenetic characterization of malignancies through sequencing. cfDNA sequencing can reveal cancer-specific nucleosome occupancy patterns, which provide insights into tumor type and differentiation status, which define prognosis and therapy recommendations. Low-coverage whole-genome cfDNA sequencing is a cost-efficient approach to assay multiple cfDNA samples, however, the low coverage limits its sensitivity. We developed NucAE, a denoising autoencoder model, to enhance nucleosome occupancy signals from low-coverage cfDNA-sequencing. Methods: We designed NucAE to estimate nucleosome occupancy genome- wide from low-coverage sequencing data. To train the model, we undersampled high-coverage cfDNA-sequencing data and produced high- and low-coverage sample pairs at varying coverages (1x-90x). Inputs to NucAE include cfDNA fragment data and the corresponding genomic nucleotide sequence, with the model leveraging a symmetrical architecture comprising two convolutional layers in both the encoder and decoder, connected by ReLU activations. Mean squared error relative to the high-coverage signal was employed as the loss function. To evaluate the model independently, we performed peak detection from the nucleosome occupancy signals. We conducted a grid search to optimize hyperparameters. Results: We found that NucAE enhances the signal to noise ratio by detecting periodicities in nucleosome occupancy data. Denoising was improved by supplementing the cfDNA fragment signals with the genomic nucleotide sequences as input, which demonstrates that the model learned the nucleotide- sequence dependence of nucleosome positioning. Independent validation through peak detection on previously unseen samples demonstrated a significant improvement (p<0.001) in accuracy compared to raw signals across all tested coverages (1x-45x). Remarkably, at extremely low coverages (1x-2x), denoised signals allowed for more accurate peak detection than raw signals from 10-20x deeper sequencing. Conclusions: By using advanced machine-learning, we were able to enhance low-coverage cfDNA sequencing data to assay nucleosome positions at a nucleotide resolution. NucAE achieves an order of magnitude improvement over the low- coverage signals and greatly improves the utility of cfDNA-sequencing data to identify cancer- specific nucleosome occupancy, while preserving its low cost. Our model is the first to use autoencoders for genome-wide signal enhancement with potential use cases in many other areas of genomics. Citation Format: Zsolt Balázs, Jean Radig, Noah Wolford, Manuel Schürch, Michael Krauthammer. NucAE: Autoencoder-based enhancement of nucleosome occupancy signals from low-coverage cfDNA sequencing [abstract]. In: Proceedings of the AACR Special Conference: Liquid Biopsy: From Discovery to Clinical Implementation; 2024 Nov 13-16; San Diego, CA. Philadelphia (PA): AACR; Clin Cancer Res 2024;30(21_Suppl):Abstract nr B053.

CTFNet: Long-Sequence Time-Series Forecasting Based on Convolution and Time-Frequency Analysis.

Although current time-series forecasting methods have significantly improved the state-of-the-art (SOTA) results for long-sequence time-series forecasting (LSTF), they still have difficulty in capturing and extracting the features and dependencies of long-term sequences and suffer from information utilization bottlenecks and high-computational complexity. To address these issues, a lightweight single-hidden layer feedforward neural network (SLFN) combining convolution mapping and time-frequency decomposition called CTFNet is proposed with three distinctive characteristics. First, time-domain (TD) feature mining-in this article, a method for extracting the long-term correlation of horizontal TD features based on matrix factorization is proposed, which can effectively capture the interdependence among different sample points of a long time series. Second, multitask frequency-domain (FD) feature mining-this can effectively extract different frequency feature information of time-series data from the FD and minimize the loss of data features. Integrating multiscale dilated convolutions, simultaneously focusing on both global and local context feature dependencies at the sequence level, and mining the long-term dependencies of the multiscale frequency information and the spatial dependencies among the different scale frequency information, break the bottleneck of data utilization, and ensure the integrity of feature extraction. Third, highly efficient-the CTFNet model has a short training time and fast inference speed. Our empirical studies with nine benchmark datasets show that compared with state-of-the-art methods, CTFNet can reduce prediction error by 64.7% and 53.7% for multivariate and univariate time series, respectively.

Neoliberalism in question: The Philippines' nurse education and labour export as liberal neo‐statist development agenda

AbstractMany scholars have used neoliberalism as an analytical framework to examine the Philippines' labour export policy. While neoliberalism entails a retreat of the state in favour of market reforms, evidence shows that state intervention of the market becomes larger and stronger over time. This paper utilises liberal neo‐statism as an alternative framework to understand the Philippines' nurse labour export by explaining that the state's role is larger than and goes beyond labour brokerage. Following the historical institutionalism approach, we show the significant timing, sequence, and path dependence that affect the emergence of institutions that govern the Philippines' nurse labour export. Our paper reveals how specific policies and regulations in labour export are tucked within the disguise of market reforms, but which are manifest within a larger state's control. These policies serve as the state's apparatus for remittance generation and protection of migrant labour rights and welfare.

Ramachandran-like Conformational Space for DNA.

DNA's ability to exist in a wide variety of structural forms, subforms, and secondary motifs is fundamental to numerous biological processes and has driven the development of biotechnological applications. Major determinants of DNA flexibility are the multiple torsional degrees of freedom around the phosphodiester backbone. This high complexity can be rationalized by using two pseudotorsional angles linking atoms P and C4', from which Ramachandran-like plots can be built. In this contribution, we explore the distribution of η (eta: C4'i-1-Pi-C4'i-Pi+1) and θ (theta: Pi-C4'i-Pi+1-C4'i+1) angles in known experimental structures retrieved from the Protein Data Bank (PDB), subdividing the conformational space into different datasets. After the removal of the canonical/helical conformations typical of the B-form, we find the existence of a conformational map with clearly permitted and forbidden regions. Some of these regions are populated with specific DNA forms, like Z- or A-DNA, or by specific secondary motifs, like G-quadruplexes and junctions. We evaluated the sequence dependency and energy relationship among the high-density regions identified in the η-θ space. Furthermore, we analyzed the effect produced by proteins and cations when bound to DNA, finding that specific proteins produce some nonhelical conformations, while other regions appear to be stabilized by divalent cations.

RS-Net: Hyperspectral Image Land Cover Classification Based on Spectral Imager Combined with Random Forest Algorithm

Recursive neural networks and transformers have recently become dominant in hyperspectral (HS) image classification due to their ability to capture long-range dependencies in spectral sequences. Despite the success of these sequential architectures, mainstream deep learning methods primarily handle two-dimensional structured data. However, challenges such as the curse of dimensionality, spectral variability, and confounding factors in hyperspectral remote sensing images limit their effectiveness, especially in remote sensing applications. To address this issue, this paper proposes a novel land cover classification algorithm that integrates random forests with a spectral transformer network structure (RS-Net). Firstly, this paper presents a combination of the Gramian Angular Field (GASF) and Gramian Angular Difference Field (GADF) algorithms, which effectively maps the multidimensional time series constructed for each pixel onto two-dimensional image features, enabling precise extraction and recognition in the backend network algorithms and improving the classification accuracy of land cover types. Secondly, to capture the relationships between features at different scales, this paper proposes a SpectralFormer network architecture using the Context and Structure Encoding (CASE) module to effectively learn dependencies between channels. This architecture enhances important features and suppresses unimportant ones, thereby addressing the semantic gap and improving the recognition capability of land cover features. Finally, the final prediction results are determined by a voting mechanism from the Random Forest algorithm, which synthesizes predictions from multiple decision trees to enhance classification stability and accuracy. To better compare the performance of RS-Net, this paper conducted extensive experiments on three benchmark HS datasets obtained from satellite and airborne imagers, comparing various classic neural network models. Surprisingly, the RS-Net algorithm achieves high performance and efficiency, offering a new and effective tool for land cover classification.

Laplace approximation of J factors for rigid base and rigid basepair models of DNA cyclization

We apply the Laplace approximation to a mathematical formulation of DNA cyclization J factors, leading to a formula that involves energies of local minima of the DNA energy, factors coming from the Hessian of the energy near each minimum, and geometric factors arising from the orientational portion of J. The approximation is derived in a quite general setting that encompasses both rigid base and rigid basepair models common in the literature. The approximation is applied to several families of 200–400 bp DNA, some relatively straight (fragments of λ-phage) and others quite bent (constructs that include up to 10 A tracts). The accuracy of the approximation is assessed by comparing with (more time-consuming) Monte Carlo computations: Laplace is within 20% of Monte Carlo for most 200 bp molecules and undershoots Monte Carlo by about 30% for 300 bp and 50% for 400 bp. We explore length and sequence dependence, both for our overall approximation of J and for its energy and entropic components, and make comparisons to a different approximation of J proposed in the literature.

MSCL-Attention: A Multi-Scale Convolutional Long Short-Term Memory (LSTM) Attention Network for Predicting CO2 Emissions from Vehicles

The transportation industry is one of the major sources of energy consumption and CO2 emissions, and these emissions have been increasing year by year. Vehicle exhaust emissions have had serious impacts on air quality and global climate change, with CO2 emissions being one of the primary causes of global warming. In order to accurately predict the CO2 emission level of automobiles, an MSCL-Attention model based on a multi-scale convolutional neural network, long short-term memory network and multi-head self-attention mechanism is proposed in this study. By combining multi-scale feature extraction, temporal sequence dependency processing, and the self-attention mechanism, the model enhances the prediction accuracy and robustness. In our experiments, the MSCL-Attention model is benchmarked against the latest state-of-the-art models in the field. The results indicate that the MSCL-Attention model demonstrates superior performance in the task of CO2 emission prediction, surpassing the leading models currently available. This study provides a new method for predicting vehicle exhaust emissions, with significant application prospects, and is expected to contribute to reducing global vehicle emissions, improving air quality, and addressing climate change.

Iterative Time Series Imputation by Maintaining Dependency Consistency

Data imputation is crucial in the analysis of incomplete time series, such as forecasting and classification, which involves learning dependencies among the observed values to infer missing ones. As there are no ground truths for missing values, the challenge of time series imputation lies in preventing the model from overfitting to spurious correlations. In this paper, we believe that ensuring dependency consistency between observed and imputed values in a sequence is paramount for data imputation. Based on this idea, we propose a model called IR 2 -Net 1 , which combines an incomplete representation mechanism ( IRM ) with an iterative reconstruction framework ( IRF ) to establish a closed-loop learning-validation imputation paradigm. Firstly, IRM facilitates the representation of dependencies in incomplete sequences while preserving their distributions and semantics, effectively preventing the model from capturing spurious correlations. Secondly, IRF enables the model to reconstruct identical complete sequences separately based on imputed and observed values, ensuring that the dependencies of imputed values remain consistent with those of the observed ones. We conduct experiments on four datasets and compare IR 2 -Net with seven state-of-the-art imputation models. The experiment results show that IR 2 -Net outperforms all the baselines by 4.1%-23.4% in terms of accuracy. Moreover, IRF and IRM are two general modules that can be easily integrated into two existing models, significantly enhancing their performance by 18.3%-42.0%.

Efficient time series adaptive representation learning via Dynamic Routing Sparse Attention

Time series prediction plays a crucial role in various fields but also faces significant challenges. Converting original 1D time series data into 2D data through dimension transformation allows capturing more hidden features but incurs high memory consumption and low time efficiency. We have designed a sparse attention mechanism with dynamic routing perception called Dynamic Routing Sparse Attention (DRSA) to address these issues. Specifically, DRSA can effectively handle variations of complex time series data. Meanwhile, under memory constraints, the Dynamic Routing Filter (DRF) module further refines it by filtering the blocked 2D time series data to identify the most relevant feature vectors in the local context. We conducted predictive experiments on six real-world time series datasets with fine granularity and long sequence dependencies. Compared to eight state-of-the-art (SOTA) models, DRSA demonstrated relative improvements ranging from 4.18% to 81.02%. Furthermore, its time efficiency is 2 to 5 times higher than the baseline. Our code and dataset will be available at https://github.com/wwy8/DRSA_main.

MFGCN: Multimodal fusion graph convolutional network for speech emotion recognition

Speech emotion recognition (SER) is challenging owing to the complexity of emotional representation. Hence, this article focuses on multimodal speech emotion recognition that analyzes the speaker’s sentiment state via audio signals and textual content. Existing multimodal approaches utilize sequential networks to capture the temporal dependency in various feature sequences, ignoring the underlying relations in acoustic and textual modalities. Moreover, current feature-level and decision-level fusion methods have unresolved limitations. Therefore, this paper develops a novel multimodal fusion graph convolutional network that comprehensively executes information interactions within and between the two modalities. Specifically, we construct the intra-modal relations to excavate exclusive intrinsic characteristics in each modality. For the inter-modal fusion, a multi-perspective fusion mechanism is devised to integrate the complementary information between the two modalities. Substantial experiments on the IEMOCAP and RAVDESS datasets and experimental results demonstrate that our approach achieves superior performance.

High Nucleotide Skew Palindromic DNA Sequences Function as Potential Replication Origins due to their Unzipping Propensity.

Locations of DNA replication initiation in prokaryotes, called "origins of replication", are well-characterized. However, a mechanistic understanding of the sequence dependence of the local unzipping of double-stranded DNA, the first step towards replication initiation, is lacking. Here, utilizing a Markov chain model that was created to address the directional nature of DNA unzipping and replication, we model the sequence dependence of local melting of double-stranded linear DNA segments. We show that generalized palindromic sequences with high nucleotide skews have a low kinetic barrier for local melting near melting temperatures. This allows for such sequences to function as potential replication origins. We support our claim with evidence for high-skew palindromic sequences within the replication origins of mitochondrial DNA, bacteria, archaea and plasmids.

A new composite heuristic to minimize the total tardiness for the single machine scheduling problem with variable and flexible maintenance

Inspired by a real-world maintenance/job scheduling issue coming from the semiconductor industry, the present paper proposes a new heuristic algorithm structure for the single machine scheduling T-problem with flexible and variable maintenance, job release dates and sequence dependence setup times. Considering the typical short-term production planning needs, a single maintenance problem has to be scheduled within a certain time interval, along with a set of jobs so as to minimize the total tardiness. A twofold contribution emerges from the present paper. First, four mixed-integer linear programming models are developed for the problem at hand and compared in terms of time to convergence and computational complexity. Second, a novel heuristic algorithm, which has been configured into three distinct variants, has been compared with 17 alternative heuristics from the relevant literature based on a comprehensive experimental campaign. The numerical results allow the selection of the most suitable MILP model and confirm the effectiveness of the proposed heuristic approach.