High Dimensional Representation Research Articles

Decoupling visual and identity features for adversarial palm-vein image attack

Palm-vein has been widely used for biometric recognition due to its resistance to theft and forgery. However, with the emergence of adversarial attacks, most existing palm-vein recognition methods are vulnerable to adversarial image attacks, and to the best of our knowledge, there is still no study specifically focusing on palm-vein image attacks. In this paper, we propose an adversarial palm-vein image attack network that generates highly similar adversarial palm-vein images to the original samples, but with altered palm-identities. Unlike most existing generator-oriented methods that directly learn image features via concatenated convolutional layers, our proposed network first maps palm-vein images into multi-scale high-dimensional shallow representation, and then develops attention-based dual-path feature learning modules to extensively exploit diverse palm-vein-specific features. After that, we design visual-consistency and identity-aware loss functions to specially decouple the visual and identity features to reconstruct the adversarial palm-vein images. By doing this, the visual characteristics of palm-vein images can be largely preserved while the identity information is removed in the adversarial palm-vein images, such that high-aggressive adversarial palm-vein samples can be obtained. Extensive white-box and black-box attack experiments conducted on three widely used databases clearly show the effectiveness of the proposed network.

Multimodal fusion for large-scale traffic prediction with heterogeneous retentive networks

Traffic speed prediction is a critical challenge in transportation research due to the complex spatiotemporal dynamics of urban mobility. This study proposes a novel framework for fusing diverse data modalities to enhance short-term traffic speed forecasting accuracy. We introduce the Heterogeneous Retentive Network (H-RetNet), which integrates multisource urban data into high-dimensional representations encoded with geospatial relationships. By combining the H-RetNet with a Gated Recurrent Unit (GRU), our model captures intricate spatial and temporal correlations. We validate the approach using a real-world Beijing traffic dataset encompassing social media, real estate, and point of interest data. Experiments demonstrate superior performance over existing methods, with the fusion architecture improving robustness. Specifically, we observe a 21.91% reduction in MSE, underscoring the potential of our framework to inform and enhance traffic management strategies.

LearnMSA2: deep protein multiple alignments with large language and hidden Markov models.

For the alignment of large numbers of protein sequences, tools are predominant that decide to align two residues using only simple prior knowledge, e.g. amino acid substitution matrices, and using only part of the available data. The accuracy of state-of-the-art programs declines with decreasing sequence identity and when increasingly large numbers of sequences are aligned. Recently, transformer-based deep-learning models started to harness the vast amount of protein sequence data, resulting in powerful pretrained language models with the main purpose of generating high-dimensional numerical representations, embeddings, for individual sites that agglomerate evolutionary, structural, and biophysical information. We extend the traditional profile hidden Markov model so that it takes as inputs unaligned protein sequences and the corresponding embeddings. We fit the model with gradient descent using our existing differentiable hidden Markov layer. All sequences and their embeddings are jointly aligned to a model of the protein family. We report that our upgraded HMM-based aligner, learnMSA2, combined with the ProtT5-XL protein language model aligns on average almost 6% points more columns correctly than the best amino acid-based competitor and scales well with sequence number. The relative advantage of learnMSA2 over other programs tends to be greater when the sequence identity is lower and when the number of sequences is larger. Our results strengthen the evidence on the rich information contained in protein language models' embeddings and their potential downstream impact on the field of bioinformatics. Availability and implementation: https://github.com/Gaius-Augustus/learnMSA, PyPI and Bioconda, evaluation: https://github.com/felbecker/snakeMSA.

Ionic species representations for materials informatics

High-dimensional representations of the elements have become common within the field of materials informatics to build useful, structure-agnostic models for the chemistry of materials. However, the characteristics of elements change when they adopt a given oxidation state, with distinct structural preferences and physical properties. We explore several methods for developing embedding vectors of elements decorated with oxidation states. Graphs generated from 110 160 crystals are used to train representations of 84 elements that form 336 species. Clustering these learned representations of ionic species in low-dimensional space reproduces expected chemical heuristics, particularly the separation of cations from anions. We show that these representations have enhanced expressive power for property prediction tasks involving inorganic compounds. We expect that ionic representations, necessary for the description of mixed valence and complex magnetic systems, will support more powerful machine learning models for materials.

The application of deep learning technology in integrated circuit design

This study addresses the intricate challenge of circuit layout optimization central to integrated circuit (IC) design, where the primary goals involve attaining an optimal balance among power consumption, performance metrics, and chip area (collectively known as PPA optimization). The complexity of this task, evolving into a multidimensional problem under multiple constraints, necessitates the exploration of advanced methodologies. In response to these challenges, our research introduces deep learning technology as an innovative strategy to revolutionize circuit layout optimization. Specifically, we employ Convolutional Neural Networks (CNNs) in developing an optimized layout strategy, a performance prediction model, and a system for fault detection and real-time monitoring. These methodologies leverage the capacity of deep learning models to learn from high-dimensional data representations and handle multiple constraints effectively. Extensive case studies and rigorous experimental validations demonstrate the efficacy of our proposed deep learning-driven approaches. The results highlight significant enhancements in optimization efficiency, with an average power consumption reduction of 120% and latency decrease by 1.5%. Furthermore, the predictive capabilities are markedly improved, evidenced by a reduction in the average absolute error for power predictions to 3%. Comparative analyses conclusively illustrate the superiority of deep learning methodologies over conventional techniques across several dimensions. Our findings underscore the potential of deep learning in achieving higher accuracy in predictions, demonstrating stronger generalization abilities, facilitating superior design quality, and ultimately enhancing user satisfaction. These advancements not only validate the applicability of deep learning in IC design optimization but also pave the way for future advancements in addressing the multidimensional challenges inherent to circuit layout optimization.

Spin Space Groups: Full Classification and Applications

In this work, we exhaust all the spin space symmetries, which fully characterize collinear, noncollinear, and commensurate spiral as well as incommensurate spiral magnetism, etc., and investigate enriched features of electronic bands that respect these symmetries. We achieve this by systematically classifying the so-called spin space groups (SSGs)—joint symmetry groups of spatial and spin operations that leave the magnetic structure unchanged. Generally speaking, they are accurate (approximate) symmetries in systems where spin-orbit coupling (SOC) is negligible (finite but weaker than the energy scale of interest), but we also show that specific SSGs could remain valid even in the presence of strong SOC. In recent years, SSGs have played increasingly pivotal roles in various fields such as altermagnetism, topological electronic states, and topological magnon, etc. However, due to its complexity, a complete SSG classification has not been completed up to now. By representing the SSGs as O(N) representations, we—for the first time—obtain the complete classifications of 1421, 9542, and 56 512 distinct SSGs for collinear (N=1), coplanar (N=2), and noncoplanar (N=3) magnetism, respectively. SSG not only fully characterizes the symmetry of spin degrees of freedom, but also gives rise to exotic electronic states, which, in general, form projective representations of magnetic space groups (MSGs). Surprisingly, electronic bands in SSGs exhibit features never seen in MSGs, such as (i) nonsymmorphic SSG Brillouin zone, where SSG operations behave as a glide or screw when acting on momentum, (ii) effective π flux, where translation operators anticommute with each other and yield duplicate bands, (iii) higher-dimensional representations unexplained by MSGs, and (iv) unconventional spin texture on a Fermi surface, which is completely determined by SSGs, independent of Hamiltonian details. To apply our theory, we identify the SSG for each of the 1595 published magnetic structures in the MAGNDATA database on the Bilbao Crystallographic Server. Material examples exhibiting the novel features (i)–(iv) are discussed with emphasis. We also investigate new types of SSG-protected topological electronic states that are unprecedented in MSGs. In particular, we propose a 3D Z2 topological insulator state with a fourfold degenerate Dirac point on the surface and a new scenario of anomalous Z2 helical states that appear on magnetic domain walls. Published by the American Physical Society 2024

ORASIS-MAE Harnesses the Potential of Self-Learning from Partially Annotated Clinical Eye Movement Records

Self-supervised learning (SSL) has gained significant attention in the past decade for its capacity to utilize non-annotated datasets to learn meaningful data representations. In the medical domain, the challenge of constructing large annotated datasets presents a significant limitation, rendering SSL an ideal approach to address this constraint. In this study, we introduce a novel pretext task tailored to stimulus-driven eye movement data, along with a denoising task to improve the robustness against simulated eye tracking failures. Our proposed task aims to capture both the characteristics of the pilot (brain) and the motor (eye) by learning to reconstruct the eye movement position signal using up to 12.5% of the unmasked eye movement signal patches, along with the entire REMOBI target signal. Thus, the encoder learns a high-dimensional representation using a multivariate time series of length 8192 points, corresponding to approximately 40 s. We evaluate the learned representation on screening eight distinct groups of pathologies, including dyslexia, reading disorder, and attention deficit disorder, across four datasets of varying complexity and size. Furthermore, we explore various head architecture designs along with different transfer learning methods, demonstrating promising results with improvements of up to approximately 15%, leading to an overall macro F1 score of 61% and 61.5% on the Saccade and the Vergence datasets, respectively. Notably, our method achieves macro F1 scores of 64.7%, 66.1%, and 61.1% for screening dyslexia, reading disorder, and attention deficit disorder, respectively, on clinical data. These findings underscore the potential of self-learning algorithms in pathology screening, particularly in domains involving complex data such as stimulus-driven eye movement analysis.

High-dimensional cortical signals reveal rich bimodal and working memory-like representations among S1 neuron populations

Complexity is important for flexibility of natural behavior and for the remarkably efficient learning of the brain. Here we assessed the signal complexity among neuron populations in somatosensory cortex (S1). To maximize our chances of capturing population-level signal complexity, we used highly repeatable resolvable visual, tactile, and visuo-tactile inputs and neuronal unit activity recorded at high temporal resolution. We found the state space of the spontaneous activity to be extremely high-dimensional in S1 populations. Their processing of tactile inputs was profoundly modulated by visual inputs and even fine nuances of visual input patterns were separated. Moreover, the dynamic activity states of the S1 neuron population signaled the preceding specific input long after the stimulation had terminated, i.e., resident information that could be a substrate for a working memory. Hence, the recorded high-dimensional representations carried rich multimodal and internal working memory-like signals supporting high complexity in cortical circuitry operation.

A General and Efficient Fault Diagnosis Approach Using Hyperdimensional Computing Enhanced by Local Feature Extraction

Abstract Ensuring the safe and reliable operation of industrial equipment is of great importance, and accordingly, machine fault diagnosis has attracted considerable attention. Recently, various deep learning-based fault diagnosis models have been widely developed due to the powerful ability of feature learning and nonlinear modeling of deep learning models, and these methods heavily rely on large amounts of data and high memory and resources. However, it is difficult to obtain enough data in practical applications, thus leading to the unsatisfactory performance of deep learning-based models and high cost. In this paper, we propose a new and efficient fault diagnosis approach based on Hyperdimensional computing (HDC), which can learn high-dimensional, random, and holographic distributed representations of input features. It has the advantages of single training, low training cost, and does not require a large amount of data. Therefore, we apply HDC to the field of few-shot fault diagnosis for the first time, and propose a simple and efficient approach for few-shot fault diagnosis based on hyperdimensional computing with local feature extraction(L-HDC). Instead of using original data, the model firstly extracts the local features of original data by hand-designed features, and then carries out HDC-encoding, training, and test classification. Through the comparison of various models and the analysis of various data, the effectiveness of our proposed model is verified. In the case of few-shot learning, the recognition accuracy is as high as 90\% and the training time of ordinary HDC model is 5-10 times that of L-HDC model, which is superior to other models.

DMRC-Net: Dual mechanism residual connection network for point cloud analysis

The purpose of point cloud analysis is to extract effective high-dimensional feature representations from point coordinates. However, existing methods only rely solely on global or local features. In this paper, we introduce the Dual Mechanism Residual Connection Network (DMRC-Net), a new end-to-end network architecture for more comprehensive representations. First, local features are encoded using graph convolution and fine-grained attention mechanisms, whereas global features are encoded using a multi-layer perceptron. Second, adaptive residual learning is designed to integrate two encoding results effectively. Furthermore, the structural information in local space is strengthened by collecting low-level geometric correlations. With these designs, DMRC-Net is able to capture both global and local details comprehensively. Experimental results show that our method performs well in point cloud classification, part segmentation, and semantic segmentation tasks.

Interpretable wind power forecasting combining seasonal-trend representations learning with temporal fusion transformers architecture

Wind power forecasting plays a critical role in the efficient integration of wind energy into power systems. Accurate short-term WPF helps maintain grid stability, optimize power dispatch, and reduce operational costs. However, current deep learning-based WPF architectures rely on mapping spatial–temporal features into a single high-dimensional representation, compromising performance and interpretability. In this work, the interpretable seasonal-trend representation algorithm (ISTR) is proposed to learn a couple of disentangled latent representations that describe seasonal-trend of wind power time series. The ISTR is developed in a self-supervised manner with no need for prior knowledge, providing sufficient interpretable information while avoiding the feature entanglement problem. The ISTR is then integrated into the temporal fusion transformer (TFT) model through seasonal time-point embedding to perform accurate multi-horizon WPF with interpretable meanings. In the case study, ISTR-TFT outperforms the current state-of-the-art methods by achieving an 8.9% improvement in mean absolute percentage error (MAPE) compared to DeepAR and a 15.7% improvement in pinball score. The proposed ISTR-TFT also shows enhanced robustness to noise and missing data scenarios. The interpretability results generated by ISTR-TFT reveal the importance of various input features and the learned temporal patterns, offering valuable insights for decision-making in power system operations.

Rethinking general time series analysis from a frequency domain perspective

Recently, Transformers and MLPs based models have dominated and made significant progress in time series analysis. However, these methods struggle to capture the complete periodic features to model the global dependencies of time series. To this end, we provide a new perspective to explore the potential of time series in the frequency domain. Specifically, we propose the DFT-based Amplitude-Phase Decoupling Network (APDNet), which learns the significance changes and periodicity characteristics of time series by decoupling the real and imaginary parts in the complex frequency domain. First, we propose the Fourier Temporal/Variable Interaction Module to model time dependency and dependency multiple variables, respectively. Secondly, we innovatively propose the Dimensional Expanded Feature Embedding Module (DEFEM), which expands time series into a higher-dimensional representation space to preserve more independent features. Finally, through extensive experiments on multiple real-world datasets, our APDNet achieves state-of-the-art performance in various tasks such as long-term and short-term forecasting, anomaly detection, imputation, and classification. Code is available at: https://github.com/CodeJester196/APDNet.

Multi-Output Prediction Model for Basic Oxygen Furnace Steelmaking Based on the Fusion of Deep Convolution and Attention Mechanisms

The objective of basic oxygen furnace (BOF) steelmaking is to achieve molten steel with final carbon content, temperature, and phosphorus content meeting the requirements. Accurate prediction of the above properties is crucial for end-point control in BOF steelmaking. Traditional prediction models typically use multi-variable input and single-variable output approaches, neglecting the coupling relationships between different property indicators, making it difficult to predict multiple outputs simultaneously. Consequently, a multi-output prediction model based on the fusion of deep convolution and attention mechanism networks (FDCAN) is proposed. The model inputs include scalar data, such as the properties of raw materials and target molten steel, and time series data, such as lance height, oxygen supply intensity, and bottom air supply intensity during the blowing process. The FDCAN model utilizes a fully connected module to extract nonlinear features from scalar data and a deep convolution module to process time series data, capturing high-dimensional feature representations. The attention mechanism then assigns greater weight to significant features. Finally, multiple multi-layer perceptron modules predict the outputs—final carbon content, temperature, and phosphorus content. This structure allows FDCAN to learn complex relationships within the input data and between input and output variables. The effectiveness of the FDCAN model is validated using actual BOF steelmaking data, achieving hit rates of 95.14% for final carbon content within ±0.015 wt%, 84.72% for final temperature within ±15 °C, and 88.89% for final phosphorus content within ±0.005 wt%.

Classification of compressed full-waveform airborne lidar data

ABSTRACT Airborne laser scanners produce 3D data that can be used for a range of applications, such as urban planning, facility monitoring, flood mapping, and forest management. Additional information on the surveyed area can be obtained from the backscattered waveforms recorded by modern light detection and ranging (lidar) sensors. However, the high-dimensional representation of full-waveform data has hindered progress in its use due to difficulties in processing and storage. This paper develops a quantized convolutional autoencoder network to compress lidar waveform data into a condensed feature representation, resulting in a compression rate of up to 20:1. This, together with height information, is fed into a U-net convolutional neural network that achieves an accuracy of 93.7% on six classes.

Research on Advanced Prediction of Major Equipment Fatigue Load Based on Deep Learning

Abstract The fatigue life of complex service structures is predicted in real-time according to dynamic random loads. Real-time monitoring of stress/strain in structure key parts is important for practical engineering. Based on the deep learning theory, this paper proposes a long and short-term memory convolutional neural network (CNN- LSTM) prediction model with attention mechanism by using the actual workload data set of key components in major equipment. Firstly, convolutional neural networks (CNN) is used to extract the high-dimensional abstract representation from the preprocessed data, and attention mechanism is added to give each channel different attention scores. Then, long and short-term memory (LSTM) is used to process the time series data and output the final prediction results. In this study, data points in the future are predicted, and the final predicted results are root mean square error (RMSE) of 33.2626 and mean absolute percentage error (MAPE) of 15.3768, which can well fit the future load change trend. At the same time, the multi-condition identification module is added to the code, which can better adapt to the changing load of mechanical equipment in the actual service process. Finally, this paper also designed an ablation experiment, which proved the effectiveness of the overall structure of the prediction model designed in this study and the necessity of the three parts of CNN, ECA-Net and LSTM. It provides theoretical support and practical engineering application for real-time monitoring and predicting the fatigue life of important equipment.

Sub-surface geospatial intelligence in carbon capture, utilization and storage: A machine learning approach for offshore storage site selection

This study introduces an innovative data-driven and machine-learning framework designed to accurately predict site scores in the site screening study for specific offshore CO2 storage sites. The framework seamlessly integrates diverse sub-surface geospatial data sources with human aided expert-weighted criteria, thereby providing a high-resolution screening tool. Tailored to accommodate varying data accessibility and the significance of criteria, this approach considers both technical and non-technical factors. Its purpose is to facilitate the identification of priority locations for projects associated with Carbon Capture, Utilization, and Storage (CCUS). Through aggregating and analyzing geospatial datasets, the study employs machine learning algorithms and an expert-weighted model to identify suitable geologic CCUS regions. This process adheres to stringent safety, risk control, and environmental guidelines, addressing situations where human analysis may fail to recognize patterns and provide detailed insights in suitable site screening techniques. The primary emphasis of this research is to bridge the gap between scientific inquiry and practical application, facilitating informed decision-making in the implementation of CCUS projects. Rigorous assessments encompassing geological, oceanographic, and eco-sensitivity metrics contribute valuable insights for policymakers and industry leaders. To ensure the accuracy, efficiency, and scalability of the established offshore CO2 storage facilities, the proposed machine learning approach undergoes benchmarking. This comprehensive evaluation includes the utilization of machine learning algorithms such as Extreme Gradient Boosting (XGBoost), Random Forest (RF), Multilayer Extreme Learning Machine (MLELM), and Deep Neural Network (DNN) for predicting more suitable site scores. Among these algorithms, the DNN algorithm emerges as the most effective in site score prediction. The strengths of the DNN algorithm encompass nonlinear modeling, feature learning, scale invariance, handling high-dimensional data, end-to-end learning, transfer learning, representation learning, and parallel processing. The evaluation results of the DNN algorithm demonstrate high accuracy in the testing subset, with values of AAPD (Average Absolute Percentage Difference) = 1.486 %, WAAPD (Weighted Average Absolute Percentage Difference) = 0.0149 %, VAF (Variance Accounted For) = 0.9937, RMSE (Root Mean Square Error) = 0.9279, RSR (Root Sum of Squares Residuals) = 0.0068, and R2 (Coefficient of Determination) = 0.9937.

Quantifying cell-state densities in single-cell phenotypic landscapes using Mellon.

Cell-state density characterizes the distribution of cells along phenotypic landscapes and is crucial for unraveling the mechanisms that drive diverse biological processes. Here, we present Mellon, an algorithm for estimation of cell-state densities from high-dimensional representations of single-cell data. We demonstrate Mellon's efficacy by dissecting the density landscape of differentiating systems, revealing a consistent pattern of high-density regions corresponding to major cell types intertwined with low-density, rare transitory states. We present evidence implicating enhancer priming and the activation of master regulators in emergence of these transitory states. Mellon offers the flexibility to perform temporal interpolation of time-series data, providing a detailed view of cell-state dynamics during developmental processes. Mellon facilitates density estimation across various single-cell data modalities, scaling linearly with the number of cells. Our work underscores the importance of cell-state density in understanding the differentiation processes, and the potential of Mellon to provide insights into mechanisms guiding biological trajectories.

Gaussian Process Regression for State-to-State Integral Cross Sections: The Case of the O + O2 Collision Dissociation Reactions.

Research on hypersonic vehicles has become increasingly important worldwide in recent years. However, accurately simulating the dynamics of the nonequilibrium high-temperature reactions that are in the hypersonic flow around the vehicles presents a significant challenge as a large number of states and transitions are accessible even for the smallest atom-diatom reaction systems. It is quite difficult, sometimes even impossible, to exhaustively investigate all relevant combinations or determine high-dimensional analytical representations for the state-to-state reaction probabilities. In this study, we used Gaussian process regression (GPR) to fit a model based on only 807 QCT data for training. The confidence interval of the GPR prediction and the Kullback-Leibler (KL) divergence were used to help minimize the sampling amount of data for fitting the converged GPR model. The model aims to predict the state-to-state integral cross section (ICS) of the O + O2 → 3O dissociation reaction under random initial conditions (Et, v, j). In total, it took almost a month to obtain this converged GPR model, but it took only a few seconds to predict the ICS value for any initial condition. For 330 initial conditions not included in the training set, the mean-square error (MSE) between the QCT-calculated ICSs and the GPR-predicted ones is only 0.08 Å2 and the R2 is 0.9986, indicating that the GPR model can replace the direct expensive QCT calculation with high accuracy. Finally, we calculated the equilibrium dissociation rate coefficients based on the StS ICS values predicted by the GPR model, and the results were in good agreement with available experimental and theoretical results. Thus, this study provides an effective and accurate approach to the extensive direct state-to-state reaction dynamic calculations.

A Spectral Clustering Algorithm for Non-Linear Graph Embedding in Information Networks

With the development of network technology, information networks have become one of the most important means for people to understand society. As the scale of information networks expands, the construction of network graphs and high-dimensional feature representation will become major factors affecting the performance of spectral clustering algorithms. To address this issue, in this paper, we propose a spectral clustering algorithm based on similarity graphs and non-linear deep embedding, named SEG_SC. This algorithm introduces a new spectral clustering model that explores the underlying structure of graphs through sparse similarity graphs and deep graph representation learning, thereby enhancing graph clustering performance. Experimental analysis with multiple types of real datasets shows that the performance of this model surpasses several advanced benchmark algorithms and performs well in clustering on medium- to large-scale information networks.

Operationalizing Clinical Speech Analytics: Moving From Features to Measures for Real-World Clinical Impact.

This research note advocates for a methodological shift in clinical speech analytics, emphasizing the transition from high-dimensional speech feature representations to clinically validated speech measures designed to operationalize clinically relevant constructs of interest. The aim is to enhance model generalizability and clinical applicability in real-world settings. We outline the challenges of using conventional supervised machine learning models in clinical speech analytics, particularly their limited generalizability and interpretability. We propose a new framework focusing on speech measures that are closely tied to specific speech constructs and have undergone rigorous validation. This research note discusses a case study involving the development of a measure for articulatory precision in amyotrophic lateral sclerosis (ALS), detailing the process from ideation through Food and Drug Administration (FDA) breakthrough status designation. The case study demonstrates how the operationalization of the articulatory precision construct into a quantifiable measure yields robust, clinically meaningful results. The measure's validation followed the V3 framework (verification, analytical validation, and clinical validation), showing high correlation with clinical status and speech intelligibility. The practical application of these measures is exemplified in a clinical trial and designation by the FDA as a breakthrough status device, underscoring their real-world impact. Transitioning from speech features to speech measures offers a more targeted approach for developing speech analytics tools in clinical settings. This shift ensures that models are not only technically sound but also clinically relevant and interpretable, thereby bridging the gap between laboratory research and practical health care applications. We encourage further exploration and adoption of this approach for developing interpretable speech representations tailored to specific clinical needs.