High Dimensional Representation Research Articles

Predicting drug and target interaction with dilated reparameterize convolution

Predicting drug-target interaction (DTI) stands as a pivotal and formidable challenge in pharmaceutical research. Many existing deep learning methods only learn the high-dimensional representation of ligands and targets on a small scale. However, it is difficult for the model to obtain the potential law of combining pockets or multiple binding sites on a large scale. To address this lacuna, we designed a large-kernel convolutional block for extracting large-scale sequence information and proposed a novel DTI prediction framework, named Rep-ConvDTI. The reparameterization method is introduced to help large-kernel convolutions capture small-scale information. We have also developed a gated attention mechanism to more efficiently characterize the interaction of drugs and targets. Extensive experiments demonstrate that Rep-ConvDTI achieves the most competitive performance against state-of-the-art baselines on the three benchmark datasets. Furthermore, we validated the potential of Rep-ConvDTI as a drug screening tool through model interpretative studies and drug screening experiments with cystathionine-β-synthase.

Traffic flow forecasting based on augmented multi-component recurrent graph attention network

ABSTRACT Accurate real-time traffic flow forecasting has been a challenge due to the complex spatial–temporal dependencies and uncertainties associated with the dynamic changes in traffic flow. To overcome this problem, a traffic flow forecasting model based on an Augmented Multi-Component Recurrent Graph Attention Network (AMR-GAT) is proposed in this paper to model the spatial–temporal correlations and periodic offset of traffic flows. This paper introduces an augmented multi-component module to address periodic temporal offset in traffic flow forecasting. It proposes an encoder-decoder architecture combining 1D convolution and LSTM via a Temporal Correlation Learner (TCL) to capture temporal characteristics, while a Graph Attention Network (GAT) handles spatial features. TCL and GAT are integrated to manage spatial-temporal correlations, and the decoder uses TCL and convolutional neural networks to generate high-dimensional representations based on spatial-temporal sequences. Experiments on two datasets demonstrate superior prediction performance of the proposed AMR-GAT model.

PSP-GEN: Stochastic inversion of the Process-Structure-Property chain in materials design through deep, generative probabilistic modeling

Inverse material design is a cornerstone challenge in materials science, with significant applications across many industries. Traditional approaches that invert the structure–property (SP) linkage to identify microstructures with targeted properties often overlook the feasibility of production processes, leading to microstructures that may not be manufacturable. Achieving both desired properties and a realizable manufacturing procedure necessitates inverting the entire Process-Structure-Property (PSP) chain. However, this task is fraught with challenges, including stochasticity along the whole modeling chain, the high dimensionality of microstructures and process parameters, and the inherent ill-posedness of the inverse problem. This paper proposes a novel framework, named PSP-GEN, for the goal-oriented material design that effectively addresses these challenges by modeling the entire PSP chain with a deep generative model. It employs two sets of continuous, microstructure- and property-aware, latent variables, the first of which provides a lower-dimensional representation that captures the stochastic aspects of microstructure generation, while the second is a direct link to processing parameters. This structured, low-dimensional embedding not only simplifies the handling of high-dimensional microstructure data but also facilitates the application of gradient-based optimization techniques. The effectiveness and efficiency of this method are demonstrated in the inverse design of two-phase materials, where the objective is to design microstructures with target effective permeability. We compare state-of-the-art alternatives in challenging settings involving limited training data, target property regions for which no training data is available, and design tasks where the process parameters and microstructures have high-dimensional representations.

Computing With Residue Numbers in High-Dimensional Representation.

We introduce residue hyperdimensional computing, a computing framework that unifies residue number systems with an algebra defined over random, high-dimensional vectors. We show how residue numbers can be represented as high-dimensional vectors in a manner that allows algebraic operations to be performed with component-wise, parallelizable operations on the vector elements. The resulting framework, when combined with an efficient method for factorizing high-dimensional vectors, can represent and operate on numerical values over a large dynamic range using resources that scale only logarithmically with the range, a vast improvement over previous methods. It also exhibits impressive robustness to noise. We demonstrate the potential for this framework to solve computationally difficult problems in visual perception and combinatorial optimization, showing improvement over baseline methods. More broadly, the framework provides a possible account for the computational operations of grid cells in the brain, and it suggests new machine learning architectures for representing and manipulating numerical data.

Numerical investigation of the bridging and current flow of a positive DC streamer using a 1.5D model

Abstract Single streamers, at nanosecond-timescales, can be simulated using detailed computational models with a high-dimensional representation. These models are computationally impractical for parametric explorations and simulation of longer times, that can follow many-streamer pulsations and the influence of one streamer burst on the next. This work develops a 1.5D model of a positive DC streamer for simulations beyond the electrode-gap bridging phase, and uses it to parametrically explore the impact of different terms and operational parameters. The geometry of interest is that of a tip-to-plane electrode configuration under DC voltage, and the simulation is followed for the duration of one current pulse (order 500 ns). The numerical model uses an axisymmetric boundary element method to solve for the electric field, as well as a ‘stack’ of 3 different transient solvers to improve efficiency and allow solving over longer timescales. The model is able to resolve the development of the cathode sheath during the streamer bridging phase using a kinetic flux boundary condition. It also gives qualitative agreement to current waveforms using an equivalent experimental setup. The different phases of the current pulse (streamer propagation, bridging, and current-flow phase) are discussed in detail.

Approximating the semantic space: word embedding techniques in psychiatric speech analysis

Large language models provide high-dimensional representations (embeddings) of word meaning, which allow quantifying changes in the geometry of the semantic space in mental disorders. A pattern of a more condensed (‘shrinking’) semantic space marked by an increase in mean semantic similarity between words has been recently documented in psychosis across several languages. We aimed to explore this pattern further in picture descriptions provided by a transdiagnostic German sample of patients with schizophrenia spectrum disorders (SSD) (n = 42), major depression (MDD, n = 43), and healthy controls (n = 44). Compared to controls, both clinical groups showed more restricted dynamic navigational patterns as captured by the time series of semantic distances crossed, while also showing differential patterns in the total distances and trajectories navigated. These findings demonstrate alterations centred on the dynamics of the flow of meaning across the semantic space in SSD and MDD, preserving previous indications towards a shrinking semantic space in both cases.

PRITrans: A Transformer-Based Approach for the Prediction of the Effects of Missense Mutation on Protein-RNA Interactions.

Protein-RNA interactions are essential to many cellular functions, and missense mutations in RNA-binding proteins can disrupt these interactions, often leading to disease. To address this, we developed PRITrans, a specialized computational method aimed at predicting the effects of missense mutations on protein-RNA interactions, which is vital for understanding disease mechanisms and advancing molecular biology research. PRITrans is a novel deep learning model designed to predict the effects of missense mutations on protein-RNA interactions, which employs a Transformer architecture enhanced with multiscale convolution modules for comprehensive feature extraction. Its primary innovation lies in integrating protein language model embeddings with a deep feature fusion strategy, effectively handling high-dimensional feature representations. By utilizing multi-layer self-attention mechanisms, PRITrans captures nuanced, high-level sequence information, while multiscale convolutions extract features across various depths, thereby enhancing predictive accuracy. Consequently, this architecture enables significant improvements in ΔΔG prediction compared to traditional approaches. We validated PRITrans using three different cross-validation strategies on two newly reconstructed mutation datasets, S315 and S630 (containing 315 forward and 315 reverse mutations). The results consistently demonstrated PRITrans's strong performance on both datasets. PRITrans demonstrated strong predictive capability, achieving a Pearson correlation coefficient of 0.741 and a root mean square error (RMSE) of 1.168 kcal/mol on the S630 dataset. Moreover, its robust performance extended to independent test sets, achieving a Pearson correlation of 0.699 and an RMSE of 1.592 kcal/mol. These results underscore PRITrans's potential as a powerful tool for protein-RNA interaction studies. Moreover, when tested against existing prediction methods on an independent dataset, PRITrans showed improved predictive accuracy and robustness.

TMAA-net: tensor-domain multi-planal anti-aliasing network for sparse-view CT image reconstruction

Objective.Among various deep-network-based sparse-view CT image reconstruction studies, the sinogram upscaling network has been predominantly employed to synthesize additional view information. However, the performance of the sinogram-based network is limited in terms of removing aliasing streak artifacts and recovering low-contrast small structures. In this study, we used a view-by-view back-projection (VVBP) tensor-domain network to overcome such limitations of the sinogram-based approaches.Approach.The proposed method offers advantages of addressing the aliasing artifacts directly in the 3D tensor domain over the 2D sinogram. In the tensor-domain network, the multi-planal anti-aliasing modules were used to remove artifacts within the coronal and sagittal tensor planes. In addition, the data-fidelity-based refinement module was also implemented to successively process output images of the tensor network to recover image sharpness and textures.Main result.The proposed method showed outperformance in terms of removing aliasing artifacts and recovering low-contrast details compared to other state-of-the-art sinogram-based networks. The performance was validated for both numerical and clinical projection data in a circular fan-beam CT configuration.Significance.We observed that view-by-view aliasing artifacts in sparse-view CT exhibit distinct patterns within the tensor planes, making them effectively removable in high-dimensional representations. Additionally, we demonstrated that the co-domain characteristics of tensor space processing offer higher generalization performance for aliasing artifact removal compared to conventional sinogram-domain processing.

Primate thalamic nuclei select abstract rules and shape prefrontal dynamics.

Flexible behavior depends on abstract rules to generalize beyond specific instances, and outcome monitoring to adjust actions. Cortical circuits are posited to read out rules from high-dimensional representations of task-relevant variables in prefrontal cortex (PFC). We instead hypothesized that converging inputs from PFC, directly or via basal ganglia (BG), enable thalamus to select rules. We measured activity across PFC and connected thalamic nuclei of monkeys applying rules. Abstract rule information first appeared in ventroanterior thalamus (VA) - the main thalamic hub between BG and PFC. Mediodorsal thalamus (MD) also represented rule information before PFC, persisting to help maintain activation of relevant PFC cell ensembles. MD, a major recipient of midbrain dopamine input, was first to represent information about behavioral outcomes. A PFC-BG-thalamus model reproduced key findings, and thalamic-lesion modeling disrupted PFC rule representations. This suggests that thalamus selects high-level cognitive information from PFC and monitors behavioral outcomes of these selections.

Incremental Classification for High-Dimensional EEG Manifold Representation Using Bidirectional Dimensionality Reduction and Prototype Learning.

In brain-computer interface (BCI) systems, symmetric positive definite (SPD) manifold within Riemannian space has been frequently utilized to extract spatial features from electroencephalogram (EEG) signals. However, the intrinsic high dimensionality of SPD matrices introduces too much computational burden to hinder the real-time applications of such BCI, especially in handling dynamic tasks, like incremental learning. Directly reducing the dimensionality of SPD matrices with conventional dimensionality reduction (DR) methods will alter the fundamental properties of SPD matrices. Moreover, current DR methods for incremental learning always necessitate retaining old data to update their representations under new mapping. To this end, a bidirectional two-dimensional principal component analysis for SPD manifold (B2DPCA-SPD) is proposed to reduce the dimensionality of SPD matrices, in such way that the reduced matrices remain on SPD manifold. Afterwards, the B2DPCA-SPD is extended to adapt to incremental learning tasks without saving old data. The incremental B2DPCA-SPD can be seamlessly integrated with the matrix-formed growing neural gas network (MF-GNG) to achieve an incremental EEG classification, where the new low-dimensional representations of the prototypes in old classifiers can be easily recalculated with the updated projection matrix. Extensive experiments are conducted on two public datasets to perform the EEG classification. The results demonstrate that our method significantly reduces computation time by 38.53% and 35.96%, and outperforms conventional methods in classification accuracy by 4.21% to 19.59%.

Dynamic graph spatial-temporal dependence information extraction for remaining useful life prediction of rolling bearings

As a powerful tool for learning high-dimensional data representation, graph neural networks (GNN) have been applied to predict the remaining useful life (RUL) of rolling bearings. Existing GNN-based RUL prediction methods predominantly rely on constant pre-constructed graphs. However, the degradation of bearings is a dynamic process, and the dependence information between features may change at different moments of degradation. This article introduces a method for RUL prediction based on dynamic graph spatial-temporal dependence information extraction. The raw signal is segmented into multiple periods, and multiple features of each period data are extracted. Then, the correlation coefficient analysis is conducted, and the feature connection graph of each period is constructed based on different analytical results, thereby dynamically mapping the degradation process. The graph data is fed into graph convolutional networks (GCN) to extract spatial dependence between the graph node features in different periods. To make up for the shortcomings of GCN in temporal dependence extraction, the TimesNet module is introduced. TimesNet considers the two-dimensional changes of time series data and can extract the temporal dependence of graph data within and between different time cycles. Experimental results based on the PHM2012 dataset show that the average RUL prediction error of the proposed method is 17.4%, outperforming other comparative methods.

Hyperdimensional Brain-Inspired Learning for Phoneme Recognition With Large-Scale Inferior Colliculus Neural Activities.

Develop a novel and highly efficient framework that decodes Inferior Colliculus (IC) neural activities for phoneme recognition. We propose using Hyperdimensional Computing (HDC) to support an efficient phoneme recognition algorithm, in contrast to widely applied Deep Neural Networks (DNN). The high-dimensional representation and operations in HDC are rooted in human brain functionalities and naturally parallelizable, showing the potential for efficient neural activity analysis. Our proposed method includes a spatial and temporal-aware HDC encoder that effectively captures global and local patterns. As part of our framework, we deploy the lightweight HDC-based algorithm on a highly customizable and flexible hardware platform, i.e., Field Programmable Gate Arrays (FPGA), for optimal algorithm speedup. To evaluate our method, we record IC neural activities on gerbils while playing the sound of different phonemes. We compare our proposed method with multiple baseline machine learning algorithms in recognition quality and learning efficiency, across different hardware platforms. The results show that our method generally achieves better classification quality than the best-performing baseline. Compared to the Deep Residual Neural Network (i.e., ResNet), our method shows a speedup up to 74×, 67×, 210× on CPU, GPU, and FPGA respectively. We achieve up to 15% (10%) higher accuracy in consonant (vowel) classification than ResNet. By leveraging brain-inspired HDC for IC neural activity encoding and phoneme classification, we achieve orders of magnitude runtime speedup while improving accuracy in various challenging task settings. Decoding IC neural activities is an important step to enhance understanding about human auditory system. However, these responses from the central auditory system are noisy and contain high variance, demanding large-scale datasets and iterative model fine-tuning. The proposed HDC-based framework is more scalable and viable for future real-world deployment thanks to its fast training and overall better quality.

PredPVP: A Stacking Model for Predicting Phage Virion Proteins Based on Feature Selection Methods

Background: Phage therapy has a broad application prospect as a novel therapeutic method, and Phage Virion Proteins (PVP) can recognize the host and bind to surface receptors, which is of great significance for the development of antimicrobial drugs for the treatment of infectious diseases caused by bacteria. In recent years, several PVP predictors based on machine learning have been developed, which usually use a single feature to train the learner. In contrast, higher dimensional feature representations tend to contain more potential sequence information. Methods: In this work, we construct a stacking model PredPVP for PVP prediction by combining multiple features and using feature selection methods. Specifically, the sequence is first encoded using seven features. For this high-dimensional feature representation, three feature selection methods wereutilized to remove redundant features, then integrated with eight machine learning algorithms. Finally, probability features and class features (PCFs) generated by 24 base models were put into logistic regression (LR) to train the model. Results: The results of the independent test set indicate that PredPVP has higher performance compared to other existing predictors, with an AUC of 93.4%. Conclusion: We expect PredPVP to be used as a tool for large-scale PVP recognition, providing a new way for the development of novel antimicrobials and accelerating its application in actual treatment. The datasets and source codes used in this study are available at https://github.com/caoqian23/PredPVP.

Language proficiency is associated with neural representational dimensionality of semantic concepts

Previous studies suggest that semantic concepts are characterized by high-dimensional neural representations and that language proficiency affects semantic processing. However, it is not clear whether language proficiency modulates the dimensional representations of semantic concepts at the neural level. To address this question, the present study adopted principal component analysis (PCA) and representational similarity analysis (RSA) to examine the differences in representational dimensionalities (RDs) and in semantic representations between words in highly proficient (Chinese) and less proficient (English) language. PCA results revealed that language proficiency increased the dimensions of lexical representations in the left inferior frontal gyrus, temporal pole, inferior temporal gyrus, supramarginal gyrus, angular gyrus, and fusiform gyrus. RSA results further showed that these regions represented semantic information and that higher semantic representations were observed in highly proficient language relative to less proficient language. These results suggest that language proficiency is associated with the neural representational dimensionality of semantic concepts.

A phenotypic drug discovery approach by latent interaction in deep learning

Contemporary drug discovery paradigms rely heavily on binding assays about the bio-physicochemical processes. However, this dominant approach suffers from overlooked higher-order interactions arising from the intricacies of molecular mechanisms, such as those involving cis -regulatory elements. It introduces potential impairments and restrains the potential development of computational methods. To address this limitation, I developed a deep learning model that leverages an end-to-end approach, relying exclusively on therapeutic information about drugs. By transforming textual representations of drug and virus genetic information into high-dimensional latent representations, this method evades the challenges arising from insufficient information about binding specificities. Its strengths lie in its ability to implicitly consider complexities such as epistasis and chemical–genetic interactions, and to handle the pervasive challenge of data scarcity. Through various modeling skills and data augmentation techniques, the proposed model demonstrates outstanding performance in out-of-sample validations, even in scenarios with unknown complex interactions. Furthermore, the study highlights the importance of chemical diversity for model training. While the method showcases the feasibility of deep learning in data-scarce scenarios, it reveals a promising alternative for drug discovery in situations where knowledge of underlying mechanisms is limited.

The prediction of bolted joint loosening state under transverse load based on a data-driven model

The impact of transverse load results in the loosening of bolted joints. The loosening condition is determined based on the transverse load and relevant parameters of the bolted joint. To efficiently forecast the loosening state of bolts under various transverse loads and relevant parameters, this study proposes a predictive model for the loosening state of bolted joints. Initially, the relationship between the loosening criteria and relevant parameters is established according to the bolt’s force conditions. Through elementary effect analysis, the high-dimensional representation model is improved, breaking down the loosening criteria function with high-dimensional parameter inputs into a series of low-dimensional functions. Subsequently, the prediction of the loosening state is achieved by training artificial neural networks to approximate multiple low-dimensional functions. The improved predictive model is utilized to determine the critical load for bolted joint loosening. Validation indicates the accuracy of the proposed model. Notably, the influence of the bearing friction coefficient and preload on the critical transverse load is more pronounced compared to the thread friction coefficient.

High-dimensional Ageome Representations of Biological Aging across Functional Modules.

The aging process involves numerous molecular changes that lead to functional decline and increased disease and mortality risk. While epigenetic aging clocks have shown accuracy in predicting biological age, they typically provide single estimates for the samples and lack mechanistic insights. In this study, we challenge the paradigm that aging can be sufficiently described with a single biological age estimate. We describe Ageome, a computational framework for measuring the epigenetic age of thousands of molecular pathways simultaneously in mice and humans. Ageome is based on the premise that an organism's overall biological age can be approximated by the collective ages of its functional modules, which may age at different rates and have different biological ages. We show that, unlike conventional clocks, Ageome provides a high-dimensional representation of biological aging across cellular functions, enabling comprehensive assessment of aging dynamics within an individual, in a population, and across species. Application of Ageome to longevity intervention models revealed distinct patterns of pathway-specific age deceleration. Notably, cell reprogramming, while rejuvenating cells, also accelerated aging of some functional modules. When applied to human cohorts, Ageome demonstrated heterogeneity in predictive power for mortality risk, and some modules showed better performance in predicting the onset of age-related diseases, especially cancer, compared to existing clocks. Together, the Ageome framework offers a comprehensive and interpretable approach for assessing aging, providing insights into mechanisms and targets for intervention.

Decoupling visual and identity features for adversarial palm-vein image attack

Palm-vein has been widely used for biometric recognition due to its resistance to theft and forgery. However, with the emergence of adversarial attacks, most existing palm-vein recognition methods are vulnerable to adversarial image attacks, and to the best of our knowledge, there is still no study specifically focusing on palm-vein image attacks. In this paper, we propose an adversarial palm-vein image attack network that generates highly similar adversarial palm-vein images to the original samples, but with altered palm-identities. Unlike most existing generator-oriented methods that directly learn image features via concatenated convolutional layers, our proposed network first maps palm-vein images into multi-scale high-dimensional shallow representation, and then develops attention-based dual-path feature learning modules to extensively exploit diverse palm-vein-specific features. After that, we design visual-consistency and identity-aware loss functions to specially decouple the visual and identity features to reconstruct the adversarial palm-vein images. By doing this, the visual characteristics of palm-vein images can be largely preserved while the identity information is removed in the adversarial palm-vein images, such that high-aggressive adversarial palm-vein samples can be obtained. Extensive white-box and black-box attack experiments conducted on three widely used databases clearly show the effectiveness of the proposed network.

Multimodal fusion for large-scale traffic prediction with heterogeneous retentive networks

Traffic speed prediction is a critical challenge in transportation research due to the complex spatiotemporal dynamics of urban mobility. This study proposes a novel framework for fusing diverse data modalities to enhance short-term traffic speed forecasting accuracy. We introduce the Heterogeneous Retentive Network (H-RetNet), which integrates multisource urban data into high-dimensional representations encoded with geospatial relationships. By combining the H-RetNet with a Gated Recurrent Unit (GRU), our model captures intricate spatial and temporal correlations. We validate the approach using a real-world Beijing traffic dataset encompassing social media, real estate, and point of interest data. Experiments demonstrate superior performance over existing methods, with the fusion architecture improving robustness. Specifically, we observe a 21.91% reduction in MSE, underscoring the potential of our framework to inform and enhance traffic management strategies.

Nested barycentric coordinate system as an explicit feature map for polyhedra approximation and learning tasks

We introduce a new embedding technique based on a nested barycentric coordinate system. We show that our embedding can be used to transform the problems of polyhedron approximation, piecewise linear classification and convex regression into one of finding a linear classifier or regressor in a higher dimensional (but nevertheless quite sparse) representation. Our embedding maps a piecewise linear function into an everywhere-linear function, and allows us to invoke well-known algorithms for the latter problem to solve the former. We explain the applications of our embedding to the problems of approximating separating polyhedra—in fact, it can approximate any convex body and unions of convex bodies—as well as to classification by separating polyhedra, and to piecewise linear regression.