Low-dimensional Vector Research Articles

Deep learning-based geological parameterization for history matching CO2 plume migration in complex aquifers

History matching is crucial for reliable numerical simulation of geological carbon storage (GCS) in deep subsurface aquifers. This study focuses on inferring highly complex aquifer permeability fields with multi- and intra-facies heterogeneity to improve the characterization of CO2 plume migration. We propose a deep learning (DL)-based parameterization strategy combined with the ensemble smoother with multiple data assimilation (ESMDA) algorithm to formulate an integrated inverse framework. The DL model is employed to parameterize non-Gaussian permeability fields using low-dimensional latent variables in a Gaussian distribution, thereby mitigating the non-Gaussianity issue faced by the ensemble-based ESMDA inverse method and simultaneously alleviating the computational burden of high-dimensional inversion. The efficacy of the integrated DL-ESMDA inverse framework is demonstrated using a 3-D GCS model, where it estimates the non-Gaussian permeability field characterized by multi- and intra-facies heterogeneity. Results show that the DL model is able to represent the highly complex and high-dimensional permeability fields using low-dimensional latent vectors. The DL-ESMDA framework sequentially updates these low-dimensional latent vectors instead of the original high-dimensional permeability field to obtain posterior estimations of the permeability field. The resulting CO2 plume migration closely matches historical measurements, suggesting a significantly improved model reliability after history matching. Additionally, a substantial reduction in uncertainty for future plume migration predictions beyond the history matching period is observed. The proposed framework provides an effective approach for reliable characterization of CO2 plume migration in highly heterogeneous aquifers, enhancing GCS project operation and risk analysis.

Accurate Prediction of Essential Proteins Using Ensemble Machine Learning

Abstract Essential proteins are crucial for biological processes and can be identified through both experimental and computational methods. While experimental approaches are highly accurate, they often demand extensive time and resources. To address these challenges, we present a computational ensemble learning framework designed to identify essential proteins more efficiently. Our method begins by using node2vec to transform proteins in the Protein-Protein Interaction (PPI) network into continuous, low-dimensional vectors. We also extract a range of features from protein sequences, including graph-theory-based, information-based, compositional, and physiochemical attributes. Additionally, we leverage deep learning techniques to analyze high-dimensional Position-Specific Scoring Matrices (PSSMs) and capture evolutionary information. We then combine these features for classification using various machine learning algorithms. To enhance performance, we integrate the outputs of these algorithms through ensemble methods such as voting, weighted averaging, and stacking. This approach effectively addresses data imbalances and improves both robustness and accuracy. Our ensemble learning framework achieved an AUC of 0.948 and an accuracy of 0.9252, outperforming other computational methods. These results demonstrate the effectiveness of our approach in accurately identifying essential proteins and highlight its superior feature extraction capabilities.

Network community detection via neural embeddings

Recent advances in machine learning research have produced powerful neural graph embedding methods, which learn useful, low-dimensional vector representations of network data. These neural methods for graph embedding excel in graph machine learning tasks and are now widely adopted. However, how and why these methods work—particularly how network structure gets encoded in the embedding—remain largely unexplained. Here, we show that node2vec—shallow, linear neural network—encodes communities into separable clusters better than random partitioning down to the information-theoretic detectability limit for the stochastic block models. We show that this is due to the equivalence between the embedding learned by node2vec and the spectral embedding via the eigenvectors of the symmetric normalized Laplacian matrix. Numerical simulations demonstrate that node2vec is capable of learning communities on sparse graphs generated by the stochastic blockmodel, as well as on sparse degree-heterogeneous networks. Our results highlight the features of graph neural networks that enable them to separate communities in the embedding space.

Exploiting common patterns in diverse cancer types via multi-task learning

Cancer prognosis requires precision to identify high-risk patients and improve survival outcomes. Conventional methods struggle with the complexity of genetic biomarkers and diverse medical data. Our study uses deep learning to distil high-dimensional medical data into low-dimensional feature vectors exploring shared patterns across cancer types. We developed a multi-task bimodal neural network integrating RNA Sequencing and clinical data from three The Cancer Genome Atlas project datasets: Breast Invasive Carcinoma, Lung Adenocarcinoma, and Colon Adenocarcinoma. Our approach significantly improved prognosis prediction, especially for Colon Adenocarcinoma, with up to 26% increase in concordance index and 41% in the area under the precision-recall curve. External validation with Small Cell Lung Cancer achieved comparable metrics, indicating that supplementing small datasets with data from other cancers can improve performance. This work represents initial strides in using multi-task learning for prognosis prediction across cancer types, potentially revealing shared mechanisms among cancers and contributing to future applications in precision medicine.

Knowledge Graph Embedding for Hierarchical Entities Based on Auto-Embedding Size

Knowledge graph embedding represents entities and relations as low-dimensional continuous vectors. Recently, researchers have attempted to leverage the potential semantic connections between entities with hierarchical relationships in the knowledge graph. However, existing knowledge embedding methods that model entity hierarchical structures face the following challenges: (1) Setting the same embedding dimension for entities at different hierarchical levels can lead to overfitting issues and the waste of storage and computational resources; (2) Manually searching the embedding dimension in discrete space makes it easy to miss the optimal parameters and increases training costs. Therefore, we propose a knowledge embedding method, HEAES, that automatically learns the embedding dimensions for entities based on their hierarchical structure. Specifically, we first propose a new modeling method to capture the hierarchical relationships of entities, and we then train the model on a triple classification task. Then, we adaptively learn the pruning threshold to trim the embedding vectors, automatically learning different embedding dimensions for entities at different hierarchical levels. Experiments on the YAGO26K and DB111K datasets verify that introducing entity hierarchical information helps boost the model performance.

Trustworthiness-Driven Graph Convolutional Networks for Signed Network Embedding

The problem of representing nodes in a signed network as low-dimensional vectors, known as signed network embedding (SNE), has garnered considerable attention in recent years. While several SNE methods based on graph convolutional networks (GCNs) have been proposed for this problem, we point out that they significantly rely on the assumption that the decades-old balance theory always holds in the real-world. To address this limitation, we propose a novel GCN-based SNE approach, named as TrustSGCN, which corrects for incorrect embedding propagation in GCN by utilizing the trustworthiness on edge signs for high-order relationships inferred by the balance theory. The proposed approach consists of three modules: (M1) generation of each node’s extended ego-network; (M2) measurement of trustworthiness on edge signs; and (M3) trustworthiness-aware propagation of embeddings. Specifically, TrustSGCN leverages topological information to measure trustworthiness on edge sign for high-order relationships inferred by balance theory. It then considers structural properties inherent to an input network, such as the ratio of triads, to correct for incorrect embedding propagation. Furthermore, TrustSGCN learns the node embeddings by leveraging two well-known social theories, i.e., balance and status, to jointly preserve the edge sign and direction between nodes connected by existing edges in the embedding space. The experiments on six real-world signed network datasets demonstrate that TrustSGCN consistently outperforms six state-of-the-art GCN-based SNE methods. The code is available at https://github.com/kmj0792/TrustSGCN .

Cluster-guided denoising graph auto-encoder for enhanced traffic data imputation and fault detection

In an era of increasing digitalization, the integration of diverse sensors has become commonplace. Within the transportation sector, state Departments of Transportation have extensively deployed traffic sensors to support various engineering endeavors. However, these sensors are prone to faults such as data loss, random noise, bias, and drift due to sensor aging, defects, or environmental influences. Therefore, detecting these faults is imperative to maintain data integrity. This paper presents a novel approach for detecting faults in traffic data based on a cluster-guided data reconstruction process. First, a traffic sensor clustering module is designed using a dual-encoding attention graph auto-encoder (DA-GAE) to identify clusters of traffic sensors. This module leverages a joint embedding of node and edge features in a low-dimensional vector space. Subsequently, utilizing the identified clusters, a cluster-guided denoising graph auto-encoder (CG-DGAE) is devised and trained for data reconstruction. The CG-DGAE employs a diffusion graph convolutional network (DGCN) and is trained with a cluster-wise sampling strategy. Extensive experiments were conducted using traffic data obtained from a real-world large sensor network, demonstrating superior performance of the CG-DGAE model in data reconstruction compared to various baseline methods. For fault detection, a score function is devised to discern potential faults by contrasting the sensor data sequence and the reconstructed data sequence. The fault detection results show that our proposed CG-DGAE model achieved an impressive overall accuracy of 99.09%, a precision of 99.13%, a recall of 99.53%, and a F1 score of 99.53%. The superior capability of CG-DGAE in data reconstruction and fault detection is attributable to the cluster-wise spatial–temporal context leveraged by the model.

Combining Dielectric and Hyperspectral Data for Apple Core Browning Detection

Apple core browning not only affects the nutritional quality of apples, but also poses a health risk to consumers. Therefore, there is an urgent need to develop a fast and reliable non-destructive detection method for apple core browning. To deal with the challenges of the long incubation period, strong infectivity, and difficulty in the prevention and control of apple core browning, a novel non-destructive detection method for apple core browning has been developed through combining hyperspectral imaging and dielectric techniques. To reduce the computational complexity of high-dimensional multi-view data, canonical correlation analysis is employed for feature dimensionality reduction. Then, the two low-dimensional vectors extracted from two different sensors are concatenated into one united feature vector; therefore, the information contained in the hyperspectral and dielectric data is fused to improve the detection accuracy of the non-destructive method. At last, five traditional classifiers, such as k-Nearest Neighbors, a support vector machine with radial basis function kernel and polynomial kernel, Decision Tree, and neural network, are trained on the fused feature vectors to discriminate apple core browning. The experimental results on our own constructed dataset have shown that the sensitivity, specificity, and precision of SVM with RBF kernel based on concatenated 70-dimensional feature vectors extracted via canonical correlation analysis reached 99.98%, 99.70%, and 99.70%, respectively, which achieved better results than other models. This study can provide theoretical assurance and technical support for further development of higher accuracy and lower-cost non-destructive detection devices for apple core browning.

Reinforcement Learning with Decoupled State Representation for Robot Manipulations

Deep reinforcement learning has significantly advanced robot manipulations by providing an alternative solution for designing control strategies using raw images as direct inputs. While images offer additional environmental information, the end-to-end policy training manner (from image to action) requires simultaneous representation and task learning by the agent. This often necessitates a substantial number of interaction samples to achieve satisfactory policy performance. Previous works has attempted to address this challenge by learning a visual representation model that encodes the entire image into a low-dimensional vector before the policy training. However, since this vector contains both robot and object information, it inevitably introduces coupling within the state, which can mislead the policy training process. In this study, a novel method called Reinforcement Learning with Decoupled State Representation is proposed to effectively decouple robot and object information within the state representation. Experimental results demonstrate that the proposed method exhibits faster learning speed and achieves superior performance compared to previous methods across various robot manipulation tasks. Moreover, with only 3096 offline images, the proposed method successfully applies to real-world robot pushing tasks, which demonstrates its high practicability.

A low-complexity adaptive speech dereverberation approach based on multichannel linear prediction

The adaptive multichannel linear prediction (AMCLP) technique is one of the most effective methods for speech dereverberation in real-time application systems. However, the computational complexity of the associated AMCLP algorithms is very high. In this paper, we propose a low-complexity speech dereverberation approach based on AMCLP. The coefficient matrix of the multichannel dereverberation filter is decomposed into a linear combination of a set of low-dimensional sub-filter matrices and a set of low-dimensional sub-filter vectors through Kronecker product. According to these sub-filter models, we define two sets of signal models and cost functions, from which the adaptive speech dereverberation algorithm is established. This solution not only leads to effective dereverberation performance, but also reduces the computational cost, which is very conducive to realistic speech systems.

Network embedding-based directed community detection with unknown community number

Community detection of network analysis plays an important role in numerous application areas, in which estimating the number of communities is a fundamental issue. However, many existing methods focus on undirected networks ignoring the directionality of edges or unrealistically assume that the number of communities is known a priori. In this article, we develop a data-dependent community detection method for the directed network to determine the number of communities and recover community structures simultaneously, which absorbs the ideas of network embedding and penalized fusion by embedding the out- and in-nodes into low-dimensional vector space and forcing the embedding vectors towards its center. The asymptotic consistency properties of the proposed method are established in terms of network embedding, directed community detection, and estimation of the number of communities. The proposed method is applied on synthetic networks and real brain functional networks, which demonstrate the superior performance of the proposed method against a number of competitors.

Disparity estimation of stereo-endoscopic images using deep generative network

A novel disparity estimation pipeline is proposed for 3D reconstruction of dynamic soft tissues in minimally invasive surgery (MIS), which uses a deep generative network to learn manifold distributions of reasonable disparity maps from past stereo images in the training phase, and transforms stereo matching into an optimization problem with respect to the low-dimensional latent vector of the learned generator in the application phase. The proposed pipeline is particularly suitable for dynamic MIS scenarios with insufficient training data, as the photometric loss is explicitly used in the application phase and the scenario priors is introduced via a deep generative network.

Designing spongy-bone-like cellular materials: Matched topology and anisotropy

Bone is a natural material with properties such as high specific stiffness and strength. These exceptional mechanical properties are attributed to the meso-scale structure and elastic anisotropy of spongy bone. Replicating the topological traits and mechanical properties of spongy bone presents a novel opportunity to develop high-performance cellular materials. To achieve this, we propose an innovative framework for designing biomimetic cellular materials that match the trabecular structure and elastic anisotropy of spongy bone. This framework introduces a forward-flow design process that utilizes gradient-based feature tuning on a low-dimensional feature vector, transforming the complex inverse design problem into an efficient iterative process. A key innovation in our approach is the use of a pre-trained generative model, SliceGAN, to reconstruct 3D unit cells from 2D micro-CT images. This significantly reduces the cost and time associated with traditional layer-by-layer CT scans typically required for 3D training data. Numerical homogenization is then used to determine the effective elastic stiffness matrix, and a Fourier neural operator is trained to predict these matrices efficiently, greatly enhancing the computational efficiency of the design process. Using this framework, we successfully designed unit cells with topological traits and elastic anisotropy that closely approximate those of natural spongy bone. This opens new avenues for developing spongy-bone-mimetic cellular materials with exceptional mechanical properties. Moreover, the framework's versatility allows it to be extended to the design of other bio-inspired cellular materials.

HiddenSinger: High-quality singing voice synthesis via neural audio codec and latent diffusion models

Recently, denoising diffusion models have demonstrated remarkable performance among generative models in various domains. However, in the speech domain, there are limitations in complexity and controllability to apply diffusion models for time-varying audio synthesis. Particularly, a singing voice synthesis (SVS) task, which has begun to emerge as a practical application in the game and entertainment industries, requires high-dimensional samples with long-term acoustic features. To alleviate the challenges posed by model complexity in the SVS task, we propose HiddenSinger, a high-quality SVS system using a neural audio codec and latent diffusion models. To ensure high-fidelity audio, we introduce an audio autoencoder that can encode audio into an audio codec as a compressed representation and reconstruct the high-fidelity audio from the low-dimensional compressed latent vector. Subsequently, we use the latent diffusion models to sample a latent representation from a musical score. In addition, our proposed model is extended to an unsupervised singing voice learning framework, HiddenSinger-U, to train the model using an unlabeled singing voice dataset. Experimental results demonstrate that our model outperforms previous models regarding audio quality. Furthermore, the HiddenSinger-U can synthesize high-quality singing voices of speakers trained solely on unlabeled data.

A Recommendation System for Trigger-Action Programming Rules via Graph Contrastive Learning.

Trigger-action programming (TAP) enables users to automate Internet of Things (IoT) devices by creating rules such as "IF Device1.TriggerState is triggered, THEN Device2.ActionState is executed". As the number of IoT devices grows, the combination space between the functions provided by devices expands, making manual rule creation time-consuming for end-users. Existing TAP recommendation systems enhance the efficiency of rule discovery but face two primary issues: they ignore the association of rules between users and fail to model collaborative information among users. To address these issues, this article proposes a graph contrastive learning-based recommendation system for TAP rules, named GCL4TAP. In GCL4TAP, we first devise a data partitioning method called DATA2DIV, which establishes cross-user rule relationships and is represented by a user-rule bipartite graph. Then, we design a user-user graph to model the similarities among users based on the categories and quantities of devices that they own. Finally, these graphs are converted into low-dimensional vector representations of users and rules using graph contrastive learning techniques. Extensive experiments conducted on a real-world smart home dataset demonstrate the superior performance of GCL4TAP compared to other state-of-the-art methods.

Chemical reaction enhanced graph learning for molecule representation.

Molecular representation learning (MRL) models molecules with low-dimensional vectors to support biological and chemical applications. Current methods primarily rely on intrinsic molecular information to learn molecular representations, but they often overlook effectively integrating domain knowledge into MRL. In this article, we develop a reaction-enhanced graph learning (RXGL) framework for MRL, utilizing chemical reactions as domain knowledge. RXGL introduces dual graph learning modules to model molecule representation. One module employs graph convolutions on molecular graphs to capture molecule structures. The other module constructs a reaction-aware graph from chemical reactions and designs a novel graph attention network on this graph to integrate reaction-level relations into molecular modeling. To refine molecule representations, we design a reaction-based relation learning task, which considers the relations between the reactant and product sides in reactions. In addition, we introduce a cross-view contrastive task to strengthen the cooperative associations between molecular and reaction-aware graph learning. Experiment results show that our RXGL achieves strong performance in various downstream tasks, including product prediction, reaction classification, and molecular property prediction. The code is publicly available at https://github.com/coder-ACAC/RLM.

Dynamic Graph Representation Learning via Coupling-Process Model.

Representation learning based on dynamic graphs has received a lot of attention in recent years due to its wide range of application scenarios. Although many discrete or continuous dynamic graph representation learning methods have been proposed, many of them ignore the role of edge types. Through the observation of dynamic graphs in the real world, it is found that the types of various edges are very different in nature. They are roughly divided into two categories according to the frequency of occurrence: evolutive edges that appear infrequently and interactive edges that appear frequently. For both types of edges, we propose a coupling-process model (DyCPM) to capture the dynamic mechanisms of them. The model not only generates low-dimensional embedding vectors of nodes, but also aggregates the structural information and temporal information of two kinds of edges. In particular, we design a neural network parameterized discrete process to depict the change law of the topology of evolutive edges and a neural network parameterized temporal point process (TPP) to characterize the temporal dynamic rule of interactive edges. More importantly, we propose a coupling mechanism to transfer the information of the two processes through a shared embedding matrix and finally generate an embedding matrix that aggregates the topology information and temporal information of the two kinds of edges for the dynamic link prediction task. We evaluate our model and several baselines on real datasets. The experimental results show that our model can better aggregate the topology information and temporal information of the two kinds of edges according to their properties and outperforms several state-of-the-art baselines in the performance of dynamic link prediction.

IMI-driver: Integrating multi-level gene networks and multi-omics for cancer driver gene identification.

The identification of cancer driver genes is crucial for early detection, effective therapy, and precision medicine of cancer. Cancer is caused by the dysregulation of several genes at various levels of regulation. However, current techniques only capture a limited amount of regulatory information, which may hinder their efficacy. In this study, we present IMI-driver, a model that integrates multi-omics data into eight biological networks and applies Multi-view Collaborative Network Embedding to embed the gene regulation information from the biological networks into a low-dimensional vector space to identify cancer drivers. We apply IMI-driver to 29 cancer types from The Cancer Genome Atlas (TCGA) and compare its performance with nine other methods on nine benchmark datasets. IMI-driver outperforms the other methods, demonstrating that multi-level network integration enhances prediction accuracy. We also perform a pan-cancer analysis using the genes identified by IMI-driver, which confirms almost all our selected candidate genes as known or potential drivers. Case studies of the new positive genes suggest their roles in cancer development and progression.

Euclidean mirrors and dynamics in network time series

– Analyzing changes in network evolution is central to statistical network inference. We consider a dynamic network model in which each node has an associated time-varying low-dimensional latent vector of feature data, and connection probabilities are functions of these vectors. Under mild assumptions, the evolution of latent vectors exhibits low-dimensional manifold structure under a suitable distance. This distance can be approximated by a measure of separation between the observed networks themselves, and there exist Euclidean representations for underlying network structure, as characterized by this distance. These Euclidean representations, called Euclidean mirrors, permit the visualization of network dynamics and lead to methods for change point and anomaly detection in networks. We illustrate our methodology with real and synthetic data, and identify change points corresponding to massive shifts in pandemic policies in a communication network of a large organization.

High-dimensional process monitoring under time-varying operating conditions via covariate-regulated principal component analysis

High-dimensional process monitoring, which aims to raise warnings for faulty events, is a fundamental tool in ensuring the safety of large-scale industrial systems. Over the last few decades, there has been significant interest in Principal Component Analysis (PCA)-based Statistical Process Control (SPC) charts. However, most of them rely on PCA’s optimal properties in homogeneous data, which often fails when dealing with heterogeneous data from industrial processes under time-varying operating conditions. To address this issue, this study proposes a Covariate-regulated PCA model (CrPCA) seamlessly integrated into a process monitoring framework. The main idea is to adjust the principal component directions smoothly according to the time-varying operating conditions, resulting in significantly less information loss in the principal component scores compared to existing methods. This approach further yields a low-dimensional feature vector that follows a distribution with fixed parameters, regardless of covariates, for real-time surveillance. We validate the performance of our methodology through a comprehensive numerical study and two real-world applications: wind turbine condition monitoring and hot rolling process monitoring.