Low-dimensional Vector Research Articles

Efficient UAV localization using combined autoencoder and SIFT

In GNSS-denied environments, accurate Unmanned Aerial (UAV) localization faces significant challenges. This paper introduces a vision-based localization method combining autoencoder and SIFT algorithms, referred to as AE+SIFT. The method compresses high-resolution map images into low-dimensional vectors, which are stored in a database for efficient retrieval. During the localization process, UAV images are encoded and matched with the database, followed by SIFT and homography projection for precise positioning. The AE+SIFT approach enhances localization accuracy, achieving an average coordinate error of 3.94 meters relative to the ground truth. Notably, when UAV images are misaligned with reference images, our method outperforms the existing AE method in terms of accuracy.

On the Compressive Power of Autoencoders With Linear and ReLU Activation Functions.

In this letter, we mainly study the depth and width of autoencoders consisting of rectified linear unit (ReLU) activation functions. An autoencoder is a layered neural network consisting of an encoder, which compresses an input vector to a lower-dimensional vector, and a decoder, which transforms the low-dimensional vector back to the original input vector exactly (or approximately). In a previous study, Melkman etal. (2023) studied the depth and width of autoencoders using linear threshold activation functions with binary input and output vectors. We show that similar theoretical results hold if autoencoders using ReLU activation functions with real input and output vectors are used. Furthermore, we show that it is possible to compress input vectors to one-dimensional vectors using ReLU activation functions, although the size of compressed vectors is trivially Ω(log n) for autoencoders with linear threshold activation functions, where n is the number of input vectors. We also study the cases of linear activation functions. The results suggest that the compressive power of autoencoders using linear activation functions is considerably limited compared with those using ReLU activation functions.

Identification of perceptive users based on the graph convolutional network

In this paper, we introduce a pioneering approach titled Identifying Perceptive Users based on Graph Convolutional Networks (IPGCN), tailored specifically for user-object bipartite networks. Our methodology incorporates rating behavior into the graph convolutional network framework, crafting a feature matrix that encapsulates the intricacies of user preferences. This feature matrix is meticulously constructed using a breadth-first search algorithm, grounded in user rating attributes, effectively transforming the bipartite network into a compact, low-dimensional vector space. This transformation not only serves as the input for our graph convolutional network but also meticulously preserves the topological structure and interconnectivity of the nodes. Furthermore, we introduce a novel training paradigm by leveraging the classification outcomes from a perceptive user quantification model. This model discriminates between perceptive users, defined as the top q% (with 0 <q< 1) most insightful users based on their ratings, and non-perceptive users. This binary classification serves as the ground truth for training our IPGCN model. Through rigorous experimentation on three real-world datasets, we demonstrate that our IPGCN method outperforms traditional machine learning algorithms like SVM, XGBoost, and Random Forest, as well as state-of-the-art graph neural network approaches including GCN and GAT. Specifically, when q = 5%, our method achieves remarkable improvements of 25.28% in Accuracy, 23.34% in F1 Score, and 25.30% in AUC, highlighting the efficacy and superiority of our proposed IPGCN approach in identifying perceptive users within complex bipartite networks.

CDVAE: VAE-guided diffusion for particle accelerator beam 6D phase space projection diagnostics

Imaging the 6D phase space of a beam in a particle accelerator in a single shot is currently impossible. Single shot beam measurements only exist for certain 2D beam projections and these methods are destructive. A virtual diagnostic that can generate an accurate prediction of a beam’s 6D phase space would be incredibly useful for precisely controlling the beam. In this work, a generative conditional diffusion- based approach to creating a virtual diagnostic of all 15 unique 2D projections of a beam’s 6D phase space is developed. The diffusion process is guided by a combination of scalar parameters and images that are converted to low-dimensional latent vector representation by a variational autoencoder (VAE). We demonstrate that conditional diffusion guided by a VAE (cDVAE) can accurately reconstruct all 15 of the unique 2D projections of a charged particle beam’s 6D phase space for the HiRES compact accelerator.

Credibility-based knowledge graph embedding for identifying social brand advocates.

Brand advocates, characterized by their enthusiasm for promoting a brand without incentives, play a crucial role in driving positive word-of-mouth (WOM) and influencing potential customers. However, there is a notable lack of intelligent systems capable of accurately identifying online advocates based on their social interactions with brands. Knowledge Graphs (KGs) offer structured and factual representations of human knowledge, providing a potential solution to gain holistic insights into customer preferences and interactions with a brand. This study presents a novel framework that leverages KG construction and embedding techniques to identify brand advocates accurately. By harnessing the power of KGs, our framework enhances the accuracy and efficiency of identifying and understanding brand advocates, providing valuable insights into customer advocacy dynamics in the online realm. Moreover, we address the critical aspect of social credibility, which significantly influences the impact of advocacy efforts. Incorporating social credibility analysis into our framework allows businesses to identify and mitigate spammers, preserving authenticity and customer trust. To achieve this, we incorporate and extend DSpamOnto, a specialized ontology designed to identify social spam, with a focus on the social commerce domain. Additionally, we employ cutting-edge embedding techniques to map the KG into a low-dimensional vector space, enabling effective link prediction, clustering, and visualization. Through a rigorous evaluation process, we demonstrate the effectiveness and performance of our proposed framework, highlighting its potential to empower businesses in cultivating brand advocates and driving meaningful customer engagement strategies.

Modeling Inflation Expectations in Forward-Looking Interest Rate and Money Growth Rules

We propose a novel approach that directly embeds rational expectations (RE) into a low-dimensional structural vector autoregression (SVAR) without the need for any mapping to a dynamic stochastic general equilibrium (DSGE) model. Beginning from a fully specified “consensus” structural model, we establish an instrumental variable procedure internal to the SVAR to obtain RE-consistent structural responses to identified monetary policy shocks. Our RE-SVAR framework facilitates a comparison across two alternative monetary policy indicators that accommodate long horizons in the formation of inflation expectations in the policy rule. We construct clouds of responses of inflation and economic activity to monetary policy shocks. We find large regions of puzzling responses to innovations in the federal funds rate. This suggests that indicator often requires being augmented with more information in standard VAR settings. A money growth rule characterization—with Divisia M4 as a policy indicator—exhibits comparatively larger regions of sensible responses within a low-dimensional textbook model of the economy.

Network Embedding-based Directed Community Detection with Unknown Community Number

Community detection of network analysis plays an important role in numerous application areas, in which estimating the number of communities is a fundamental issue. However, many existing methods focus on undirected networks ignoring the directionality of edges or unrealistically assume that the number of communities is known a priori. In this article, we develop a data-dependent community detection method for the directed network to determine the number of communities and recover community structures simultaneously, which absorbs the ideas of network embedding and penalized fusion by embedding the out- and in-nodes into low-dimensional vector space and forcing the embedding vectors toward its center. The asymptotic consistency properties of the proposed method are established in terms of network embedding, directed community detection, and estimation of the number of communities. The proposed method is applied on synthetic networks and real brain functional networks, which demonstrate the superior performance of the proposed method against a number of competitors. Supplementary materials for this article are available online.

GeoPredict-LLM: Intelligent tunnel advanced geological prediction by reprogramming large language models

With the improvement of multisource information sensing and data acquisition capabilities inside tunnels, the availability of multimodal data in tunnel engineering has significantly increased. However, due to structural differences in multimodal data, traditional intelligent advanced geological prediction models have limited capacity for data fusion. Furthermore, the lack of pre-trained models makes it difficult for neural networks trained from scratch to deeply explore the features of multimodal data. To address these challenges, we utilize the fusion capability of knowledge graph for multimodal data and the pre-trained knowledge of large language models (LLMs) to establish an intelligent advanced geological prediction model (GeoPredict-LLM). First, we develop an advanced geological prediction ontology model, forming a knowledge graph database. Using knowledge graph embeddings, multisource and multimodal data are transformed into low-dimensional vectors with a unified structure. Secondly, pre-trained LLMs, through reprogramming, reconstruct these low-dimensional vectors, imparting linguistic characteristics to the data. This transformation effectively reframes the complex task of advanced geological prediction as a "language-based" problem, enabling the model to approach the task from a linguistic perspective. Moreover, we propose the prompt-as-prefix method, which enables output generation, while freezing the core of the LLM, thereby significantly reduces the number of training parameters. Finally, evaluations show that compared to neural network models without pre-trained models, GeoPredict-LLM significantly improves prediction accuracy. It is worth noting that as long as a knowledge graph database can be established, GeoPredict-LLM can be adapted to multimodal data mining tasks with minimal modifications.

Accurate Prediction of Essential Proteins Using Ensemble Machine Learning

Abstract Essential proteins are crucial for biological processes and can be identified through both experimental and computational methods. While experimental approaches are highly accurate, they often demand extensive time and resources. To address these challenges, we present a computational ensemble learning framework designed to identify essential proteins more efficiently. Our method begins by using node2vec to transform proteins in the Protein-Protein Interaction (PPI) network into continuous, low-dimensional vectors. We also extract a range of features from protein sequences, including graph-theory-based, information-based, compositional, and physiochemical attributes. Additionally, we leverage deep learning techniques to analyze high-dimensional Position-Specific Scoring Matrices (PSSMs) and capture evolutionary information. We then combine these features for classification using various machine learning algorithms. To enhance performance, we integrate the outputs of these algorithms through ensemble methods such as voting, weighted averaging, and stacking. This approach effectively addresses data imbalances and improves both robustness and accuracy. Our ensemble learning framework achieved an AUC of 0.948 and an accuracy of 0.9252, outperforming other computational methods. These results demonstrate the effectiveness of our approach in accurately identifying essential proteins and highlight its superior feature extraction capabilities.

Network community detection via neural embeddings

Recent advances in machine learning research have produced powerful neural graph embedding methods, which learn useful, low-dimensional vector representations of network data. These neural methods for graph embedding excel in graph machine learning tasks and are now widely adopted. However, how and why these methods work—particularly how network structure gets encoded in the embedding—remain largely unexplained. Here, we show that node2vec—shallow, linear neural network—encodes communities into separable clusters better than random partitioning down to the information-theoretic detectability limit for the stochastic block models. We show that this is due to the equivalence between the embedding learned by node2vec and the spectral embedding via the eigenvectors of the symmetric normalized Laplacian matrix. Numerical simulations demonstrate that node2vec is capable of learning communities on sparse graphs generated by the stochastic blockmodel, as well as on sparse degree-heterogeneous networks. Our results highlight the features of graph neural networks that enable them to separate communities in the embedding space.

Exploiting common patterns in diverse cancer types via multi-task learning

Cancer prognosis requires precision to identify high-risk patients and improve survival outcomes. Conventional methods struggle with the complexity of genetic biomarkers and diverse medical data. Our study uses deep learning to distil high-dimensional medical data into low-dimensional feature vectors exploring shared patterns across cancer types. We developed a multi-task bimodal neural network integrating RNA Sequencing and clinical data from three The Cancer Genome Atlas project datasets: Breast Invasive Carcinoma, Lung Adenocarcinoma, and Colon Adenocarcinoma. Our approach significantly improved prognosis prediction, especially for Colon Adenocarcinoma, with up to 26% increase in concordance index and 41% in the area under the precision-recall curve. External validation with Small Cell Lung Cancer achieved comparable metrics, indicating that supplementing small datasets with data from other cancers can improve performance. This work represents initial strides in using multi-task learning for prognosis prediction across cancer types, potentially revealing shared mechanisms among cancers and contributing to future applications in precision medicine.

Ensemble learning model for Protein-Protein interaction prediction with multiple Machine learning techniques

Protein-Protein Interactions (PPIs) encompass the physical interactions or chemical combinations between proteins, which cells employ to carry out diverse biological functions. PPIs Prediction is of great significance for understanding and studying molecular mechanisms and disease mechanisms in organisms. Traditional machine learning algorithms for predicting PPIs face challenges related to imbalanced sample data, loss of long sequence features, and variations in datasets across different species. This paper takes advantage of the attention mechanism in processing sequence tasks and proposes the fusion of Bi-directional Long Short- Term Memory, an integrated learning model of convolutional neural network and multi-head attention mechanism, referred to as CNN BiLSTM Multi-head Att. The model first performs one-hot encoding on the protein sequence and obtains a low-dimensional word vector representation, and then calculates the encoding matrix through the embedding matrix; it uses the convolutional neural network (CNN) and the bidirectional long short-term memory network (Bi-LSTM) to extract the amino acids in the protein sequence. The feature matrix is obtained by collecting information on the time, site, and physical and chemical characteristics; the multi-head attention mechanism is used in weighted calculation and merger of the subspace features of each sequence, and classification is performed after global average pooling and the PPI prediction is output. Cross-validation experiments on different proportions of positive and negative protein data sets and proteins of different species show that CNN BiLSTM Multi-head Att can better predict imbalanced data sets and protein interactions of different species. Compared to the state-of-the-art model, our model improves accuracy by 0.025, precision by 0.046, F1-score by 0.052, and recall by 0.014. Exploring the scalability and adaptability of the proposed model for large-scale biological datasets, along with investigating its potential in real-time applications, could lead to significant advancements in the field of protein–protein interaction prediction.

Designing spongy-bone-like cellular materials: Matched topology and anisotropy

Bone is a natural material with properties such as high specific stiffness and strength. These exceptional mechanical properties are attributed to the meso-scale structure and elastic anisotropy of spongy bone. Replicating the topological traits and mechanical properties of spongy bone presents a novel opportunity to develop high-performance cellular materials. To achieve this, we propose an innovative framework for designing biomimetic cellular materials that match the trabecular structure and elastic anisotropy of spongy bone. This framework introduces a forward-flow design process that utilizes gradient-based feature tuning on a low-dimensional feature vector, transforming the complex inverse design problem into an efficient iterative process. A key innovation in our approach is the use of a pre-trained generative model, SliceGAN, to reconstruct 3D unit cells from 2D micro-CT images. This significantly reduces the cost and time associated with traditional layer-by-layer CT scans typically required for 3D training data. Numerical homogenization is then used to determine the effective elastic stiffness matrix, and a Fourier neural operator is trained to predict these matrices efficiently, greatly enhancing the computational efficiency of the design process. Using this framework, we successfully designed unit cells with topological traits and elastic anisotropy that closely approximate those of natural spongy bone. This opens new avenues for developing spongy-bone-mimetic cellular materials with exceptional mechanical properties. Moreover, the framework's versatility allows it to be extended to the design of other bio-inspired cellular materials.

Knowledge Graph Embedding for Hierarchical Entities Based on Auto-Embedding Size

Knowledge graph embedding represents entities and relations as low-dimensional continuous vectors. Recently, researchers have attempted to leverage the potential semantic connections between entities with hierarchical relationships in the knowledge graph. However, existing knowledge embedding methods that model entity hierarchical structures face the following challenges: (1) Setting the same embedding dimension for entities at different hierarchical levels can lead to overfitting issues and the waste of storage and computational resources; (2) Manually searching the embedding dimension in discrete space makes it easy to miss the optimal parameters and increases training costs. Therefore, we propose a knowledge embedding method, HEAES, that automatically learns the embedding dimensions for entities based on their hierarchical structure. Specifically, we first propose a new modeling method to capture the hierarchical relationships of entities, and we then train the model on a triple classification task. Then, we adaptively learn the pruning threshold to trim the embedding vectors, automatically learning different embedding dimensions for entities at different hierarchical levels. Experiments on the YAGO26K and DB111K datasets verify that introducing entity hierarchical information helps boost the model performance.

Trustworthiness-Driven Graph Convolutional Networks for Signed Network Embedding

The problem of representing nodes in a signed network as low-dimensional vectors, known as signed network embedding (SNE), has garnered considerable attention in recent years. While several SNE methods based on graph convolutional networks (GCNs) have been proposed for this problem, we point out that they significantly rely on the assumption that the decades-old balance theory always holds in the real-world. To address this limitation, we propose a novel GCN-based SNE approach, named as TrustSGCN, which corrects for incorrect embedding propagation in GCN by utilizing the trustworthiness on edge signs for high-order relationships inferred by the balance theory. The proposed approach consists of three modules: (M1) generation of each node’s extended ego-network; (M2) measurement of trustworthiness on edge signs; and (M3) trustworthiness-aware propagation of embeddings. Specifically, TrustSGCN leverages topological information to measure trustworthiness on edge sign for high-order relationships inferred by balance theory. It then considers structural properties inherent to an input network, such as the ratio of triads, to correct for incorrect embedding propagation. Furthermore, TrustSGCN learns the node embeddings by leveraging two well-known social theories, i.e., balance and status, to jointly preserve the edge sign and direction between nodes connected by existing edges in the embedding space. The experiments on six real-world signed network datasets demonstrate that TrustSGCN consistently outperforms six state-of-the-art GCN-based SNE methods. The code is available at https://github.com/kmj0792/TrustSGCN .

Deep learning-based geological parameterization for history matching CO2 plume migration in complex aquifers

History matching is crucial for reliable numerical simulation of geological carbon storage (GCS) in deep subsurface aquifers. This study focuses on inferring highly complex aquifer permeability fields with multi- and intra-facies heterogeneity to improve the characterization of CO2 plume migration. We propose a deep learning (DL)-based parameterization strategy combined with the ensemble smoother with multiple data assimilation (ESMDA) algorithm to formulate an integrated inverse framework. The DL model is employed to parameterize non-Gaussian permeability fields using low-dimensional latent variables in a Gaussian distribution, thereby mitigating the non-Gaussianity issue faced by the ensemble-based ESMDA inverse method and simultaneously alleviating the computational burden of high-dimensional inversion. The efficacy of the integrated DL-ESMDA inverse framework is demonstrated using a 3-D GCS model, where it estimates the non-Gaussian permeability field characterized by multi- and intra-facies heterogeneity. Results show that the DL model is able to represent the highly complex and high-dimensional permeability fields using low-dimensional latent vectors. The DL-ESMDA framework sequentially updates these low-dimensional latent vectors instead of the original high-dimensional permeability field to obtain posterior estimations of the permeability field. The resulting CO2 plume migration closely matches historical measurements, suggesting a significantly improved model reliability after history matching. Additionally, a substantial reduction in uncertainty for future plume migration predictions beyond the history matching period is observed. The proposed framework provides an effective approach for reliable characterization of CO2 plume migration in highly heterogeneous aquifers, enhancing GCS project operation and risk analysis.

Cluster-guided denoising graph auto-encoder for enhanced traffic data imputation and fault detection

In an era of increasing digitalization, the integration of diverse sensors has become commonplace. Within the transportation sector, state Departments of Transportation have extensively deployed traffic sensors to support various engineering endeavors. However, these sensors are prone to faults such as data loss, random noise, bias, and drift due to sensor aging, defects, or environmental influences. Therefore, detecting these faults is imperative to maintain data integrity. This paper presents a novel approach for detecting faults in traffic data based on a cluster-guided data reconstruction process. First, a traffic sensor clustering module is designed using a dual-encoding attention graph auto-encoder (DA-GAE) to identify clusters of traffic sensors. This module leverages a joint embedding of node and edge features in a low-dimensional vector space. Subsequently, utilizing the identified clusters, a cluster-guided denoising graph auto-encoder (CG-DGAE) is devised and trained for data reconstruction. The CG-DGAE employs a diffusion graph convolutional network (DGCN) and is trained with a cluster-wise sampling strategy. Extensive experiments were conducted using traffic data obtained from a real-world large sensor network, demonstrating superior performance of the CG-DGAE model in data reconstruction compared to various baseline methods. For fault detection, a score function is devised to discern potential faults by contrasting the sensor data sequence and the reconstructed data sequence. The fault detection results show that our proposed CG-DGAE model achieved an impressive overall accuracy of 99.09%, a precision of 99.13%, a recall of 99.53%, and a F1 score of 99.53%. The superior capability of CG-DGAE in data reconstruction and fault detection is attributable to the cluster-wise spatial–temporal context leveraged by the model.

Combining Dielectric and Hyperspectral Data for Apple Core Browning Detection

Apple core browning not only affects the nutritional quality of apples, but also poses a health risk to consumers. Therefore, there is an urgent need to develop a fast and reliable non-destructive detection method for apple core browning. To deal with the challenges of the long incubation period, strong infectivity, and difficulty in the prevention and control of apple core browning, a novel non-destructive detection method for apple core browning has been developed through combining hyperspectral imaging and dielectric techniques. To reduce the computational complexity of high-dimensional multi-view data, canonical correlation analysis is employed for feature dimensionality reduction. Then, the two low-dimensional vectors extracted from two different sensors are concatenated into one united feature vector; therefore, the information contained in the hyperspectral and dielectric data is fused to improve the detection accuracy of the non-destructive method. At last, five traditional classifiers, such as k-Nearest Neighbors, a support vector machine with radial basis function kernel and polynomial kernel, Decision Tree, and neural network, are trained on the fused feature vectors to discriminate apple core browning. The experimental results on our own constructed dataset have shown that the sensitivity, specificity, and precision of SVM with RBF kernel based on concatenated 70-dimensional feature vectors extracted via canonical correlation analysis reached 99.98%, 99.70%, and 99.70%, respectively, which achieved better results than other models. This study can provide theoretical assurance and technical support for further development of higher accuracy and lower-cost non-destructive detection devices for apple core browning.

Reinforcement Learning with Decoupled State Representation for Robot Manipulations

Deep reinforcement learning has significantly advanced robot manipulations by providing an alternative solution for designing control strategies using raw images as direct inputs. While images offer additional environmental information, the end-to-end policy training manner (from image to action) requires simultaneous representation and task learning by the agent. This often necessitates a substantial number of interaction samples to achieve satisfactory policy performance. Previous works has attempted to address this challenge by learning a visual representation model that encodes the entire image into a low-dimensional vector before the policy training. However, since this vector contains both robot and object information, it inevitably introduces coupling within the state, which can mislead the policy training process. In this study, a novel method called Reinforcement Learning with Decoupled State Representation is proposed to effectively decouple robot and object information within the state representation. Experimental results demonstrate that the proposed method exhibits faster learning speed and achieves superior performance compared to previous methods across various robot manipulation tasks. Moreover, with only 3096 offline images, the proposed method successfully applies to real-world robot pushing tasks, which demonstrates its high practicability.

A low-complexity adaptive speech dereverberation approach based on multichannel linear prediction

The adaptive multichannel linear prediction (AMCLP) technique is one of the most effective methods for speech dereverberation in real-time application systems. However, the computational complexity of the associated AMCLP algorithms is very high. In this paper, we propose a low-complexity speech dereverberation approach based on AMCLP. The coefficient matrix of the multichannel dereverberation filter is decomposed into a linear combination of a set of low-dimensional sub-filter matrices and a set of low-dimensional sub-filter vectors through Kronecker product. According to these sub-filter models, we define two sets of signal models and cost functions, from which the adaptive speech dereverberation algorithm is established. This solution not only leads to effective dereverberation performance, but also reduces the computational cost, which is very conducive to realistic speech systems.