High-dimensional Vector Space Research Articles

Multi-view network embedding with node similarity ensemble

Node similarity is utilized as the most popular guidance for network embedding: nodes more similar in a network should still be more similar when mapping node information from a high-dimensional vector space to a low-dimensional vector space. Most existing methods preserve a single node similarity in the network embedding, which can merely preserve one-side network structural information. Though some works try to utilize several node similarities to preserve more network information, they fail to consider the interrelationships between the latent spaces preserving different node similarities. This causes both network information insufficiency and network information redundancy. To solve the problems of existing works, we propose a novel multi-view network embedding model with node similarity ensemble. Node similarities are first selected to maximize the represented network information while minimizing the information redundancy. For each combination of the selected node similarities, a latent space is generated as a view of the network. A Canonical Correlation Analysis based approach is then used to extract the common structure of the latent spaces alignment, and a neural network based approach is used to extract the view-specific latent structure by measuring the asymmetric KL divergence of nodes’ Gaussian distribution. The common structure and the view-specific structure of multiple views are merged to perverse the overall network information. Experiments held on the real-world networks verified the superiority of the proposed method to existing works.

Quantitative structure of stable sets in finite abelian groups

We prove an arithmetic regularity lemma for stable subsets of finite abelian groups, generalising our previous result for high-dimensional vector spaces over finite fields of prime order. A qualitative version of this generalisation was recently obtained by the first author in joint work with Conant and Pillay, using model-theoretic techniques. In contrast, the approach in the present paper is highly quantitative and relies on several key ingredients from arithmetic combinatorics.

SqueezeFit: Label-Aware Dimensionality Reduction by Semidefinite Programming

Given labeled points in a high-dimensional vector space, we seek a low-dimensional subspace such that projecting onto this subspace maintains some prescribed distance between points of differing labels. Intended applications include compressive classification. Taking inspiration from large margin nearest neighbor classification, this paper introduces a semidefinite relaxation of this problem. Unlike its predecessors, this relaxation is amenable to theoretical analysis, allowing us to provably recover a planted projection operator from the data.

Providing loyalty discounts to customers in retail stores

This paper describes the study of different types of algorithms being commonly used for face recognition. We will use the algorithm that yields better accuracy to detect potential customers purchasing in a retail store and accordingly providing discounts based on their loyalty.. In recent times discounts to customers are provided in terms of sale or as a general discounts but there is no scope of discounts based on loyalty or trust that customers show on their purchase irrespective of any age group. Thus the set of all faces in the feature space is treated as a high-dimensional vector space. The faces of different subjects as being in different classes for all subjects in the training set, we establish a framework for performing a cluster separation analysis in the feature space. Also, having labeled all instances in the training set and having defined all the classes, we compute the within and between-class scatter matrices. The most commonly and used mechanism for face detection is Eigen faces algorithm but however we will go through a couple of other algorithms in order to determine better accuracy in face recognition which will our prime goal. Then we will use Fisher's LDA mechanism in order to classify the buyer's as high, mid and low potential groups and accordingly take decision based on that. Hence it can overcome the problem of varying illumination. It is used to extract the magnitude of edge information and it works well even during the variations in poses and the illumination conditions. HOG works well under such challenging situations as it represents directionality of edge information thereby making it significant for the study of pattern and structure of the interested object.

Generative topographic mapping in drug design

This is a review article of Generative Topographic Mapping (GTM) - a non-linear dimensionality reduction technique producing generative 2D maps of high-dimensional vector spaces - and its specific applications in Drug Design (chemical space cartography, compound library design and analysis, virtual screening, pharmacological profiling, de novo drug design, conformational space & docking interaction cartography, etc.) Written by chemoinformaticians for potential users among medicinal chemists and biologists, the article purposely avoids all underlying mathematics. First, the GTM concept is intuitively explained, based on the strong analogies with the rather popular Self-Organizing Maps (SOMs), which are well established library analysis tools. GTM is basically a fuzzy-logics-based generalization of SOMs. The second part of the review, some of published GTM applications in drug design are briefly revisited.

Reducing Redundancy of Musculoskeletal Robot With Convex Hull Vertexes Selection

It has been a long-term dream of roboticist to create a robot that has similar behavior and appearance to human. With highly redundant joints and actuators, the musculoskeletal system of human can fulfill movement and operation with high precision, flexibility, and reliability. However, the redundancy of the musculoskeletal system also brings great difficulties to the control and fabrication of a human-like robot. Therefore, we propose an algorithm based on the convex hull theory to effectively reduce the number of redundant muscles. Specifically, taking the strength limitation and state-dependent characteristics of muscles into consideration, by analyzing the torque contribution of muscles for the discrete motion of joints, each muscle can be represented by a feature vector, and the convex hulls are formed by these vectors in high-dimensional space. By applying the proposed algorithm, the muscles corresponding to the vertexes of the convex hull are reserved while the redundant muscles within the convex hull are deleted, and a simplified model can be obtained. Sets of motion experiments demonstrate that the simplified model can achieve high-precision movement as a complete model. Based on the simplified model, a hardware platform of a robotic arm is constructed, which consists of six degrees of freedom and 11 pneumatic muscles.

How meaningful are similarities in deep trajectory representations?

Finding similar trajectories is an important task in moving object databases. However, classical similarity models face several limitations, including scalability and robustness. Recently, an approach named t2vec proposed transforming trajectories into points in a high dimensional vector space, and this transformation approximately keeps distances between trajectories. t2vec overcomes that scalability limitation: Now it is possible to cluster millions of trajectories. However, the semantics of the learned similarity values – and whether they are meaningful – is an open issue. One can ask: How does the configuration of t2vec affect the similarity values of trajectories? Is the notion of similarity in t2vec similar, different, or even superior to existing models? As for any neural-network-based approach, inspecting the network does not help to answer these questions. So the problem we address in this paper is how to assess the meaningfulness of similarity in deep trajectory representations. Our solution is a methodology based on a set of well-defined, systematic experiments. We compare t2vec to classical models in terms of robustness and their semantics of similarity, using two real-world datasets. We give recommendations which model to use in possible application scenarios and use cases. We conclude that using t2vec in combination with classical models may be the best way to identify similar trajectories. Finally, to foster scientific advancement, we give the public access to all trained t2vec models and experiment scripts. To our knowledge, this is the biggest collection of its kind.

Lörres, Möppes, and the Swiss. (Re)Discovering regional patterns in anonymous social media data

AbstractWe study regional similarities and differences in language use on an anonymous mobile chat application in the German-speaking area. We use a neural network on 2.3 million online conversations to automatically learn representations of words and cities. These linguistic-use-based representations capture regional distinctions in a high-dimensional vector space that can be clustered and visualized to discover patterns in the data. We find that the resulting regional patterns are closely linked to the traditional division of German dialects, even though most of the conversations are written in standard German. The resulting maps correspond to traditional dialect divisions and language-external spatial structures, with a few notable exceptions that can be explained through external factors.Our method also facilitates two qualitative analyses, allowing us to discover geographically-pertinent words for various regional levels, as well as creating regional group-specific style profiles based on various linguistic resources. The results of our study strongly suggest the existence of region-specific patterns of language use (“digital regiolects”) representing distinctive strategies of linguistic stylization in relation to linguistic resources and topics. As a methodological contribution, we show how linguistic theory can drive the application and direction of neural network-based representation learning, and how their judicious application provides the basis for qualitative analysis of large-scale data collections.

An Introduction to Hyperdimensional Computing for Robotics

Hyperdimensional computing combines very high-dimensional vector spaces (e.g. 10,000 dimensional) with a set of carefully designed operators to perform symbolic computations with large numerical vectors. The goal is to exploit their representational power and noise robustness for a broad range of computational tasks. Although there are surprising and impressive results in the literature, the application to practical problems in the area of robotics is so far very limited. In this work, we aim at providing an easy to access introduction to the underlying mathematical concepts and describe the existing computational implementations in form of vector symbolic architectures (VSAs). This is accompanied by references to existing applications of VSAs in the literature. To bridge the gap to practical applications, we describe and experimentally demonstrate the application of VSAs to three different robotic tasks: viewpoint invariant object recognition, place recognition and learning of simple reactive behaviors. The paper closes with a discussion of current limitations and open questions.

One-trial correction of legacy AI systems and stochastic separation theorems

We consider the problem of efficient “on the fly” tuning of existing, or legacy, Artificial Intelligence (AI) systems. The legacy AI systems are allowed to be of arbitrary class, albeit the data they are using for computing interim or final decision responses should posses an underlying structure of a high-dimensional topological real vector space. The tuning method that we propose enables dealing with errors without the need to re-train the system. Instead of re-training a simple cascade of perceptron nodes is added to the legacy system. The added cascade modulates the AI legacy system’s decisions. If applied repeatedly, the process results in a network of modulating rules “dressing up” and improving performance of existing AI systems. Mathematical rationale behind the method is based on the fundamental property of measure concentration in high dimensional spaces. The method is illustrated with an example of fine-tuning a deep convolutional network that has been pre-trained to detect pedestrians in images.

A Novel Multi-Stage Fusion based Approach for Gene Expression Profiling in Non-Small Cell Lung Cancer

Background: Non-small cell lung cancer is defined at the molecular level by mutations and alterations to oncogenes, including AKT1, ALK, BRAF, EGFR, HER2, KRAS, MEK1, MET, NRAS, PIK3CA, RET, and ROS1. A better understanding of non-small cell lung cancer requires a thorough consideration of these oncogenes. However, the complexity of the problem arises from high-dimensional gene vector space, which complicates the identification of cluster boundaries, and hence gene expression cluster membership. This paper aims to analyze potential biological biomarkers for tumorigenesis in lung cancer based on different treatment solutions. Results: Genes BRAF, RET, and ROS1 show an overexpression transition by one cluster from non-treatment to treatment states, followed by a stabilization in the 3 treatment states at the same cluster. Genes MET, ALK, and PIK3CA show an overexpression transition by two clusters from non-treatment to treatment states, followed by a stabilization in the 3 treatment states at the same cluster. SME1 shows an under-expression transition by two clusters from non-treatment to the treatment states, a stabilization in the 3 treatment states at the same cluster. Conclusions: We present a novel fusion-based approach for gene expression profiling of non-small cell lung cancer under non-thermal plasma treatment. The main contribution of the proposed approach is to exploit Dempster-Shafer evidence theory-based data fusion to combine information from different samples in the considered dataset. This minimizes uncertainty and enhances the reliability and validity of decisions, leading to a better description of genes related to non-small cell lung cancer. We also propose use of fuzzy c-means-with-range clustering to track changes of genes' states under different non-thermal plasma treatments.

Random selection of factors preserves the correlation structure in a linear factor model to a high degree.

In a very high-dimensional vector space, two randomly-chosen vectors are almost orthogonal with high probability. Starting from this observation, we develop a statistical factor model, the random factor model, in which factors are chosen stochastically based on the random projection method. Randomness of factors has the consequence that correlation and covariance matrices are well preserved in a linear factor representation. It also enables derivation of probabilistic bounds for the accuracy of the random factor representation of time-series, their cross-correlations and covariances. As an application, we analyze reproduction of time-series and their cross-correlation coefficients in the well-diversified Russell 3,000 equity index.

Measuring Entity Relatedness via Entity and Text Joint Embedding

As unique identifiers of objects and basic components of knowledge graphs, entities are crucial to many natural language processing related works, such as entity linking and question answering, in which the estimation of entity relatedness is required. Current entity relatedness measures either consider entities as words, which neglects the rich semantics entities contain, or are integrated into extrinsic applications, which fail to evaluate the intrinsic effectiveness. In this work, we propose E5, an effective entity relatedness measure taking into account of entity description text in a neural embedding manner. We first jointly map words and entities to the same high-dimensional vector space, the output of which is utilized as the input for the following joint entity and text embedding training. The well-trained entity and text embedding network is then leveraged to measure similarity between entities and entity descriptions, which in combination with a graph structure based method, constitute the eventual entity relatedness measure. The experimental results validate the usefulness of E5.

Exploring the Value of Learned Representations for Automated Syndromic Definitions

ObjectiveTo better define and automate biosurveillance syndrome categorization using modern unsupervised vector embedding techniques.IntroductionComprehensive medical syndrome definitions are critical for outbreak investigation, disease trend monitoring, and public health surveillance. However, because current definitions are based on keyword string-matching, they may miss important distributional information in free text and medical codes that could be used to build a more general classifier. Here, we explore the idea that individual ICD codes can be categorized by examining their contextual relationships across all other ICD codes. We extend previous work in representation learning with medical data [1] by generating dense vector embeddings of these ICD codes found in emergency department (ED) visit records. The resulting representations capture information about disease co-occurrence that would typically require SME involvement and support the development of more robust syndrome definitions.MethodsWe evaluate our method on anonymized ED visit records obtained from the New York City Department of Health and Mental Hygiene. The data set consists of approximately 3 million records spanning January 2016 to December 2016, each containing from one to ten ICD-9 or ICD-10 codes.We use these data to embed each ICD code into a high-dimensional vector space following techniques described in Mikolov, et al. [2], colloquially known as word2vec. We define an individual code’s context window as the entirety of its current health record. Final vector embeddings are generated using the gensim machine learning library in Python. We generate 300-dimensional embeddings using a skip-gram network for qualitative evaluation.We use the TensorFlow Embedding Projector to visualize the resulting embedding space. We generate a three-dimensional t-SNE visualization with a perplexity of 32 and a learning rate of 10, run for 1,000 iterations (Figure 1). Finally, we use cosine distance to measure the nearest neighbors of common ICD-10 codes to evaluate the consistency of the generated vector embeddings (Table 1).ResultsT-SNE visualization of the generated vector embeddings confirms our hypothesis that ICD codes can be contextually grouped into distinct syndrome clusters (Figure 1). Manual examination of the resulting embeddings confirms consistency across codes from the same top-level category but also reveals cross-category relationships that would be missed from a strictly hierarchical analysis (Table 1). For example, not only does the method appropriately discover the close relationship between influenza codes J10.1 and A49.2, it also reveals a link between asthma code J45.20 and obesity code E66.09. We believe these learned relationships will be useful both for refining existing syndrome categories and developing new ones.ConclusionsThe embedding structure supports the hypothesis of distinct syndrome clusters, and nearest-neighbor results expose relationships between categorically unrelated codes (appropriate upon examination). The method works automatically without the need for SME analysis and it provides an objective, data-driven baseline for the development of syndrome definitions and their refinement.

Machine health monitoring based on locally linear embedding with kernel sparse representation for neighborhood optimization

Abstract Manifold is considered to be a low dimensional surface embedded in a high dimensional vector space, and manifold learning is to find this surface based on data points sampled from this vector space. Neighborhood construction is a critical step in manifold learning to retain local relationship of data, i.e., neighbors and the connection weights. Current methods for manifold learning, including locally linear embedding, locality preserving projection, etc., assume fixed and linear neighborhood, thus lacking in adaptability for handling nonlinear system states caused by variations in machine condition or operation. To overcome this limitation, an enhanced manifold learning method is developed by utilizing kernel sparse representation to determine data neighbors and connecting weights. This enhanced manifold learning method maps data into a feature space where a kernel function is adopted to represent data by its neighbors nonlinearly. The number of data neighbors and connecting weights are determined adaptively by kernel sparse representation. It is found that the developed method enables state-related feature fusion and redundant feature elimination, thus is more effective for dimensionality reduction and feature extraction than traditional manifold learning. Analysis using vibration data measured on a gearbox with multiple faults of varying severity degrees confirmed the performance of the developed method.

Term weighting scheme for short-text classification: Twitter corpuses

Term weighting is a well-known preprocessing step in text classification that assigns appropriate weights to each term in all documents to enhance the performance of text classification. Most methods proposed in the literature use traditional approaches that emphasize term frequency. These methods perform reasonably with traditional documents. However, these approaches are unsuitable for social network data with limited length and where sparsity and noise are characteristics of short text. A simple supervised term weighting approach, i.e., SW, which considers the special nature of short texts based on term strength and term distribution, is introduced in these study, and its effect in a high-dimensional vector space over term weighting schemes, which represent baseline term weighting in traditional text classification, are assessed. Two datasets are employed with support vector machine, decision tree, k-nearest neighbor, and logistic regression algorithms. The first dataset, Sanders dataset, is a benchmark dataset that includes approximately 5000 tweets in four categories. The second self-collected dataset contains roughly 1500 tweets distributed in six classes collected using Twitter API. The evaluation applied tenfold cross-validation on the labeled data to compare the proposed approach with state-of-the-art methods. The experimental results indicate that supervised approaches perform varied performance, predominantly better than the unsupervised approaches. However, the proposed approach SW has better performance than other ones in terms of accuracy. SW can deal with the limitations of short texts and mitigate the limitations of traditional approaches in the literature, thus improving performance to 80.83 and 90.64 (F-measure) on Sanders dataset and a self-collected dataset, respectively.

Cylindrical overbounding for quadratic integrity monitors with non-Gaussian inputs

We develop tools for conservatively approximating the missed-detection probability of quadratic integrity monitors, commonly used for safety-critical navigation, for example, in the signal deformation monitor for global positioning system satellites. A derivation is presented that guarantees that a cylindrical overbounding approach conservatively approximates the true worst-case missed-detection probability for a quadratic monitor. A simulation study demonstrates that the cylindrical overbound is a relatively accurate bound, with mild overconservatism, within 5% of the correct answer for two-dimensional vector spaces with moderate-sized faults. Overconservatism increases somewhat for high-dimensional vector spaces and large fault-induced biases.

高次元ベクトル空間モデルによるテキスト分類問題について : 分類性能と距離構造の漸近解析(理論・技術)

高次元ベクトル空間モデルによるテキスト分類問題について : 分類性能と距離構造の漸近解析(理論・技術)

Link prediction based on non-negative matrix factorization.

With the rapid expansion of internet, the complex networks has become high-dimensional, sparse and redundant. Besides, the problem of link prediction in such networks has also obatined increasingly attention from different types of domains like information science, anthropology, sociology and computer sciences. It makes requirements for effective link prediction techniques to extract the most essential and relevant information for online users in internet. Therefore, this paper attempts to put forward a link prediction algorithm based on non-negative matrix factorization. In the algorithm, we reconstruct the correlation between different types of matrix through the projection of high-dimensional vector space to a low-dimensional one, and then use the similarity between the column vectors of the weight matrix as the scoring matrix. The experiment results demonstrate that the algorithm not only reduces data storage space but also effectively makes the improvements of the prediction performance during the process of sustaining a low time complexity.

Collaborative PCA/DCA Learning Methods for Compressive Privacy

In the Internet era, the data being collected on consumers like us are growing exponentially, and attacks on our privacy are becoming a real threat. To better ensure our privacy, it is safer to let the data owner control the data to be uploaded to the network as opposed to taking chance with data servers or third parties. To this end, we propose compressive privacy , a privacy-preserving technique to enable the data creator to compress data via collaborative learning so that the compressed data uploaded onto the Internet will be useful only for the intended utility and not be easily diverted to malicious applications. For data in a high-dimensional feature vector space, a common approach to data compression is dimension reduction or, equivalently, subspace projection. The most prominent tool is principal component analysis (PCA). For unsupervised learning, PCA can best recover the original data given a specific reduced dimensionality. However, for the supervised learning environment, it is more effective to adopt a supervised PCA, known as discriminant component analysis (DCA), to maximize the discriminant capability. The DCA subspace analysis embraces two different subspaces. The signal-subspace components of DCA are associated with the discriminant distance/power (related to the classification effectiveness), whereas the noise subspace components of DCA are tightly coupled with recoverability and/or privacy protection. This article presents three DCA-related data compression methods useful for privacy-preserving applications: — Utility-driven DCA : Because the rank of the signal subspace is limited by the number of classes, DCA can effectively support classification using a relatively small dimensionality (i.e., high compression). — Desensitized PCA : By incorporating a signal-subspace ridge into DCA, it leads to a variant especially effective for extracting privacy-preserving components. In this case, the eigenvalues of the noise-space are made to become insensitive to the privacy labels and are ordered according to their corresponding component powers. — Desensitized K-means/SOM : Since the revelation of the K-means or SOM cluster structure could leak sensitive information, it is safer to perform K-means or SOM clustering on a desensitized PCA subspace.