K-nearest Neighbor Graph Research Articles

Discovering communities of malapps on Android-based mobile cyber-physical systems

Android-based devices like smartphones have become ideal mobile cyber-physical systems (MCPS) due to their powerful processors and variety of sensors. In recent years, an explosively and continuously growing number of malicious applications (malapps) have posed a great threat to Android-based MCPS as well as users’ privacy. The effective detection of malapps is an emerging yet crucial task. How to establish relationships among malapps, discover their potential communities, and explore their evolution process has become a challenging issue in effective detection of malapps. To deal with this issue, in this work, we are motivated to propose an automated community detection method for Android malapps by building a relation graph based on their static features. First, we construct a large feature set to profile the behaviors of malapps. Second, we propose an E-N algorithm for graph construction by combining epsilon graph and k-nearest neighbor (k-NN) graph. It solves the problem of an incomplete graph led by epsilon method and the problem of noise generated by k-NN graph. Finally, a community detection method, Infomap, is employed to explore the underlying structures of the relation graph, and obtain the communities of malapps. We evaluate our community detection method with 3996 malapp samples. Extensive experimental results show that our method outperforms the traditional clustering methods and achieves the best performance with rand statistic of 94.93% and accuracy of 79.53%.

Fast Approximate Nearest Neighbor Search via k-Diverse Nearest Neighbor Graph

Approximate nearest neighbor search is a fundamental problem and has been studied for a few decades. Recently graph-based indexing methods have demonstrated their great efficiency, whose main idea is to construct neighborhood graph offline and perform a greedy search starting from some sampled points of the graph online. Most existing graph-based methods focus on either the precise k-nearest neighbor (k-NN) graph which has good exploitation ability, or the diverse graph which has good exploration ability. In this paper, we propose the k-diverse nearest neighbor (k-DNN) graph, which balances the precision and diversity of the graph, leading to good exploitation and exploration abilities simultaneously. We introduce an efficient indexing algorithm for the construction of the k-DNN graph inspired by a well-known diverse ranking algorithm in information retrieval (IR). Experimental results show that our method can outperform both state-of-the-art precise graph and diverse graph methods.

The out-of-core KNN awakens: the light side of computation force on large datasets

K-nearest neighbors (KNN) is a crucial tool for many applications, e.g. recommender systems, image classification and web-related applications. However, KNN is a resource greedy operation particularly for large datasets. We focus on the challenge of KNN computation over large datasets on a single commodity PC with limited memory. We propose a novel approach to compute KNN on large datasets by leveraging both disk and main memory efficiently. The main rationale of our approach is to minimize random accesses to disk, maximize sequential accesses to data and efficient usage of only the available memory. We evaluate our approach on large datasets, in terms of performance and memory consumption. The evaluation shows that our approach requires only 7% of the time needed by an in-memory baseline to compute a KNN graph.

Refinement of LiDAR point clouds using a super voxel based approach

We propose a new approach for automatic refinement of unorganized point clouds captured by LiDAR scanning systems. Given a point cloud, our method first abstracts the input data into super voxels via over segmentations, and then builds a K-nearest neighbor graph on these voxel nodes. Abstracting into voxel representation provides a means to generate an elastic wireframe over the original data. An iterative resampling method is then introduced to project resampling points to all potential surfaces considering repulsion constraints from both interior and exterior of voxels. Our point consolidation process contributes to accurate normal estimation, uniform point distribution, and sufficient sampling density. Experiments and comparisons have demonstrated that the proposed method is effective on point clouds from a variety of datasets.

Quantifying the Effects of Increasing Mechanical Stress on Knee Acoustical Emissions Using Unsupervised Graph Mining.

In this paper, we investigate the effects of increasing mechanical stress on the knee joints by recording knee acoustical emissions and analyze them using an unsupervised graph mining algorithm. We placed miniature contact microphones on four different locations: on the lateral and medial sides of the patella and superficial to the lateral and medial meniscus. We extracted audio features in both time and frequency domains from the acoustical signals and calculated the graph community factor (GCF): an index of heterogeneity (variation) in the sounds due to different loading conditions enforced on the knee. To determine the GCF, a k-nearest neighbor graph was constructed and an Infomap community detection algorithm was used to extract all potential clusters within the graph-the number of detected communities were then quantified with GCF. Measurements from 12 healthy subjects showed that the GCF increased monotonically and significantly with vertical loading forces (mean GCF for no load = 30 and mean GCF for maximum load [body weight] = 39). This suggests that the increased complexity of the emitted sounds is related to the increased forces on the joint. In addition, microphones placed on the medial side of the patella and superficial to the lateral meniscus produced the most variation in the joint sounds. This information can be used to determine the optimal location for the microphones to obtain acoustical emissions with greatest sensitivity to loading. In future work, joint loading quantification based on acoustical emissions and derived GCF can be used for assessing cumulative knee usage and loading during activities, for example for patients rehabilitating knee injuries.

Acoustical Emission Analysis by Unsupervised Graph Mining: A Novel Biomarker of Knee Health Status.

To study knee acoustical emission patterns in subjects with acute knee injury immediately following injury and several months after surgery and rehabilitation. We employed an unsupervised graph mining algorithm to visualize heterogeneity of the high-dimensional acoustical emission data, and then to derive a quantitative metric capturing this heterogeneity-the graph community factor (GCF). A total of 42 subjects participated in the studies. Measurements were taken once each from 33 healthy subjects with no known previous knee injury, and twice each from 9 subjects with unilateral knee injury: first, within seven days of the injury, and second, 4-6 months after surgery when the subjects were determined to start functional activities. Acoustical signals were processed to extract time and frequency domain features from multiple time windows of the recordings from both knees, and k-nearest neighbor graphs were then constructed based on these features. The GCF calculated from these graphs was found to be 18.5 ± 3.5 for healthy subjects, 24.8 ± 4.4 (p = 0.01) for recently injured, and 16.5 ± 4.7 (p = 0.01) at 4-6 months recovery from surgery. The objective GCF scores changes were consistent with a medical professional's subjective evaluations and subjective functional scores of knee recovery. Unsupervised graph mining to extract GCF from knee acoustical emissions provides a novel, objective, and quantitative biomarker of knee injury and recovery that can be incorporated with a wearable joint health system for use outside of clinical settings, and austere/under resourced conditions, to aid treatment/therapy.

Mutual proximity graphs for improved reachability in music recommendation

This paper is concerned with the impact of hubness, a general problem of machine learning in high-dimensional spaces, on a real-world music recommendation system based on visualisation of a k-nearest neighbour (knn) graph. Due to a problem of measuring distances in high dimensions, hub objects are recommended over and over again while anti-hubs are nonexistent in recommendation lists, resulting in poor reachability of the music catalogue. We present mutual proximity graphs, which are an alternative to knn and mutual knn graphs, and are able to avoid hub vertices having abnormally high connectivity. We show that mutual proximity graphs yield much better graph connectivity resulting in improved reachability compared to knn graphs, mutual knn graphs and mutual knn graphs enhanced with minimum spanning trees, while simultaneously reducing the negative effects of hubness.

A Novel Graph Constructor for Semisupervised Discriminant Analysis: Combined Low-Rank and k-Nearest Neighbor Graph

Semisupervised Discriminant Analysis (SDA) is a semisupervised dimensionality reduction algorithm, which can easily resolve the out-of-sample problem. Relative works usually focus on the geometric relationships of data points, which are not obvious, to enhance the performance of SDA. Different from these relative works, the regularized graph construction is researched here, which is important in the graph-based semisupervised learning methods. In this paper, we propose a novel graph for Semisupervised Discriminant Analysis, which is called combined low-rank and k-nearest neighbor (LRKNN) graph. In our LRKNN graph, we map the data to the LR feature space and then the kNN is adopted to satisfy the algorithmic requirements of SDA. Since the low-rank representation can capture the global structure and the k-nearest neighbor algorithm can maximally preserve the local geometrical structure of the data, the LRKNN graph can significantly improve the performance of SDA. Extensive experiments on several real-world databases show that the proposed LRKNN graph is an efficient graph constructor, which can largely outperform other commonly used baselines.

A Hybrid Semi-Supervised Anomaly Detection Model for High-Dimensional Data.

Anomaly detection, which aims to identify observations that deviate from a nominal sample, is a challenging task for high-dimensional data. Traditional distance-based anomaly detection methods compute the neighborhood distance between each observation and suffer from the curse of dimensionality in high-dimensional space; for example, the distances between any pair of samples are similar and each sample may perform like an outlier. In this paper, we propose a hybrid semi-supervised anomaly detection model for high-dimensional data that consists of two parts: a deep autoencoder (DAE) and an ensemble k-nearest neighbor graphs- (K-NNG-) based anomaly detector. Benefiting from the ability of nonlinear mapping, the DAE is first trained to learn the intrinsic features of a high-dimensional dataset to represent the high-dimensional data in a more compact subspace. Several nonparametric KNN-based anomaly detectors are then built from different subsets that are randomly sampled from the whole dataset. The final prediction is made by all the anomaly detectors. The performance of the proposed method is evaluated on several real-life datasets, and the results confirm that the proposed hybrid model improves the detection accuracy and reduces the computational complexity.

Reconstructing Early Erythroid Development In Vivo Using Single-Cell Transcriptomics

Erythroid differentiation may be divided into two broad stages: early development, and terminal differentiation. Early development was first explored using the colony-formation potential of hematopoietic tissue. This approach identified multi-potential progenitors (MPP) and unipotential erythroid progenitors that form 'bursts' (BFUe) and smaller colonies (CFUe). Early erythroid development is followed by erythroid terminal differentiation (ETD), which profoundly remodels erythroblasts into enucleated red cells. The molecular study of ETD was fundamentally transformed with the development of cell-surface marker strategies that identify sequential stage-specific erythroblasts in hematopoietic tissue. By contrast, there have been no strategies that systematically identify the entire cellular and molecular trajectory of the early erythroid lineage as it first arises from the MPP and progresses to the point where the ETD program is activated.To address this gap, we undertook single-cell transcriptomics using the InDrop-seq platform (Klein et al. Cell 161:1187 2015). We analyzed Kit+ cells in the bone marrow of mice in the basal state, mice stimulated with Erythropoietin (Epo) for 48 hours, or fetal liver cells. We used Next-Generation Sequencing data to construct k-nearest neighbor (kNN) graphs of cell states for each condition, and extracted the erythroid trajectory using Population Balance Analysis (PBA), a novel computational approach that predicts differentiation fates from single cell RNA profiles.We identified early erythroid developmental stages based on the modeled probability of erythroid commitment, gene expression dynamics, and the architecture of the kNN graph. By screening for appropriate cell surface markers, we developed a flow-cytometric strategy that isolates sub-populations corresponding to regions of the kNN graph, including sub-regions of the erythroid trajectory. This allowed validation of gene expression patterns and of cell fate predictions using in vitro colony formation assays.The earliest stage in the erythroid trajectory, the Erythroid/Basophil MPP stage (E/B-MPP) contains progenitors that emerge from MPPs, predicted to remain mutli-potential but be biased towards bipotential erythroid/basophil and eryrthroid/megakaryocytic fates. This stage is characterized by rapidly changing gene expression profiles. Genes whose expression correlates with the probability of erythroid commitment include both known transcriptional regulators of erythropoiesis such as GATA1, GATA2, Ldb1 and Klf1, as well as novel candidates.Downstream from the E/B-MPP, the two-dimensional projection of the kNN graph becomes a narrow bottleneck, indicating a transient stage. Here the erythroid trajectory contains cells with rapidly increasing probability of erythroid commitment (Emerging Erythroid Progenitors, EEP). The bottleneck connects to a bulge-like region in which progenitors have an extremely high probability of attaining the erythroid fate (committed erythroid progenitors, CEP). This region contains the majority of the marrow's committed erythroid progenitors, and functions as an amplification module, increasing in size in Epo-stimulated marrow and in the fetal liver. Functionally, cells in the bottleneck region give rise to multifocal erythroid colonies (early and late BFUe), whereas cells in the CEP amplification module give rise to unifocal erythroid colonies, including the CFUe. Therefore, the ability of a progenitor to give rise to either multifocal or unifocal colonies correlates closely with molecular stage.Cells in the CEP module express a unique set of genes, induced at the module entry, and repressed at its exit. These include growth-factor receptors mst1r, ryk and il17ra. Their ligands, MSP, Wnt5a and IL17a, are novel regulators of erythropoiesis, either stimulating or inhibiting the formation of erythroid colonies. Exit from the CEP module is marked by a rapid switch, in which the repression of CEP genes coincides with induction of the ETD program. Remarkably, this switch is synchronized with expression of G1/S and S phase genes, underlying a role for S phase progression in ETD activation (Pop et al., PLoS Biology 2010).Our work charts the erythroid trajectory of murine hematopoietic tissue, identifying developmental milestones, setting the stage for their molecular study and for discovery of novel erythroid regulators. [Display omitted] DisclosuresKlein:OneCell Bio: Equity Ownership, Membership on an entity's Board of Directors or advisory committees.

Semisupervised Dimension Reduction Based on Pairwise Constraint Propagation for Hyperspectral Images

This letter presents a semisupervised dimension reduction method based on pairwise constraint propagation (SSDR-PCP) for hyperspectral images (HSIs). SSDR-PCP first utilizes pairwise constraint propagation, which is based on the labeled samples and k-nearest neighbor graphs to obtain more similarity information. Then SSDR-PCP applies the obtained weak supervised information of the entire training data set to construct a new similarity matrix. At last, we embed the similarity matrix to local preserving projection to achieve dimension reduction by finding the optimal transformation matrix for HSIs. The experimental results demonstrate that SSDR-PCP achieves better performance than the previous methods on two HSIs.

Active graph based semi-supervised learning using image matching: Application to handwritten digit recognition

With the availability of large amounts of documents and multimedia content to be classified, the creation of new databases with labeled examples is an expensive task. Efficient supervised classifiers often require large training databases that are not always immediately available. Active learning approaches solve this issue by querying an expert to set a label to particular instances. In this paper, we present a novel active learning strategy for the classification of handwritten digits. The proposed method is based on a k-nearest neighbor graph obtained with an image deformation model, which takes into account local deformations. During the active learning procedure, the user is first asked to label the vertices with the highest number of neighbors. Thus, the expert sets the label to the examples that are more likely to propagate their labels to a high number of close neighbors. Then, a label propagation function is performed to automatically label the examples. The procedure is repeated until all the images are labeled. We evaluate the performance of the method on four databases corresponding to different scripts (Latin, Bangla, Devnagari, and Oriya). We show that it is possible to label only 332 images in the MNIST training database to obtain an accuracy of 98.54% on this same database (60000 images). The robustness of the method is highlighted by the performance of handwritten digit recognition in different scripts.

Kinetic k-Semi-Yao graph and its applications

This paper introduces a new supergraph, called the k-Semi-Yao graph (k-SYG), of the k-nearest neighbor graph (k-NNG) of a set P of points in Rd. We provide a kinetic data structure (KDS) to maintain the k-SYG on moving points, where the trajectory of each point is a polynomial function whose degree is bounded by some constant. Our technique gives the first KDS for the theta graph (i.e., 1-SYG) in Rd. It generalizes and improves on previous work on maintaining the theta graph in R2.As an application, we use the kinetic k-SYG to provide the first KDS for maintenance of all the k-nearest neighbors in Rd, for any k≥1. Previous works considered the k=1 case only.Our KDS for all the 1-nearest neighbors is deterministic. The best previous KDS for all the 1-nearest neighbors in Rd is randomized. Our structure and analysis are simpler and improve on this work for the k=1 case. We also provide a KDS for all the (1+ϵ)-nearest neighbors, which in fact gives better performance than previous KDSs for maintenance of all the exact 1-nearest neighbors.As another application, we present the first KDS for answering reversek-nearest neighbor queries on moving points in Rd, for any k≥1.

Spatial Outlier Detection of CO2 Monitoring Data Based on Spatial Local Outlier Factor

Spatial local outlier factor (SLOF) algorithm was adopted in this study for spatial outlier detection because of the limitations of the traditional static threshold detection. Based on the spatial characteristics of CO 2 monitoring data obtained in the carbon capture and storage (CCS) project, the K-Nearest Neighbour (KNN) graph was constructed using the latitude and longitude information of the monitoring points to identify the spatial neighbourhood of the monitoring points. Then SLOF was adopted to calculate the outlier degrees of the monitoring points and the 3σ rule was employed to identify the spatial outlier. Finally, the selection of K value was analysed and the optimal one was selected. The results show that, compared with the static threshold method, the proposed algorithm has a higher detection precision. It can overcome the shortcomings of the static threshold method and improve the accuracy and diversity of local outlier detection, which provides a reliable reference for the safety assessment and warning of CCS monitoring.

A novel algorithm for scalable k-nearest neighbour graph construction

Finding the k-nearest neighbours of every node in a dataset is one of the most important data operations with wide application in various areas such as recommendation and information retrieval. However, a major challenge is that the execution time of existing approaches grows rapidly as the number of nodes or dimensions increases. In this paper, we present greedy filtering, an efficient and scalable algorithm for finding an approximate k-nearest neighbour graph. It selects a fixed number of nodes as candidates for every node by filtering out node pairs that do not have any matching dimensions with large values. Greedy filtering achieves consistent approximation accuracy across nodes in linear execution time. We also present a faster version of greedy filtering that uses inverted indices on the node prefixes. Through theoretical analysis, we show that greedy filtering is effective for datasets whose features have Zipfian distribution, a characteristic observed in majority of large datasets. We also conduct extensive comparative experiments against (a) three state-of-the-art algorithms, and (b) three algorithms in related research domains. Our experimental results show that greedy filtering consistently outperforms other algorithms in various types of high-dimensional datasets.

Normal approximation on Poisson spaces: Mehler’s formula, second order Poincaré inequalities and stabilization

We prove a new class of inequalities, yielding bounds for the normal approximation in the Wasserstein and the Kolmogorov distance of functionals of a general Poisson process (Poisson random measure). Our approach is based on an iteration of the classical Poincare inequality, as well as on the use of Malliavin operators, of Stein’s method, and of an (integrated) Mehler’s formula, providing a representation of the Ornstein-Uhlenbeck semigroup in terms of thinned Poisson processes. Our estimates only involve first and second order difference operators, and have consequently a clear geometric interpretation. In particular we will show that our results are perfectly tailored to deal with the normal approximation of geometric functionals displaying a weak form of stabilization, and with non-linear functionals of Poisson shot-noise processes. We discuss two examples of stabilizing functionals in great detail: (i) the edge length of the k-nearest neighbour graph, (ii) intrinsic volumes of k-faces of Voronoi tessellations. In all these examples we obtain rates of convergence (in the Kolmogorov and the Wasserstein distance) that one can reasonably conjecture to be optimal, thus significantly improving previous findings in the literature. As a necessary step in our analysis, we also derive new lower bounds for variances of Poisson functionals.

Balanced Canopy Clustering에 기반한 일반적 k-인접 이웃 그래프 생성 알고리즘

Constructing a k-nearest neighbor (k-NN) graph is a primitive operation in the field of recommender systems, information retrieval, data mining and machine learning. Although there have been many algorithms proposed for constructing a k-NN graph, either the existing approaches cannot be used for various types of similarity measures, or the performance of the approaches is decreased as the number of nodes or dimensions increases. In this paper, we present a novel algorithm for k-NN graph construction based on balanced canopy clustering. The experimental results show that irrespective of the number of nodes or dimensions, our algorithm is at least five times faster than the brute-force approach while retaining an accuracy of approximately 92%.

Parallel Chameleon Clustering Based on MapReduce

With the enlarging volumes of datasets in various areas and the rapid development of distributed technologies, parallel clustering is becoming increasingly important. To cluster large-scale data of various shapes, this paper proposes a parallel Chameleon clustering algorithm. The key idea is using a parallel minimum spanning tree algorithm to generate the initial clusters after obtaining the k-nearest neighbor graph of the original dataset in a parallel way inspired by matrix multiplication, and then using strategies suggested by the primary Chameleon clustering to combine clusters and obtain the final clusters. Finally, we design the parallel Chameleon clustering based on MapReduce. Experiments show that this algorithm is efficient and well-performed.

Graph-connected components for filtering urban LiDAR data

A graph-based approach for modeling and solving the LiDAR filtering problem in urban areas is established. Our method consists of three steps. In the first step, we construct a graph-based representation of the LiDAR data, where either the Delaunay triangulation or the K-nearest neighbors graph is used. Given a set of features extracted from LiDAR data, we introduce an algorithm to label the edges of this graph. In this second step, we define criteria to eliminate some of the graph edges and then use a connected components algorithm to detect the different components in the graph representation. Finally, these components are classified into terrain or objects. Different datasets with different characteristics have been used to analyze the performance of our method. We compared our method against two other methods, and results show that our method outperforms the other methods in most tests cases.

Exploiting Variable Associations to Configure Efficient Local Search in Large-Scale Set Partitioning Problems

We present a data mining approach for reducing the search space of local search algorithms in large-scale set partitioning problems (SPPs). We construct a k-nearest neighbor graph by extracting variable associations from the instance to be solved, in order to identify promising pairs of flipping variables in the large neighborhood search. We incorporate the search space reduction technique into a 2-flip neighborhood local search algorithm with an efficient incremental evaluation of solutions and an adaptive control of penalty weights. We also develop a 4-flip neighborhood local search algorithm that flips four variables alternately along 4-paths or 4-cycles in the k-nearest neighbor graph. According to computational comparison with the latest solvers, our algorithm performs effectively for large-scale SPP instances with up to 2.57 million variables.