Probabilistic K-nearest Neighbor Research Articles

A similarity-based Bayesian mixture-of-experts model

We present a new nonparametric mixture-of-experts model for multivariate regression problems, inspired by the probabilistic k-nearest neighbors algorithm. Using a conditionally specified model, predictions for out-of-sample inputs are based on similarities to each observed data point, yielding predictive distributions represented by Gaussian mixtures. Posterior inference is performed on the parameters of the mixture components as well as the distance metric using a mean-field variational Bayes algorithm accompanied with a stochastic gradient-based optimization procedure. The proposed method is especially advantageous in settings where inputs are of relatively high dimension in comparison to the data size, where input–output relationships are complex, and where predictive distributions may be skewed or multimodal. Computational studies on five datasets, of which two are synthetically generated, illustrate clear advantages of our mixture-of-experts method for high-dimensional inputs, outperforming competitor models both in terms of validation metrics and visual inspection.

Voronoi Partition to Support Data Search in Uncertain Database with k-Bound Filtering

With the development of mobile technology, one of its main functions is to have mobile navigation capabilities, this is one of the Location-Based Services (LBS). Mobile navigation is designed as a mobile device to be able to monitor access objects from users. Because of the mobile characteristic, the impact on the points of objects to be found always occur a not certain change. Efficient query processing is needed on a set of mobile data due to the movement of users. This movement has an impact on searching in an uncertain database. For this uncertain database, the important query method is Probabilistic k-Nearest Neighbor query (PkNN), which calculates the probability of the set of k objects to be closest to the given query point. Several studies have been conducted at this time in order to contribute to the search results in a mobile database and uncertain with the support of certain algorithms to produce better performance. In this study, we propose a method called voronoi partitioning to support searching in uncertain database (Partition threshold k Aggregate Nearest Neighbor query method-Partition_PANN). In designing the partition of the voronoi region, it starts by using voronoi local networks to calculate query answers for small areas around the request point. A query point that meets the threshold used as a value as a set of threshold queries. So, all of the query points that meet the thresholds are the basis for forming local network voronoi partitions. Thus, this method does not require preliminary calculations also evaluation of distances at each intersection. The purpose of this research is to improve the performance of queries in an uncertain database by making the aggregate process and trimming the probability value as one phase of the search algorithm. In the first stage, objects which not able to form the answers are filtered by calculating the minimum closed circle from the dataset of query which prepared for the trimming phase. The second step called probabilistic candidate selection, its cut significant set of candidates for inspected as the aggregate function of the nearest neighbor’s demand. The remaining set is sent for verification which obtains possible answers at the lower and upper limits so that candidates whose probabilities are not less than the user-specified threshold are saved in the result set and returned to the user. We also examine the efficient data structures spatially which support this method. Our solution can be applied to uncertain data with an ever-changing probability density function.

SDR Based Indoor Beacon Localization Using 3D Probabilistic Multipath Exploitation and Deep Learning

Wireless indoor positioning systems (IPS) are ever-growing as traditional global positioning systems (GPS) are ineffective due to non-line-of-sight (NLoS) signal propagation. In this paper, we present a novel approach to learning three-dimensional (3D) multipath channel characteristics in a probabilistic manner for providing high performance indoor localization of wireless beacons. The proposed system employs a single triad dipole vector sensor (TDVS) for polarization diversity, a deep learning model deemed the denoising autoencoder to extract unique fingerprints from 3D multipath channel information, and a probabilistic k-nearest-neighbor (PkNN) to exploit the 3D multipath characteristics. The proposed system is the first to exploit 3D multipath channel characteristics for indoor wireless beacon localization via vector sensing methodologies, a software defined radio (SDR) platform, and multipath channel estimation.

Energy efficient tracking in uncertain sensor networks

Uncertainty existed in sensor networks presents new challenges for target tracking. Besides energy conservation of the network, target tracking has to deal with different kinds of uncertainty, such as the impreciseness of positioning systems, environment noise and limited sensitivity of sensors. In this paper, we solve the problem of energy efficient tracking in uncertain sensor networks. We first investigate the uncertainty existed in sensor networks and propose a series of uncertain models. Then, we construct a grid based network model and incorporate it into tracking procedure, which makes nodes near the vertexes of involved grid units work and others sleep to save energy with tracking quality guarantee. To optimize the tracking algorithm with uncertainty consideration, we further introduce the problem of probabilistic k-nearest neighbors (PkNN) and provide an efficient tracking algorithm based on PkNN retrieval. Finally, a comprehensive set of simulations is presented. From the experimental results, we conclude that the proposed target tracking algorithm can yield excellent performance in terms of tracking accuracy and energy saving in wireless sensor networks.

안드로이드 모바일 악성 앱 탐지를 위한 확률적 K-인접 이웃 분류기

ABSTRACT In this modern society, people are having a close relationship with smartphone. This makes easier for hackers to gain the user’s information by installing the malware in the user’s smar tphone without the user’s authority. This kind of action are th reats to the user’s privacy. The malware characteristics are different to the general applications. It requires the user’s authority . In this paper, we proposed a new classification method of user requirem ents method by each application using the Principle Component Analysis(PCA) and Probabilistic K-Nearest Neighbor(PKNN) method s. The combination of those method outputs the improved result to classify between malware and general applications. By using the K-fold Cross Validation, the measurement precision o f PKNN is improved compare to the previous K-Nearest Neighbor(KNN ). The classification which difficult to solve by KNN also can be solve by PKNN with optimizing the discovering the parame ter k and  . Also the sample that has being use in this experiment is based on the Contagio.Keywords: Malware Detection, Android Permissions, Principal Component Ana lysis, KNN, PKNN

Web-Based Supervised Thematic Mapping

We introduce a methodology for semiautomatic thematic map generation from remotely sensed Earth Observation raster image data based on user-selected examples. The methodology is based on a probabilistic k-nearest neighbor supervised classification algorithm. Efficient operation is attained by exploiting data structures for high-dimensional indexing. The methodology is integrated in a Web-mapping server that is coupled to an HTML supervision interface that supports interactive navigation as well as model training and tuning. Quantitative classification quality and performance measurements are extracted for real optical data with 0.25 m resolution on a highly diverse training area.

Efficient model selection for probabilistic K nearest neighbour classification

Probabilistic K-nearest neighbour (PKNN) classification has been introduced to improve the performance of the original K-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, K. The contribution of this paper is to incorporate the uncertainty in K into the decision making, and consequently to provide improved classification with Bayesian model averaging. Indeed the problem of assessing the uncertainty in K can be viewed as one of statistical model selection which is one of the most important technical issues in the statistics and machine learning domain. In this paper, we develop a new functional approximation algorithm to reconstruct the density of the model (order) without relying on time consuming Monte Carlo simulations. In addition, the algorithms avoid cross validation by adopting Bayesian framework. The performance of the proposed approaches is evaluated on several real experimental datasets. HighlightsIt finds the optimal number of neighbours K for probabilistic KNN (PKNN).It reconstructs the density of K for the PKNN.It incorporates the uncertainty in K into the decision making.It provides improved PKNN classification using Bayesian model averaging.

Approximate nearest neighbor algorithm based on navigable small world graphs

We propose a novel approach to solving the approximate k-nearest neighbor search problem in metric spaces. The search structure is based on a navigable small world graph with vertices corresponding to the stored elements, edges to links between them, and a variation of greedy algorithm for searching. The navigable small world is created simply by keeping old Delaunay graph approximation links produced at the start of construction. The approach is very universal, defined in terms of arbitrary metric spaces and at the same time it is very simple. The algorithm handles insertions in the same way as queries: by finding approximate neighbors for the inserted element and connecting it to them. Both search and insertion can be done in parallel requiring only local information from the structure. The structure can be made distributed. The accuracy of the probabilistic k-nearest neighbor queries can be adjusted without rebuilding the structure.The performed simulation for data in the Euclidean spaces shows that the structure built using the proposed algorithm has small world navigation properties with log2(n) insertion and search complexity at fixed accuracy, and performs well at high dimensionality. Simulation on a CoPHiR dataset revealed its high efficiency in case of large datasets (more than an order of magnitude less metric computations at fixed recall) compared to permutation indexes. Only 0.03% of the 10 million 208-dimensional vector dataset is needed to be evaluated to achieve 0.999 recall (virtually exact search). For recall 0.93 processing speed 2800queries/s can be achieved on a dual Intel X5675 Xenon server node with Java implementation.

An empirical analysis of the probabilistic K-nearest neighbour classifier

The probabilistic nearest neighbour (PNN) method for pattern recognition was introduced to overcome a number of perceived shortcomings of the nearest neighbour (NN) classifiers namely the lack of any probabilistic semantics when making predictions of class membership. In addition the NN method possesses no inherent principled framework for inferring the number of neighbours, K, nor indeed associated parameters related to the chosen metric. Whilst the Bayesian inferential methodology underlying the PNN classifier undoubtedly overcomes these shortcomings there has been to date no extensive systematic study of the performance of the PNN method nor any comparison with the standard non-probabilistic approach. We address this issue by undertaking an extensive empirical study which highlights the essential characteristics of PNN when compared to a cross-validated K-NN.