Sparse Random Projection Research Articles

Efficient $\chi ^{2}$ Kernel Linearization via Random Feature Maps.

Explicit feature mapping is an appealing way to linearize additive kernels, such as χ2 kernel for training large-scale support vector machines (SVMs). Although accurate in approximation, feature mapping could pose computational challenges in high-dimensional settings as it expands the original features to a higher dimensional space. To handle this issue in the context of χ2 kernel SVMs learning, we introduce a simple yet efficient method to approximately linearize χ2 kernel through random feature maps. The main idea is to use sparse random projection to reduce the dimensionality of feature maps while preserving their approximation capability to the original kernel. We provide approximation error bound for the proposed method. Furthermore, we extend our method to χ2 multiple kernel SVMs learning. Extensive experiments on large-scale image classification tasks confirm that the proposed approach is able to significantly speed up the training process of the χ2 kernel SVMs at almost no cost of testing accuracy.

Adaptive sparse random projections for wireless sensor networks with energy harvesting constraints

Considering a large-scale energy-harvesting wireless sensor network (EH-WSN) measuring compressible data, sparse random projections are feasible for data well-approximation, and the sparsity of random projections impacts the mean square error (MSE) as well as the system delay. In this paper, we propose an adaptive algorithm for sparse random projections in order to achieve a better tradeoff between the MSE and the system delay. With the energy-harvesting constraints, the sparsity is adapted to channel conditions via an optimal power allocation algorithm, and the structure of the optimal power allocation solution is analyzed for some special case. The performance is illustrated by numerical simulations.

Robust people counting using sparse representation and random projection

Estimating the number of people present in an image has many practical applications including visual surveillance and public resource management. Recently, regression-based methods for people counting have gained considerable importance, principally due to the capability of these methods to handle crowded scenes. However, the principal drawback of regression-based methods is to find an optimal set of features and a model, which is usually dependent on the crowd density. Encouraged by the recent success of sparse representation, here, we develop a robust and scalable people counting method. Sparse representation allows us to capture the hidden structure and semantic information in visual data and leads to faster processing algorithms. In order to reduce the complexity of solving l1-minimization problem, which resides at the heart of the sparse representation, a dimensionality reduction method based on random projection is employed. The sparse representation framework provides new insight that if sparsity in the classification problem is properly harnessed, feature extraction is no longer critical. So, in addition to several hand-crafted features, we exploit the features obtained from pre-trained deep Convolutional neural network and show these features perform competitively. Further, to render the proposed method user friendly, we employ a semi-supervised elastic net to automatically annotate unlabelled data with only a handful of user-labelled image frames. Our semi-supervised method exploits temporal continuity in videos. We use extensive evaluations on the crowd analysis benchmark datasets to demonstrate the effectiveness of our approach as well as its superiority over the state-of-the-art regression-based people counting methods, in terms of accuracy and time.

Wireless Compressive Sensing Over Fading Channels With Distributed Sparse Random Projections

We address the problem of recovering a sparse signal observed by a resource constrained wireless sensor network with fading channels. Sparse random matrices are exploited to reduce the communication cost in forwarding information to a fusion center. The presence of channel fading leads to inhomogeneity and non-Gaussian statistics in the effective measurement matrix that relates the measurements collected at the fusion center and the sparse signal being observed. We analyze the impact of channel fading on recovery of a given sparse signal by leveraging the properties of heavy-tailed random matrices. We quantify the additional number of measurements required to ensure reliable signal recovery in the presence of nonidentical fading channels compared to that is required with identical Gaussian channels. Our analysis provides insights into how to control the probability of sensor transmissions at each node based on the channel fading statistics to minimize the number of measurements collected at the fusion center for reliable sparse signal recovery. We further discuss recovery guarantees of a given sparse signal with any random projection matrix where the elements are subexponential with a given subexponential norm. Numerical results are provided to corroborate the theoretical findings.

Data Gathering with Compressive Sensing in Wireless Sensor Networks: A Random Walk Based Approach

In this paper, we study the problem of data gathering with compressive sensing (CS) in wireless sensor networks (WSNs). Unlike the conventional approaches, which require uniform sampling in the traditional CS theory, we propose a random walk algorithm for data gathering in WSNs. However, such an approach will conform to path constraints in networks and result in the non-uniform selection of measurements. It is still unknown whether such a non-uniform method can be used for CS to recover sparse signals in WSNs. In this paper, from the perspectives of CS theory and graph theory, we provide mathematical foundations to allow random measurements to be collected in a random walk based manner. We find that the random matrix constructed from our random walk algorithm can satisfy the expansion property of expander graphs. The theoretical analysis shows that a k-sparse signal can be recovered using `1 minimization decoding algorithm when it takes m = O(k log(n=k)) independent random walks with the length of each walk t = O(n=k) in a random geometric network with n nodes. We also carry out simulations to demonstrate the effectiveness of the proposed scheme. Simulation results show that our proposed scheme can significantly reduce communication cost compared to the conventional schemes using dense random projections and sparse random projections, indicating that our scheme can be a more practical alternative for data gathering applications in WSNs.

Sparse Random Projection Algorithm Based on Minimum Energy Tree in Wireless Sensor Network

Sparse Random Projection Algorithm Based on Minimum Energy Tree in Wireless Sensor Network

Sparse random projection for χ2 kernel linearization: Algorithm and applications to image classification

The χ2 kernel based support vector machines (SVMs) have achieved impressive performances in many image and text classification tasks. As a nonlinear kernel method, however, it does not scale well to large scale data, because the computation of the χ2 kernel matrix is intractable. To address this challenge, we propose a sparse random projection method to linearly approximate the χ2 kernel, so that the original nonlinear SVMs could be converted to linear ones. Then we are able to make use of the existing large scale linear SVMs training method efficiently. Experimental results on three popular image data sets (MNIST, rcv1.binary, Caltech-101) show that the proposed method can significantly improve the learning efficiency of the χ2 kernel SVMs and the improvement comes at almost no cost of accuracy.

Research on Data Compression of WSN Based on Compressed Sensing

For Wireless Sensor Networks (WSN) is responsible for sensing, collecting, processing and monitoring of environmental data, but it might be limited in resources. This paper describes in detail the compressed sensing theory, study the wireless sensor network data conventional compression and network coding method. The linear network coding scheme based on sparse random projection theory of compressed sensing. Simulation results show that this system satisfies the requirements of the reconstruction error of packets needed to reduce the number of nodes to the total number of 30%, improves the efficiency of data communications in wireless sensor network, reduce the energy consumption of the system. With other wireless sensor network data compression algorithm, the proposed algorithm has the advantages of simple realization, the compression effect is good, especially suitable for resource limited, and the accuracy requirements are not particularly stringent in wireless sensor networks.

Exploring Data-Independent Dimensionality Reduction in Sparse Representation-Based Speaker Identification

The sparse representation classification (SRC) has attracted the attention of many signal processing domains in past few years. Recently, it has been successfully explored for the speaker recognition task with Gaussian mixture model (GMM) mean supervectors which are typically of the order of tens of thousands as speaker representations. As a result of this, the complexity of such systems become very high. With the use of the state-of-the-art i-vector representations, the dimension of GMM mean supervectors can be reduced effectively. But the i-vector approach involves a high dimensional data projection matrix which is learned using the factor analysis approach over huge amount of data from a large number of speakers. Also, the estimation of i-vector for a given utterance involves a computationally complex procedure. Motivated by these facts, we explore the use of data-independent projection approaches for reducing the dimensionality of GMM mean supervectors. The data-independent projection methods studied in this work include a normal random projection and two kinds of sparse random projections. The study is performed on SRC-based speaker identification using the NIST SRE 2005 dataset which includes channel matched and mismatched conditions. We find that the use of data-independent random projections for the dimensionality reduction of the supervectors results in only 3 % absolute loss in performance compared to that of the data-dependent (i-vector) approach. It is highlighted that with the use of highly sparse random projection matrices having $$\pm $$ ± 1 as non-zero coefficients, a significant reduction in computational complexity is achieved in finding the projections. Further, as these matrices do not require floating point representations, their storage requirement is also very small compared to that of the data-dependent or the normal random projection matrices. These reduced complexity sparse random projections would be of interest in context of the speaker recognition applications implemented on platforms having low computational power.

基于压缩感知与尺度不变特征变换的图像配准算法

基于压缩感知与尺度不变特征变换的图像配准算法

Compressive detection with sparse random projections

Compressive detection with sparse random projections

A Method of Seabed Soil Image Reconstruction Based on Compressed Sensing

Compressed Sensing(CS) can project a high dimensional signal to a low dimensional signal by a random measurement matrix . As the projection calculation is time-consuming in the process of reconstruction, the reconstruction speed is greatly affected.In order to improve the reconstruction speed , some improvement in the selection of the measurement matrix and the design of the reconstruction algorithm is made. The wavelet transform is used to sparse decompose the image, and the very sparse random projection matrix is used as the measurement matrix, after the image block processing we use the OMP algorithm to reconstruct the image. The experimental result shows that this method could reduce the algorithm time and improved the reconstruction speed greatly.

Efficient sampling and compressive sensing for urban monitoring vehicular sensor networks

Vehicular sensor network (VSN) using vehicle-based sensors is an emerging technology that can provide an inexpensive solution for surveillance and urban monitoring applications. For the constantly moving vehicles, resulting in unpredictable network topology, data transmission in VSN is vulnerable to packet losses, thus deteriorating the surveillance quality. To resolve this problem, a cooperative data sampling and compression approach is proposed. Based on compressive sensing, this approach does not require inter-sensor communication and adopts sparse random projections to remove redundancy in spatially neighbouring measurements. It is experimentally shown that the proposed algorithm provides fairly accurate reconstruction of the field under surveillance, and incurs much less communication traffic load compared to conventional sampling strategies. Practical data sets, including the temperature distribution in Beijing and the global position system (GPS) tracking data of over 6000 taxis in the city, are used in our experiments to verify the reconstruction accuracy and energy efficiency of the scheme. Different vehicular mobility models are also employed to study the impact of movement behavior. Simulation results show that our proposed approach outperforms the conventional sampling and interpolation strategy, which propagates data in uncompressed format, by 5 dB in reconstruction quality and by 50% in communication complexity reduction for the same sampling rate.

An empirical evaluation of rotation-based ensemble classifiers for customer churn prediction

Several studies have demonstrated the superior performance of ensemble classification algorithms, whereby multiple member classifiers are combined into one aggregated and powerful classification model, over single models. In this paper, two rotation-based ensemble classifiers are proposed as modeling techniques for customer churn prediction. In Rotation Forests, feature extraction is applied to feature subsets in order to rotate the input data for training base classifiers, while RotBoost combines Rotation Forest with AdaBoost. In an experimental validation based on data sets from four real-life customer churn prediction projects, Rotation Forest and RotBoost are compared to a set of well-known benchmark classifiers. Moreover, variations of Rotation Forest and RotBoost are compared, implementing three alternative feature extraction algorithms: principal component analysis (PCA), independent component analysis (ICA) and sparse random projections (SRP). The performance of rotation-based ensemble classifier is found to depend upon: (i) the performance criterion used to measure classification performance, and (ii) the implemented feature extraction algorithm. In terms of accuracy, RotBoost outperforms Rotation Forest, but none of the considered variations offers a clear advantage over the benchmark algorithms. However, in terms of AUC and top-decile lift, results clearly demonstrate the competitive performance of Rotation Forests compared to the benchmark algorithms. Moreover, ICA-based Rotation Forests outperform all other considered classifiers and are therefore recommended as a well-suited alternative classification technique for the prediction of customer churn that allows for improved marketing decision making.

Faster least squares approximation

Least squares approximation is a technique to find an approximate solution to a system of linear equations that has no exact solution. In a typical setting, one lets n be the number of constraints and d be the number of variables, with $${n \gg d}$$. Then, existing exact methods find a solution vector in O(nd 2) time. We present two randomized algorithms that provide accurate relative-error approximations to the optimal value and the solution vector of a least squares approximation problem more rapidly than existing exact algorithms. Both of our algorithms preprocess the data with the Randomized Hadamard transform. One then uniformly randomly samples constraints and solves the smaller problem on those constraints, and the other performs a sparse random projection and solves the smaller problem on those projected coordinates. In both cases, solving the smaller problem provides relative-error approximations, and, if n is sufficiently larger than d, the approximate solution can be computed in O(nd ln d) time.

High-dimensional Variable Selection with Sparse Random Projections: Measurement Sparsity and Statistical Efficiency

We consider the problem of high-dimensional variable selection: given n noisy observations of a k-sparse vector β* ∈ Rp, estimate the subset of non-zero entries of β*. A significant body of work has studied behavior of l1-relaxations when applied to random measurement matrices that are dense (e.g., Gaussian, Bernoulli). In this paper, we analyze sparsified measurement ensembles, and consider the trade-off between measurement sparsity, as measured by the fraction γ of non-zero entries, and the statistical efficiency, as measured by the minimal number of observations n required for correct variable selection with probability converging to one. Our main result is to prove that it is possible to let the fraction on non-zero entries γ → 0 at some rate, yielding measurement matrices with a vanishing fraction of non-zeros per row, while retaining the same statistical efficiency as dense ensembles. A variety of simulation results confirm the sharpness of our theoretical predictions.

A performance driven methodology for cancelable face templates generation

In this paper, we propose a performance driven methodology for cancelable face templates generation. This is to address the issue of satisfying both the security and performance requirements at the same time. Essentially, the methodology consists of two transformations namely, an efficient feature extraction transformation and an error minimizing template transformation. The first transformation is achieved via a modified sparse random projection which extracts and transforms essential face features into cancelable templates. The second transformation is realized through a direct objective formulation to minimize the system's total error rate. In order to facilitate convergence of the resulted minimization search, a modified sigmoid is proposed for an error counting step function approximation. Using two publicly available face databases, we empirically show an improved verification performance in terms of the equal error rate while hiding the face identity simultaneously.

Practical recipes for the model order reduction, dynamical simulation and compressive sampling of large-scale open quantum systems

Practical recipes are presented for simulating high-temperature and nonequilibrium quantum spin systems that are continuously measured and controlled. The notion of a spin system is broadly conceived, in order to encompass macroscopic test masses as the limiting case of large-j spins. The simulation technique has three stages: first the deliberate introduction of noise into the simulation, then the conversion of that noise into an equivalent continuous measurement and control process, and finally, projection of the trajectory onto state-space manifolds having reduced dimensionality and possessing a Kähler potential of multilinear algebraic form. These state-spaces can be regarded as ruled algebraic varieties upon which a projective quantum model order reduction (MOR) is performed. The Riemannian sectional curvature of ruled Kählerian varieties is analyzed, and proved to be non-positive upon all sections that contain a rule. These manifolds are shown to contain Slater determinants as a special case and their identity with Grassmannian varieties is demonstrated. The resulting simulation formalism is used to construct a positive P-representation for the thermal density matrix. Single-spin detection by magnetic resonance force microscopy (MRFM) is simulated, and the data statistics are shown to be those of a random telegraph signal with additive white noise. Larger-scale spin-dust models are simulated, having no spatial symmetry and no spatial ordering; the high-fidelity projection of numerically computed quantum trajectories onto low dimensionality Kähler state-space manifolds is demonstrated. The reconstruction of quantum trajectories from sparse random projections is demonstrated, the onset of Donoho–Stodden breakdown at the Candès–Tao sparsity limit is observed, a deterministic construction for sampling matrices is given and methods for quantum state optimization by Dantzig selection are given.

The Fast Johnson–Lindenstrauss Transform and Approximate Nearest Neighbors

We introduce a new low-distortion embedding of $\ell_2^d$ into $\ell_p^{O(\log n)}$ ($p=1,2$) called the fast Johnson-Lindenstrauss transform (FJLT). The FJLT is faster than standard random projections and just as easy to implement. It is based upon the preconditioning of a sparse projection matrix with a randomized Fourier transform. Sparse random projections are unsuitable for low-distortion embeddings. We overcome this handicap by exploiting the “Heisenberg principle” of the Fourier transform, i.e., its local-global duality. The FJLT can be used to speed up search algorithms based on low-distortion embeddings in $\ell_1$ and $\ell_2$. We consider the case of approximate nearest neighbors in $\ell_2^d$. We provide a faster algorithm using classical projections, which we then speed up further by plugging in the FJLT. We also give a faster algorithm for searching over the hypercube.

Cortical Representation of Bimanual Movements

It is well established that the discharge of neurons in primate motor cortex is tuned to the movement direction of the contralateral arm. Interestingly, it has been found that these neurons exhibit a directional tuning to the ipsilateral arm as well and that the preferred directions to both arms tend to be similar. A recent study showed that motor cortex cells are also directionally selective to bimanual movements, but the relationship between the bimanual and unimanual representations remains unclear. To address this issue, we analyzed the responses of motor cortical neurons recorded from two macaque monkeys during unimanual and bimanual reaching movements. We decomposed the bimanual movement representation into contralateral and ipsilateral directionally tuned components. Our major finding is that the movement of the contralateral arm modifies the tuning of the cells to the ipsilateral arm such that: (1) the offset and modulation depth of the tuning are suppressed; and (2) the preferred directions are randomly shifted. Both these effects eliminate the correlation between the contralateral and ipsilateral representations during bimanual movements. We suggest that the modification of the ipsilateral arm representation is caused by the recruitment of local inhibition that conveys callosal inputs during bimanual movements. This hypothesis is supported by the analysis of a model of two motor cortical networks, coupled with sparse random interhemispheric projections that reproduce the main features observed in the data. Finally, we show that the modification of the ipsilateral arm representation reduces the interference between the movements of both arms.