Covariance Matrix Descriptor Research Articles

Hyperspectral image classification using multiple weighted local kernel matrix descriptors

ABSTRACT Local covariance matrix descriptor is a new spatial-spectral feature generation method. It has been successfully applied for remote sensing image classification. Meanwhile, there are some critiques of it because it neglects nonlinear relationships between features, which are serious when applied to hyperspectral images (HSIs). So, the present paper aims to develop weighted local kernel matrix (WLKM) descriptors for the spatial-spectral classification of HSI. The developed weighted local kernel matrix features, including spectral-textural-geometrical aspects, have been used in two classification schemes proposed. In the first approach, called ‘early fusion’, the weighted sum of WLKM descriptors derived from spectral and spatial features is classified using the log-Euclidean kernel SVM. In the second approach, called ‘late fusion’, a multiple log-Euclidean kernel SVM strategy based on the WLKM descriptors of spatial and spectral features is developed for HSI classification. Experiments on three widely used HSI datasets have proved the superiority of the proposed approaches over some recent spatial-spectral HSI classification techniques.

Developing a Reassembling Algorithm for Broken Objects

The research on reassembling broken objects has many important applications, such as cultural relics restoration, medical surgery and solving puzzle. Because of the complicated surfaces of the fractured object pieces, it is not easy to extract salient features from them. It becomes even more difficult and very time-consuming to reassemble broken objects when the fragments are severely corroded or some of them are lost. In order to improve the accuracy and speed of 3D fragment reassembling, an effective and efficient fragment reassembling algorithm based on point clouds is proposed this article. This method first extracts keypoints and their concavity and convexity according to the symbolic projection distance of the point cloud, and then uses the local neighborhood information of the keypoints to construct a multi-scale covariance matrix descriptor. Furthermore, by calculating the similarity of the covariance matrix descriptors, the initial pairs of match points are obtained. Finally, the geometric constraints are gradually added to optimize the sampling so as to find good hypotheses as quickly as possible. By doing so, the search space is narrowed continuously in each iteration of the process to speed up the hypothesis test. We have conducted extensive experiments. The results show that the proposed method can fuse multiple features of the fragments effectively and achieve an outstanding matching effect on the defected fragments, and that the proposed method is faster than the existing methods in literature.

Adaptive multi-degree of freedom Brain Computer Interface using online feedback: Towards novel methods and metrics of mutual adaptation between humans and machines for BCI.

This paper proposes a novel adaptive online-feedback methodology for Brain Computer Interfaces (BCI). The method uses ElectroEncephaloGraphic (EEG) signals and combines motor with speech imagery to allow for tasks that involve multiple degrees of freedom (DoF). The main approach utilizes the covariance matrix descriptor as feature, and the Relevance Vector Machines (RVM) classifier. The novel contributions include, (1) a new method to select representative data to update the RVM model, and (2) an online classifier which is an adaptively-weighted mixture of RVM models to account for the users’ exploration and exploitation processes during the learning phase. Instead of evaluating the subjects’ performance solely based on the conventional metric of accuracy, we analyze their skill’s improvement based on 3 other criteria, namely the confusion matrix’s quality, the separability of the data, and their instability. After collecting calibration data for 8 minutes in the first run, 8 participants were able to control the system while receiving visual feedback in the subsequent runs. We observed significant improvement in all subjects, including two of them who fell into the BCI illiteracy category. Our proposed BCI system complements the existing approaches in several aspects. First, the co-adaptation paradigm not only adapts the classifiers, but also allows the users to actively discover their own way to use the BCI through their exploration and exploitation processes. Furthermore, the auto-calibrating system can be used immediately with a minimal calibration time. Finally, this is the first work to combine motor and speech imagery in an online feedback experiment to provide multiple DoF for BCI control applications.

Matching Algorithm of 3D Point Clouds Based on Multiscale Features and Covariance Matrix Descriptors

The three-dimensional (3D) point cloud is one of the most promising tools for representing and identifying 3D objects. The critical step for matching is to find the appropriate feature descriptors. Two prevalent descriptors are global feature descriptor and local feature descriptor. The former represents the geometric and topological properties of the neighborhood in the entire 3D model, but it can not recognize the covered areas. The local descriptor focuses on narrow neighborhoods, while coarse areas are still present for disambiguation. In this paper, we present a novel matching algorithm of 3D point clouds based on multiple scale features and covariance matrix descriptors. By the combination of the curvature and eigenvalue variation, the key points are detected precisely under multiple scales. Furthermore, we develop a three-scale covariance matrix descriptor to demonstrate local features of the key points. The three-scale covariance matrix descriptor includes the geometric angles, dimensionality, the ratio of projection length and the difference of the curvature, which can describe the local geometric features of the key points more clearly and make feature descriptors more distinguished, especially for key points which are similar in a small range but are not similar in a large range. Besides, a bidirectional proportion strategy is used to find the optimal matching pairs. The algorithm efficiently reduces the mismatching error compared with some local descriptor. Moreover, it is more robust to high noise. Experiments show the efficiency and the robustness of the proposed algorithm for matching three-dimensional point clouds with Gaussian noise and deformed shapes.

Robust Visual Tracking via Smooth Manifold Kernel Sparse Learning

Various sparse-representation-based tracking meth-ods have been proposed to tackle visual tracking problems, and most of them use simple intensity feature as the observation of target object. Moreover, most of them only take into account either global or local image representation and only exploit the underlying relationship among target candidates in a single frame. All of these may make their appearance models less robust to deal with complex scenes. To overcome these problems, we propose a smooth manifold kernel sparse tracker under the framework of particle filter. The proposed method characterizes targets and candidates with region covariance matrix descriptors, and constructs object tracking as a kernel sparse learning model based on symmetric positive-definite (SPD) manifolds. The spatial-temporal interdependencies among candidates and global-local representations of candidates are jointly considered and unified via the kernel sparse learning model. Moreover, in order to make the model more robust, the detection of outlier tasks is also taken into account. To handle the variation of object appearance, we develop a robust and efficient online dictionary learning algorithm on SPD manifolds. Extensive experiments on multiple benchmark datasets demonstrate that our tracker performs favorably against state-of-the-art trackers.

Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features

Objective. In this paper, we investigate the suitability of imagined speech for brain–computer interface (BCI) applications. Approach. A novel method based on covariance matrix descriptors, which lie in Riemannian manifold, and the relevance vector machines classifier is proposed. The method is applied on electroencephalographic (EEG) signals and tested in multiple subjects. Main results. The method is shown to outperform other approaches in the field with respect to accuracy and robustness. The algorithm is validated on various categories of speech, such as imagined pronunciation of vowels, short words and long words. The classification accuracy of our methodology is in all cases significantly above chance level, reaching a maximum of 70% for cases where we classify three words and 95% for cases of two words. Significance. The results reveal certain aspects that may affect the success of speech imagery classification from EEG signals, such as sound, meaning and word complexity. This can potentially extend the capability of utilizing speech imagery in future BCI applications. The dataset of speech imagery collected from total 15 subjects is also published.

Pairwise registration of TLS point clouds using covariance descriptors and a non-cooperative game

It is challenging to automatically register TLS point clouds with noise, outliers and varying overlap. In this paper, we propose a new method for pairwise registration of TLS point clouds. We first generate covariance matrix descriptors with an adaptive neighborhood size from point clouds to find candidate correspondences, we then construct a non-cooperative game to isolate mutual compatible correspondences, which are considered as true positives. The method was tested on three models acquired by two different TLS systems. Experimental results demonstrate that our proposed adaptive covariance (ACOV) descriptor is invariant to rigid transformation and robust to noise and varying resolutions. The average registration errors achieved on three models are 0.46 cm, 0.32 cm and 1.73 cm, respectively. The computational times cost on these models are about 288 s, 184 s and 903 s, respectively. Besides, our registration framework using ACOV descriptors and a game theoretic method is superior to the state-of-the-art methods in terms of both registration error and computational time. The experiment on a large outdoor scene further demonstrates the feasibility and effectiveness of our proposed pairwise registration framework.

Towards semantic segmentation of orthophoto images using graph-based community identification

We present an unsupervised framework that automatically detects objects of interest in images by formulating the general problem of semantic segmentation as community detection problem in graphs. The framework broadly follows a four-step procedure. First, we perform an over-segmentation of the original image using the well-known statistical region merging algorithm which presents the advantage of not requiring any quantization or colour space transformations. Second, we compute the feature descriptors of the resulting segmented regions. For encoding colour and other textural information, each region is described by an hybrid descriptor based on colour histograms and covariance matrix descriptor. Third, from the set of descriptors we construct different weighted graphs using various graph construction algorithms. Finally, the resulting graphs are then divided into groups or communities using a community detection algorithm based on spectral modularity maximization. This algorithm makes use of the eigenspectrum of matrices such as the graph Laplacian matrix and the modularity matrix which are more likely to reveal the community structure of the graph. Experiments conducted on large orthophotos depicting several zones in the region of Belfort city situated on the north-eastern of France provide promising results. The proposed framework can be used by semi-automatic approaches to handle the challenging problems of scene parsing.

A 3D Surface Matching Method Using Keypoint- Based Covariance Matrix Descriptors

A 3D surface is considered one of the most promising tools for representing and recognizing 3D objects. Therefore, 3D surface matching is widely applied to 3D object recognition, retrieval, and so on. In this paper, a 3D surface matching method using a keypoint-based covariance matrix descriptor is proposed, whose purpose is to find correspondences between 3D surfaces (e.g., 3D model and scene) by matching feature points, which are highly repeatable keypoints described by a multi-scale covariance matrix descriptor. A keypoint is detected by analyzing the surface variation index and eigenvalue variation index of a local neighborhood centered at a point. A multi-scale covariance matrix descriptor of a keypoint describes the geometric relation, surface variation gradient, and eigenvalue variation gradient between the keypoint and its neighborhood. The rationale for adopting the keypoint-based covariance matrix descriptor in our proposed 3D surface matching method is that a small number of keypoints with high repeatability can greatly enhance the matching effect of a 3D surface, after being described by a multi-scale covariance matrix descriptor with high descriptiveness. The experimental results also show that our proposed keypoint detection algorithm has higher repeatability than the surface variation index-based and eigenvalue variation index-based detection algorithms; our proposed multi-scale covariance matrix descriptor has higher descriptiveness than spin image, PFH, and 3DSC; our proposed bidirectional nearest-neighbor distance ratio algorithm can obtain better feature matching effect than the nearest-neighbor-based and nearest-neighbor distance-ratio-based feature matching. Finally, our proposed 3D surface matching method has a better matching effect than 3D surface matching methods based on spin image, PFH, and 3DSC on the Stanford 3D Scanning Repository, UWA data set and Bologna data set.

ENCODING STRUCTURAL SIMILARITY BY CROSS-COVARIANCE TENSORS FOR IMAGE CLASSIFICATION

In computer vision, an object can be modeled in two main ways: by explicitly measuring its characteristics in terms of feature vectors, and by capturing the relations which link an object with some exemplars, that is, in terms of similarities. In this paper, we propose a new similarity-based descriptor, dubbed structural similarity cross-covariance tensor (SS-CCT), where self-similarities come into play: Here the entity to be measured and the exemplar are regions of the same object, and their similarities are encoded in terms of cross-covariance matrices. These matrices are computed from a set of low-level feature vectors extracted from pairs of regions that cover the entire image. SS-CCT shares some similarities with the widely used covariance matrix descriptor, but extends its power focusing on structural similarities across multiple parts of an image, instead of capturing local similarities in a single region. The effectiveness of SS-CCT is tested on many diverse classification scenarios, considering objects and scenes on widely known benchmarks (Caltech-101, Caltech-256, PASCAL VOC 2007 and SenseCam). In all the cases, the results obtained demonstrate the superiority of our new descriptor against diverse competitors. Furthermore, we also reported an analysis on the reduced computational burden achieved by using and efficient implementation that takes advantage from the integral image representation.

Online Least Squares One-Class Support Vector Machines-Based Abnormal Visual Event Detection

The abnormal event detection problem is an important subject in real-time video surveillance. In this paper, we propose a novel online one-class classification algorithm, online least squares one-class support vector machine (online LS-OC-SVM), combined with its sparsified version (sparse online LS-OC-SVM). LS-OC-SVM extracts a hyperplane as an optimal description of training objects in a regularized least squares sense. The online LS-OC-SVM learns a training set with a limited number of samples to provide a basic normal model, then updates the model through remaining data. In the sparse online scheme, the model complexity is controlled by the coherence criterion. The online LS-OC-SVM is adopted to handle the abnormal event detection problem. Each frame of the video is characterized by the covariance matrix descriptor encoding the moving information, then is classified into a normal or an abnormal frame. Experiments are conducted, on a two-dimensional synthetic distribution dataset and a benchmark video surveillance dataset, to demonstrate the promising results of the proposed online LS-OC-SVM method.

Spatially correlated background subtraction, based on adaptive background maintenance

Moving object detection in dynamic backgrounds remains a challenging problem. Our earlier work established that the background subtraction using the covariance matrix descriptor is robust for dynamic backgrounds. The work proposed herein extends this approach further, using just two features-Hu moment and intensity. An improved local Hu moment is proposed, where the moment calculation of a pixel, involving neighboring pixels, are used in a weighted manner to reduce the effects of background moving pixels and the accurate shape localization of moving objects simultaneously. To further counter the erratic labeling of dynamic pixels, the fact that the neighboring pixels are spatially correlated is exploited for model construction and foreground detection. An adaptive model updating rate is calculated as a function of model distance. The proposed approach models each pixel with a covariance matrix and a mean feature vector and is dynamically updated. Extensive studies are made with the proposed technique to demonstrate its effectiveness.

Fast pedestrian detection system with a two layer cascade of classifiers

This work presents a novel pedestrian detection system that uses Haar-like feature extraction and a covariance matrix descriptor to identify the distinctions of pedestrians. An approach that adopts an integral image is also applied to reduce the computational loads required in both the Haar-like feature extraction and evaluation of the covariance matrix descriptor. Based on the Fisher linear discriminant analysis (FLDA) classification algorithm, the proposed system can classify pedestrians efficiently. Additionally, the detection procedure of the proposed system is accelerated using a two-layer cascade of classifiers. The front end, constructed based on Haar-like features, can select candidate regions quickly wherever pedestrians may be present. Moreover, the back end, constructed based on the covariance matrix descriptor, can determine accurately whether pedestrians are positioned in candidate regions. If a region tests positive through the two-layer cascade classifiers, pedestrian images are likely captured.Test video sequences during the experiments are taken from a test set of the INRIA person database, using 30 input frames per second, each with a resolution of 320×240 pixels. Experimental results demonstrate that the proposed system can detect pedestrians efficiently and accurately, significantly contributing to efforts to develop a real time system.

Moving object detection in the presence of dynamic backgrounds using intensity and textural features

Moving object detection in the presence of dynamic backgrounds remains a challenging problem in video surveillance. Earlier work established that the background subtraction technique based on a covariance matrix descriptor is effective and robust for dynamic backgrounds. The work proposed herein extends this concept further, using the covariance-matrix descriptor derived from local textural properties, instead of directly computing from the local image features. The proposed approach models each pixel with a covariance matrix and a mean feature vector and the model is dynamically updated. We made extensive studies with the proposed technique to demonstrate the effectiveness of statistics on local textural properties.