Local Subspace Research Articles

Discriminative Structured Dictionary Learning on Grassmann Manifolds and Its Application on Image Restoration.

Image restoration is a difficult and challenging problem in various imaging applications. However, despite of the benefits of a single overcomplete dictionary, there are still several challenges for capturing the geometric structure of image of interest. To more accurately represent the local structures of the underlying signals, we propose a new problem formulation for sparse representation with block-orthogonal constraint. There are three contributions. First, a framework for discriminative structured dictionary learning is proposed, which leads to a smooth manifold structure and quotient search spaces. Second, an alternating minimization scheme is proposed after taking both the cost function and the constraints into account. This is achieved by iteratively alternating between updating the block structure of the dictionary defined on Grassmann manifold and sparsifying the dictionary atoms automatically. Third, Riemannian conjugate gradient is considered to track local subspaces efficiently with a convergence guarantee. Extensive experiments on various datasets demonstrate that the proposed method outperforms the state-of-the-art methods on the removal of mixed Gaussian-impulse noise.

Differential spatial representation of precision and power grasps in the human motor system

Power and precision grasps are two interrelated, kinematically distinct types of finger movements. We examined whether these types of motor actions may be spatially differently represented in the human central nervous system.In healthy participants representations of finger movements were mapped by delivering single pulse TMS to multiple scalp regions covering the left primary motor cortex (M1). Finger joint motions were recorded from the right hand using a data glove. Principal component analysis was used to extract local subspaces representing the TMS-evoked movement data from each scalp region. Voluntary power and precision grasps were reconstructed with these subspaces. The spatial properties of these reconstructions were analyzed for each grasp type using a general linear model.We found overlapping, yet distinct spatial representations for precision and power grasps with precision grasps represented slightly posterior compared to a more uniform distribution for power grasps. Differential spatial encoding of both grasp types may point towards a representation of power grasps within a phylogenetically older M1 area at the crown of the precentral gyrus and of precision grasps in a newer area in the depth of the central sulcus. Results also support the idea of separate synergistic movement representations in the human motor system.

Locally Linear Embedding on Grassmann Manifold for Performance Degradation Assessment of Bearings

In recent years, a significant amount of research work has been undertaken to address the problem about prognostic and health management (PHM) systems. Performance degradation assessment, an essential part of PHM systems, is still a challenge. Subspaces, forming a non-Euclidean and curved manifold that is known as Grassmann manifold, are able to capture dynamic behaviors and accommodate the effects of variations. In this paper, we propose a novel local subspace model for performance degradation assessment termed locally linear embedding on Grassmann manifold (GM-LLE), where subspaces are treated as points on Grassmann manifold. Due to the nonstationary property of vibration signal, second generation wavelet package is used to decompose the vibration signal into different levels. Subspaces are modeled by optimal statistical features of different frequency bands, and then GM-LLE is used to assess bearing performance degradation by embedding the subspaces into reproducing kernel Hilbert spaces. Finally, simulated and experimental vibration signals are used to validate the effectiveness of the proposed method. The results show that the proposed method can assess the bearing's degradation effectively, and performs better compared with locally linear embedding.

Confidence-Weighted Local Expression Predictions for Occlusion Handling in Expression Recognition and Action Unit Detection

Fully-automatic facial expression recognition (FER) is a key component of human behavior analysis. Performing FER from still images is a challenging task as it involves handling large interpersonal morphological differences, and as partial occlusions can occasionally happen. Furthermore, labelling expressions is a time-consuming process that is prone to subjectivity, thus the variability may not be fully covered by the training data. In this work, we propose to train random forests upon spatially-constrained random local subspaces of the face. The output local predictions form a categorical expression-driven high-level representation that we call local expression predictions (LEPs). LEPs can be combined to describe categorical facial expressions as well as action units (AUs). Furthermore, LEPs can be weighted by confidence scores provided by an autoencoder network. Such network is trained to locally capture the manifold of the non-occluded training data in a hierarchical way. Extensive experiments show that the proposed LEP representation yields high descriptive power for categorical expressions and AU occurrence prediction, and leads to interesting perspectives towards the design of occlusion-robust and confidence-aware FER systems.

Multi-View Structural Local Subspace Tracking.

In this paper, we propose a multi-view structural local subspace tracking algorithm based on sparse representation. We approximate the optimal state from three views: (1) the template view; (2) the PCA (principal component analysis) basis view; and (3) the target candidate view. Then we propose a unified objective function to integrate these three view problems together. The proposed model not only exploits the intrinsic relationship among target candidates and their local patches, but also takes advantages of both sparse representation and incremental subspace learning. The optimization problem can be well solved by the customized APG (accelerated proximal gradient) methods together with an iteration manner. Then, we propose an alignment-weighting average method to obtain the optimal state of the target. Furthermore, an occlusion detection strategy is proposed to accurately update the model. Both qualitative and quantitative evaluations demonstrate that our tracker outperforms the state-of-the-art trackers in a wide range of tracking scenarios.

The smoother extension of the nonlinear ensemble transform filter

The recently-proposed nonlinear ensemble transform filter (NETF) is extended to a fixed-lag smoother. The NETF approximates Bayes’ theorem by applying a square root update. The smoother (NETS) is derived and formulated in a joint framework with the filter. The new smoother method is evaluated using the low-dimensional, highly nonlinear Lorenz-96 model and a square-box configuration of the NEMO ocean model, which is nonlinear and has a higher dimensionality. The new smoother is evaluated within the same assimilation framework against the local error subspace transform Kalman filter (LESTKF) and its smoother extension (LESTKS), which are state-of-the-art ensemble square-root Kalman techniques. In the case of the Lorenz-96 model, both the filter NETF and its smoother extension NETS provide lower errors than the LESTKF and LESTKS for sufficiently large ensembles. In addition, the NETS shows a distinct dependence on the smoother lag, which results in a stronger error increase beyond the optimal lag of minimum error. For the experiment using NEMO, the smoothing in the NETS effectively reduces the errors in the state estimates, compared to the filter. For different state variables very similar optimal smoothing lags are found, which allows for a simultaneous tuning of the lag. In comparison to the LESTKS, the smoothing with the NETS yields a smaller relative error reduction with respect to the filter result, and the optimal lag of the NETS is shorter in both experiments. This is explained by the distinct update mechanisms of both filters. The comparison of both experiments shows that the NETS can provide better state estimates with similar smoother lags if the model exhibits a sufficiently high degree of nonlinearity or if the observations are not restricted to be Gaussian with a linear observation operator.

A clustering algorithm for stream data with LDA-based unsupervised localized dimension reduction

We present an algorithm for clustering high dimensional streaming data. The algorithm incorporates dimension reduction into the stream clustering framework. When a new datum arrives, the algorithm performs dimension reduction to find a local projected subspace using unsupervised LDA (Linear Discriminant Analysis)-based method. The obtained local subspace would maximally separate the nearby micro-clusters with respect to the incoming point. Then, the incoming point is assigned to a micro-cluster in the projected space, rather than in the full dimensional space. The experimental results show that the proposed algorithm outperforms its counterpart streaming clustering algorithms. Moreover, when compared with traditional clustering algorithms which require the whole data set, the proposed algorithms shows comparable clustering performances with much less computation time for large data sets.

Enhancing perceptrons with contrastive biclusters

An enhanced perceptron model, referred to as BicNeuron (BN), which is based on the novel concept of contrastive biclusters is investigated. According to this viewpoint, a coherent bicluster (i.e. a subset of data patterns showing high similarity across a particular subset of features) belonging to one class is required to show high discrimination from the nearest patterns of the other class when projected on the same feature set. On each local subspace associated with a different pair of contrastive biclusters a perceptron is induced and the model with highest area under the receiver operating characteristic curve value is selected as the final classifier. Experiments involving the discrimination of epileptic and non-epileptic electroencephalogram (EEG) signals show that the BN approach can be very useful, prevailing significantly on standard and kernel perceptrons in terms of classification accuracy.

Application of Machine Learning to Systematic Allocation Strategies

We investigate the use of machine learning techniques into building statistically stable systematic allocation strategies. Traditionally, allocation processes usually rely on variations of Markowitz framework such as Mean Variance allocation, Maximum Diversity, Risk Allocation , Value at Risk, Expected Shortfall, in other words convex frontier optimization. Although those methods show some efficiency to allocate assets through the convex efficient frontier, they usually rely deeply on the estimation and the usage of the covariance matrix. Being no stationary and having multiple range memory (ie FIGARCH using Fractional Brownian Motion), the statistical estimation of covariance may lead to biases and errors and in the end, bias conclusions. Very extensive literature in econo-metrics, econo-physics, quantitative allocation cover this problem in order to remedy to the statistical estimation of covariance and his bias and issues.Here, our emphasis is not a new estimator of the covariance matrix, or a variant of Mean Variance framework but an application of Machine Learning techniques to infer no-linear relationships and long range memory between the assets.It has the advantage to remove the linear projection of the assets onto the covariance framework and then capture no-linear relationships between at various time periods.Recent advances in Neural Network, Deep Learning and Machine Learning allows a more efficient modeling of the no-linear statistical relationships between data (ie price, dividends,...). Among them, we can mention Restricted Boltzman Machines, Variationnal Auto-encoders and variations of Recurrent Neural Network, Attention and Highway Long Short Term Memory as well as Factorization Machines for projection on local sub-spaces.Thus, we investigate some of the techniques to develop practical systematic allocation strategies by reducing risks and estimations biases and show the results.

Semisupervised Subspace-Based DNA Encoding and Matching Classifier for Hyperspectral Remote Sensing Imagery

Hyperspectral remote sensing images, which are characterized by their high dimensionality, provide us with the capability to accurately identify objects on the ground. They can also be used to identify subclasses of objects. However, these subclasses are usually embedded in different subspaces due to the complex distribution of pixels in the feature space. In the literature, few hyperspectral image classification methods can take both the subclass and subspace into consideration at the same time. Motivated by the fact that natural DNA can distinguish biological subspecies (subclasses in hyperspectral images) using critical DNA fragments (subspaces in hyperspectral images), a semisupervised subspace-based DNA encoding and matching classifier for hyperspectral remote sensing imagery (SSDNA) is proposed in this paper. First, in the process of DNA encoding, the hyperspectral remote sensing image is transformed into a DNA cube, in which the first-order spectral curve of the hyperspectral remote sensing image is utilized in order to take the gradient information of the spectral curve into consideration. Second, in the process of DNA optimization, evolutionary algorithms are used to obtain the best DNA library of the typical objects, which includes the following: 1) A multicenter individual representation is designed in order to consider the existence of subclasses in the hyperspectral remote sensing image; 2) the unlabeled samples are utilized in the process of population initialization and fitness calculation to enhance the diversity of the population and the generalization of the classification performance; and 3) the different classes are embedded in different subspaces. A semisupervised technique is used to extract the subspaces, including the global subspace for all the classes and the local subspace for each class. Three hyperspectral data sets were tested and confirm that SSDNA performs better than the other supervised or semisupervised classifiers.

LRSR: Low-Rank-Sparse representation for subspace clustering

High-dimensional data in the real world often resides in low-dimensional subspaces. The state-of-the-art methods for subspace segmentation include Low Rank Representation (LRR) and Sparse Representation (SR). The former seeks the global lowest rank representation but restrictively assumes the independence among subspaces, whereas the latter seeks the clustering of disjoint or overlapped subspaces through locality measure, which may cause failure in the case of large noises. To this end, a Low Rank subspace Sparse Representation framework, hereafter referred to as LRSR, is proposed in this paper to recover and segment embedding subspaces simultaneously. Three major contributions can be claimed in this paper: First, a clean dictionary is constructed by optimizing its nuclear norm, low-rank-sparse coefficient matrix obtained using linearized alternating direction method (LADM). Second, both the convergence proof and the complexity analysis are given to prove the effectiveness and efficiency of our proposed LRSR algorithm. Third, the experiments on synthetic data and two benchmark datasets further verify that the LRSR enjoys the capability of clustering disjoint subspaces as well as the robustness against large noises, thanks to its considerations of both global and local subspace information. Therefore, it has been demonstrated in this research that our proposed LRSR algorithm outperforms the state-of-the-art subspace clustering methods, verified by both theoretical analysis and the empirical studies.

Local subspace smoothness alignment for constrained local model fitting

Constrained local model (CLM) is a classic method for facial landmarks estimation. While the CLM enhances the well-known active shape model with discriminative local appearance models, its shape model is based on the point distribution model, which is essentially principal component analysis over the training facial shape vectors and hence the nonlinear manifold of facial shapes is not well embedded. In this paper, we propose a novel manifold learning method, i.e., local subspace smoothness alignment (LSSA), to address this issue. The LSSA approach smoothes the nonlinear structure directly in the original feature space, with a newly defined geometric measure for the curvature of the local structures. We then proceed to apply this method for face alignment, with an ensemble of correlated local subspaces derived from LSSA. The proposed method is demonstrated on both toy data and real-world datasets that it yields reasonable manifold embedding and leads to encouraging performance for face alignment even under difficult conditions.

Non-linear shape optimization using local subspace projections

In this paper we present a novel method for non-linear shape optimization of 3d objects given by their surface representation. Our method takes advantage of the fact that various shape properties of interest give rise to underdetermined design spaces implying the existence of many good solutions. Our algorithm exploits this by performing iterative projections of the problem to local subspaces where it can be solved much more efficiently using standard numerical routines. We demonstrate how this approach can be utilized for various shape optimization tasks using different shape parameterizations. In particular, we show how to efficiently optimize natural frequencies, mass properties, as well as the structural yield strength of a solid body. Our method is flexible, easy to implement, and very fast.

Localized model order reduction in porous media flow simulation

This paper introduces a new approach to construct an efficient reduced order model for fluid flow simulation and optimization in porous media. For nonlinear systems, one of the most common methodology used is the proper orthogonal decomposition (POD) combined with discrete empirical interpolation method (DEIM) due to its computational efficiency and good approximation. Whereas regular POD-DEIM approximates the fine scale model with just one single reduced subspace, the localized POD (LPOD) and localized DEIM (LDEIM) are constructed by computing several local subspaces. Each subspace characterize a period of solutions in time and all together they not only can approximate the high fidelity model better, but also can reduce the computational cost of simulation. LPOD and LDEIM use classification approach to find these regions in the offline computational phase. After obtaining each class, POD and DEIM is applied to construct the basis vectors of the reduced subspace. In the online phase, at each time step, the reduced states and nonlinear functions will be used to find the most representative basis vectors for POD and DEIM without requiring fine scale information. The advantages of LPOD and LDEIM are shown in a numerical example of two phase flow in porous media.

Image segmentation by adaptive nonconvex local and global subspace representation

We formulate image segmentation as subspace clustering of image feature vectors. We propose a subspace representation model using a nonconvex extension of trace Lasso and a nonconvex approximation of rank function to regularize the subspace representation. The proposed model can adaptively capture the local and the global structures of the subspace representation so that the subspace representation can reveal the real subspace structure of the data and obtains excellent clustering performance. Experimental results show that the proposed model is better than the previous models in clustering and natural image segmentation.

Low-rank plus sparse decomposition for exoplanet detection in direct-imaging ADI sequences

Data processing constitutes a critical component of high-contrast exoplanet imaging. Its role is almost as important as the choice of a coronagraph or a wavefront control system, and it is intertwined with the chosen observing strategy. Among the data processing techniques for angular differential imaging (ADI), the most recent is the family of principal component analysis (PCA) based algorithms. PCA serves, in this case, as a subspace projection technique for constructing a reference point spread function (PSF) that can be subtracted from the science data for boosting the detectability of potential companions present in the data. Unfortunately, when building this reference PSF from the science data itself, PCA comes with certain limitations such as the sensitivity of the lower dimensional orthogonal subspace to non-Gaussian noise. Inspired by recent advances in machine learning algorithms such as robust PCA, we aim to propose a localized subspace projection technique that surpasses current PCA-based post-processing algorithms in terms of the detectability of companions at near real-time speed, a quality that will be useful for future direct imaging surveys. We used randomized low-rank approximation methods recently proposed in the machine learning literature, coupled with entry-wise thresholding to decompose an ADI image sequence locally into low-rank, sparse, and Gaussian noise components (LLSG). This local three-term decomposition separates the starlight and the associated speckle noise from the planetary signal, which mostly remains in the sparse term. We tested the performance of our new algorithm on a long ADI sequence obtained on beta Pictoris with VLT/NACO. Compared to a standard PCA approach, LLSG decomposition reaches a higher signal-to-noise ratio and has an overall better performance in the receiver operating characteristic space. (abridged).

PEBL-ROM: Projection-error based local reduced-order models

Projection-based model order reduction (MOR) using local subspaces is becoming an increasingly important topic in the context of the fast simulation of complex nonlinear models. Most approaches rely on multiple local spaces constructed using parameter, time or state-space partitioning. State-space partitioning is usually based on Euclidean distances. This work highlights the fact that the Euclidean distance is suboptimal and that local MOR procedures can be improved by the use of a metric directly related to the projections underlying the reduction. More specifically, scale-invariances of the underlying model can be captured by the use of a true projection error as a dissimilarity criterion instead of the Euclidean distance. The capability of the proposed approach to construct local and compact reduced subspaces is illustrated by approximation experiments of several data sets and by the model reduction of two nonlinear systems.

Evaluating the utility of mid-infrared spectral subspaces for predicting soil properties

We propose four methods for finding local subspaces in large spectral libraries. The proposed four methods include (a) cosine angle spectral matching; (b) hit quality index spectral matching; (c) self-organizing maps and (d) archetypal analysis methods. Then evaluate prediction accuracies for global and subspaces calibration models. These methods were tested on a mid-infrared spectral library containing 1907 soil samples collected from 19 different countries under the Africa Soil Information Service project. Calibration models for pH, Mehlich-3 Ca, Mehlich-3 Al, total carbon and clay soil properties were developed for the whole library and for the subspace. Root mean square error of prediction was used to evaluate predictive performance of subspace and global models. The root mean square error of prediction was computed using a one-third-holdout validation set. Effect of pretreating spectra with different methods was tested for 1st and 2nd derivative Savitzky–Golay algorithm, multiplicative scatter correction, standard normal variate and standard normal variate followed by detrending methods. In summary, the results show that global models outperformed the subspace models. We, therefore, conclude that global models are more accurate than the local models except in few cases. For instance, sand and clay root mean square error values from local models from archetypal analysis method were 50% poorer than the global models except for subspace models obtained using multiplicative scatter corrected spectra with which were 12% better. However, the subspace approach provides novel methods for discovering data pattern that may exist in large spectral libraries.

Spectral based Discontinuous Galerkin Reduced Basis Element method for parametrized Stokes problems

In this work we extend to the Stokes problem the Discontinuous Galerkin Reduced Basis Element (DGRBE) method introduced in Antonietti et al. (2015). By this method we aim at reducing the computational cost for the approximation of a parametrized Stokes problem on a domain partitioned into subdomains. During an offline stage, expensive but performed only once, a low-dimensional approximation space is built on each subdomain. For any new value of the parameter, the rapid evaluation of the solution takes place during the online stage and consists in a Galerkin projection onto the low-dimensional subspaces computed offline. The high-fidelity discretization on each subdomain, used to build the local low-dimensional subspaces, is based on spectral element methods. The continuity of both the velocity and the normal component of the Cauchy stress tensor at subdomain interfaces is weakly enforced by a discontinuous Galerkin approach.

Image Classification by Selective Regularized Subspace Learning

Feature learning is an intensively studied research topic in image classification. Although existing methods like sparse coding, locality-constrained linear coding, fisher vector encoding, etc., have shown their effectiveness in image representation, most of them overlook a phenomenon called thesmall sample size problem, where the number of training samples is relatively smaller than the dimensionality of the features, which may limit the predictive power of the classifier. Subspace learning is a strategy to mitigate this problem by reducing the dimensionality of the features. However, most conventional subspace learning methods attempt to learn a global subspace to discriminate all the classes, which proves to be difficult and ineffective in multi-class classification task. To this end, we propose to learn a local subspace for each sample instead of learning a global subspace for all samples. Our key observation is that, in multi-class image classification, the label of each testing sample is only confused by a few classes which have very similar visual appearance to it. Thus, in this work, we propose a coarse-to-fine strategy, which first picks out such classes, and then conducts a local subspace learning to discriminate them. As the subspace learning method is regularized and conducted within some selected classes, we term it selective regularized subspace learning (SRSL), and we term our classification pipeline selective regularized subspace learning based multi-class image classification (SRSL_MIC). Experimental results on four representative datasets (Caltech-101, Indoor-67, ORL Faces and AR Faces) demonstrate the effectiveness of the proposed method.