Local Subspace Research Articles

A Riemannian gossip approach to subspace learning on Grassmann manifold

In this paper, we focus on subspace learning problems on the Grassmann manifold. Interesting applications in this setting include low-rank matrix completion and low-dimensional multivariate regression, among others. Motivated by privacy concerns, we aim to solve such problems in a decentralized setting where multiple agents have access to (and solve) only a part of the whole optimization problem. The agents communicate with each other to arrive at a consensus, i.e., agree on a common quantity, via the gossip protocol. We propose a novel cost function for subspace learning on the Grassmann manifold, which is a weighted sum of several sub-problems (each solved by an agent) and the communication cost among the agents. The cost function has a finite-sum structure. In the proposed modeling approach, different agents learn individual local subspaces but they achieve asymptotic consensus on the global learned subspace. The approach is scalable and parallelizable. Numerical experiments show the efficacy of the proposed decentralized algorithms on various matrix completion and multivariate regression benchmarks.

Meeting the Challenge of Magnetic Coupling in a Triply-Bridged Chromium Dimer: Complementary Broken-Symmetry Density Functional Theory and Multireference Density Matrix Renormalization Group Perspectives.

Face-sharing octahedral dinuclear Cr(III) compounds with d3–d3 electronic configurations represent nontrivial examples of electronic complexity, posing particular challenges for theoretical and computational studies. A tris-hydroxy-bridged Cr(III)–Cr(III) system has proven to be a richly rewarding target for studies of magnetism and electron paramagnetic resonance spectroscopy. It was also reported to be a peculiarly difficult system to treat with density functional theory (DFT). In this work the magnetic coupling problem for this dimer is approached with broken-symmetry (BS)-DFT and multireference calculations that utilize the density matrix renormalization group (DMRG) to handle full-valence active spaces. BS-DFT is shown to recover the correct ordering and energy spacing of Heisenberg spin states if used in conjunction with appropriate spin projection procedures, albeit with pronounced functional sensitivity. The contrasting conclusions of previous studies are traced to incorrect inclusion of electronically excited configurations. Analysis of the direct and differential overlap of corresponding orbital pairs from the BS-DFT solution indicates that metal–metal through-space interaction is the dominant contributor to antiferromagnetic coupling. At the DFT level a procedure that utilizes pseudopotential substitution is demonstrated that allows evaluation of the direct exchange vs superexchange contributions. A complementary description is obtained with DMRG-SCF calculations that enable state-averaged CASSCF calculations with both metal and bridge orbitals in the active space. A localized orbital subspace analysis supports the DFT conclusions that in contrast to doubly bridged isoelectronic analogues, antiferromagnetic coupling in the chromium dimer arises primarily from direct metal–metal interaction but is significantly enhanced by ligand-mediated superexchange.

Object Tracking Using Sparsity Based Discriminative Minimum-Spanning-Tree

An object tracking method based on a minimum-spanning-tree one-class classifier is proposed where the background is regarded as a collection of all other classes except the target. First, two kinds of different target class examples and outlier examples are collected in the recent few frames. Meanwhile, the candidates are sampled by a particle filter in the current frame. Second, using the sparsity-based local discriminant analysis, all examples and candidates are projected into a local discriminative subspace to improve the discriminative ability of one-class classifier. Then, a graph is built on all the target examples by adopting one-to-all sparse reconstruction coefficients rather than pair-wise Euclidean distance and a minimum-spanning-tree one-class classifier is trained on the graph. Finally, the similarity scores of the candidates are evaluated by the trained classifier. The one with the highest score is determined as the tracking result and used to update the training set. The experimental results demonstrate the superior performance of our method in robustness and accuracy to the state-of-the-art methods.

Personal Identification Using a Robust Eigen ECG Network Based on Time-Frequency Representations of ECG Signals

This paper is concerned with personal identification using a robust EigenECG network (REECGNet) based on time-frequency representations of electrocardiogram (ECG) signals. For this purpose, we use a robust principal component analysis network (RPCANet) and wavelet analysis. In general, PCA performance and applicability in real case scenarios is limited by the lack of robustness to outliers and corrupted observations. However, in a real nonstationary ECG noise environment, RPCA shows good performance when the method is applied with variable dimensions of local signal subspaces. That is why RPCA-based ECG identification is extremely robust with nonlinear data. Also, a REECGNet performs well without back-propagation to obtain features from the visual content. We constructed a Chosun University ECG Database (CU-ECG DB) and compared with the Physikalisch-Technische Bundesanstalt ECG database (PTB-ECG DB), which is shared data. Finally, the experimental results show the advantages and effectiveness of the applied recognition scheme with 98.25% performance. In addition, to demonstrate the superiority of REECGNet, we experimented with adding noise and the experimental result showed 97.5% recognition rate.

Low-Complexity Off-Grid STAP Algorithm Based on Local Search Clutter Subspace Estimation

Space–time adaptive processing (STAP) based on sparse recovery (SR-STAP) techniques exhibits significantly better performance than conventional STAP algorithms within a very small number of snapshots. However, when the clutter patches do not locate exactly on the discrete space–time grid points, the performances of SR-STAP algorithms degrade severely. In this letter, a low-complexity off-grid STAP algorithm based on local search clutter subspace estimation is proposed to overcome this issue. In the proposed algorithm, the global atoms are first selected from the reduced-dimension global STAP dictionary using the design selection criterion. Then, the optimal atoms are searched from the local STAP dictionary. Finally, these space–time steering vectors corresponding to the optimal atoms are used to construct the clutter subspace iteratively, and the STAP weight is obtained by projecting the snapshot on the subspace orthogonal to the clutter subspace. Numerical experiments with both simulated and Mountain-Top data are carried out to demonstrate the effectiveness of the proposed algorithm.

Arctic‐Wide Sea Ice Thickness Estimates From Combining Satellite Remote Sensing Data and a Dynamic Ice‐Ocean Model with Data Assimilation During the CryoSat‐2 Period

AbstractExploiting the complementary character of CryoSat‐2 and Soil Moisture and Ocean Salinity satellite sea ice thickness products, daily Arctic sea ice thickness estimates from October 2010 to December 2016 are generated by an Arctic regional ice‐ocean model with satellite thickness assimilated. The assimilation is performed by a Local Error Subspace Transform Kalman filter coded in the Parallel Data Assimilation Framework. The new estimates can be generally thought of as combined model and satellite thickness (CMST). It combines the skill of satellite thickness assimilation in the freezing season with the model skill in the melting season, when neither CryoSat‐2 nor Soil Moisture and Ocean Salinity sea ice thickness is available. Comparisons with in situ observations from the Beaufort Gyre Exploration Project, Ice Mass Balance Buoys, and the NASA Operation IceBridge demonstrate that CMST reproduces most of the observed temporal and spatial variations. Results also show that CMST compares favorably to the Pan‐Arctic Ice‐Ocean Modeling and Assimilation System product and even appears to correct known thickness biases in the Pan‐Arctic Ice‐Ocean Modeling and Assimilation System. Due to imperfect parameterizations in the sea ice model and satellite thickness retrievals, CMST does not reproduce the heavily deformed and ridged sea ice along the northern coast of the Canadian Arctic Archipelago and Greenland. With the new Arctic sea ice thickness estimates sea ice volume changes in recent years can be further assessed.

Student’s t-Hidden Markov Model for Unsupervised Learning Using Localized Feature Selection

Recently, the hidden Markov model (HMM) with student’s t-mixture model (SMM), called student’s t-HMM (SHMM) for short, has received much attention in unsupervised learning of sequential data. However, the current existing SHMMs fail to take into consideration of the relevant features embedded in local subspaces, thus influencing their performances in clustering. To address the problem, a novel SHMM is proposed by combining the measure of localized feature saliency (LFS) with SMM and utilizing two student’s t-distributions as subcomponents to respectively describe the distributions of useful features and non-salient “features,” with the purpose of accurately modeling the hidden state observation emission distributions of SHMM. Moreover, we exploit the variational Bayesian learning technique to simultaneously estimate the LFS, the number of components and other parameters of the herein proposed SHMM. Experimental results on both synthetic and real data sets demonstrate the improved robustness, effectiveness, and accuracy of our model.

Deep Localized Metric Learning

Metric learning has been widely used in many visual analysis applications, which learns new distance metrics to measure the similarities of samples effectively. Conventional metric learning methods learn a single linear Mahalanobis metric, yet such linear projections are not powerful enough to capture the nonlinear relationships. Recently, deep metric learning approaches, such as discriminative deep metric learning and deep transfer metric learning, have been introduced to fully exploit the nonlinearity of samples by learning hierarchical nonlinear transformations. However, these methods only learn holistic metrics over the input space and are limited for the heterogeneous data sets, where data varies locally. In this paper, we propose a deep localized metric learning approach for visual recognition by learning multiple fine-grained deep localized metrics. We first learn $K$ local subspaces and one holistic subspace with the K-auto-encoders-based clustering. Then, given an input pair, we compute its localized distance on each learned subspace and obtain the final distance representation. Finally, we train the entire neural networks to ensure the distances of positive pairs smaller than negative pairs by a large margin. Experimental results on three visual recognition applications, including face recognition, person re-identification, and scene recognition, show that our DLML outperforms most existing metric learning approaches.

3D Human Action Recognition Using a Single Depth Feature and Locality-Constrained Affine Subspace Coding

This paper addresses the problem of recognizing human actions from depth videos. We propose a depth-based local descriptor and affine subspace coding representation with locality-constrained affine subspace coding (LASC) for 3D action recognition. First, each depth video sequence is divided into a set of subsequences (i.e., multi-scale sub-actions) based on the normalized motion energy vector. Next, depth motion map-based gradient local auto-correlation features are employed to capture the shape information and motion cues of each sub-action. In order to obtain discriminative and compact representation, we extract the local high-order information of the depth video using LASC. Through experiments, we show that the use of LASC exhibits better performance compared with existing methods such as locality-constrained linear coding. We compared LASC with the state-of-the-art methods based on similar principle, using features extracted from a single modality, on four datasets, and with those using multiple features or nonlinear recognition machines. The results on four datasets clearly show the effectiveness of the proposed method.

Robust Dimension Reduction for Clustering With Local Adaptive Learning.

In pattern recognition and data mining, clustering is a classical technique to group matters of interest and has been widely employed to numerous applications. Among various clustering algorithms, K-means (KM) clustering is most popular for its simplicity and efficiency. However, with the rapid development of the social network, high-dimensional data are frequently generated, which poses a considerable challenge to the traditional KM clustering as the curse of dimensionality. In such scenarios, it is difficult to directly cluster such high-dimensional data that always contain redundant features and noises. Although the existing approaches try to solve this problem using joint subspace learning and KM clustering, there are still the following limitations: 1) the discriminative information in low-dimensional subspace is not well captured; 2) the intrinsic geometric information is seldom considered; and 3) the optimizing procedure of a discrete cluster indicator matrix is vulnerable to noises. In this paper, we propose a novel clustering model to cope with the above-mentioned challenges. Within the proposed model, discriminative information is adaptively explored by unifying local adaptive subspace learning and KM clustering. We extend the proposed model using a robust l2,1 -norm loss function, where the robust cluster centroid can be calculated in a weighted iterative procedure. We also explore and discuss the relationships between the proposed algorithm and several related studies. Extensive experiments on kinds of benchmark data sets demonstrate the advantage of the proposed model compared with the state-of-the-art clustering approaches.

Accelerated MR parameter mapping with a union of local subspaces constraint.

A new reconstruction method for multi-contrast imaging and parameter mapping based on a union of local subspaces constraint is presented. Subspace constrained reconstructions use a predetermined subspace to explicitly constrain the relaxation signals. The choice of subspace size impacts the approximation error vs noise-amplification tradeoff associated with these methods. A different approach is used in the model consistency constraint (MOCCO) framework to leverage the subspace model to enforce a softer penalty. Our proposed method, MOCCO-LS, augments the MOCCO model with a union of local subspaces (LS) approach. The union of local subspaces model is coupled with spatial support constraints and incorporated into the MOCCO framework to regularize the contrast signals in the scene. The performance of the MOCCO-LS method was evaluated in vivo on T1 and T2 mapping of the human brain and with Monte-Carlo simulations and compared against MOCCO and the explicit subspace constrained models. The results demonstrate a clear improvement in the multi-contrast images and parameter maps. We sweep across the model order space to compare the different reconstructions and demonstrate that the reconstructions have different preferential operating points. Experiments on T2 mapping show that the proposed method yields substantial improvements in performance even when operating at very high acceleration rates. The use of a union of local subspace constraints coupled with a sparsity promoting penalty leads to improved reconstruction quality of multi-contrast images and parameter maps.

Structured AutoEncoders for Subspace Clustering.

Existing subspace clustering methods typically employ shallow models to estimate underlying subspaces of unlabeled data points and cluster them into corresponding groups. However, due to the limited representative capacity of the employed shallow models, those methods may fail in handling realistic data without the linear subspace structure. To address this issue, we propose a novel subspace clustering approach by introducing a new deep model-Structured AutoEncoder (StructAE). The StructAE learns a set of explicit transformations to progressively map input data points into nonlinear latent spaces while preserving the local and global subspace structure. In particular, to preserve local structure, the StructAE learns representations for each data point by minimizing reconstruction error w.r.t. itself. To preserve global structure, the StructAE incorporates a prior structured information by encouraging the learned representation to preserve specified reconstruction patterns over the entire data set. To the best of our knowledge, StructAE is one of first deep subspace clustering approaches. Extensive experiments show that the proposed StructAE significantly outperforms 15 state-of-the-art subspace clustering approaches in terms of five evaluation metrics.

Local spectral subspace preservers

Let $${\mathcal {B}}(X)$$ be the algebra of all bounded linear operators on Banach space X. For $$T\in {\mathcal {B}}(X)$$ and $$\lambda \in \mathbb {C}$$ , let $$X_{T}(\{ \lambda \})$$ denotes the local spectral subspace of T associated with $$\{\lambda \}$$ . We determine the forms of mappings (not necessarily linear) $$\phi :{\mathcal {B}}(X)\rightarrow {{\mathcal {B}}(X)}$$ that preserve the local spectral subspace of either sum, product of operators or triple product of operators associated with a singleton.

MILFM: Multiple index latent factor model based on high-dimensional features.

The aim of this article is to develop a multiple-index latent factor modeling (MILFM) framework to build an accurate prediction model for clinical outcomes based on a massive number of features. We develop a three-stage estimation procedure to build the prediction model. MILFM uses an independent screening method to select a set of informative features, which may have a complex nonlinear relationship with the outcome variables. Moreover, we develop a latent factor model to project all informative predictors onto a small number of local subspaces, which lead to a few key features that capture reliable and informative covariate information. Finally, we fit the regularized empirical estimate to those key features in order to accurately predict clinical outcomes. We systematically investigate the theoretical properties of MILFM, such as risk bounds and selection consistency. Our simulation results and real data analysis show that MILFM outperforms many state-of-the-art methods in terms of prediction accuracy.

Multimode processes monitoring based on hierarchical mode division and subspace decomposition

AbstractTo handle multimodal uncertainty and dynamics which are common in actual industrial processes, this paper proposes a multimode process monitoring method based on hierarchical mode division and subspace decomposition. First, hierarchical mode division is presented to achieve hierarchical mode information for multimode process modelling, which can improve the adaptability of industrial processes with multimodal uncertainty. Then, hierarchical subspace decomposition is presented to decompose multimode processes into multiple global subspaces and multiple local subspaces for monitoring. Lastly, the most similar mode with various hierarchical modes at the current sampling point is dynamically determined for monitoring, which considers multimodal dynamics. The simulation tests of the penicillin fermentation processes are developed to show feasibility and superiority of the proposed method.

Locally Linear Embedding of Local Orthogonal Least Squares Images for Face Recognition

Dimensionality reduction is very important in face recognition since it ensures that high-dimensionality data can be mapped to lower dimensional space without losing salient and integral facial information. Locally Linear Embedding (LLE) has been previously used to serve this purpose, however, the process of acquiring LLE features requires high computation and resources. To overcome this limitation, we propose a locally-applied Local Orthogonal Least Squares (LOLS) model can be used as initial feature extraction before the application of LLE. By construction of least squares regression under orthogonal constraints we can preserve more discriminant information in the local subspace of facial features while reducing the overall features into a more compact form that we called LOLS images. LLE can then be applied on the LOLS images to maps its representation into a global coordinate system of much lower dimensionality. Several experiments carried out using publicly available face datasets such as AR, ORL, YaleB, and FERET under Single Sample Per Person (SSPP) constraint demonstrates that our proposed method can reduce the time required to compute LLE features while delivering better accuracy when compared to when either LLE or OLS alone is used. Comparison against several other feature extraction methods and more recent feature-learning method such as state-of-the-art Convolutional Neural Networks (CNN) also reveal the superiority of the proposed method under SSPP constraint.

Low and high frequency wavelet sub-band-based feature extraction

In a biometric system, feature extraction is an important task for faster and efficient identification of a person. A new feature extraction method, sub-band PCA+LDA is proposed to extract distinct features from low frequency and high frequency wavelet sub-bands. The proposed method captures both local and global features of two biometrics under consideration, face and iris. The matching scores of face and iris are normalised using minmax and tanh techniques, and fused using sum rule, product rule and weighted sum rule. For unimodal systems, the proposed method gives better recognition rate in comparison to other existing methods, like DWT, DWT+PCA, DWT+LDA, local binary pattern and subspace LDA. The performance of the proposed multimodal biometric system is superior to unimodal system in terms of attaining maximum of 100% recognition rate and minimum equal error rate (EER) of 0.017 for standard biometric databases.

Nonlinear maps preserving the local spectral subspace

Let X be a complex Banach space, and let be the space of all bounded operators on X. For and , let denote the local spectral subspace of T associated with . In this paper, we prove that if is a surjective weakly continuous map which satisfies for all and , there exists a nonzero scalar c such that for all . We do not assume to be linear, or even additive.

Dynamic Texture Comparison Using Derivative Sparse Representation: Application to Video-Based Face Recognition

Video-based face, expression, and scene recognition are fundamental problems in human–machine interaction, especially when there is a short-length video. In this paper, we present a new derivative sparse representation approach for face and texture recognition using short-length videos. First, it builds local linear subspaces of dynamic texture segments by computing spatiotemporal directional derivatives in a cylinder neighborhood within dynamic textures. Unlike traditional methods, a nonbinary texture coding technique is proposed to extract high-order derivatives using continuous circular and cylinder regions to avoid aliasing effects. Then, these local linear subspaces of texture segments are mapped onto a Grassmann manifold via sparse representation. A new joint sparse representation algorithm is developed to establish the correspondences of subspace points on the manifold for measuring the similarity between two dynamic textures. Extensive experiments on the Honda/UCSD, the CMU motion of body, the YouTube, and the DynTex datasets show that the proposed method consistently outperforms the state-of-the-art methods in dynamic texture recognition, and achieved the encouraging highest accuracy reported to date on the challenging YouTube face dataset. The encouraging experimental results show the effectiveness of the proposed method in video-based face recognition in human–machine system applications.

Impacts of Assimilating Satellite Sea Ice Concentration and Thickness on Arctic Sea Ice Prediction in the NCEP Climate Forecast System

Here sea ice concentration derived from the Special Sensor Microwave Imager/Sounder and thickness derived from the Soil Moisture and Ocean Salinity and CryoSat-2 satellites are assimilated in the National Centers for Environmental Prediction Climate Forecast System using a localized error subspace transform ensemble Kalman filter (LESTKF). Three ensemble-based hindcasts are conducted to examine impacts of the assimilation on Arctic sea ice prediction, including CTL (without any assimilation), LESTKF-1 (with initial sea ice assimilation only), and LESTKF-E5 (with every 5-day sea ice assimilation). Assessment with the assimilated satellite products and independent sea ice thickness datasets shows that assimilating sea ice concentration and thickness leads to improved Arctic sea ice prediction. LESTKF-1 improves sea ice forecast initially. The initial improvement gradually diminishes after ~3-week integration for sea ice extent but remains quite steady through the integration for sea ice thickness. Large biases in both the ice extent and thickness in CTL are remarkably reduced through the hindcast in LESTKF-E5. Additional numerical experiments suggest that the hindcast with sea ice thickness assimilation dramatically reduces systematic bias in the predicted ice thickness compared with sea ice concentration assimilation only or without any assimilation, which also benefits the prediction of sea ice extent and concentration due to their covariability. Hence, the corrected state of sea ice thickness would aid in the forecast procedure. Increasing the number of ensemble members or extending the integration period to generate estimates of initial model states and uncertainties seems to have small impacts on sea ice prediction relative to LESTKF-E5.