Low-dimensional Subspace Research Articles

Self-adaptive subspace representation from a geometric intuition

In this paper, we address the unsupervised subspace learning task from a geometric intuition, which offers a direct understanding of the data representation. First, we consider all subspaces of the Euclidean space as a series of graded Grassmannian manifolds, and represent them by orbits of rotation group action. Then, we reformulate the unsupervised subspace learning by a least square problem with respect to rotation and projection operator. Second, we introduce a low-rank regularization to obtain a low-dimensional and robust subspace representation. Then, the model is translated into a minimization problem on the Grassmannian manifold. By dividing the model into two subproblems of solving the optimal rotation and subspace dimension, we design an alternating iteration strategy, where the locally geodesic structure of the rotation group and the unconstrained quadratic 0-1 programming are utilized. Finally, we apply the proposed method to the image classification problem on five benchmark datasets and compare it with nine state-of-the-art methods. Numerical results show that our proposed method has better feature representation ability and almost achieves the best performance in terms of classification accuracy and robustness.

Unsupervised feature selection with high-order similarity learning

Graph-based unsupervised feature selection methods have successfully processed high-dimensional data since they can effectively preserve data structure information. However, most existing methods are facing two challenges: (1) it is unreliable to exploit the intrinsic spatial structure from raw data because of the existence of redundant information and noise; (2) they only consider the first-order similar information but ignore the high-order similar information (i.e., samples with similar neighborhood network structure should be similar to each other). This study proposes an unsupervised feature selection with high-order similarity learning (HSL) to tackle the above problems. The projection matrix, first-order similar information, and high-order similar information are simultaneously learned in a unified framework, such that the intrinsic structure information can be exploited in the clean data that come from a latent low-dimensional embedding subspace. Specifically, we project raw data into a low-dimensional embedding subspace to effectively eliminate redundant information and noise. Moreover, we simultaneously consider the first-order similarity and the high-order similarity in the low-dimensional embedding space so that a better graph Laplacian matrix can be learned to fully preserve the intrinsic structure information of raw data and simultaneously utilize the label information. Furthermore, we design an effective optimization algorithm to tackle the proposed HSL. Comprehensive experiments on real-world datasets verify the superiority and effectiveness of HSL.

Sensory input to cortex encoded on low-dimensional periphery-correlated subspaces.

As information about the world is conveyed from the sensory periphery to central neural circuits, it mixes with complex ongoing cortical activity. How do neural populations keep track of sensory signals, separating them from noisy ongoing activity? Here, we show that sensory signals are encoded more reliably in certain low-dimensional subspaces. These coding subspaces are defined by correlations between neural activity in the primary sensory cortex and upstream sensory brain regions; the most correlated dimensions were best for decoding. We analytically show that these correlation-based coding subspaces improve, reaching optimal limits (without an ideal observer), as noise correlations between cortex and upstream regions are reduced. We show that this principle generalizes across diverse sensory stimuli in the olfactory system and the visual system of awake mice. Our results demonstrate an algorithm the cortex may use to multiplex different functions, processing sensory input in low-dimensional subspaces separate from other ongoing functions.

Non-negative Tucker decomposition with graph regularization and smooth constraint for clustering

Non-negative Tucker decomposition (NTD) and its graph regularized extensions are the most popular techniques for representing high-dimensional non-negative data, which are typically found in a low-dimensional sub-manifold of ambient space, from a geometric perspective. Therefore, the performance of the graph-based NTD methods relies heavily on the low-dimensional representation of the original data. However, most existing approaches treat the last factor matrix in NTD as a low-dimensional representation of the original data. This treatment leads to the loss of the original data’s multi-linear structure in the low-dimensional subspace. To remedy this defect, we propose a novel graph regularized Lp smooth NTD (GSNTD) method for high-dimensional data representation by incorporating graph regularization and an Lp smoothing constraint into NTD. The new graph regularization term constructed by the product of the core tensor and the last factor matrix in NTD, and it is used to uncover hidden semantics while maintaining the intrinsic multi-linear geometric structure of the data. The addition of the Lp smoothing constraint to NTD may produce a more accurate and smoother solution to the optimization problem. The update rules and the convergence of the GSNTD method are proposed. In addition, a randomized variant of the GSNTD algorithm based on fiber sampling is proposed. Finally, the experimental results on four standard image databases show that the proposed method and its randomized variant have better performance than some other state-of-the-art graph-based regularization methods for image clustering.

Persistent Homology-Based Classification of Chaotic Multi-variate Time Series: Application to Electroencephalograms

In this work, we present a combination of dimension reduction techniques and persistent homology for detection of epileptic events in electroencephalograms for a special kind of epilepsy called petit-mal epilepsy. Persistent homology, one of the main methods in topological data analysis, extracts information about the structures appearing in a given data set. Since during an epileptic seizure of the above type the electrical brain activity is more synchronized, we take the resulting structure in the EEG signal as classification feature that is analyzed topologically by means of persistent homology. As preprocessing step, the dimension of the data is reduced by two alternative techniques, principal component analysis and dynamical component analysis, and their performance is compared. Our results show that in comparison to principal component analysis, dynamical component analysis captures the dynamics of the system when projected onto a low-dimensional subspace. Furthermore, the results prove that persistent homology is well-suited for the detection of petit-mal epileptic seizures by means of their inherent structure.

Dual Discriminative Low-Rank Projection Learning for Robust Image Classification

Numerous methods have exploited projection learning to extract low-dimensional features for image classification. Some projection learning methods integrate low-rank matrix recovery into classification models to equip the projection subspace with discrimination and robustness against corruption. However, these methods cannot directly recover “clean” components from the new datum in a low-dimensional subspace. Additionally, they are sensitive to the selection of projection dimensions. To overcome these shortcomings, we propose a dual discriminative low-rank projection learning framework for robust image classification. Specifically, the proposed method learns a low-rank projection and a semi-orthogonal projection to recover “clean” components from the original data and simultaneously obtain a low-dimensional subspace. Thereafter, to preserve discriminative information in the low-dimensional subspace, an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm term is constructed by concentrating the projected intra-class samples around their adaptive class centroids. Regression-based terms are appended using the low-dimensional features extracted from the recovered clean data and the class centroids for more accurate classification. Experiments on five public databases with various corruptions demonstrate that the proposed method can robustly classify image data despite a small training sample sizes and gross corruption. The superiority of the proposed method is further verified on the large-scale PubFig83 database, on which it achieves an 87.58% classification accuracy.

Modular CSI Quantization for FDD Massive MIMO Communication

We consider high-dimensional MIMO transmissions in frequency division duplexing (FDD) systems. For precoding, the frequency selective channel has to be measured, quantized and fed back to the base station by the users. When the number of antennas is very high this typically leads to prohibitively high quantization complexity and large feedback. In 5G New Radio (NR), a modular quantization approach has been applied for this, where first a low-dimensional subspace is identified for the whole frequency selective channel, and then subband channels are linearly mapped to this subspace and quantized. We analyze how the components in such a modular scheme contribute to the overall quantization distortion. Based on this analysis we improve the technology components in the modular approach and propose an orthonormalized wideband precoding scheme and a sequential wideband precoding approach which provide considerable gains over the conventional method. We compare the performance of the developed quantization schemes to prior art by simulations in terms of the projection distortion, overall distortion and spectral efficiency, in a scenario with a realistic spatial channel model.

Breaking the curse of dimensional collapse in graph contrastive learning: A whitening perspective

Dimensional collapse in graph contrastive learning (GCL) confines node embeddings to their lower-dimensional subspace, diminishing their distinguishability. However, the causes and solutions of this curse remain relatively underexplored. In statistics, whitening presents a powerful tool to eliminate correlations among multiple variables. This motivates us to relieve the dimensional collapse of GCL from a whitening perspective. In this paper, we propose an intuitive analysis suggesting that high similarity scores of node embeddings may cause dimensional collapse, providing more evidence for its presence. Considering the success of whitening in statistics, we introduce a new plug-and-play module called the ▪hitening ▪raph ▪ontrastive ▪earning (WGCL) to address the dimensional collapse issue in existing GCL methods. WGCL plugin standardises the covariance matrices of dimensions, eliminating correlations among node embeddings' dimensions. Additionally, we enhance the conventional GCL training objective by introducing a mutual information maximisation loss between input features and node embeddings to maintain information capacity. Our experiments demonstrate that WGCL effectively addresses dimensional collapse, leading to an average improvement of 0.93% (up to 2.0%) in classification accuracy across three GCL backbones on nine widely-used datasets. The code to reproduce the experiments is available at https://github.com/acboyty/WGCL.

Comparison of synergy patterns between the right and left hand while performing postures and object grasps

The human hand, with many degrees of freedom, serves as an excellent tool for dexterous manipulation. Previous research has demonstrated that there exists a lower-dimensional subspace that synergistically controls the full hand kinematics. The elements of this subspace, also called synergies, have been viewed as the strategy developed by the CNS in the control of finger movements. Considering that the control of fingers is lateralized to the contralateral hemisphere, how the synergies differ for the control of the dominant and the non-dominant hand has not been widely addressed. In this paper, hand kinematics was recorded using electromagnetic tracking system sensors as participants made various postures and object grasps with their dominant hand and non-dominant hand separately. Synergies that explain 90% of variance in data of both hands were analyzed for similarity at the individual level as well as at the population level. The results showed no differences in synergies between the hands at both these levels. PC scores and cross-reconstruction errors were analyzed to further support the prevalence of similarity between the synergies of the hands. Future work is proposed, and implications of the results to the treatment and diagnosis of neuromotor disorders are discussed.

Aspect-based sentiment score and star rating prediction for travel destination using Multinomial Logistic Regression with fuzzy domain ontology algorithm

Due to its significant advantages for downstream applications, such as recommender systems, In the context of travel and tourism, user reviews on TripAdvisor have an impact on other travellers' judgments regarding a variety of travel-related issues, including the choice of a vacation spot, lodging, and places to visit. In graded user reviews, the model must specifically forecast the user's review score after receiving the textual review. The purpose of this article is to present a predictive outline for aspect-based extraction and classification that can estimate the users' optimal travel destination. This impression helps travellers in a variety of ways, such as by recommending better locations that also include expensive destinations. The underlying classification algorithm's processing time is significantly impacted by more dimensions. The term “curse of dimensionality” is often used in statistics and machine learning to describe these issues. By projecting the high-dimensional input data into the low-dimensional subspace while roughly maintaining the distance between the data points with a higher probability, the Random Projection (RP) ensemble classifier decreases the complexity of multivariate data. Extract the crucial information from the reviews, then use Glove word vector representation to categorise the relevant emotions. Further, the article proposed Multinomial Logistic Regression (MNLR) with a Fuzzy Domain Ontology (FDO) algorithm for aspect-based sentiment analysis. More intricate aspects than just the products themselves impact how satisfied people are with tourist destinations. The most important factor in evaluating how convenient a tourist route will be is typically the weather. The combined form of predicted sentiment score, start ratings and environment factor has to be calculated to predict the travel destination based on the measurement of personalized search results. The simulation was processed in Python software. The presented work has utilized some performance measures to evaluate the classification model such as F1-score, Recall, Precision, Mean Absolute Error (MAE), Mean Squared Error (MSE), Cohen Score and Matthew Score. The accuracy with GloVe word vector representation was 90% and after the GloVe representation, the classification model accuracy was about 94%. The proposed strategy outperforms in terms of classification accuracy, according to simulated results and analyses of real-world data.

Numerical homogenization of spatial network models

We present and analyze a methodology for numerical homogenization of spatial networks models, e.g. heat conduction and linear deformation in large networks of slender objects, such as paper fibers. The aim is to construct a coarse model of the problem that maintains high accuracy also on the micro-scale. By solving decoupled problems on local subgraphs we construct a low dimensional subspace of the solution space with good approximation properties. The coarse model of the network is expressed by a Galerkin formulation and can be used to perform simulations with different source and boundary data, at a low computational cost. We prove optimal convergence to the micro-scale solution of the proposed method under mild assumptions on the homogeneity, connectivity, and locality of the network on the coarse scale. The theoretical findings are numerically confirmed for both scalar-valued (heat conduction) and vector-valued (linear deformation) models.

Predicting Arrhythmia Based on Machine Learning Using Improved Harris Hawk Algorithm

Arrhythmia disease is widely recognized as a prominent and lethal ailment on a global scale, resulting in a significant number of fatalities annually. The timely identification of this ailment is crucial for preserving individuals' lives. Machine Learning (ML), a branch of artificial intelligence (AI), has emerged as a highly efficient and cost-effective method for illness detection. The objective of this work is to develop a machine learning (ML) model capable of accurately predicting heart illness by using the Arrhythmia disease dataset, with the purpose of achieving optimal performance. The performance of the model is greatly influenced by the selection of the machine learning method and the features in the dataset for training purposes. In order to mitigate the issue of overfitting caused by the high dimensionality of the features in the Arrhythmia dataset, a reduction of the dataset to a lower dimensional subspace was performed via the improved Harris hawk optimization algorithm (iHHO). The Harris hawk algorithm exhibits a rapid convergence rate and possesses a notable degree of adaptability in its ability to identify optimal characteristics. The performance of the models created with the feature-selected dataset using various machine learning techniques was evaluated and compared. In this work, total seven classifiers like SVM, GB, GNB, RF, LR, DT, and KNN are used to classify the data produced by the iHHO algorithm. The results clearly show the improvement of 3%, 4%, 4%, 9%, 8%, 3%, and 9% with the classifiers KNN, RF, GB, SVM, LR, DT, and GNB respectively.

Heterogeneous Regularization-Based Tensor Subspace Clustering for Hyperspectral Band Selection.

Band selection (BS) reduces effectively the spectral dimension of a hyperspectral image (HSI) by selecting relatively few representative bands, which allows efficient processing in subsequent tasks. Existing unsupervised BS methods based on subspace clustering are built on matrix-based models, where each band is reshaped as a vector. They encode the correlation of data only in the spectral mode (dimension) and neglect strong correlations between different modes, i.e., spatial modes and spectral mode. Another issue is that the subspace representation of bands is performed in the raw data space, where the dimension is often excessively high, resulting in a less efficient and less robust performance. To address these issues, in this article, we propose a tensor-based subspace clustering model for hyperspectral BS. Our model is developed on the well-known Tucker decomposition. The three factor matrices and a core tensor in our model encode jointly the multimode correlations of HSI, avoiding effectively to destroy the tensor structure and information loss. In addition, we propose well-motivated heterogeneous regularizations (HRs) on the factor matrices by taking into account the important local and global properties of HSI along three dimensions, which facilitates the learning of the intrinsic cluster structure of bands in the low-dimensional subspaces. Instead of learning the correlations of bands in the original domain, a common way for the matrix-based models, our model learns naturally the band correlations in a low-dimensional latent feature space, which is derived by the projections of two factor matrices associated with spatial dimensions, leading to a computationally efficient model. More importantly, the latent feature space is learned in a unified framework. We also develop an efficient algorithm to solve the resulting model. Experimental results on benchmark datasets demonstrate that our model yields improved performance compared to the state-of-the-art.

A comprehensive survey of sparse regularization: Fundamental, state-of-the-art methodologies and applications on fault diagnosis

Sparse regularization has been attracting much attention in industrial applications over the past few decades. By exploiting the latent data structure in low-dimensional subspaces, a significant amount of research achievements have been realized in signal/image processing, pattern recognition and system identification, etc. However, very few systematic review or comprehensive survey are reported for sparse regularization including fundamentals, state-of-the-art methodologies, and applications on fault diagnosis. To fill this gap, this article conducts an in-depth review of the state-of-the-art technologies of sparse regularization, and the R&D of sparse regularization applied to fault diagnosis will also be summarized. Specifically, we discuss the rationales of cause formulation, algorithm idea, algorithm merits, algorithm demerits and computing techniques for each category. The availability and practicability of several representative models of sparse regularization are investigated with real-world experimental datasets. Finally, benefiting from theoretical developments of the sparse regularization, open/upcoming challenges, instructive perspectives, as well as possible future trends of the sparse regularization for prognostic and health management (PHM) are discussed.

Accelerating inverse inference of ensemble Kalman filter via reduced-order model trained using adaptive sparse observations

The implementation of data assimilation often struggles with low computational efficiency due to the complexity of high-fidelity numerical modeling and the large size of the observation vectors. To address this challenge, it is necessary to employ reduced-order strategies for both the full-order dynamical model and observation operator. Proper orthogonal decomposition (POD) based reduced-order model (ROM) and discrete empirical interpolation method (DEIM) based sparse observation identification are often considered as primary solutions. However, their effective combination requires the determination of high-quality POD reduced-order basis vectors. The current research focuses on incorporating the ROM and DEIM into a Kalman filter framework, resulting in a reduced-order Kalman filter with sparse observations. The key aspect is identifying the optimal POD basis, which can be achieved iteratively by introducing a relative root-mean-square-error (RMSE) for a near-optimal exploration of both the desired low-order POD basis and observation locations. The singular value decomposition (SVD) of the snapshots is performed in a low-dimensional subspace, leading to significant computational savings. This improved POD basis not only enhances the reconstruction of the ROM, but also helps build the DEIM observation matrix, resulting in simultaneous dimensionality reduction of the state and observation space. The effectiveness of this algorithm is demonstrated through the retrieval of initial conditions for one-dimensional tsunami wave equations with real data scenarios. Experiments are conducted for both smooth and nonsmoothed initial conditions. The results show that when the observation data is available at 3 and 6 spatial observation locations, our method only requires fewer vectors with the first 45 and 90 dominant modes derived using the RMSE, compared to as many as 210 and 252 derived from the relative energy criterion. Our method enables a low-modal ROM to effectively reconstruct the wave height field, leading to considerable prediction capability for potential long-wave dynamics with comparable accuracy. The CPU time savings achieved through our method is at least one to two orders of magnitude compared to the CPU time required by the full-order model.

Nonlinear characterization of enhanced and generalized Hjorth’s feature space for bearing condition monitoring

The degradation assessment of rolling bearings provides a reasonable maintenance plan for the safe operation of mechanical equipment. The general strategy for bearing condition monitoring is to construct a health indicator (HI) to characterize different degradation stages. A preferable HI that can sensitively detect initial faults and track machine degradation is crucial to developing timely maintenance strategies for mechanical equipment to avoid catastrophic accidents. However, many developed and reported HIs are still insensitive to early faults, resulting in delayed maintenance schedules. To identify the incipient defects as early as possible, a novel HI constructed by nonlinear characterization of enhanced and generalized Hjorth’s feature space based on extended probability entropy is proposed in this paper. Firstly, the time-frequency spectral amplitude modulation helps to enhance the characteristics of the original signal with the amplitude editing in the time-frequency domain. Then, three new features of generalized Hjorth’s parameter combinations are designed and combined with other similar feature combinations to construct a high-dimensional enhanced and generalized Hjorth’s feature space. On this basis, a set of low-dimensional sensitive features is obtained by nonlinearly characterizing high-dimensional features through extended probability entropy after these features are standardized. Finally, a novel HI is developed by calculating the distance between the minimum volume ellipse (MVE) center of the low-dimensional feature subspace based on nonlinear characterization and the low-dimensional feature vector of the real-time monitoring signal. The performance of the proposed approach is verified in three cases, whose experimental results indicate that the proposed HI is more sensitive and significant in detecting early faults compared to some current HIs.

Stokesian processes : inferring Stokes flows using physics-informed Gaussian processes

We develop a probabilistic Stokes flow framework, using physics informed Gaussian processes, which can be used to solve both forward/inverse flow problems with missing and/or noisy data. The physics of the problem, specified by the Stokes and continuity equations, is exactly encoded into the inference framework. Crucially, this means that we do not need to explicitly solve the Poisson equation for the pressure field, as a physically meaningful (divergence-free) velocity field will automatically be selected. We test our method on a simple pressure driven flow problem, i.e. flow through a sinusoidal channel, and compare against standard numerical methods (Finite Element and Direct Numerical Simulations). We obtain excellent agreement, even when solving inverse problems given only sub-sampled velocity data on low dimensional sub-spaces (i.e. 1 component of the velocity on 1D domains to reconstruct 2D flows). The proposed method will be a valuable tool for analyzing experimental data, where noisy/missing data is the norm.

P2S distance induced locally conjugated orthogonal subspace learning for feature extraction

When performing data classification tasks, it often occurs to them the curse of dimensionality problem. To address the issue, a manifold learning method termed locally conjugated orthogonal subspace (LCOS) is put forward for dimensionality reduction or feature extraction in this paper. Note that point to feature space (P2S) distance contributes to mining local geometry information, both a local margin characterizing data apartness and a locally conjugated orthogonal constraint beneficial to removing data redundancy are well studied from the P2S distance metric. They are all exploited to model the proposed LCOS. Then, a low dimensional subspace can be explored by maximizing the P2S distance induced local margin under the constraint. Compared with some other related dimensionality reduction methods, experimental results on benchmark face and object data sets validate the performance of the proposed method.

Coupled discriminative manifold alignment for low-resolution face recognition

In practical applications, due to a long distance between the monitored population and monitoring equipment, the face images or human pose captured by the cameras often incur low-resolution (LR), small size, and poor quality, which leads to extreme difficulty in directly matching an LR face with the high-resolution (HR) ones in the gallery. In this paper, we propose a novel coupled discriminative manifold alignment (CDMA) method for LR face recognition. Specifically, in the training stage the principal component analysis (PCA) is first used to reduce the dimensional gap between LR and HR facial features. Next the LR face images and the corresponding HR face images are converted into a common shared feature subspace by learning two linear mappings in a supervised manner, where the neighborhood samples within the same class and from different classes are jointly exploited to align the manifold structures of LR and HR faces. In the test stage, for a given LR face in the probe set, two learned coupled mappings (CMs) are applied to match the HR images in the gallery set through the correlative metric. Thorough experimental results on three representative face databases verify the effectiveness of the proposed method in comparing with other state-of-the-art competitors. In particular, the proposed method is capable of yielding more competitive recognition performance than other predecessors when lower dimensional feature subspaces are applied to match the expected HR faces.

Cross-study analyses of microbial abundance using generalized common factor methods

BackgroundBy creating networks of biochemical pathways, communities of micro-organisms are able to modulate the properties of their environment and even the metabolic processes within their hosts. Next-generation high-throughput sequencing has led to a new frontier in microbial ecology, promising the ability to leverage the microbiome to make crucial advancements in the environmental and biomedical sciences. However, this is challenging, as genomic data are high-dimensional, sparse, and noisy. Much of this noise reflects the exact conditions under which sequencing took place, and is so significant that it limits consensus-based validation of study results.ResultsWe propose an ensemble approach for cross-study exploratory analyses of microbial abundance data in which we first estimate the variance-covariance matrix of the underlying abundances from each dataset on the log scale assuming Poisson sampling, and subsequently model these covariances jointly so as to find a shared low-dimensional subspace of the feature space.ConclusionsBy viewing the projection of the latent true abundances onto this common structure, the variation is pared down to that which is shared among all datasets, and is likely to reflect more generalizable biological signal than can be inferred from individual datasets. We investigate several ways of achieving this, demonstrate that they work well on simulated and real metagenomic data in terms of signal retention and interpretability, and recommend a particular implementation.