Low-rank Feature Research Articles

Stabilization of a matrix via a low-rank-adaptive ODE

Let A be a square matrix with a given structure (e.g. real matrix, sparsity pattern, Toeplitz structure, etc.) and assume that it is unstable, i.e. at least one of its eigenvalues lies in the complex right half-plane. The problem of stabilizing A consists in the computation of a matrix B, whose eigenvalues have all negative real part and such that the perturbation Δ=B-A\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Delta =B-A$$\\end{document} has minimal norm. The structured stabilization further requires that the perturbation preserves the structural pattern of A. This non-convex problem is solved by a two-level procedure which involves the computation of the stationary points of a matrix ODE. It is possible to exploit the underlying low-rank features of the problem by using an adaptive-rank integrator that follows rigidly the rank of the solution. Some benefits derived from the low-rank setting are shown in several numerical examples. These computational advantages also allow to deal with high dimensional problems.

Missing data recovery based on temporal smoothness and time-varying similarity for wireless sensor network

Wireless Sensor Networks (WSN) play a vital role in the Internet of Things (IoT) and show great potential in monitoring applications. However, due to harsh environmental conditions and unreliable communication links, WSN often encounter partial data loss during data collection, which inevitably affects the quality of service. To address this challenge, researchers have employed matrix completion techniques to recover missing data by exploiting the low-rank features in the data, but its accuracy is not satisfactory. This paper argues that the spatiotemporal characteristics of the data underlie its low-rank nature, enabling a more accurate capture of the intrinsic patterns within the data. Drawing on this insight, we propose a missing data recovery algorithm based on Temporal Smoothness and Time-Varying Similarity (TSTVS). Unlike traditional low-rank methods, the TSTVS algorithm directly utilizes the structural features of data in the spatiotemporal domain to establish a missing data completion model. Subsequently, the model is converted into an unconstrained optimization problem using the penalty function method, and the gradient descent method is applied to solve it, reconstructing the complete data matrix. Finally, simulation experiments were conducted on three real-world monitoring datasets, comparing the TSTVS with three low-rank methods, Efficient Data Collection Approach (EDCA), Matrix factorization with Smoothness constraints (MFS) and Data Recovery Based on Low Rank and Short-Term Stability(DRLRSS). The experimental results indicate that the proposed TSTVS algorithm consistently outperforms the three low-rank based algorithms in terms of recovery accuracy across different datasets and missing rate scenarios.

Adversarial client detection via non-parametric subspace monitoring in the internet of federated things

The Internet of Federated Things (IoFT) represents a network of interconnected systems with federated learning as the backbone, facilitating collaborative knowledge acquisition while ensuring data privacy for individual systems. The wide adoption of IoFT, however, is hindered by security concerns, particularly the susceptibility of federated learning networks to adversarial attacks. In this article, we propose an effective non-parametric approach FedRR, which leverages the low-rank features of the transmitted parameter updates generated by federated learning to address the adversarial attack problem. In addition, our proposed method is capable of accurately detecting adversarial clients and controlling the false alarm rate under the scenario with no attack occurring. Experiments based on digit recognition using the MNIST datasets validated the advantages of our approach.

Label generation with consistency on the graph for multi-label feature selection

Multi-label feature selection involves the selection of informative features in high-dimensional data sets based on the relationships among different variables. However, large-scale data sets often contain unknown labels that hold latent information, posing a challenge for discovery. Explicitly uncovering latent information among labels not only helps establish robust mappings between features and labels but also expands the applicable knowledge. This paper introduces a novel feature selection method called Label Generation with Consistency on the Graph for Multi-label Feature Selection (LGCM) to effectively integrate label generation and feature selection processes. The proposed method incorporates a two-way flipping mechanism that leverages the consistency on the global graph to guide label generation for each instance. Additionally, the feature selection model utilizes a shared low-rank feature space to minimize deviation from the label generation, providing feedback to the label generation process. Extensive experiments validate the effectiveness of LGCM in improving state-of-the-art feature selection methods in multi-label learning.

Towards Robust Subspace Clustering via Joint Feature Extraction and Cauchy Loss Function

The purpose of the subspace clustering approach is to discover the similarity between samples by learning a self-representation matrix, and it has been widely employed in machine learning and pattern recognition. Most existing subspace clustering techniques discover subspace structures from raw data and simply adopt L2 loss to characterize the reconstruction error. To break through these limitations, a novel robust model named Feature extraction and Cauchy loss function-based Subspace Clustering (FCSC) is proposed. FCSC performs low dimensional and low-rank feature extraction at the same time, as well as processing large noise in the data to generate a more ideal similarity matrix. Furthermore, we provide an efficient iterative strategy to solve the resultant problem. Extensive experiments on benchmark datasets confirm its superiority in the robustness of some advanced subspace clustering algorithms.

Bearing Damage Detection With Orthogonal and Nonnegative Low-Rank Feature Extraction

Bearing Damage Detection With Orthogonal and Nonnegative Low-Rank Feature Extraction

Green Space Reverse Pixel Shuffle Network: Urban Green Space Segmentation Using Reverse Pixel Shuffle for Down-Sampling from High-Resolution Remote Sensing Images

Urban green spaces (UGS) play a crucial role in the urban environmental system by aiding in mitigating the urban heat island effect, promoting sustainable urban development, and ensuring the physical and mental well-being of residents. The utilization of remote sensing imagery enables the real-time surveying and mapping of UGS. By analyzing the spatial distribution and spectral information of a UGS, it can be found that the UGS constitutes a kind of low-rank feature. Thus, the accuracy of the UGS segmentation model is not heavily dependent on the depth of neural networks. On the contrary, emphasizing the preservation of more surface texture features and color information contributes significantly to enhancing the model’s segmentation accuracy. In this paper, we proposed a UGS segmentation model, which was specifically designed according to the unique characteristics of a UGS, named the Green Space Reverse Pixel Shuffle Network (GSRPnet). GSRPnet is a straightforward but effective model, which uses an improved RPS-ResNet as the feature extraction backbone network to enhance its ability to extract UGS features. Experiments conducted on GaoFen-2 remote sensing imagery and the Wuhan Dense Labeling Dataset (WHDLD) demonstrate that, in comparison with other methods, GSRPnet achieves superior results in terms of precision, F1-score, intersection over union, and overall accuracy. It demonstrates smoother edge performance in UGS border regions and excels at identifying discrete small-scale UGS. Meanwhile, the ablation experiments validated the correctness of the hypotheses and methods we proposed in this paper. Additionally, GSRPnet’s parameters are merely 17.999 M, and this effectively demonstrates that the improvement in accuracy of GSRPnet is not only determined by an increase in model parameters.

Robust Ranking Kernel Support Vector Machine via Manifold Regularized Matrix Factorization for Multi-Label Classification

Multi-label classification has been extensively researched and utilized for several decades. However, the performance of these methods is highly susceptible to the presence of noisy data samples, resulting in a significant decrease in accuracy when noise levels are high. To address this issue, we propose a robust ranking support vector machine (Rank-SVM) method that incorporates manifold regularized matrix factorization. Unlike traditional Rank-SVM methods, our approach integrates feature selection and multi-label learning into a unified framework. Within this framework, we employ matrix factorization to learn a low-rank robust subspace within the input space, thereby enhancing the robustness of data representation in high-noise conditions. Additionally, we incorporate manifold structure regularization into the framework to preserve manifold relationships among low-rank samples, which further improves the robustness of the low-rank representation. Leveraging on this robust low-rank representation, we extract a resilient low-rank features and employ them to construct a more effective classifier. Finally, the proposed framework is extended to derive a kernelized ranking approach, for the creation of nonlinear multi-label classifiers. To effectively solve this non-convex kernelized method, we employ the augmented Lagrangian multiplier (ALM) and alternating direction method of multipliers (ADMM) techniques to obtain the optimal solution. Experimental evaluations conducted on various datasets demonstrate that our framework achieves superior classification results and significantly enhances performance in high-noise scenarios.

Predicting potential microbe-disease associations based on auto-encoder and graph convolution network

The increasing body of research has consistently demonstrated the intricate correlation between the human microbiome and human well-being. Microbes can impact the efficacy and toxicity of drugs through various pathways, as well as influence the occurrence and metastasis of tumors. In clinical practice, it is crucial to elucidate the association between microbes and diseases. Although traditional biological experiments accurately identify this association, they are time-consuming, expensive, and susceptible to experimental conditions. Consequently, conducting extensive biological experiments to screen potential microbe-disease associations becomes challenging. The computational methods can solve the above problems well, but the previous computational methods still have the problems of low utilization of node features and the prediction accuracy needs to be improved. To address this issue, we propose the DAEGCNDF model predicting potential associations between microbes and diseases. Our model calculates four similar features for each microbe and disease. These features are fused to obtain a comprehensive feature matrix representing microbes and diseases. Our model first uses the graph convolutional network module to extract low-rank features with graph information of microbes and diseases, and then uses a deep sparse Auto-Encoder to extract high-rank features of microbe-disease pairs, after which the low-rank and high-rank features are spliced to improve the utilization of node features. Finally, Deep Forest was used for microbe-disease potential relationship prediction. The experimental results show that combining low-rank and high-rank features helps to improve the model performance and Deep Forest has better classification performance than the baseline model.

Robust Label and Feature Space Co-Learning for Multi-Label Classification

Multi-label classification remains a challenging task for high-dimensional data samples and their labels both increase the complexity of training models. In this paper, we propose a Robust Label and Feature Space Co-Learning method, referred to as RLFSCL, for multi-label classification. Different from traditional multi-label classification methods which focus on feature space learning through regression directly between data samples and labels, our proposed method can further learn robust low rank label space from this traditional regression method. Therefore, our RLFSCL can learn better low rank feature and label representations simultaneously in original noisy and high dimensional spaces. Experimental comparison on five benchmark datasets, including Rcv1s5, Cal500, and Corel16k4 shows that the proposed RLFSCL algorithm outperforms state-of-the-art multi-label classification methods. The code of RLFSCL is made available on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/JingChuanTang/RLFSCL.</uri>

Face recognition technology based on low-rank joint sparse representation algorithm

With the improvement of computer computing power and the development of artificial intelligence technology, face recognition technology has made a major breakthrough, and has been popularized and applied in all areas of life. However, different face structure and pose will affect the accuracy of face recognition. To overcome the problem, a low rank joint sparse representation algorithm for face recognition is proposed. The low rank features of images are extracted by structure independent and pairwise rank decomposition methods. The extracted low rank features of the first level image and the low rank features of the second level image are sparsely represented. Finally, the residual rate model is used to classify the images, and the final result of face recognition is obtained. The experimental results show that the proposed SRP algorithm has a recognition accuracy of more than 92% in two different face recognition tests. In the mixed multi face pose test, PRS algorithm performs best in the recognition of 1, 2, 3, 4, and 5 multi face pose types, with recognition rates of 95%, 94%, 93%, 91%, and 90% respectively. The algorithm also has excellent recognition performance and robustness in identifying harsh environments such as fuzzy environments. The research content focuses on complex face recognition scenes, innovatively uses low rank to complete the extraction of face feature data, and combines sparse selection of classification features to improve the overall effect of face recognition. It has important reference value for improving the overall security and recognition rate of face recognition.

LoRA-NCL: Neighborhood-Enriched Contrastive Learning with Low-Rank Dimensionality Reduction for Graph Collaborative Filtering

Graph Collaborative Filtering (GCF) methods have emerged as an effective recommendation approach, capturing users’ preferences over items by modeling user–item interaction graphs. However, these methods suffer from data sparsity in real scenarios, and their performance can be improved using contrastive learning. In this paper, we propose an optimized method, named LoRA-NCL, for GCF based on Neighborhood-enriched Contrastive Learning (NCL) and low-rank dimensionality reduction. We incorporate low-rank features obtained through matrix factorization into the NCL framework and employ LightGCN to extract high-dimensional representations. Extensive experiments on five public datasets demonstrate that the proposed method outperforms a competitive graph collaborative filtering base model, achieving 4.6% performance gains on the MovieLens dataset, respectively.

Weakly-supervised content-based video moment retrieval using low-rank video representation

Content-based video moment retrieval (CVMR) aims to localize a successive sequence of frames in an untrimmed reference video, called target moment, that is semantically corresponding to a given query video. Current state-of-the-art CVMR methods are mainly developed using frame-level annotation, which is often quite expensive to collect. In this paper, we aim to develop a weakly-supervised CVMR method, which uses coarse-grained video-level annotations during training. Under weak supervision, video localizers require more discriminative frame-level video features. To achieve this goal, we proposed a novel prior, termed low-rank prior, based on an observation that the frame-level feature of a video should have low-rank properties. We demonstrated that the low-rank features are more discriminative and are beneficial to accurately localize the action boundaries. To produce a low-rank feature, we designed a low-rank feature reconstruction (LFR) operator. A new differentiable matrix decomposition approach is proposed to generate the low-rank reconstruction of the input matrix, meanwhile ensuring that the matrix decomposition process is differentiable. Based on the LFR, we developed a new weakly-supervised CVMR model which produces low-rank video representation and performs semantic consistency measures to discover the semantically matched segment in the reference video to the query video. Extensive experiments demonstrate that our method outperforms state-of-the-art weakly-supervised methods consistently and even achieves competing performance to fully-supervised baselines.

Tensor Distribution Regression Based on the 3D Conventional Neural Networks

Dear Editor, This letter presents a novel tensor-distribution-regression model based on 3D conventional neural networks (3D-TDR) with an application to clinical score prediction in aging-related diagnosis. The estimation of clinical scores of subjects using brain magnetic resonance imaging (MRI) helps understand the pathological stage of dementia. However, clinical scores prediction is still unsolved due to the reasons of: 1) Analyzing the whole-brain MRI is extremely difficult as the high-dimensional MRI data contains millions of voxels; 2) The clinical scores prediction is formulated as a one-dimensional regression issue in the current deep-learning-based algorithms, which ignores the implicit label information between subjects with different score levels. Motivated by the above discoveries, the proposed 3D-TDR model innovatively establishes the following three-fold ideas: a) incorporating a tensor regression layer (TRL) into a 3D conventional neural network (3D-CNN) to enable its extraction of more discriminative structural changes from the high-dimensional whole-brain magnetic resonance (MR) data; b) adopting the label distribution learning (LDL) to fully utilize the label correlation among the MR images, thus emphasizing the diversity of subjects' scores; and c) combining the TRL and LDL for an end-to-end deep learning framework, thereby achieving jointly low-rank feature extraction and clinical scores prediction. Experimental results on two real-world MRI datasets of two typical clinical prediction tasks indicate that the 3D-TDR outperforms the benchmark and state-of-the-art models. The proposed 3D-TDR model can achieve significant accuracy gain in dementia score and brain age prediction.

Visible light fingerprint database recovery algorithm based on CP decomposition.

Visible light communication(VLC) is a new method of indoor communication. It can provide an effective solution for indoor positioning. Fingerprint-based visible light positioning(VLP) has been widely studied for its feasibility and high accuracy. The acquisition of 'fingerprint database' is crucial for accurate VLP. However, sparse sensors such as photodiode(PD) can only be arranged because of the space-limited scenario and high costs. Correspondingly, it results in the loss of the fingerprint database. Therefore, it is indispensable to solve the problem of how to effectively and accurately recover the fingerprint database from measurements of sparsely arranged sensors. In this paper, we propose a spatio-temporal constraint tensor completion (SCTC) algorithm based on CANDECOMP/PARAFAC (CP) decomposition to recover the fingerprint database from measurements of sparsely arranged sensors. Specifically, we model the measurements from the spatial and temporal dimensions as a tensor, and formulate the optimization problem based on the low-rank feature of the tensor. To improve the recovery accuracy, spatial and temporal constraint matrices are introduced to effectively constrain the optimization direction when completing the tensor. Spatial constraint matrices are constructed by utilizing the mode-n expansion matrix of the tensor based on the undirected graph theory. Accordingly, the Toeplitz matrix is used as the temporal constraint matrix to excavate the temporal correlation of the tensor. Since the optimization problem is non-convex and difficult to solve, we introduce CP decomposition to decompose the tensor into several factor matrices. By solving the factor matrices, the original tensor is reconstructed. The performance of the proposed SCTC algorithm is confirmed via experimental measured data.

Automatic stabilization of finite-element simulations using neural networks and hierarchical matrices

Petrov–Galerkin formulations with optimal test functions allow for the stabilization of finite element simulations. In particular, given a discrete trial space, the optimal test space induces a numerical scheme delivering the best approximation in terms of a problem-dependent energy norm. This ideal approach has two shortcomings: first, we need to explicitly know the set of optimal test functions; and second, the optimal test functions may have large supports inducing expensive dense linear systems. A concise proposal on how to overcome these shortcomings has been raised during the last decade by the Discontinuous Petrov–Galerkin (DPG) methodology. However, DPG has also some limitations and difficulties: the method requires ultraweak variational formulations, obtained through a hybridization process, which is not trivial to implement at the discrete level.Our motivation is to offer a simpler alternative for the case of parametric PDEs, which can be used with any variational formulation. Indeed, parametric families of PDEs are an example where it is worth investing some (offline) computational effort to obtain stabilized linear systems that can be solved efficiently in an online stage, for a given range of parameters. Therefore, as a remedy for the first shortcoming, we explicitly compute (offline) a function mapping any PDE parameter, to the matrix of coefficients of optimal test functions (in some basis expansion) associated with that PDE parameter. Next, as a remedy for the second shortcoming, we use the low-rank approximation to hierarchically compress the (non-square) matrix of coefficients of optimal test functions. In order to accelerate this process, we train a neural network to learn a critical bottleneck of the compression algorithm (for a given set of PDE parameters). When solving online the resulting (compressed) Petrov–Galerkin formulation, we employ a GMRES iterative solver with inexpensive matrix–vector multiplications thanks to the low-rank features of the compressed matrix. We perform experiments showing that the full online procedure is as fast as an (unstable) Galerkin approach. We illustrate our findings by means of 2D–3D Eriksson–Johnson problems, together with 2D Helmholtz equation.

Multilayer Network Community Detection: A Novel Multi-Objective Evolutionary Algorithm Based on Consensus Prior Information [Feature

In recent years, multilayer networks have served as effective models for addressing and analyzing real-world systems with multiple relationships. Among these scenarios, the community detection (CD) problem is one of the most prominent research hotspots. Although some research on multilayer network CD (MCD) has been proposed to address this problem, most studies focus only on topological structures. Therefore, their algorithms cannot extract the most out of complementary network information, such as node similarities and low-rank features, which may lead to unsatisfactory accuracy. To tackle this problem, this paper proposes a novel multi-objective evolutionary algorithm based on consensus prior information (MOEA-CPI). The proposed algorithm takes full advantage of prior information to guide the MOEA with respect to topological structures, initializations, and the optimization process. More specifically, this paper first extracts two kinds of prior information, i.e., graph-level and node-level information, based on Node2vec and Jaccard similarity, respectively. Then, the prior layer and a high-quality initial population are constructed on the basis of the graph-level information. During the optimization process, the genetic operator, which integrates the weighting strategy and node-level information, is applied to guide the algorithm to distribute similar nodes into the same community. Extensive experiments are implemented to prove the superior performance of MOEA-CPI over the state-of-the-art methods.

Similarity matrix enhanced collaborative filtering for e-government recommendation

Personalized e-government recommendation aims to predict the user’s next action in a near future based on his or her historical behavior data. Similarity measures and distances are widely used for powering personalized recommendations. The classical similarity-based models in the recommender systems are known as K-Nearest Neighbor Collaborative Filtering (KNN-based CF). Most prior work adopts a new similarity formula to obtain the k-nearest neighbors for the user and/or item. This is a good try but falls short of solving the problem of expensive computation and introducing the order of users’ interaction sequences. In this paper, we propose SACF, a similarity matrix enhanced the e-government recommendation model that takes advantage of pairwise learning to introduce the order of uses’ interaction sequences and quickly obtain the similarity matrix using matrix factorization. In other words, the similarity matrix can be presented as the production of two low-rank feature matrices. Therefore, it is possible to effectively estimate the similarity between two users even with few co-rated items. Experimental results on a real-world e-government dataset show that the proposed approach improves the performance significantly compared with other counterparts.

Tripartite Graph Aided Tensor Completion For Sparse Network Measurement

Network measurements provide critical inputs for a wide range of network management. Existing network-wide monitoring methods face the challenge of incurring a high measurement cost. Some recent studies show that network-wide measurement data such as end-to-end latency and flow traffic, have hidden spatio-temporal correlations and thus low-rank features. Taking advantage of the low-rank feature, enlightened by tensor model's strong capability of information representation and extracting, this paper studies a novel sparse measurement scheduling problem which selects a proportion of Origin and Destination (OD) pairs to take measurements in the future time slots, while ensuring the data of the remaining un-measured OD pairs be accurately inferred through tensor completion. It is challenging to find the optimal sampling points (OD pairs) without knowing the structure of the future data and also infer the un-measured data in the presence of noise in the measurement samples. To conquer the challenges, we propose several techniques: a tripartite graph to illustrate the relationship between sample locations and tensor factorization, a graph-based sample selection algorithm, and a graph-based robust tensor completion algorithm. We have conducted extensive experiments based on two real network latency monitoring traces (PlanetLab and Harvard) and two other network monitoring traces (including a traffic trace Abilene and a throughput trace WS-Dream). Our results demonstrate that, even with a sampling ratio of less than 5%, our scheme can accurately obtain the complete network-wide monitoring data by inferring the missing ones based on the samples taken. To achieve similar recovery performance, the best peer tensor completion algorithm needs a significantly larger number of samples, with the sampling ratio up to 25-150 times ours.

Tensor Singular Spectrum Analysis for 3-D Feature Extraction in Hyperspectral Images

Due to the cubic structure of a hyperspectral image (HSI), how to characterize its spectral and spatial properties in three dimensions is challenging. Conventional spectral-spatial methods usually extract spectral and spatial information separately, ignoring their intrinsic correlations. Recently, some 3D feature extraction methods are developed for the extraction of spectral and spatial features simultaneously, although they rely on local spatial-spectral regions and thus ignore the global spectral similarity and spatial consistency. Meanwhile, some of these methods contain huge model parameters which require a large number of training samples. In this paper, a novel Tensor Singular Spectral Analysis (TensorSSA) method is proposed to extract global and low-rank features of HSI. In TensorSSA, an adaptive embedding operation is first proposed to construct a trajectory tensor corresponding to the entire HSI, which takes full advantage of the spatial similarity and improves the adequate representation of the global low-rank properties of the HSI. Moreover, the obtained trajectory tensor, which contains the global and local spatial and spectral information of the HSI, is decomposed by the Tensor singular value decomposition (t-SVD) to explore its low-rank intrinsic features. Finally, the efficacy of the extracted features is evaluated using the accuracy of image classification with a support vector machine (SVM) classifier. Experimental results on three publicly available datasets have fully demonstrated the superiority of the proposed TensorSSA over a few state-of-the-art 2D/3D feature extraction and deep learning algorithms, even with a limited number of training samples.