Sparse Learning Method Research Articles

Hyperspectral anomaly detection based on weighted low-rank sparse dictionary learning

The geometric model-based hyperspectral anomaly detection techniques have garnered a lot of interest recently. These methods are predicated on the assumption that the background can be represented by a dictionary of spectral vectors, but the anomaly cannot. Building an objective model with high representational capabilities and obtaining precise background modeling are therefore key aspects of this type of method. The background dictionary is also readily tainted by anomalies due to the limitations of the current approaches, making the detection findings less reliable. To address the above problems, this paper proposes a weighted low-rank sparse dictionary learning method (WLSDL). This model organically combines sparse representation with low-rank representation in order to effectively depict the intricate background distribution. The weights relating to the eigenvalues of the image are designed by mining the physical characteristics of the HSI. This can enhance the nuclear norm’s capacity for representation, resulting in a background dictionary with more representativeness. Furthermore, the capped norm constraint is incorporated in the objective function to decrease the effect of anomalies and noises on background modeling, hence minimizing anomaly pollution on the background dictionary. The residual image effectively increases the accuracy of anomaly identification since it makes it easier to distinguish between anomalies and background. We use three hyperspectral datasets to evaluate the proposed method. The experimental findings demonstrate the effectiveness and superiority of the proposed method over state-of-the-art methods.

Identification of genetic basis of brain imaging by group sparse multi-task learning leveraging summary statistics

Brain imaging genetics is an evolving neuroscience topic aiming to identify genetic variations related to neuroimaging measurements of interest. Traditional linear regression methods have shown success, but their reliance on individual-level imaging and genetic data limits their applicability. Herein, we proposed S-GsMTLR, a group sparse multi-task linear regression method designed to harness summary statistics from genome-wide association studies (GWAS) of neuroimaging quantitative traits. S-GsMTLR directly employs GWAS summary statistics, bypassing the requirement for raw imaging genetic data, and applies multivariate multi-task sparse learning to these univariate GWAS results. It amalgamates the strengths of conventional sparse learning methods, including sophisticated modeling techniques and efficient feature selection. Additionally, we implemented a rapid optimization strategy to alleviate computational burdens by identifying genetic variants associated with phenotypes of interest across the entire chromosome. We first evaluated S-GsMTLR using summary statistics derived from the Alzheimer's Disease Neuroimaging Initiative. The results were remarkably encouraging, demonstrating its comparability to conventional methods in modeling and identification of risk loci. Furthermore, our method was evaluated with two additional GWAS summary statistics datasets: One focused on white matter microstructures and the other on whole brain imaging phenotypes, where the original individual-level data was unavailable. The results not only highlighted S-GsMTLR's ability to pinpoint significant loci but also revealed intriguing structures within genetic variations and loci that went unnoticed by GWAS. These findings suggest that S-GsMTLR is a promising multivariate sparse learning method in brain imaging genetics. It eliminates the need for original individual-level imaging and genetic data while demonstrating commendable modeling and feature selection capabilities.

Dual-mask: Progressively sparse multi-task architecture learning

Multi-task architecture learning has achieved significant success by learning optimal sharing architectures for different tasks. However, previous works to learn branched architectures for different tasks can sometimes lead to unsatisfying multi-task performance, as not all detailed branches are relevant to a specific task. Task-relevant architectures can be sparse, including only partial channels or layers in the entire architecture (i.e., a sub-network). In addition, most previous works rely on a heuristic architecture selection procedure that could not support continuous architecture optimization. To this end, in this paper, we propose dual-mask, a progressively sparse multi-task architecture learning method. Starting with a task-free architecture, it identifies the informative features along two-level, channels and layers, for each task, while suppressing conflicting or noisy parts in a differentiable manner, so that better task-specific sub-networks are captured. Specifically, the channel and layer selection modules produce respective hybrid binary and real value masks, designed to pick salient channels and layers for each task, respectively. To jointly optimize masks with model parameters, we propose an importance-guided relaxation method for solving the stochastic binary optimization problem, after which the interference or noise parts can be pruned by masks. Additionally, a progressive training strategy with continuation is provided that gradually sparsity the task-specific sub-networks. Experiments show that dual-mask achieves superior performance than SOTA multi-task methods.

Multi-cluster nonlinear unsupervised feature selection via joint manifold learning and generalized Lasso

Due to the scarcity of data labels, unsupervised feature selection has received a lot of attention in recent years. While many unsupervised feature selection methods are capable of selecting relevant features, they often fail to comprehensively consider the impact of both local and global information of the data on feature selection, nor can they effectively handle the complex nonlinear relationships commonly found in real-world data. As a result, suboptimal feature subsets are often selected. In this paper, inspired by the Uniform Manifold Approximation and Projection (UMAP) manifold learning technique and the nonlinear sparse learning method based on Feature-Wise Kernelized Lasso, we propose a novel unsupervised feature selection method called Multi-Cluster Unsupervised Nonlinear Feature Selection based on UMAP and block HSIC Lasso (MUNFS). MUNFS greatly improves the representation of high-dimensional data during dimensionality reduction and effectively handles complex nonlinear relationships in such data. Specifically, by capturing the intrinsic topology of the data, MUNFS accurately preserves the local structure of the data while keeping as much of the global structure as possible. Furthermore, the kernel-based Hilbert–Schmidt Independence Criterion (HSIC) may measure the nonlinear dependency between the features and the target variables, while applying the l1 regularization term in feature selection to achieve sparsity. This allows for a more precise assessment of the significance of each feature. Extensive experimental results on five benchmark datasets and eight hyperspectral datasets demonstrate that the MUNFS method performs much better than several other feature selection methods.

Generalized nonlinear hybrid-norm parallel sparse filtering for bearing fault diagnosis under complex interference

Abstract Considerable attention has been garnered by the bearing fault diagnostic method, founded on sparse feature learning, for its exceptional robustness and adaptability to noise. Being an effective sparse learning method, parallel sparse filtering (PSF) has found successful applications in intelligent fault diagnosis. However, influenced by mechanical structural collisions and complex transmission paths, the obtained bearing signals come with abnormal impacts and noise, resulting in a reduced reliability of the PSF. Additionally, PSF constrains features and samples with l 1 / 2 norm, which is significantly disturbed by noise. To address these issues, generalized nonlinear hybrid-norm PSF (GNHNPSF) is proposed. Firstly, the Sigmoid nonlinear function is examined to suppress the amplitude of the anomalous shock. Subsequently, the hybrid-norm constraints, which involve the application of a l 1 / 2 norm to the column features of the input signal and a l 1 / 3 norm to the row features, can be employed to effectively extract distinctive information from the original data. The GNHNPSF is demonstrated to improve the accuracy of bearing fault diagnosis under complex interference, as evidenced by the results of simulation and experiments.

Physics-informed machine learning of multi-source monitoring data sparsely measured within a 2D vertical cross-section

Conventional machine learning (ML) methods suffer from several limitations, e.g., overreliance on large training datasets and poor performance in extrapolating beyond observations. Geotechnical monitoring data at specific sites are multi-sourced (e.g., settlement and horizontal displacement data). Moreover, each source of monitoring data may vary temporally and be sparsely measured, e.g., within a two-dimensional (2D) geological cross-section. Thus, effectively learning multi-source and limited monitoring data is difficult using existing ML methods. To address this challenge, this study proposes a physics-informed ML method combining geotechnical numerical models (e.g., the finite element model (FEM)) with sparse dictionary learning (SDL) methods. It uses a range of probable numerical model results corresponding to multiple, temporally varying responses in a 2D spatial range to construct a “dictionary” in SDL and employs multi-source and limited monitoring data to identify a few “atoms” from the constructed dictionary to improve model predictions. An engineering project for embankment construction is used to illustrate the proposed approach. Our results indicate that the proposed approach effectively integrates FEM results with multi-source and sparsely measured monitoring data and improves the predictions of multiple spatio-temporally varying geotechnical responses significantly, particularly those at subsequent time steps and unmonitored spatial locations within a 2D vertical cross-section.

Discriminative sparse subspace learning with manifold regularization

Common subspace learning methods only utilize local or global structure in feature extraction, and cannot obtain the global optimal discriminative projection matrix. For this reason, this paper proposes a discriminative sparse subspace learning method based on the manifold regularization framework (DSSL-MR), which introduces the graph Laplacian matrix that reflects the intrinsic geometric structure of the sample as a penalty term. DSSL-MR simultaneously uses both sub-manifold and multi-manifold information of samples for obtaining optimal projection to enhance the discriminability of different classes in subspace. DSSL-MR uses the sparse property of the L2,1-norm to constrain the projection matrix, which can eliminate redundant features and select features that are significant for classification. It is a linear supervised method, which belongs to the Fisher discriminant analysis framework. Experimental results on multiple real-world datasets show that the algorithm is very effective in classification and has high recognition rates.

Learning Multi-Task Sparse Representation Based on Fisher Information

Multi-task learning deals with multiple related tasks simultaneously by sharing knowledge. In a typical deep multi-task learning model, all tasks use the same feature space and share the latent knowledge. If the tasks are weakly correlated or some features are negatively correlated, sharing all knowledge often leads to negative knowledge transfer among. To overcome this issue, this paper proposes a Fisher sparse multi-task learning method. It can obtain a sparse sharing representation for each task. In such a way, tasks share features on a sparse subspace. Our method can ensure that the knowledge transferred among tasks is beneficial. Specifically, we first propose a sparse deep multi-task learning model, and then introduce Fisher sparse module into traditional deep multi-task learning to learn the sparse variables of task. By alternately updating the neural network parameters and sparse variables, a sparse sharing representation can be learned for each task. In addition, in order to reduce the computational overhead, an heuristic method is used to estimate the Fisher information of neural network parameters. Experimental results show that, comparing with other methods, our proposed method can improve the performance for all tasks, and has high sparsity in multi-task learning.

Prediction of MicroRNA-Disease Potential Association Based on Sparse Learning and Multilayer Random Walks.

More and more studies have shown that microRNAs (miRNAs) play an indispensable role in the study of complex diseases in humans. Traditional biological experiments to detect miRNA-disease associations are expensive and time-consuming. Therefore, it is necessary to propose efficient and meaningful computational models to predict miRNA-disease associations. In this study, we aim to propose a miRNA-disease association prediction model based on sparse learning and multilayer random walks (SLMRWMDA). The miRNA-disease association matrix is decomposed and reconstructed by the sparse learning method to obtain richer association information, and at the same time, the initial probability matrix for the random walk with restart algorithm is obtained. The disease similarity network, miRNA similarity network, and miRNA-disease association network are used to construct heterogeneous networks, and the stable probability is obtained based on the topological structure features of diseases and miRNAs through a multilayer random walk algorithm to predict miRNA-disease potential association. The experimental results show that the prediction accuracy of this model is significantly improved compared with the previous related models. We evaluated the model using global leave-one-out cross-validation (global LOOCV) and fivefold cross-validation (5-fold CV). The area under the curve (AUC) value for the LOOCV is 0.9368. The mean AUC value for 5-fold CV is 0.9335 and the variance is 0.0004. In the case study, the results show that SLMRWMDA is effective in inferring the potential association of miRNA-disease.

Estimating Functional Brain Networks by Low-Rank Representation With Local Constraint.

The functional architecture undergoes alterations during the preclinical phase of Alzheimer's disease. Consequently, the primary research focus has shifted towards identifying Alzheimer's disease and its early stages by constructing a functional connectivity network based on resting-state fMRI data. Recent investigations show that as Alzheimer's Disease (AD) progresses, modular tissue and connections in the core brain areas of AD patients diminish. Sparse learning methods are powerful tools for understanding Functional Brain Networks (FBNs) with Regions of Interest (ROIs) and a connectivity matrix measuring functional coherence between them. However, these tools often focus exclusively on functional connectivity measures, neglecting the brain network's modularity. Modularity orchestrates dynamic activities within the FBN to execute intricate cognitive tasks. To provide a comprehensive delineation of the FBN, we propose a local similarity-constrained low-rank sparse representation (LSLRSR) method that encodes modularity information under a manifold-regularized network learning framework and further formulate it as a low-rank sparse graph learning problem, which can be solved by an efficient optimization algorithm. Specifically, for each modularity structure, the Schatten p-norm regularizer reduces the reconstruction error and provides a better approximation of the low-rank constraint. Furthermore, we adopt a manifold-regularized local similarity prior to infer the intricate relationship between subnetwork similarity and modularity, guiding the modeling of FBN. Additionally, the proximal average method approximates the joint solution's proximal map, and the resulting nonconvex optimization problems are solved using the alternating direction multiplier method (ADMM). Compared to state-of-the-art methods for constructing FBNs, our algorithm generates a more modular FBN. This lays the groundwork for further research into alterations in brain network modularity resulting from diseases.

Sparse learning method with feature selection for sensor placement and response prediction

Sparse learning method with feature selection for sensor placement and response prediction

Nectarine Disease Identification Based on Color Features and Label Sparse Dictionary Learning with Hyperspectral Images

Fruit cracking and rust spots are common diseases of nectarines that seriously affect their yield and quality. Therefore, it is essential to construct fast and accurate disease-identification models for agricultural products. In this paper, a sparse dictionary learning method was proposed to realize the rapid and nondestructive identification of nectarine disease based on multiple color features combined with improved LK-SVD (Label K-Singular Value Decomposition). According to the color characteristics of the nectarine itself and the significant color differences existing in the three categories of nectarine (diseased, normal, and background parts), multiple color spaces of RGB, HSV, Lab, and YCbCr were studied. It was concluded that the G channel in RGB space, Y channel in YCbCr space, and L channel in Lab space can better distinguish the diseased part from the other parts. At the model-training stage, pixels of the diseased, normal, and background parts in the nectarine image were randomly selected as the initial training sets, and then, the neighboring image blocks of the pixels were selected to construct the feature vectors based on the above color space channels. An improved LK-SVD dictionary learning algorithm was proposed that integrated the category label into the process of dictionary learning, and thus, an over-complete feature dictionary with significant discrimination was obtained. At the model-testing stage, the orthogonal matching pursuit (OMP) algorithm was used for sparse reconstruction of the original data, which can obtain the classification categories based on the optimized feature dictionary. The experimental results show that the sparse dictionary learning method based on multi-color features combined with improved LK-SVD can identify fruit cracking and rust spot diseases of nectarines quickly and accurately, and the average overall classification accuracies were 92.06% and 88.98%, respectively, which were better than those of k-nearest neighbor (KNN), support vector machine (SVM), DeepLabV3+, and Unet++; the identification results of DeepLabV3+ and Unet++ were also relatively high, but their average time costs were much higher, requiring 126.46~265.65 s. It is demonstrated that this study can provide technical support for disease identification in agricultural products.

A Sparse Learning Method with Regularization Parameter as a Self-Adaptation Strategy for Rolling Bearing Fault Diagnosis

Sparsity-based fault diagnosis methods have achieved great success. However, fault classification is still challenging because of neglected potential knowledge. This paper proposes a combined sparse representation deep learning (SR-DEEP) method for rolling bearing fault diagnosis. Firstly, the SR-DEEP method utilizes prior domain knowledge to establish a sparsity-based fault model. Then, based on this model, the corresponding regularization parameter regression networks are trained for different running states, whose core is to explore the latent relationship between the regularization parameters and running states. Subsequently, the performance of the fault classification is improved by embedding the trained regularization parameter regression networks into the sparse representation classification method. This strategy improves the adaptability of the sparse regularization parameter, further improving the performance of the fault classification method. Finally, the applicability of the SR-DEEP method for rolling bearing fault diagnosis is validated with the CWRU platform and QPZZ-II platform, demonstrating that SR-DEEP yields superior accuracies of 100% and 99.20% for diagnosing four and five running states, respectively. Comparative studies show that the SR-DEEP method outperforms four sparse representation methods and seven classical deep learning classification methods in terms of the classification performance.

Seq-SVF: An unsupervised data-driven method for automatically identifying hidden governing equations

In this work, an unsupervised data-driven method based on sequential singular value filtering (Seq-SVF) is proposed to simultaneously identify multiple partial differential equations from observed data considering potential noises. This method is aimed to extend the Sparse Identification of Nonlinear Dynamics (SINDy) to the identification of general nonlinear partial differential equations by transforming the paradigm based on regression to an unsupervised paradigm. To discover the complex coupled equations of vector or tensor forms without prior knowledge, the techniques of singular value decomposition (SVD) and strong rank-revealing QR factorization (sRRQR) are applied to the data matrix, which ensures that the method can automatically identify the number and the corresponding linearly independent terms as the left-hand terms of governing equations. To balance the complexity and the precision of modeling, a strategy for filtering singular values is designed to determine the sparse structure of governing equations from a large number of nonlinear basis functions. We show the success of the method to extract explicit and succinct models from many complex linear, nonlinear, and multiphysics mechanical systems, and the examples show more accuracy compared with using traditional sparse learning methods.

Sparse representation learning for fault feature extraction and diagnosis of rotating machinery

Early fault feature extraction and fault diagnosis are of great importance for predictive maintenance of rotating machinery. To accurately extract early fault features from original noisy signals, a novel joint sparse representation learning method is developed in this paper, this method is based on the proposed nonlocal generalized minimax-concave (GMC) penalty and generalized fraction-order total variation (FTV) regularization. The motivation for this research is to leverage the benefits of joint regularizations. The proposed nonlocal GMC penalty regularization tends to preserve weak fault features, promote sparsity and avoid underestimating the amplitude of periodic fault impulses. Simultaneously, the proposed generalized FTV regularization tends to remove fault irrelevant noise and reduce staircase artifacts. Therefore, the proposed model can effectively extract early fault features from original noisy signals. The performance of the proposed model is verified by a series of experiments. In two fault diagnosis tasks, the peak signal-to-noise ratio (PSNR) of the proposed method reaches − 5 dB and − 8 dB, respectively. Compared with state-of-the-art methods, the PSNR has been improved by at least 2 dB, comparison results show that the proposed model has superior performance for early fault feature extraction.

Molecular kinematic viscosity prediction of natural ester insulating oil based on sparse Machine learning models

The high viscosity property of natural ester insulating oils can be improved by means of molecular structure modification. However, it is difficult to find the optimal molecular structure that meets the conditions in the vast chemical space by relying on macroscopic experiments. Herein, the kinematic viscosity prediction models of triglyceride molecules based on sparse machine learning methods are proposed in this paper. Firstly, the molecular dynamics technique is used to simulate the kinematic viscosity (-20℃ to 40℃) of the main 20 triglyceride molecules in natural ester insulating oil, which provides extensive characterization data for subsequent model training. Secondly, the molecular descriptors are calculated for each triglyceride molecule based on density functional theory (DFT). Thirdly, four sparse feature selection methods (Boruta, RFECV, Pearson correlation coefficient, and mutual information) are used to identify the most relevant molecular descriptors for the kinematic viscosity of natural ester insulating oil and further analysis is performed. Finally, quantitative structure–property relationship (QSPR) models are constructed using automated machine learning methods to achieve efficient and accurate prediction of kinematic viscosity (average R2 reached 0.96). This study provides certain mechanistic explanations from the perspective of molecular descriptors and provides a predictive model basis for the viscosity modification of natural ester molecules, which serves as a theoretical guide to improve the kinematic viscosity of natural ester insulating oil.

Patient-independent epileptic seizure detection by stable feature selection

The detection of epileptic seizures in EEG signals is a challenging task because it requires careful review of multi-channel EEG recordings over a lengthy time interval. In general, EEG-based seizure detection is strongly dependent on the ability to select descriptive features that are also stable in the sense that they are not sensitive to changes in the training data. This study proposes and investigates a patient-independent seizure detection model that uses stable EEG-based features obtained by comparing multiple feature selection methods. The schemes considered can be divided into five categories, often referred to as similarity, information theoretic, sparse learning, statistical, and graph centrality feature selection methods. The stability of the features these schemes produce was evaluated by random forest classification, using the intersection, frequency, correlation, and similarity measures. In experiments with Temple University Hospital’s Seizure database of EEG records of 341 patients, unsupervised graph centrality feature selection was the most effective method, with a correct classification rate of 91.5%.

Group Sparse Reduced Rank Tensor Regression for Micro-Expression Recognition

To overcome the challenge in micro-expression recognition that it only emerge in several small facial regions with low intensity, some researchers proposed facial region partition mechanisms and introduced group sparse learning methods for feature selection. However, such methods have some shortcomings, including the complexity of region division and insufficient utilization of critical facial regions. To address these problems, we propose a novel Group Sparse Reduced Rank Tensor Regression (GSRRTR) to transform the fearure matrix into a tensor by laying blocks and features in different dimensions. So we can process grids and texture features separately and avoid interference between grids and features. Furthermore, with the use of Tucker decomposition, the feature tensor can be decomposed into a product of core tensor and a set of matrix so that the number of parameters and the computational complexity of the scheme will decreased. To evaluate the performance of the proposed micro-expression recognition method, extensive experiments are conducted on two micro expression databases: CASME2 and SMIC. The experimental results show that the proposed method achieves comparable recognition rate with less parameters than state-of-the-art methods.

Automatic Knowledge Integration Method of English Translation Corpus Based on Kmeans Algorithm

Abstract We propose a feature extraction method based on the Kmeans algorithm based on the text characteristics in the English translation corpus. The article first uses a sparse autoencoder unsupervised learning method to reduce dimensionality. It then uses the Kmeans clustering algorithm for text clustering. The experimental results prove that the text features extracted by the sparse autoencoder based on the Kmeans algorithm can be used for English translation corpus knowledge clustering to achieve automatic integration. And this method can effectively solve the problems of high-dimensional, sparse, and noisy texts in the English translation corpus. The algorithm mentioned in the article can significantly improve the accuracy of the clustering results.

OCD diagnosis via smooth sparse network and fused sparse auto-encoder learning

Obsessive-compulsive disorder (OCD) brings many problems to patients. Redundant information in the OCD data can be removed to preserve valuable biological functions through sparse learning methods. Therefore, constructing a brain functional connectivity network (BFCN) with sparse learning is beneficial to objectively diagnose OCD. However, most studies ignore the relationship between subjects. Therefore, a new smooth sparse network (SSN) model is proposed to construct BFCN. Specifically, a smoothing term is designed in the objective function to capture the relationship between subjects. Then, a fused sparse auto-encoder (FSAE) model is proposed to learn the deep feature and decrease feature dimension of BFCN. The FASE is able to fuse the regularized sparse auto-encoder (RSAE) features and regularized stacking SAE (RSSAE) features for diagnosing OCD. Specifically, the l2-norm regularization is integrated in RSAE and RSSAE to address overfitting. Our proposed method combines the traditional machine learning with deep learning, which can achieve promising OCD diagnosis performance on our self- collected data.