Sparse Learning Method Research Articles

Robust sparse and low-redundancy multi-label feature selection with dynamic local and global structure preservation

Recent years, joint feature selection and multi-label learning have received extensive attention as an open problem. However, there exist three general issues in previous multi-label feature selection methods. First of all, existing methods either consider local label correlations or global label correlations when they design multi-label feature selection methods, in fact, both two types of label correlations are significant for feature selection; second, previous methods use the low-quality graph to excavate local label correlations so that the results of these methods are under-performing; third, feature redundancy is ignored by most of the sparse learning methods. To overcome these challenges, we preserve global label correlations and dynamic local label correlations by preserving graph structure. Additionally, the l2,1-norm and an inner product regularization term are imposed onto the objective function to preserve robust high row-sparsity and to select low redundant features. All the above terms are integrated into one learning framework, and then we utilize a simple yet effective scheme to optimize the framework. Experimental results demonstrate the classification superiority of the proposed method.

Performance and complexity analysis using a sparse deep learning method for indoor terahertz transmission.

In this Letter, we propose and experimentally validate a sparse deep learning method (SDLM) for terahertz indoor wireless-over-fiber by transmitting a 16-quadrature amplitude modulation (QAM) orthogonal frequency-division multiplexing (OFDM) signal over a 15-km single-mode fiber (SMF) and a wireless link distance of 60 cm at 135 GHz through a cost-effective intensity-modulated direct detection (IM-DD) communications system. The proposed SDLM imposes the L1-regularized mechanism on the cost function, which not only improves performance but also reduces complexity when compared with traditional Volterra nonlinear equalizer (VNLE), sparse VNLE, and conventional DLM. Our experimental findings show that the proposed SDLM provides viable options for successfully mitigating nonlinear distortions and outperforms conventional VNLE, conventional DLM, and SVNLE with a 76%, 72%, and 61% complexity reduction, respectively, for 8-QAM without losing signal integrity.

Image Genetic Analysis and Application Research Based on QRFPR and Other Neural Network-Related SNP Loci.

The development of neuroimaging technology and molecular genetics has produced a large amount of imaging genetics data, which has greatly promoted the study of complex mental diseases. However, because the feature dimension of the data is too high, the correlation measure assumes that the data obey Gaussian distribution, and traditional algorithms often cannot explain these two types of data well. This article mainly studies image genetics analysis and its application based on neural network. In this paper, based on the theory and application technology of neural network, the tree structure is established by prior knowledge, that is, each SNP site is used as a leaf node of the tree, and the LD block and genome formed by the linkage imbalance of multiple SNP sites are used as intermediate nodes. Then, the hierarchical relationship of features was introduced. On this basis, a sparse learning method based on tree structure guidance is used to select features from multiple features of multiple SNPs locus regression candidate brain regions. Finally, the identification of SNPs in feature selection is used to predict quantitative traits of brain regions. The distribution of the typical vector values obtained by the algorithm in the experimental data is basically consistent with the distribution of the median of the actual data, and the correlation coefficient obtained is closest to the actual correlation coefficient in the data set. The average correlation coefficient of the algorithm reaches 82.3%, which is about 4.2% higher than the control algorithm. Experimental results show that this method can not only significantly improve the regression performance but also detect the risk gene SNPs loci with spatial clustering features and functional interpretation significance. It is practical and effective to use it in clinical trials.

Hierarchical multi-class Alzheimer’s disease diagnostic framework using imaging and clinical features

Due to the clinical continuum of Alzheimer’s disease (AD), the accuracy of early diagnostic remains unsatisfactory and warrants further research. The objectives of this study were: (1) to develop an effective hierarchical multi-class framework for clinical populations, namely, normal cognition (NC), early mild cognitive impairment (EMCI), late mild cognitive impairment (LMCI), and AD, and (2) to explore the geometric properties of cognition-related anatomical structures in the cerebral cortex. A total of 1,670 participants were enrolled in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, comprising 985 participants (314 NC, 208 EMCI, 258 LMCI, and 205 AD) in the model development set and 685 participants (417 NC, 110 EMCI, 83 LMCI, and 75 AD) after 2017 in the temporal validation set. Four cortical geometric properties for 148 anatomical structures were extracted, namely, cortical thickness (CTh), fractal dimension (FD), gyrification index (GI), and sulcus depth (SD). By integrating these imaging features with Mini-Mental State Examination (MMSE) scores at four-time points after the initial visit, we identified an optimal subset of 40 imaging features using the temporally constrained group sparse learning method. The combination of selected imaging features and clinical variables improved the multi-class performance using the AdaBoost algorithm, with overall accuracy rates of 0.877 in the temporal validation set. Clinical Dementia Rating (CDR) was the primary clinical variable associated with AD-related populations. The most discriminative imaging features included the bilateral CTh of the dorsal part of the posterior cingulate gyrus, parahippocampal gyrus (PHG), parahippocampal part of the medial occipito-temporal gyrus, and angular gyrus, the GI of the left inferior segment of the insula circular sulcus, and the CTh and SD of the left superior temporal sulcus (STS). Our hierarchical multi-class framework underscores the utility of combining cognitive variables with imaging features and the reliability of surface-based morphometry, facilitating more accurate early diagnosis of AD in clinical practice.

DCSRL: a change detection method for remote sensing images based on deep coupled sparse representation learning

ABSTRACT Change detection methods include model-driven methods and data-driven methods. Model-driven methods are rigorous in theory but have difficulties in establishing accurate models for complex problems. Data-driven methods have strong learning abilities, but it is difficult to interpret their mechanism. In order to comprehensively utilize their advantages to highlight changes in multi-temporal remote sensing images. We proposed a change detection method for remote sensing images based on deep coupled sparse representation learning, in which the coupled sparse representation learning is implicitly expressed by convolutional neural networks (CNNs). A coupled sparse representation learning method is first proposed, and then it is unfolded into a CNNs network, the coupled sparse coefficient and dictionary are learned from the training data. Unlike fully data-driven CNNs, auxiliary coupled sparse representation is utilized to guide CNNs to identify the changed areas. Finally, the experiments on two datasets verified the validity of the proposed method.

An adaptive dimension-reduction method-based sparse polynomial chaos expansion via sparse Bayesian learning and Bayesian model averaging

Polynomial chaos expansion (PCE) is a popular surrogate method for stochastic uncertainty quantification (UQ). Nevertheless, when dealing with high-dimensional problems, the well-known curse of dimensionality and overfitting pose great challenges to the PCE. This study proposes an adaptive sparse learning method based on the automatic relevance determination (ARD) and the Bayesian model averaging (BMA) to solve this problem. Firstly, we construct the sparse PCE model in the framework of sparse Bayesian learning. Secondly, the dimension-reduction method (DRM) is used to reduce the size of candidate PCE bases. Different from the common univariate and bivariate DRMs, this study considers the high-dimensional components in an adaptive way. Thirdly, we propose a novel ARD method based on the analytical B-LASSO to prune the candidate PCE bases. The pruned PCE is regarded as an initial model, and then the BMA is used to refine the high-fidelity sparse PCE. Compared with the direct use of the noninformation prior, the intervention of the pruned PCE model can enhance the rationality of the prior, and improve the accuracy of the posterior estimation. We illustrate the validity of the proposed method from different angles using five classical numerical examples and solve one practical engineering problem. The results indicate that the proposed method is a good choice for UQ under limited design samples.

Maintaining the Status Quo: Simultaneous Estimation and Elimination for Multiple Interference in Transform Domain Vehicular Communication

Wireless communication in vehicular networks is highly vulnerable to electromagnetic interference, which seriously undermines the communication effectiveness. Thus, interference estimation and elimination are essential in vehicular communications. To solve the multitasking problem of interference estimation and cancelation within the transformation domain of vehicular communications where they are divided into the consistency and inconsistency based on the different conditions of spectrum sensing, the present study proposes an iterative Bayesian sparse learning approach for simultaneous interference recovery and elimination which applies multiple Bayesian learning for measured signals, and the structured sparse learning method for simultaneous interference mitigation with signal recovery. Furthermore, the updating optimization based on Bayesian estimation framework is performed to facilitate the learning process and improve the performance of interference estimation and signal reconstruction. Simulation results show that the proposed algorithms achieved more rapid convergence and more accurate recovery (31.84% and 27.88% higher in spectrum sensing with consistency and inconsistency, respectively) than conventional algorithms. During all iterations, the proposed algorithms, because of updating optimization, were more robust than conventional ones, but the computational complexity remained largely the same. To explore the correlation between the parameters and the interference recovery performance, we performed experiments, which revealed linear, logarithmic, and exponential regression correlations for particular forms of interference. We also compared the demodulation performance of the proposed algorithm with existing methods, and it turned out the proposed algorithm also obtained higher gains on average in the bit error rate.

Image Classification Based on Sparse Representation in the Quaternion Wavelet Domain

In this study, we propose a novel sparse representation learning method in the Quaternion Wavelet (QW) domain for multi-class image classification. The proposed method takes advantages from: i) the QW decomposition, which promotes sparsity and provides structural information about the image data while allowing approximate shift-invariance, to extract meaningful features from low-frequency QW sub-bands, ii) the dimensionality reduction method using Principal Component Analysis (PCA) for reducing the complexity of the problem, and iii) the sparse representation of the generated QW features to efficiently learn and capture the meaningful and compact information of this data. After the QW decomposition, the features extracted from low-frequency image sub-bands information are projected, by the PCA, into a new feature space with lower dimensionality. The features extracted from the training samples are used to construct a dictionary, while the features of the test samples are sparsely coded for the classification step. The sparse coding problem is formulated in a QW Least Absolute Shrinkage and Selection Operator (QWLasso) model applying quaternion <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{1}$ </tex-math></inline-formula> minimization. A novel Quaternion Fast Iterative Shrinkage-Thresholding Algorithm (QFISTA) is developed to solve the QWLasso model. The experiments conducted on various public image datasets validated that the proposed method possesses higher accuracy, sparsity, and robustness in comparison with several contemporary methods in the field including Neural Networks.

Image emotion distribution learning based on enhanced fuzzy KNN algorithm with sparse learning

With the trend of people expressing opinions and emotions via images online, increasing attention has been paid to affective analysis of visual content. Traditional image affective analysis mainly focuses on single-label classification, but an image usually evokes multiple emotions. To this end, emotion distribution learning is proposed to describe emotions more explicitly. However, most current studies ignore the ambiguity included in emotions and the elusive correlations with complex visual features. Considering that emotions evoked by images are delivered through various visual features, and each feature in the image may have multiple emotion attributes, this paper develops a novel model that extracts multiple features and proposes an enhanced fuzzy k-nearest neighbor (EFKNN) to calculate the fuzzy emotional memberships. Specifically, the multiple visual features are converted into fuzzy emotional memberships of each feature belonging to emotion classes, which can be regarded as an intermediate representation to bridge the affective gap. Then, the fuzzy emotional memberships are fed into a fully connected neural network to learn the relationships between the fuzzy memberships and image emotion distributions. To obtain the fuzzy memberships of test images, a novel sparse learning method is introduced by learning the combination coefficients of test images and training images. Extensive experimental results on several datasets verify the superiority of our proposed approach for emotion distribution learning of images.

Auto-weighted centralised multi-task learning via integrating functional and structural connectivity for subjective cognitive decline diagnosis.

Early diagnosis and intervention of mild cognitive impairment (MCI) and its early stage (i.e., subjective cognitive decline (SCD)) is able to delay or reverse the disease progression. However, discrimination between SCD, MCI and healthy subjects accurately remains challenging. This paper proposes an auto-weighted centralised multi-task (AWCMT) learning framework for differential diagnosis of SCD and MCI. AWCMT is based on structural and functional connectivity information inferred from magnetic resonance imaging (MRI). To be specific, we devise a novel multi-task learning algorithm to combine neuroimaging functional and structural connective information. We construct a functional brain network through a sparse and low-rank machine learning method, and also a structural brain network via fibre bundle tracking. Those two networks are constructed separately and independently. Multi-task learning is then used to identify features integration of functional and structural connectivity. Hence, we can learn each task's significance automatically in a balanced way. By combining the functional and structural information, the most informative features of SCD and MCI are obtained for diagnosis. The extensive experiments on the public and self-collected datasets demonstrate that the proposed algorithm obtains better performance in classifying SCD, MCI and healthy people than traditional algorithms. The newly proposed method has good interpretability as it is able to discover the most disease-related brain regions and their connectivity. The results agree well with current clinical findings and provide new insights into early AD detection based on the multi-modal neuroimaging technique.

A novel intelligent fault diagnosis method based on variant sparse filtering and back-propagation

Detecting the mechanical faults of rotating machinery in time plays a key role in avoiding accidents. With the coming of the big data era, intelligent fault diagnosis methods based on machine learning models have become promising tools. To improve the feature learning ability, an unsupervised sparse feature learning method called variant sparse filtering is developed. Then, a fault diagnosis method combining variant sparse filtering with a back-propagation algorithm is presented. The involvement of the back-propagation algorithm can further optimize the weight matrix of variant sparse filtering using label data. At last, the developed diagnosis method is validated by rolling bearing and planetary gearbox experiments. The experiment results indicate that the developed method can achieve high accuracy and good stability in rotating machinery fault diagnosis.

Case Histories on 2D/3D Underground Stratification Using Sparse Machine Learning

This paper showcases novel underground stratification based on sparse machine learning (SML) methods, including sparse Bayesian learning (SBL) and least absolute shrinkage selection operator (Lasso). The SML methods proposed by the authors were applied to two- and three-dimensional underground stratification of actual sites, Odagawa Riverbank (Okayama, Japan) and New Lock (Terneuzen, the Netherlands), to demonstrate their performances. Cone penetration test (CPT) data were available in both sites, and they were converted to the soil behavior type (SBT) index for the underground stratification analysis. The trends of Ic/SBT profiles and their distribution colormaps estimated by the two methods were compared to discuss the methodological characteristics. For the Odagawa Riverbank case, the detection ratio of SBT obtained by the two methods was also compared to investigate the estimation accuracy in terms of stratification ability.

Robust Rank-Constrained Sparse Learning: A Graph-Based Framework for Single View and Multiview Clustering.

Graph-based clustering aims to partition the data according to a similarity graph, which has shown impressive performance on various kinds of tasks. The quality of similarity graph largely determines the clustering results, but it is difficult to produce a high-quality one, especially when data contain noises and outliers. To solve this problem, we propose a robust rank constrained sparse learning (RRCSL) method in this article. The L2,1 -norm is adopted into the objective function of sparse representation to learn the optimal graph with robustness. To preserve the data structure, we construct an initial graph and search the graph within its neighborhood. By incorporating a rank constraint, the learned graph can be directly used as the cluster indicator, and the final results are obtained without additional postprocessing. In addition, the proposed method cannot only be applied to single-view clustering but also extended to multiview clustering. Plenty of experiments on synthetic and real-world datasets have demonstrated the superiority and robustness of the proposed framework.

Fusing functional connectivity with network nodal information for sparse network pattern learning of functional brain networks

Sparse learning methods have been powerful tools for learning compact representations of functional brain networks consisting of a set of brain network nodes and a connectivity matrix measuring functional coherence between the nodes. However, these tools typically focus on the functional connectivity measures alone, ignoring the brain network nodal information that is complementary to the functional connectivity measures for comprehensively characterizing the functional brain networks. In order to provide a comprehensive delineation of the functional brain networks, we develop a new data fusion method for heterogeneous data, aiming at learning sparse network patterns to characterize both the functional connectivity measures and their complementary network nodal information within a unified framework. Experimental results have demonstrated that our method outperforms the best alternative method under comparison in terms of accuracy on simulated data as well as both reproducibility and prediction performance of brain age on real resting state functional magnetic resonance imaging data.

Joint sparse neural network compression via multi-application multi-objective optimization

Over the pass decade, deep neural network (DNN) has been widely applied in various applications. To alleviate the storage and computation requirement of the complicated DNNs, network compression methods are developed. The sparse structure learning methods based on multi-objective optimization have been proven to be valid to balance the sparsity of the network model and network performance. However, when multiple applications are deployed on one single platform simultaneously, these methods become inefficient because each network model for each application needs to be trained and optimized individually. In this article, a multi-objective, multi-application sparse learning model is proposed to optimize multiple targets from a set of applications together. The joint network structure is first proposed. After a pre-training of the network model, a joint multi-objective evolutionary algorithm is derived to solve the optimization problems. Note that an improved initialization method for parent model generation is also developed. Finally, based on the joint loss between the objectives, fine tuning is used to compute the final models with good performance. The proposed method is evaluated under different datasets with a comparison to the state-of-the-art approaches, and experimental results demonstrate that the multi-application optimization model can give much better performance than the single-application optimization ones, especially in the case that different datasets are involved simultaneously.

Robust Long-Term Visual Object Tracking via Low-Rank Sparse Learning for Re-Detection

The Discriminative Correlation Filter (DCF) has been universally recognized in visual object tracking, thanks to its excellent accuracy and high speed. Nevertheless, these DCF-based trackers perform poorly in long-term tracking. The reasons include the following aspects—first, they have low adaptability to significant appearance changes in long-term tracking and are prone to tracking failure; second, these trackers lack a practical re-detection module to find the target again after tracking failure. In our work, we propose a new long-term tracking strategy to solve these issues. First, we make the best of the static and dynamic information of the target by introducing the motion features to our long-term tracker and obtain a more robust tracker. Second, we introduce a low-rank sparse dictionary learning method for re-detection. This re-detection module can exploit a correlation among these training samples and alleviate the impact of occlusion and noise. Third, we propose a new reliability evaluation method to model an adaptive update, which can switch expediently between the tracking module and the re-detection module. Massive experiments demonstrate that our proposed approach has an obvious improvement in precision and success rate over these state-of-the-art trackers.

Multi-Block Color-Binarized Statistical Images for Single-Sample Face Recognition.

Single-Sample Face Recognition (SSFR) is a computer vision challenge. In this scenario, there is only one example from each individual on which to train the system, making it difficult to identify persons in unconstrained environments, mainly when dealing with changes in facial expression, posture, lighting, and occlusion. This paper discusses the relevance of an original method for SSFR, called Multi-Block Color-Binarized Statistical Image Features (MB-C-BSIF), which exploits several kinds of features, namely, local, regional, global, and textured-color characteristics. First, the MB-C-BSIF method decomposes a facial image into three channels (e.g., red, green, and blue), then it divides each channel into equal non-overlapping blocks to select the local facial characteristics that are consequently employed in the classification phase. Finally, the identity is determined by calculating the similarities among the characteristic vectors adopting a distance measurement of the K-nearest neighbors (K-NN) classifier. Extensive experiments on several subsets of the unconstrained Alex and Robert (AR) and Labeled Faces in the Wild (LFW) databases show that the MB-C-BSIF achieves superior and competitive results in unconstrained situations when compared to current state-of-the-art methods, especially when dealing with changes in facial expression, lighting, and occlusion. The average classification accuracies are 96.17% and 99% for the AR database with two specific protocols (i.e., Protocols I and II, respectively), and 38.01% for the challenging LFW database. These performances are clearly superior to those obtained by state-of-the-art methods. Furthermore, the proposed method uses algorithms based only on simple and elementary image processing operations that do not imply higher computational costs as in holistic, sparse or deep learning methods, making it ideal for real-time identification.

A framework to identify structured behavioral patterns within rodent spatial trajectories

Animal behavior is highly structured. Yet, structured behavioral patterns—or “statistical ethograms”—are not immediately apparent from the full spatiotemporal data that behavioral scientists usually collect. Here, we introduce a framework to quantitatively characterize rodent behavior during spatial (e.g., maze) navigation, in terms of movement building blocks or motor primitives. The hypothesis that we pursue is that rodent behavior is characterized by a small number of motor primitives, which are combined over time to produce open-ended movements. We assume motor primitives to be organized in terms of two sparsity principles: each movement is controlled using a limited subset of motor primitives (sparse superposition) and each primitive is active only for time-limited, time-contiguous portions of movements (sparse activity). We formalize this hypothesis using a sparse dictionary learning method, which we use to extract motor primitives from rodent position and velocity data collected during spatial navigation, and successively to reconstruct past trajectories and predict novel ones. Three main results validate our approach. First, rodent behavioral trajectories are robustly reconstructed from incomplete data, performing better than approaches based on standard dimensionality reduction methods, such as principal component analysis, or single sparsity. Second, the motor primitives extracted during one experimental session generalize and afford the accurate reconstruction of rodent behavior across successive experimental sessions in the same or in modified mazes. Third, in our approach the number of motor primitives associated with each maze correlates with independent measures of maze complexity, hence showing that our formalism is sensitive to essential aspects of task structure. The framework introduced here can be used by behavioral scientists and neuroscientists as an aid for behavioral and neural data analysis. Indeed, the extracted motor primitives enable the quantitative characterization of the complexity and similarity between different mazes and behavioral patterns across multiple trials (i.e., habit formation). We provide example uses of this computational framework, showing how it can be used to identify behavioural effects of maze complexity, analyze stereotyped behavior, classify behavioral choices and predict place and grid cell displacement in novel environments.

Improved atmospheric weighted mean temperature modeling using sparse kernel learning

As a crucial parameter in the process of converting the zenith wet delay into precipitable water vapor at Global Navigation Satellite System (GNSS) stations, the weighted mean temperature (Tm) influences the accuracy of GNSS-based water vapor retrievals. We propose an improved atmospheric Tm modeling method by introducing sparse kernel learning to obtain high-accuracy and high-temporal-resolution Tm data. To establish the model, the temporal variation characteristics of Tm time series are first analyzed by spectral analysis using the Lomb–Scargle periodogram. Second, the sparse kernel learning method is introduced to model the residuals from spectral analysis, for which the Gauss radial basis function kernel, L1-norm regularization, and the highly efficient fast iterative shrinkage thresholding algorithm are employed. To verify the performance of the proposed method, we used ERA5 hourly data from China’s 9 International GNSS Service stations produced by the European Centre for Medium-Range Weather Forecasts. ERA5 Tm data with a temporal resolution of 12 h are used as the training data, and the ERA5 hourly Tm data (excluding the data employed for modeling) are used for testing. The results show that compared with the spectral analysis accuracy, the accuracy of the calculated Tm can be improved by approximately 48.3% when the residual sparse kernel learning method is used. Thus, this proposed method for GNSS-based water vapor retrievals can provide high-accuracy and high-temporal-resolution Tm data.

Sovereign credit default swaps and macroeconomic fundamentals

We provide evidence in favour of a significant non-linear, time-varying dependence between sovereign credit default swap (CDS) spreads and macroeconomic fundamentals for OECD countries. Macroeconomic conditions alone explain more than 80% of the out-of-sample variation in CDS spreads when non-linear random forest regressions are used. This is almost double the predictive performance of sparse and dense linear statistical learning methods. We test the coherence of the expected CDS spreads across different pure out-of-sample scenarios, e.g. training a random forest regression on EU countries and predicting CDS spreads for non-EU economies. The results suggest that non-linear machine learning methods enable “shadow” sovereign CDS pricing for countries and historical periods in which tradable sovereign CDS contracts are not available.