Multi-task Sparse Learning Research Articles

Multiple ocular diseases detection based on joint sparse multi-task learning.

In this paper, we present a multiple ocular diseases detection scheme based on joint sparse multi-task learning. Glaucoma, Pathological Myopia (PM), and Age-related Macular Degeneration (AMD) are three major causes of vision impairment and blindness worldwide. The proposed joint sparse multitask learning framework aims to reconstruct a test fundus image with multiple features from as few training subjects as possible. The linear version of this problem could be casted into a multi-task joint covariate selection model, which can be very efficiently optimized via kernelizable accelerated proximal gradient method. Extensive experiments are conducted in order to validate the proposed framework on the SiMES dataset. From the Area Under Curve (AUC) results in multiple ocular diseases classification, our method is shown to outperform the state-of-the-art algorithms.

Sparse multi-task reinforcement learning

In multi-task reinforcement learning (MTRL), the objective is to simultaneously learn multiple tasks and exploit their similarity to improve the performance w.r.t. single-task learning. In this paper we investigate the case when all the tasks can be accurately represented in a linear approximation space using the same small subset of the original (large) set of features. This is equivalent to assuming that the weight vectors of the task value functions are jointly sparse, i.e., the set of their non-zero components is small and it is shared across tasks. Building on existing results in multi-task regression, we develop two multi-task extensions of the fitted Q-iteration algorithm. While the first algorithm assumes that the tasks are jointly sparse in the given representation, the second one learns a transformation of the features in the attempt of finding a more sparse representation. For both algorithms we provide a sample complexity analysis and numerical simulations.

Deep sparse multi-task learning for feature selection in Alzheimer's disease diagnosis.

Recently, neuroimaging-based Alzheimer's disease (AD) or mild cognitive impairment (MCI) diagnosis has attracted researchers in the field, due to the increasing prevalence of the diseases. Unfortunately, the unfavorable high-dimensional nature of neuroimagingdata, but a limited small number of samples available, makes it challenging to build a robust computer-aided diagnosis system. Machine learning techniques have been considered as a useful tool in this respect and, among various methods, sparse regression has shown its validity in the literature. However, to our best knowledge, the existing sparse regression methods mostly try to select features based on the optimal regression coefficients in one step. We argue that since the training feature vectors are composed of both informative and uninformative or less informative features, the resulting optimal regression coefficients are inevidently affected by the uninformative or less informative features. To this end, we first propose a novel deep architecture to recursively discard uninformative features by performing sparse multi-task learning in a hierarchical fashion. We further hypothesize that the optimal regression coefficients reflect the relative importance of features in representing the target response variables. In this regard, we use the optimal regression coefficients learned in one hierarchy as feature weighting factors in the following hierarchy, and formulate a weighted sparse multi-task learning method. Lastly, we also take into account the distributional characteristics of samples per class and use clustering-induced subclass label vectors as target response values in our sparse regression model. In our experiments on the ADNI cohort, we performed both binary and multi-class classification tasks in AD/MCI diagnosis and showed the superiority of the proposed method by comparing with the state-of-the-art methods.

Visual tracking via robust multitask sparse prototypes

Sparse representation has been applied to an online subspace learning-based tracking problem. To handle partial occlusion effectively, some researchers introduce l1 regularization to principal component analysis (PCA) reconstruction. However, in these traditional tracking methods, the representation of each object observation is often viewed as an individual task so the inter-relationship between PCA basis vectors is ignored. We propose a new online visual tracking algorithm with multitask sparse prototypes, which combines multitask sparse learning with PCA-based subspace representation. We first extend a visual tracking algorithm with sparse prototypes in multitask learning framework to mine inter-relations between subtasks. Then, to avoid the problem that enforcing all subtasks to share the same structure may result in degraded tracking results, we impose group sparse constraints on the coefficients of PCA basis vectors and element-wise sparse constraints on the error coefficients, respectively. Finally, we show that the proposed optimization problem can be effectively solved using the accelerated proximal gradient method with the fast convergence. Experimental results compared with the state-of-the-art tracking methods demonstrate that the proposed algorithm achieves favorable performance when the object undergoes partial occlusion, motion blur, and illumination changes.

Network traffic classification via non-convex multi-task feature learning

Machine learning has been used in network traffic classification and statistical features are used to represent flows. However, conventional feature selection may work out in face of dynamic and complex traffic data. Multi-Task Learning has obtained quite wide attention nowadays, and one important form of multi-task learning is to exploit the features shared by tasks by sparse models. We propose a fast multi-task sparse feature learning method, using a non-convex Capped-ℓ1,ℓ1 as the regularizer to learn a set of shared features in traffic data. Specifically, the non-convex multi-task feature learning model can learn features belonging to each task as well as the common features shared among tasks. We use the iterative shrinkage and thresholding (IST) algorithm to solve the problem, which has a closed-form solution for one of the crucial steps in the whole iteration. Experiment on real traffic data captured from backbone network as well as synthetic data and other popular real-world data show the effectiveness the method, compared with state-of-the-art methods.

Learning Multiscale Active Facial Patches for Expression Analysis.

In this paper, we present a new idea to analyze facial expression by exploring some common and specific information among different expressions. Inspired by the observation that only a few facial parts are active in expression disclosure (e.g., around mouth, eye), we try to discover the common and specific patches which are important to discriminate all the expressions and only a particular expression, respectively. A two-stage multitask sparse learning (MTSL) framework is proposed to efficiently locate those discriminative patches. In the first stage MTSL, expression recognition tasks are combined to located common patches. Each of the tasks aims to find dominant patches for each expression. Secondly, two related tasks, facial expression recognition and face verification tasks, are coupled to learn specific facial patches for individual expression. The two-stage patch learning is performed on patches sampled by multiscale strategy. Extensive experiments validate the existence and significance of common and specific patches. Utilizing these learned patches, we achieve superior performances on expression recognition compared to the state-of-the-arts.

Cortical surface biomarkers for predicting cognitive outcomes using group l2,1 norm

Regression models have been widely studied to investigate the prediction power of neuroimaging measures as biomarkers for inferring cognitive outcomes in the Alzheimer's disease study. Most of these models ignore the interrelated structures either within neuroimaging measures or between cognitive outcomes, and thus may have limited power to yield optimal solutions. To address this issue, we propose to use a new sparse multitask learning model called Group-Sparse Multi-task Regression and Feature Selection (G-SMuRFS) and demonstrate its effectiveness by examining the predictive power of detailed cortical thickness measures toward 3 types of cognitive scores in a large cohort. G-SMuRFS proposes a group-level l2,1-norm strategy to group relevant features together in an anatomically meaningful manner and use this prior knowledge to guide the learning process. This approach also takes into account the correlation among cognitive outcomes for building a more appropriate predictive model. Compared with traditional methods, G-SMuRFS not only demonstrates a superior performance but also identifies a small set of surface markers that are biologically meaningful.

Sparse multi-stage regularized feature learning for robust face recognition

The major limitation in current facial recognition systems is that they do not perform very well in uncontrolled environments, that is, when faces present variations in pose, illumination, facial expressions and environment. This is a serious obstacle in applications such as law enforcement and surveillance systems. To address this limitation, in this paper we introduce an improved approach to ensure robust face recognition, that relies on two innovative ideas. First, we apply a new multiscale directional framework, called Shearlet Network (SN), to extract facial features. The advantage of this approach comes from the highly sparse representation properties of the shearlet framework that is especially designed to robustly extract the fundamental geometric content of an image. Second, we apply a refinement of the Multi-Task Sparse Learning (MTSL) framework to exploit the relationships among multiple shared tasks generated by changing the regularization parameter during the recognition stage. We provide extensive numerical tests to show that our Sparse Multi-Regularized Shearlet Network (SMRSN) algorithm performs very competitively when compared against different state-of-the-art methods on different experimental protocols, including face recognition in uncontrolled conditions and single-sample-per-person.

Kernel Sparse Multitask Learning for Hyperspectral Image Classification With Empirical Mode Decomposition and Morphological Wavelet-Based Features

Recently, many researchers have attempted to exploit spectral–spatial features and sparsity-based hyperspectral image classifiers for higher classification accuracy. However, challenges remain for efficient spectral–spatial feature generation and combination in the sparsity-based classifiers. This paper utilizes the empirical mode decomposition (EMD) and morphological wavelet transform (MWT) to gain spectral–spatial features, which can be significantly integrated by the sparse multitask learning (MTL). In the feature extraction step, the sum of the intrinsic mode functions extracted by an optimized EMD is taken as spectral features, whereas the spatial features are formed by the low-frequency components of one-level MWT. In the classification step, a kernel-based sparse MTL solved by the accelerated proximal gradient is applied to analyze both the spectral and spatial features simultaneously. Experiments are conducted on two benchmark data sets with different spectral and spatial resolutions. It is found that the proposed methods provide more accurate classification results compared to the state-of-the-art techniques with various ratio of training samples.

Multi-modality canonical feature selection for Alzheimer's disease diagnosis.

Feature selection has been commonly regarded as an effective method to lessen the problem of high dimension and low sample size in medical image analysis. In this paper, we propose a novel multimodality canonical feature selection method. Unlike the conventional sparse Multi-Task Learning (MTL) based feature selection method that mostly considered only the relationship between target response variables, we further consider the correlations between features of different modalities by projecting them into a canonical space determined by canonical correlation analysis. We call the projections as canonical representations. By setting the canonical representations as regressors in a sparse least square regression framework and by further penalizing the objective function with a new canonical regularizer on the weight coefficient matrix, we formulate a multi-modality canonical feature selection method. With the help of the canonical information of canonical representations and also a canonical regularizer, the proposed method selects canonical-cross-modality features that are useful for the tasks of clinical scores regression and multi-class disease identification. In our experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, we combine Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) images to jointly predict clinical scores of Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog) and Mini-Mental State Examination (MMSE) and also identify multiclass disease status for Alzheimer's disease diagnosis.

Multi-stage multi-task feature learning

Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including comp...

Robust Visual Tracking via Structured Multi-Task Sparse Learning

In this paper, we formulate object tracking in a particle filter framework as a structured multi-task sparse learning problem, which we denote as Structured Multi-Task Tracking (S-MTT). Since we model particles as linear combinations of dictionary templates that are updated dynamically, learning the representation of each particle is considered a single task in Multi-Task Tracking (MTT). By employing popular sparsity-inducing $$\ell _{p,q}$$ mixed norms $$(\text{ specifically} p\in \{2,\infty \}$$ and $$q=1),$$ we regularize the representation problem to enforce joint sparsity and learn the particle representations together. As compared to previous methods that handle particles independently, our results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity. Interestingly, we show that the popular $$L_1$$ tracker (Mei and Ling, IEEE Trans Pattern Anal Mach Intel 33(11):2259---2272, 2011) is a special case of our MTT formulation (denoted as the $$L_{11}$$ tracker) when $$p=q=1.$$ Under the MTT framework, some of the tasks (particle representations) are often more closely related and more likely to share common relevant covariates than other tasks. Therefore, we extend the MTT framework to take into account pairwise structural correlations between particles (e.g. spatial smoothness of representation) and denote the novel framework as S-MTT. The problem of learning the regularized sparse representation in MTT and S-MTT can be solved efficiently using an Accelerated Proximal Gradient (APG) method that yields a sequence of closed form updates. As such, S-MTT and MTT are computationally attractive. We test our proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that S-MTT is much better than MTT, and both methods consistently outperform state-of-the-art trackers.

Minimum Description Length Penalization for Group and Multi-Task Sparse Learning

We propose a framework MIC (Multiple Inclusion Criterion) for learning sparse models based on the information theoretic Minimum Description Length (MDL) principle. MIC provides an elegant way of incorporating arbitrary sparsity patterns in the feature space by using two-part MDL coding schemes. We present MIC based models for the problems of grouped feature selection (MIC-GROUP) and multi-task feature selection (MIC-MULTI). MIC-GROUP assumes that the features are divided into groups and induces two level sparsity, selecting a subset of the feature groups, and also selecting features within each selected group. MIC-MULTI applies when there are multiple related tasks that share the same set of potentially predictive features. It also induces two level sparsity, selecting a subset of the features, and then selecting which of the tasks each feature should be added to. Lastly, we propose a model, TRANSFEAT, that can be used to transfer knowledge from a set of previously learned tasks to a new task that is expected to share similar features. All three methods are designed for selecting a small set of predictive features from a large pool of candidate features. We demonstrate the effectiveness of our approach with experimental results on data from genomics and from word sense disambiguation problems.