Multi-task Feature Selection Research Articles

DMTFS-FO: Dynamic multi-task feature selection based on flexible loss and orthogonal constraint

Multi-task feature selection (MTFS) has been proven effective for reducing the curse of dimensionality in large-scale classification. Many existing MTFS methods assume that all tasks learn concurrently in a static environment without considering the dynamism of tasks in real-world scenarios. However, new tasks emerge dynamically in practical applications, meaning that the aforementioned static assumption is insufficient. In this paper, we construct a dynamic multi-task feature selection framework to achieve feature reduction for constantly arriving new tasks. First, we modify the traditional mapping by shifting from hard labels to soft labels. Unlike the conventional rigid mapping, the new flexible loss function changes the direct mapping strategy to an indirect one. Second, we use the orthogonal regularization term to constrain the independent relationship between new and old tasks. This ensures that the selected relevant features for new tasks differ from prior tasks. Finally, we integrate the flexible loss and the orthogonal regularization term in the dynamic multi-task feature selection framework. Our method outperforms nine other advanced feature selection methods in terms of effectiveness and efficiency across six datasets. For example, the ACC value of our method is almost 1% higher than the next-best method on the large-scale SUN dataset.

Surgery Duration Prediction Using Multi-Task Feature Selection.

Efficient optimization of operating room (OR) activity poses a significant challenge for hospital managers due to the complex and risky nature of the environment. The traditional "one size fits all" approach to OR scheduling is no longer practical, and personalized medicine is required to meet the diverse needs of patients, care providers, medical procedures, and system constraints within limited resources. This paper aims to introduce a scientific and practical tool for predicting surgery durations and improving OR performance for maximum benefit to patients and the hospital. Previous works used machine-learning models for surgery duration prediction based on preoperative data. The models consider covariates known to the medical staff at the time of scheduling the surgery. Given a large number of covariates, model selection becomes crucial, and the number of covariates used for prediction depends on the available sample size. Our proposed approach utilizes multi-task regression to select a common subset of predicting covariates for all tasks with the same sample size while allowing the model's coefficients to vary between them. A regression task can refer to a single surgeon or operation type or the interaction between them. By considering these diverse factors, our method provides an overall more accurate estimation of the surgery durations, and the selected covariates that enter the model may help to identify the resources required for a specific surgery. We found that when the regression tasks were surgeon-based or based on the pair of operation type and surgeon, our suggested approach outperformed the compared baseline suggested in a previous study. However, our approach failed to reach the baseline for an operation-type-based task. By accurately estimating surgery durations, hospital managers can provide care to a greater number of patients, optimize resource allocation and utilization, and reduce waste. This research contributes to the advancement of personalized medicine and provides a valuable tool for improving operational efficiency in the dynamic world of medicine.

Hypergraph-Based Multitask Feature Selection with Temporally Constrained Group Sparsity Learning on fMRI

Localizing the brain regions affected by tasks is crucial to understanding the mechanisms of brain function. However, traditional statistical analysis does not accurately identify the brain regions of interest due to factors such as sample size, task design, and statistical effects. Here, we propose a hypergraph-based multitask feature selection framework, referred to as HMTFS, which we apply to a functional magnetic resonance imaging (fMRI) dataset to extract task-related brain regions. HMTFS is characterized by its ability to construct a hypergraph through correlations between subjects, treating each subject as a node to preserve high-order information of time-varying signals. Additionally, it manages feature selection across different time windows in fMRI data as multiple tasks, facilitating time-constrained group sparse learning with a smoothness constraint. We utilize a large fMRI dataset from the Human Connectome Project (HCP) to validate the performance of HMTFS in feature selection. Experimental results demonstrate that brain regions selected by HMTFS can provide higher accuracy for downstream classification tasks compared to other competing feature selection methods and align with findings from previous neuroscience studies.

Multitask feature selection within structural datasets

Abstract Population-based structural health monitoring (PBSHM) systems use data from multiple structures to make inferences of health states. An area of PBSHM that has recently been recognized for potential development is the use of multitask learning (MTL) algorithms that differ from traditional single-task learning. This study presents an application of the MTL approach, Joint Feature Selection with LASSO, to provide automatic feature selection. The algorithm is applied to two structural datasets. The first dataset covers a binary classification between the port and starboard side of an aircraft tailplane, for samples from two aircraft of the same model. The second dataset covers normal and damaged conditions for pre- and postrepair of the same aircraft wing. Both case studies demonstrate that the MTL results are interpretable, highlighting features that relate to structural differences by considering the patterns shared between tasks. This is opposed to single-task learning, which improved accuracy at the cost of interpretability and selected features, which failed to generalize in previously unobserved experiments.

Joint multi-site domain adaptation and multi-modality feature selection for the diagnosis of psychiatric disorders

Identifying biomarkers for computer-aided diagnosis (CAD) is crucial for early intervention of psychiatric disorders. Multi-site data have been utilized to increase the sample size and improve statistical power, while multi-modality classification offers significant advantages over traditional single-modality based approaches for diagnosing psychiatric disorders. However, inter-site heterogeneity and intra-modality heterogeneity present challenges to multi-site and multi-modality based classification. In this paper, brain functional and structural networks (BFNs/BSNs) from multiple sites were constructed to establish a joint multi-site multi-modality framework for psychiatric diagnosis. To do this we developed a hypergraph based multi-source domain adaptation (HMSDA) which allowed us to transform source domain subjects into a target domain. A local ordinal structure based multi-task feature selection (LOSMFS) approach was developed by integrating the transformed functional and structural connections (FCs/SCs). The effectiveness of our method was validated by evaluating diagnosis of both schizophrenia (SZ) and autism spectrum disorder (ASD). The proposed method obtained accuracies of 92.2 %±2.22 % and 84.8 %±2.68 % for the diagnosis of SZ and ASD, respectively. We also compared with 6 DA, 10 multi-modality feature selection, and 8 multi-site and multi-modality methods. Results showed the proposed HMSDA+LOSMFS effectively integrated multi-site and multi-modality data to enhance psychiatric diagnosis and identify disorder-specific diagnostic brain connections.

Diagnosis of Alzheimer’s disease using hypergraph [formula omitted]-Laplacian regularized multi-task feature learning

Multimodal data-based classification methods have been widely used in the diagnosis of Alzheimer’s disease (AD) and have achieved better performance than single-modal-based methods. However, most classification methods based on multimodal data tend to consider only the correlation between different modal data and ignore the inherent non-linear higher-order relationships between similar data, which can improve the robustness of the model. Therefore, this study proposes a hypergraph p-Laplacian regularized multi-task feature selection (HpMTFS) method for AD classification. Specifically, feature selection for each modal data is considered as a distinct task and the common features of multimodal data are extracted jointly by group-sparsity regularizer. In particular, two regularization terms are introduced in this study, namely (1) a hypergraph p-Laplacian regularization term to retain higher-order structural information for similar data, and (2) a Frobenius norm regularization term to improve the noise immunity of the model. Finally, using a multi-kernel support vector machine to fuse multimodal features and perform the final classification. We used baseline sMRI, FDG-PET, and AV-45 PET imaging data from 528 subjects in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) to evaluate our approach. Experimental results show that our HpMTFS method outperforms existing multimodal-based classification methods.

Predictive classification of Alzheimer\u2019s disease using brain imaging and genetic data

For now, Alzheimer’s disease (AD) is incurable. But if it can be diagnosed early, the correct treatment can be used to delay the disease. Most of the existing research methods use single or multi-modal imaging features for prediction, relatively few studies combine brain imaging with genetic features for disease diagnosis. In order to accurately identify AD, healthy control (HC) and the two stages of mild cognitive impairment (MCI: early MCI, late MCI) combined with brain imaging and genetic characteristics, we proposed an integrated Fisher score and multi-modal multi-task feature selection research method. We learned first genetic features with Fisher score to perform dimensionality reduction in order to solve the problem of the large difference between the feature scales of genetic and brain imaging. Then we learned the potential related features of brain imaging and genetic data, and multiplied the selected features with the learned weight coefficients. Through the feature selection program, five imaging and five genetic features were selected to achieve an average classification accuracy of 98% for HC and AD, 82% for HC and EMCI, 86% for HC and LMCI, 80% for EMCI and LMCI, 88% for EMCI and AD, and 72% for LMCI and AD. Compared with only using imaging features, the classification accuracy has been improved to a certain extent, and a set of interrelated features of brain imaging phenotypes and genetic factors were selected.

Multi-Task Feature Selection for Speech Emotion Recognition: Common Speaker-Independent Features Among Emotions

Feature selection is the one of the most important steps in designing speech emotion recognition systems. Because there is uncertainty as to which speech feature is related to which emotion, many features must be taken into account and, for this purpose, identifying the most discriminative features is necessary. In the interest of selecting appropriate emotion-related speech features, the current paper focuses on a multi-task approach. For this reason, the study considers each speaker as a task and proposes a multi-task objective function to select features. As a result, the proposed method chooses one set of speaker-independent features of which the selected features are discriminative in all emotion classes. Correspondingly, multi-class classifiers are utilized directly or binary classifications simply perform multi-class classifications. In addition, the present work employs two well-known datasets, the Berlin and Enterface. The experiments also applied the openSmile toolkit to extract more than 6500 features. After feature selection phase, the results illustrated that the proposed method selects the features which is common in the different runs. Also, the runtime of proposed method is the lowest in comparison to other methods. Finally, 7 classifiers are employed and the best achieved performance is 73.76% for the Berlin dataset and 72.17% for the Enterface dataset, in the faced of a new speaker .These experimental results then show that the proposed method is superior to existing state-of-the-art methods.

Improving the Accuracy of Subseasonal Forecasting of China Precipitation With a Machine Learning Approach

Precipitation change, which is closely related to drought and flood disasters in China, affects billions of people every year, and the demand for subseasonal forecasting of precipitation is even more urgent. Subseasonal forecasting, which is more difficult than weather forecasting, however, has remained as a blank area in meteorological service for a long period of time. To improve the accuracy of subseasonal forecasting of China precipitation, this work introduces the machine learning method proposed by Hwang et al. in 2019 to predict the precipitation in China 2–6 weeks in advance. The authors used a non-linear regression model called local linear regression together with multitask feature election (MultiLLR) model and chosen 21 meteorological elements as candidate predictors to integrate diverse meteorological observation data. This method automatically eliminates irrelevant predictors so as to establish the forecast equations using multitask feature selection process. The experiments demonstrate that the pressure and Madden–Julian Oscillation (MJO) are the most important physical factors. The average prediction skill is 0.11 during 2011–2016, and there are seasonal differences in forecasting skills, evidenced by higher forecast skills of winter and spring seasons than summer and autumn seasons. The proposed method can provide effective and indicative guidance for the subseasonal prediction of precipitation in China. By adding another three factors, Arctic Oscillation (AO) index, Western North Pacific Monsoon (WNPM) index and Western North Pacific Subtropical High (WNPSH) index into the MultiLLR model, the authors find that AO can improve the forecast skill of China precipitation to the maximum extent from 0.11 to 0.13, followed by WNPSH. Moreover, the ensemble skill of our model and CFSv2 is 0.16. This work shows that our subseasonal prediction of China precipitation should be benefited from the MultiLLR model.

A method of credit evaluation modeling based on block-wise missing data

Missing data is a common problem in credit evaluation practice and can obstruct the development and application of an evaluation model. Block-wise missing data is a particularly troublesome issue. Based on multi-task feature selection approach, this paper proposes a method called MMPFS to build a model for credit evaluation that primarily includes two steps: (1) dividing the dataset into several nonoverlapping subsets based on missing patterns, and (2) integrating the multi-task feature selection approach using logistic regression to perform joint feature learning on all subsets. The proposed method has the following advantages: (1) missing data do not need to be managed in advance, (2) available data can be fully used for model learning, (3) information loss or bias caused by general missing data processing methods can be avoided, and (4) overfitting risk caused by redundant features can be reduced. The implementation framework and algorithm principle of the proposed method are described, and three credit datasets from UCI are investigated to compare the proposed method with other commonly used missing data treatments. The results show that MMPFS can produce a better credit evaluation model than data preprocessing methods, such as sample deletion and data imputation.

Identification of early mild cognitive impairment using multi-modal data and graph convolutional networks

BackgroundThe identification of early mild cognitive impairment (EMCI), which is an early stage of Alzheimer’s disease (AD) and is associated with brain structural and functional changes, is still a challenging task. Recent studies show great promises for improving the performance of EMCI identification by combining multiple structural and functional features, such as grey matter volume and shortest path length. However, extracting which features and how to combine multiple features to improve the performance of EMCI identification have always been a challenging problem. To address this problem, in this study we propose a new EMCI identification framework using multi-modal data and graph convolutional networks (GCNs). Firstly, we extract grey matter volume and shortest path length of each brain region based on automated anatomical labeling (AAL) atlas as feature representation from T1w MRI and rs-fMRI data of each subject, respectively. Then, in order to obtain features that are more helpful in identifying EMCI, a common multi-task feature selection method is applied. Afterwards, we construct a non-fully labelled subject graph using imaging and non-imaging phenotypic measures of each subject. Finally, a GCN model is adopted to perform the EMCI identification task.ResultsOur proposed EMCI identification method is evaluated on 210 subjects, including 105 subjects with EMCI and 105 normal controls (NCs), with both T1w MRI and rs-fMRI data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Experimental results show that our proposed framework achieves an accuracy of 84.1% and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.856 for EMCI/NC classification. In addition, by comparison, the accuracy and AUC values of our proposed framework are better than those of some existing methods in EMCI identification.ConclusionOur proposed EMCI identification framework is effective and promising for automatic diagnosis of EMCI in clinical practice.

A novel semisupervised classification framework for coupling faults in hot rolling mill process

Compared with single fault, the occurrence and composition of coupling faults have more uncertainties and diversities, which make fault classification a challenging topic in academic research and industrial application areas. In this paper, the classification problems of coupling faults are addressed from a new perspective, which will provide diagnostic decisions for online operators to take immediate remedial measures to bring the abnormal operation back to an incontrol state. Specifically, the main innovations are: (1) a semisupervised classification scheme for coupling faults is first proposed, which combines adaptive classification with multi-task feature selection; (2) number of classifications can be learned adaptively and automatically; (3) common and specific features among single and the associated coupling faults can be captured, which are crucial for improving classification performance. A case study on hot rolling mill process is finally given to validate the effectiveness of the proposed scheme, and several competitive methods are employed to carry out the classification process. It can be observed that the obtained classification results for two different cases are more successful than the traditional methods.

Improved ASD classification using dynamic functional connectivity and multi-task feature selection

Abstract Accurate diagnosis of autism spectrum disorder (ASD), which is a neurodevelopmental disorder and often accompanied by abnormal social skills, communication skills, interests and behavior patterns, has always been a challenging task in clinical practice. Recent studies have shown great potential for using fMRI data to distinguish ASD from typical control (TC). However, it has always been a challenging problem to extract which features from fMRI data and how to combine these different types of features to achieve improved ASD/TC classification performance. To address this problem, in this study we propose an improved ASD/TC classification framework based on dynamic functional connectivity (DFC) and multi-task feature selection. Our proposed ASD/TC classification framework is evaluated on 871 subjects with fMRI data from the Autism Brain Imaging Data Exchange I (ABIDE I) via a 10-fold cross validation strategy. Experimental results show that our proposed method achieves an accuracy of 76.8% and an area under the receiver operating characteristic curve (AUC) of 0.81 for ASD/TC classification. In addition, compared with some existing state-of-the-art methods, our proposed method achieves better accuracy and AUC for ASD/TC classification. Overall, our proposed ASD/TC classification framework is effective and promising for automatic diagnosis of ASD in clinical practice.

Enhancing the feature representation of multi-modal MRI data by combining multi-view information for MCI classification

The classification of mild cognitive impairment (MCI), which is a early stage of Alzheimer’s disease and is associated with brain structural and functional changes, is still a challenging task. Recent studies have shown great promise for improving the performance of MCI classification by combining multiple structural and functional features, such as grey matter volume and clustering coefficient. However, extracting which features and how to combine multiple features to improve the performance of MCI classification have always been challenging problems. To address these problems, in this study we propose a new method to enhance the feature representation of multi-modal MRI data by combining multi-view information to improve the performance of MCI classification. Firstly, we extract two structural features (including grey matter volume and cortical thickness) and two functional features (including clustering coefficient and shortest path length) of each cortical brain region based on automated anatomical labeling (AAL) atlas from both T1w MRI and rs-fMRI data of each subject. Then, in order to obtain features that are more helpful in distinguishing MCI subjects, an improved multi-task feature selection method, namely MTFS-gLASSO-TTR, is proposed. Finally, a multi-kernel learning algorithm is adopted to combine multiple features to perform the MCI classification task. Our proposed MCI classification method is evaluated on 315 subjects (including 105 LMCI subjects, 105 EMCI subjects and 105 NCs) with both T1w MRI and rs-fMRI data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Experimental results show that our proposed method achieves an accuracy of 88.5% and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.897 for LMCI/NC classification, an accuracy of 82.7% and an AUC of 0.832 for EMCI/NC classification, and an accuracy of 79.6% and an AUC of 0.803 for LMCI/EMCI classification, respectively. In addition, by comparison, the accuracy and AUC values of our proposed method are better than those of some existing state-of-the-art methods in MCI classification. Overall, our proposed MCI classification method is effective and promising for automatic diagnosis of MCI in clinical practice.

Hypergraph based multi-task feature selection for multimodal classification of Alzheimer's disease

Multi-modality based classification methods are superior to the single modality based approaches for the automatic diagnosis of the Alzheimer's disease (AD) and mild cognitive impairment (MCI). However, most of the multi-modality based methods usually ignore the structure information of data and simply squeeze them to pairwise relationships. In real-world applications, the relationships among subjects are much more complex than pairwise, and the high-order structure containing more discriminative information will be intuitively beneficial to our learning tasks. In light of this, a hypergraph based multi-task feature selection method for AD/MCI classification is proposed in this paper. Specifically, we first perform feature selection on each modality as a single task and incorporate group-sparsity regularizer to jointly select common features across multiple modalities. Then, we introduce a hypergraph based regularization term for the standard multi-task feature selection to model the high-order structure relationship among subjects. Finally, a multi-kernel support vector machine is adopted to fuse the features selected from different modalities for the final classification. The experimental results on the Alzheimer's Disease Neuroimaging Initiative (ADNI) demonstrate that our proposed method achieves better classification performance than the start-of-art multi-modality based methods.

A Novel Robust Semisupervised Classification Framework for Quality-Related Coupling Faults in Manufacturing Industries

An imbalanced number of faulty and normal samples make the traditional supervised classification methods difficult to ensure their classification performance. Accordingly, semisupervised learning methods have recently become hotspots both in academic research and practical application domains. Different from previous schemes, this paper dedicates on the correlations, common features, and specific features among quality-related coupling faults in manufacturing industries. The main innovations are as follows: first, it is the first time to develop a robust semisupervised classification framework for quality-related coupling faults, which integrates semisupervised multitask feature selection and manifold learning; second, manifold structures and local discriminant information of unlabeled and limited labeled faulty samples are sufficiently explored to improve the classification performance; and third, correlations among quality-related coupling faults are accurately captured, which are crucial for understanding the uniqueness and relationships of them at the feature level. The proposed method is finally validated in a representative manufacturing industry, i.e., hot strip mill process, where detailed simulation processes are presented and better classification performance is shown compared with the existing approaches.

Multi-task feature selection with sparse regularization to extract common and task-specific features

Multi-task learning (MTL), with the help of the relationship among tasks, is able to improve the generalization performance of all tasks by learning multiple tasks simultaneously. Multi-task sparse feature learning, formulated under the regularization framework, is one of the main approaches for MTL. Hence, the regularization term is crucial for multi-task sparse feature models. While most of the existing models utilize convex sparse regularization, a non-convex capped-ℓ1 regularization is extended into MTL and proven as a powerful sparse term. In this paper, we propose a novel regularization term for multi-task learning by extending the non-convex ℓ1−2 regularization to multi-task learning. The regularization term can not only realize group sparsity to extract the common features shared by all tasks, but also learn task-specific features through the relaxation of the second term in regularization. Although the model formulation is similar to a proposed one for multi-class problem, we first extend ℓ1−2 regularization to multi-task learning so that both common features and task-specific features can be extracted. A classical multi-task learning model (Multi-task Feature Selection, MTFS) can be viewed as a special case of our proposed model. Due to the complexity of regularization, we approximate the original problem by a locally linear subproblem and then use the Alternating Direction Method of Multipliers (ADMM) to solve this subproblem. The theoretical analysis shows the convergence of the proposed algorithm and the time complexity of the algorithm is provided. Experimental results demonstrate the effectiveness of our proposed method.

Multiview Laplacian semisupervised feature selection by leveraging shared knowledge among multiple tasks

Recently, considerable advancement has been achieved in semisupervised multitask feature selection methods, which they exploit the shared information from multiple related tasks. Besides, these algorithms have adopted manifold learning to leverage both the unlabeled and labeled data since its laborious to obtain adequate labeled training data. However, these semisupervised multitask selection feature algorithms are unable to naturally handle the multiview data since they are designed to deal single-view data. Existing studies have demonstrated that mining information enclosed in multiple views can drastically enhance the performance of feature selection. Multiview learning is capable of exploring the complementary and correlated knowledge from different views. In this paper, we incorporate multiview learning into semisupervised multitask feature selection framework and present a novel semisupervised multiview multitask feature selection framework. Our proposed algorithm is capable of exploiting complementary information from different feature views in each task while exploring the shared knowledge between multiple related tasks in a joint framework when the labeled training data is sparse. We develop an efficient iterative algorithm to optimize it since the objective function of the proposed method is non-smooth and difficult to solve. Experiment results on several multimedia applications have shown that the proposed algorithm is competitive compared with the other single-view feature selection algorithms.

Parkinson's Disease Diagnosis via Joint Learning From Multiple Modalities and Relations.

Parkinson's disease (PD) is a neurodegenerative progressive disease that mainly affects the motor systems of patients. To slow this disease deterioration, early and accurate diagnosis of PD is an effective way, which alleviates mental and physical sufferings by clinical intervention. In this paper, we propose a joint regression and classification framework for PD diagnosis via magnetic resonance and diffusion tensor imaging data. Specifically, we devise a unified multitask feature selection model to explore multiple relationships among features, samples, and clinical scores. We regress four clinical variables of depression, sleep, olfaction, cognition scores, as well as perform the classification of PD disease from the multimodal data. The multitask model explores the relationships at the level of clinical scores, image features, and subjects, to select the most informative and diseased-related features for diagnosis. The proposed method is evaluated on the public Parkinson's progression markers initiative dataset. The extensive experimental results show that the multitask framework can effectively boost the performance of regression and classification and outperforms other state-of-the-art methods. The computerized predictions of clinical scores and label for PD diagnosis may offer quantitative reference for decision support as well.

Multitask Feature Selection by Graph-Clustered Feature Sharing.

Multitask feature selection (MTFS) methods have become more important for many real world applications, especially in a high-dimensional setting. The most widely used assumption is that all tasks share the same features, and the l2,1 regularization method is usually applied. However, this assumption may not hold when the correlations among tasks are not obvious. Learning with unrelated tasks together may result in negative transfer and degrade the performance. In this paper, we present a flexible MTFS by graph-clustered feature sharing approach. To avoid the above limitation, we adopt a graph to represent the relevance among tasks instead of adopting a hard task set partition. Furthermore, we propose a graph-guided regularization approach such that the sparsity of the solution can be achieved on both the task level and the feature level, and a variant of the smooth proximal gradient method is developed to solve the corresponding optimization problem. An evaluation of the proposed method on multitask regression and multitask binary classification problem has been performed. Extensive experiments on synthetic datasets and real-world datasets demonstrate the effectiveness of the proposed approach to capture task structure.