Multi-task Learning Research Articles

PulseNet: Multi-task learning-based non-contact pulse condition diagnosis using multi-scale fusion and transformer

The pulse condition, also called mai xiang, a significant diagnostic indicator in traditional Chinese medicine (TCM), has increased demand for intelligent diagnosis in recent years. However, the existing intelligent pulse condition diagnosis is mostly sensor-based or contact-dependent, resulting in numerous inconveniences, especially in mobile healthcare. Recently, remote photoplethysmography(rPPG) with the information captured from facial videos has been widely used in non-contact physiological measurements, such as blood pressure and heart rate monitoring. Inspired by these studies, a novel end-to-end rPPG-based method, named PulseNet, is proposed for non-contact pulse condition diagnosis in this paper. In PulseNet, the transformer is employed to extract spatio-temporal features from facial videos, multi-scale fusion is adopted to enhance the feature representation progressively, and multi-task learning is proposed for final pulse condition diagnosis. Besides, we propose the spatio-temporal difference attention (STDA) block in the transformer to aggregate spatio-temporal difference clues of the local region to provide fine-grained spatio-temporal context. We conduct extensive experiments with the proposed method. The results show that PulseNet outperforms the state-of-the-art rPPG-based methods. To the best of our knowledge, our method is the first attempt to detect pulse conditions with the information captured from facial videos, making it a promising non-contact diagnosis approach for mobile healthcare in TCM.

Liver vessel MRI image segmentation based on dual-path diffusion model

Accurate segmentation of liver blood vessels in magnetic resonance imaging (MRI) images is a challenging task due to the complex tree-like structure and anisotropic diffusion properties of blood vessels. To solve this problem, we propose a new Dual-Path Diffusion Model (DPDM) framework. The framework consists of two collaborative diffusion paths: a local feature learning path based on convolution operations and a global context modeling path based on transform blocks. Local path encodes rich shape priors and preserve spatial details, while global path captures long-distance dependencies and enhance representation. In the decoding phase, the boundary features from the boundary path are fused with the features of the ordinary path decoding, which further enhances the shape sensitivity. In addition, we leverage a multi-task learning scheme to jointly optimize vascular segmentation and boundary prediction tasks in an end-to-end manner. Experiments on retrospective clinical dataset demonstrate that the proposed DPDM framework achieves excellent performance on the liver vessel segmentation task. Compared with state-of-the-art methods, our approach achieved a 6.0% and 7.3% performance improvement in Dice coefficient and IoU index, respectively. Our approach offers a promising solution for automated blood vessel segmentation in precision medicine.

A novel multitask learning algorithm for tasks with distinct chemical space: zebrafish toxicity prediction as an example

Data scarcity is one of the most critical issues impeding the development of prediction models for chemical effects. Multitask learning algorithms leveraging knowledge from relevant tasks showed potential for dealing with tasks with limited data. However, current multitask methods mainly focus on learning from datasets whose task labels are available for most of the training samples. Since datasets were generated for different purposes with distinct chemical spaces, the conventional multitask learning methods may not be suitable. This study presents a novel multitask learning method MTForestNet that can deal with data scarcity problems and learn from tasks with distinct chemical space. The MTForestNet consists of nodes of random forest classifiers organized in the form of a progressive network, where each node represents a random forest model learned from a specific task. To demonstrate the effectiveness of the MTForestNet, 48 zebrafish toxicity datasets were collected and utilized as an example. Among them, two tasks are very different from other tasks with only 1.3% common chemicals shared with other tasks. In an independent test, MTForestNet with a high area under the receiver operating characteristic curve (AUC) value of 0.911 provided superior performance over compared single-task and multitask methods. The overall toxicity derived from the developed models of zebrafish toxicity is well correlated with the experimentally determined overall toxicity. In addition, the outputs from the developed models of zebrafish toxicity can be utilized as features to boost the prediction of developmental toxicity. The developed models are effective for predicting zebrafish toxicity and the proposed MTForestNet is expected to be useful for tasks with distinct chemical space that can be applied in other tasks.Scieific contributionA novel multitask learning algorithm MTForestNet was proposed to address the challenges of developing models using datasets with distinct chemical space that is a common issue of cheminformatics tasks. As an example, zebrafish toxicity prediction models were developed using the proposed MTForestNet which provide superior performance over conventional single-task and multitask learning methods. In addition, the developed zebrafish toxicity prediction models can reduce animal testing.

Multi-task prompt tuning with soft context sharing for vision–language models

Vision–language models have recently shown great potential on many tasks in computer vision. Meanwhile, prior work demonstrates prompt tuning designed for vision–language models could acquire superior performance on few-shot image recognition compared to linear probe, a strong baseline. In practice, many few-shot tasks are inherently correlated, particularly within specialized domains. However, such information is overlooked previously. Inspired by the fact that modeling task relationship by multi-task learning can usually boost performance, we propose a novel method SoftCPT (Soft Context Sharing for Prompt Tuning) to tune pre-trained vision–language models on multiple target few-shot tasks jointly. Specifically, we design a task-shared meta network to generate prompt context for each task using task name together with a learnable task context as input. The parameters of this meta network as well as the task context are tuned on the joint training set of all tasks. As such, the prompt context of all tasks will be shared in a soft manner. Extensive experiments across four multi-task few-shot datasets covering 44 tasks and 1593 categories demonstrate that SoftCPT significantly outperforms single-task prompt tuning methods, highlighting the effectiveness of multi-task learning for vision–language prompt tuning.

Improve the downscaling accuracy of high-resolution precipitation field using classification mask

Due to the low spatial and temporal density of early meteorological observations in China, The China Meteorological Administration Land Data Assimilation System V3.0 (CLDAS3.0) could not obtain high-quality historical, high-resolution assimilated data. In particular, with high-resolution precipitation assimilation data, it is difficult to estimate the precipitation range accurately because of its substantial spatiotemporal heterogeneity. To solve this problem, our team proposes a two-stage downscaling framework for the precipitation field based on multi-task learning, CLDASSD-PRCP. It improves the accuracy of the downscaling task by predicting the precipitation classification mask first. We verify the algorithm using CLDAS3.0 and site observations as ground truth in Yunnan, China. The results show that CLDASSD-PRCP can accurately generate three types of precipitation masks and reduce the MAE of downscaling results by 0.32 mm/h after adding multi-classification masks (MutiClassesMask). In addition, the effects of light and extreme precipitation on precipitation spatial downscaling are also revealed. CLDASSD-PRCP is significant for establishing statistical downscaling models based on deep learning and provides a new way to deal with atmospheric elements with significant spatiotemporal heterogeneity.

OmniJet-α: the first cross-task foundation model for particle physics

Abstract Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of training time and data. We report significant progress on this challenge on several fronts. First, a comprehensive set of evaluation methods is introduced to judge the quality of an encoding from physics data into a representation suitable for the autoregressive generation of particle jets with transformer architectures (the common backbone of foundation models). These measures motivate the choice of a higher-fidelity tokenization compared to previous works. Finally, we demonstrate transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging) with our new OmniJet-α model. This is the first successful transfer between two different and actively studied classes of tasks and constitutes a major step in the building of foundation models for particle physics.

BertSNR: an interpretable deep learning framework for single-nucleotide resolution identification of transcription factor binding sites based on DNA language model.

Transcription factors are pivotal in the regulation of gene expression, and accurate identification of transcription factor binding sites (TFBSs) at high resolution is crucial for understanding the mechanisms underlying gene regulation. The task of identifying TFBSs from DNA sequences is a significant challenge in the field of computational biology today. To address this challenge, a variety of computational approaches have been developed. However, these methods face limitations in their ability to achieve high-resolution identification and often lack interpretability. We propose BertSNR, an interpretable deep learning framework for identifying TFBSs at single-nucleotide resolution. BertSNR integrates sequence-level and token-level information by multi-task learning based on pre-trained DNA language models. Benchmarking comparisons show that our BertSNR outperforms the existing state-of-the-art methods in TFBS predictions. Importantly, we enhanced the interpretability of the model through attentional weight visualization and motif analysis, and discovered the subtle relationship between attention weight and motif. Moreover, BertSNR effectively identifies TFBSs in promoter regions, facilitating the study of intricate gene regulation. The BertSNR source code can be found at https://github.com/lhy0322/BertSNR.

A comprehensive multi-task deep learning approach for predicting metabolic syndrome with genetic, nutritional, and clinical data

Metabolic syndrome (MetS) is a complex disorder characterized by a cluster of metabolic abnormalities, including abdominal obesity, hypertension, elevated triglycerides, reduced high-density lipoprotein cholesterol, and impaired glucose tolerance. It poses a significant public health concern, as individuals with MetS are at an increased risk of developing cardiovascular diseases and type 2 diabetes. Early and accurate identification of individuals at risk for MetS is essential. Various machine learning approaches have been employed to predict MetS, such as logistic regression, support vector machines, and several boosting techniques. However, these methods use MetS as a binary status and do not consider that MetS comprises five components. Therefore, a method that focuses on these characteristics of MetS is needed. In this study, we propose a multi-task deep learning model designed to predict MetS and its five components simultaneously. The benefit of multi-task learning is that it can manage multiple tasks with a single model, and learning related tasks may enhance the model's predictive performance. To assess the efficacy of our proposed method, we compared its performance with that of several single-task approaches, including logistic regression, support vector machine, CatBoost, LightGBM, XGBoost and one-dimensional convolutional neural network. For the construction of our multi-task deep learning model, we utilized data from the Korean Association Resource (KARE) project, which includes 352,228 single nucleotide polymorphisms (SNPs) from 7729 individuals. We also considered lifestyle, dietary, and socio-economic factors that affect chronic diseases, in addition to genomic data. By evaluating metrics such as accuracy, precision, F1-score, and the area under the receiver operating characteristic curve, we demonstrate that our multi-task learning model surpasses traditional single-task machine learning models in predicting MetS.

Multi-Task Credible Pseudo-Label Learning for Semi-Supervised Crowd Counting.

As a widely used semi-supervised learning strategy, self-training generates pseudo-labels to alleviate the labor-intensive and time-consuming annotation problems in crowd counting while boosting the model performance with limited labeled data and massive unlabeled data. However, the noise in the pseudo-labels of the density maps greatly hinders the performance of semi-supervised crowd counting. Although auxiliary tasks, e.g., binary segmentation, are utilized to help improve the feature representation learning ability, they are isolated from the main task, i.e., density map regression and the multi-task relationships are totally ignored. To address the above issues, we develop a multi-task credible pseudo-label learning (MTCP) framework for crowd counting, consisting of three multi-task branches, i.e., density regression as the main task, and binary segmentation and confidence prediction as the auxiliary tasks. Multi-task learning is conducted on the labeled data by sharing the same feature extractor for all three tasks and taking multi-task relations into account. To reduce epistemic uncertainty, the labeled data are further expanded, by trimming the labeled data according to the predicted confidence map for low-confidence regions, which can be regarded as an effective data augmentation strategy. For unlabeled data, compared with the existing works that only use the pseudo-labels of binary segmentation, we generate credible pseudo-labels of density maps directly, which can reduce the noise in pseudo-labels and therefore decrease aleatoric uncertainty. Extensive comparisons on four crowd-counting datasets demonstrate the superiority of our proposed model over the competing methods. The code is available at: https://github.com/ljq2000/MTCP.

Enhancing Robot Learning with Transformer-based Morphology Modeling

Abstract The transformer model has made significant progress in various areas through large-scale training. In contrast, the traditional robot performs a single task, and there is an issue with migrating the strategic model. In this study, a Robot Morphology Learning (RML) method is proposed to enhance efficiency and generalization performance by learning multiple tasks in a transformer model. RML constructs the robot’s morphology as a graph and utilizes a graph neural network to handle graphs of arbitrary connections and sizes, addressing the disparity in state and action space dimensions. RML breaks through the limitation of non-migration of models, realizes efficient training, and improves the generalization performance of models, enabling quick adaptation to new tasks. Experimental results show that the proposed method outperforms previous methods in both multi-task learning and transfer learning experiments.

Multi-Objective Software Defect Prediction via Multi-Source Uncertain Information Fusion and Multi-Task Multi-View Learning

Multi-Objective Software Defect Prediction via Multi-Source Uncertain Information Fusion and Multi-Task Multi-View Learning

Self-supervised cognitive learning for multifaced interest in large-scale industrial recommender systems

Recommender systems play a vital role in e-commerce. Existing algorithms usually learn ranking scores from a user's historical sequence, but they generally pay little attentions to mining the user's precise and varied interests to jointly optimize multiple tasks. In this study, we argue that it is crucial to formulate different user interests based on self-supervised learning (SSL) enhanced data representation for multi-scene objectives. We propose self-supervised cognitive learning (SSCL), a novel framework that can simultaneously model a user's multifaceted interest representation and linked multitask information learning. SSCL exploits SSL to enhance the value of training data. In addition, multi-interest-aggregation and multi-interest-select units are introduced and a customized gate-control model is used to optimize multiple objectives. We evaluate our framework using three real-world datasets, and our results demonstrate its effectiveness and superior performance over the state-of-the-art techniques. Compared with the baselines in three datasets, the click-through rate increased by +7.58%, and the conversion rate increased by +6.04%.

Segmentation Guided Crossing Dual Decoding Generative Adversarial Network for Synthesizing Contrast-Enhanced Computed Tomography Images.

Although contrast-enhanced computed tomography (CE-CT) images significantly improve the accuracy of diagnosing focal liver lesions (FLLs), the administration of contrast agents imposes a considerable physical burden on patients. The utilization of generative models to synthesize CE-CT images from non-contrasted CT images offers a promising solution. However, existing image synthesis models tend to overlook the importance of critical regions, inevitably reducing their effectiveness in downstream tasks. To overcome this challenge, we propose an innovative CE-CT image synthesis model called Segmentation Guided Crossing Dual Decoding Generative Adversarial Network (SGCDD-GAN). Specifically, the SGCDD-GAN involves a crossing dual decoding generator including an attention decoder and an improved transformation decoder. The attention decoder is designed to highlight some critical regions within the abdominal cavity, while the improved transformation decoder is responsible for synthesizing CE-CT images. These two decoders are interconnected using a crossing technique to enhance each other's capabilities. Furthermore, we employ a multi-task learning strategy to guide the generator to focus more on the lesion area. To evaluate the performance of proposed SGCDD-GAN, we test it on an in-house CE-CT dataset. In both CE-CT image synthesis tasks-namely, synthesizing ART images and synthesizing PV images-the proposed SGCDD-GAN demonstrates superior performance metrics across the entire image and liver region, including SSIM, PSNR, MSE, and PCC scores. Furthermore, CE-CT images synthetized from our SGCDD-GAN achieve remarkable accuracy rates of 82.68%, 94.11%, and 94.11% in a deep learning-based FLLs classification task, along with a pilot assessment conducted by two radiologists.

EC number prediction of protein sequences based on combination of hierarchical and global features.

The identification of enzyme functions plays a crucial role in understanding the mechanisms of biological activities and advancing the development of life sciences. However, existing enzyme EC number prediction methods did not fully utilize protein sequence information and still had shortcomings in identification accuracy. To address this issue, we proposed an EC number prediction network using hierarchical features and global features (ECPN-HFGF). This method first utilized residual networks to extract generic features from protein sequences, and then employed hierarchical feature extraction modules and global feature extraction modules to further extract hierarchical and global features of protein sequences. Subsequently, the prediction results of both feature types were combined, and a multitask learning framework was utilized to achieve accurate prediction of enzyme EC numbers. Experimental results indicated that the ECPN-HFGF method performed best in the task of predicting EC numbers for protein sequences, achieving macro F1 and micro F1 scores of 95.5% and 99.0%, respectively. The ECPN-HFGF method effectively combined hierarchical and global features of protein sequences, allowing for rapid and accurate EC number prediction. Compared to current commonly used methods, this method offers significantly higher prediction accuracy, providing an efficient approach for the advancement of enzymology research and enzyme engineering applications.

A Self-Supervised Few-Shot Semantic Segmentation Method Based on Multi-Task Learning and Dense Attention Computation.

Nowadays, autonomous driving technology has become widely prevalent. The intelligent vehicles have been equipped with various sensors (e.g., vision sensors, LiDAR, depth cameras etc.). Among them, the vision systems with tailored semantic segmentation and perception algorithms play critical roles in scene understanding. However, the traditional supervised semantic segmentation needs a large number of pixel-level manual annotations to complete model training. Although few-shot methods reduce the annotation work to some extent, they are still labor intensive. In this paper, a self-supervised few-shot semantic segmentation method based on Multi-task Learning and Dense Attention Computation (dubbed MLDAC) is proposed. The salient part of an image is split into two parts; one of them serves as the support mask for few-shot segmentation, while cross-entropy losses are calculated between the other part and the entire region with the predicted results separately as multi-task learning so as to improve the model's generalization ability. Swin Transformer is used as our backbone to extract feature maps at different scales. These feature maps are then input to multiple levels of dense attention computation blocks to enhance pixel-level correspondence. The final prediction results are obtained through inter-scale mixing and feature skip connection. The experimental results indicate that MLDAC obtains 55.1% and 26.8% one-shot mIoU self-supervised few-shot segmentation on the PASCAL-5i and COCO-20i datasets, respectively. In addition, it achieves 78.1% on the FSS-1000 few-shot dataset, proving its efficacy.

Multitask learning of a biophysically-detailed neuron model.

The human brain operates at multiple levels, from molecules to circuits, and understanding these complex processes requires integrated research efforts. Simulating biophysically-detailed neuron models is a computationally expensive but effective method for studying local neural circuits. Recent innovations have shown that artificial neural networks (ANNs) can accurately predict the behavior of these detailed models in terms of spikes, electrical potentials, and optical readouts. While these methods have the potential to accelerate large network simulations by several orders of magnitude compared to conventional differential equation based modelling, they currently only predict voltage outputs for the soma or a select few neuron compartments. Our novel approach, based on enhanced state-of-the-art architectures for multitask learning (MTL), allows for the simultaneous prediction of membrane potentials in each compartment of a neuron model, at a speed of up to two orders of magnitude faster than classical simulation methods. By predicting all membrane potentials together, our approach not only allows for comparison of model output with a wider range of experimental recordings (patch-electrode, voltage-sensitive dye imaging), it also provides the first stepping stone towards predicting local field potentials (LFPs), electroencephalogram (EEG) signals, and magnetoencephalography (MEG) signals from ANN-based simulations. While LFP and EEG are an important downstream application, the main focus of this paper lies in predicting dendritic voltages within each compartment to capture the entire electrophysiology of a biophysically-detailed neuron model. It further presents a challenging benchmark for MTL architectures due to the large amount of data involved, the presence of correlations between neighbouring compartments, and the non-Gaussian distribution of membrane potentials.

Real-Time Anomaly Detection in Smart Grid Networks Using Deep Learning with Cross-Domain Generalization and Multi-Task Learning

Real-Time Anomaly Detection in Smart Grid Networks Using Deep Learning with Cross-Domain Generalization and Multi-Task Learning

Mechanical Quantities Prediction of Metal Cutting by Machine Learning and Simulation Data

Metal cutting is an important process in industrial manufacturing. Using the mechanical quantities of metal cutting to optimize process design is helpful to improve productivity. However, it is expensive to obtain these quantities due to the complexity of the cutting process, including material nonlinearity, geometric nonlinearity, state nonlinearity and their interactions. In this paper, a prediction model is constructed by combining machine learning (ML) and simulation data to quickly acquire multi-difficult-to-obtain metal cutting mechanical quantities to solve this problem. First, Adaptive Smoothed Particle Hydrodynamics (ASPH) is used to generate a simulation dataset of 2000 metal cutting cases. Based on the simulation data, six machine learning (ML) methods are employed to establish two prediction models, single-task learning and multi-task learning, to predict the mechanical quantities of metal cutting. The experimental results demonstrate that the ML method can predict abundant reference data efficiently after understanding the relationship between simulation parameters and mechanical quantities from simulation data, which is expected to replace some similar and repetitive simulation work. The Multilayer Perceptron (MLP) model under the multi-task setting provides the best prediction performance, fastest prediction time efficiency, and stable model behavior. Additionally, input erasure experiments reveal that the prediction of maximum equivalent plastic strain is significantly affected by particle spacing, and cutting speed plays a vital role in predicting maximum velocity. This work highlights the promotion of the data-driven ML method in quickly obtaining abundant reference data for the metal cutting process, and provides an auxiliary means for process optimization.

Acoustic emission source localization in complex pipe structure using multi-task deep learning models

Localizing defects in long-range pipelines is essential to reduce the inspection time and develop timely repair strategies. The acoustic emission (AE) method is employed to pinpoint the position of defects in pipelines. The conventional 1-D localization algorithm requires time of arrival differences between two sensors, which may not be accurately captured due to the dispersive nature of the pipe structures. The geometric variations such as elbows and welds can influence the propagating elastic waves and, consequently, arrival time. In this study, an AE source localization approach using a deep learning model is developed to tackle the influences of sensor-source distance and geometric variables. The multi-task learning model first identifies the impact of the elbow and subsequently integrates this information when predicting the source location. The proposed model is evaluated on a complex piping system, which features welded elbows in its connections. Incorporating the elbow effect into the model shows a notable improvement in overall accuracy, rising from 53% (conventional method) to 94% (proposed multi-task learning method).

Enhancing Hydrological Variable Prediction through Multitask LSTM Models

Deep learning models possess the capacity to accurately forecast various hydrological variables, encompassing flow, temperature, and runoff, notably leveraging Long Short-Term Memory (LSTM) networks to exhibit exceptional performance in capturing long-term dynamics. Nonetheless, these deep learning models often fixate solely on singular predictive tasks, thus overlooking the interdependencies among variables within the hydrological cycle. To address this gap, our study introduces a model that amalgamates Multitask Learning (MTL) and LSTM, harnessing inter-variable information to achieve high-precision forecasting across multiple tasks. We evaluate our proposed model on the global ERA5-Land dataset and juxtapose the results against those of a single-task model predicting a sole variable. Furthermore, experiments explore the impact of task weight allocation on the performance of multitask learning. The results indicate that when there is positive transfer among variables, multitask learning aids in enhancing predictive performance. When jointly forecasting first-layer soil moisture (SM1) and evapotranspiration (ET), the Nash–Sutcliffe Efficiency (NSE) increases by 19.6% and 4.1%, respectively, compared to the single-task baseline model; Kling–Gupta Efficiency (KGE) improves by 8.4% and 6.1%. Additionally, the model exhibits greater forecast stability when confronted with extreme data variations in tropical monsoon regions (AM). In conclusion, our study substantiates the applicability of multitask learning in the realm of hydrological variable prediction.