Multi-task Learning Research Articles

Multivariate quality prediction of thin-walled parts machining using multi-task parallel deep transfer learning

Multivariate machining quality prediction of thin-walled parts with multiple machining features is a complex problem due to different data distribution between training and unlabelled test samples. Traditional quality prediction methods ignore the correlation of multiple quality labels and do not consider changes in data distribution, resulting in low accuracy of multivariate quality prediction. Therefore, a multivariate quality prediction method using multi-task parallel deep transfer learning is proposed to solve this problem. Specifically, a multi-output quality prediction model of cross-machining features is constructed through the joint design of multi-task parallel learning and deep transfer learning. Furthermore, a domain matcher is designed to form multiple transfer strategies, which can be used for dynamic matching of multi-source and multi-target machining features with multiple quality labels. The domain invariant data features through dynamic domain adaptation are extracted to deal with data distribution discrepancy between the source and target domains. Finally, the results of multiple comparison experiments show that the proposed method can effectively achieve the accurate quality prediction of the target domain with unlabelled labels and different distributions. Compared with the traditional methods, the proposed method has improved by 8.34%, 7.14%, and 9.09%, respectively, in MAE, RMSE, and score on average.

Magnetostrictive bi-perceptive flexible sensor for tracking bend and position of human and robot hand

The sensor that simultaneously perceives bending strain and magnetic field has the potential to detect the finger bending state and hand position of the human and robot. Based on unique magneto-mechanical coupling effect of magnetostrictive materials, the proposed a bi-perceptive flexible sensor, consisting of the Co–Fe film and magnetic sensing plane coils, can realize dual information perception of strain/magnetic field through the change of magnetization state. The sensor structure and interface circuit of the sensing system are designed to provide high sensitivity and fast response, based on the input–output characteristics of the simulation model. An asynchronous multi-task deep learning method is proposed, which takes the output of the position task as the partial input of the bending state task to analyze the output information of the sensor quickly and accurately. The sensing system, integrating with the proposed model, can better predict the bending state and approach distance of human or robot hand.

Preventing future zoonosis: SARS-CoV-2 mutations enhance human–animal cross-transmission

The COVID-19 pandemic has driven substantial evolution of the SARS-CoV-2 virus, yielding subvariants that exhibit enhanced infectiousness in humans. However, this adaptive advantage may not universally extend to zoonotic transmission. In this work, we hypothesize that viral adaptations favoring animal hosts do not necessarily correlate with increased human infectivity. In addition, we consider the potential for gain-of-function mutations that could facilitate the virus’s rapid evolution in humans following adaptation in animal hosts. Specifically, we identify the SARS-CoV-2 receptor-binding domain (RBD) mutations that enhance human–animal cross-transmission. To this end, we construct a multitask deep learning model, MT-TopLap trained on multiple deep mutational scanning datasets, to accurately predict the binding free energy changes upon mutation for the RBD to ACE2 of various species, including humans, cats, bats, deer, and hamsters. By analyzing these changes, we identified key RBD mutations such as Q498H in SARS-CoV-2 and R493K in the BA.2 variant that are likely to increase the potential for human–animal cross-transmission.

A Multi-Task Network: Improving Unmanned Underwater Vehicle Self-Noise Separation via Sound Event Recognition

The performance of an Unmanned Underwater Vehicle (UUV) is significantly influenced by the magnitude of self-generated noise, making it a crucial factor in advancing acoustic load technologies. Effective noise management, through the identification and separation of various self-noise types, is essential for enhancing a UUV’s reception capabilities. This paper concentrates on the development of UUV self-noise separation techniques, with a particular emphasis on feature extraction and separation in multi-task learning environments. We introduce an enhancement module designed to leverage noise categorization for improved network efficiency. Furthermore, we propose a neural network-based multi-task framework for the identification and separation of self-noise, the efficacy of which is substantiated by experimental trials conducted in a lake setting. The results demonstrate that our network outperforms the Conv-tasnet baseline, achieving a 0.99 dB increase in Signal-to-Interference-plus-Noise Ratio (SINR) and a 0.05 enhancement in the recognized energy ratio.

Automatic ploidy prediction and quality assessment of human blastocysts using time-lapse imaging

Assessing fertilized human embryos is crucial for in vitro fertilization, a task being revolutionized by artificial intelligence. Existing models used for embryo quality assessment and ploidy detection could be significantly improved by effectively utilizing time-lapse imaging to identify critical developmental time points for maximizing prediction accuracy. Addressing this, we develop and compare various embryo ploidy status prediction models across distinct embryo development stages. We present BELA, a state-of-the-art ploidy prediction model that surpasses previous image- and video-based models without necessitating input from embryologists. BELA uses multitask learning to predict quality scores that are thereafter used to predict ploidy status. By achieving an area under the receiver operating characteristic curve of 0.76 for discriminating between euploidy and aneuploidy embryos on the Weill Cornell dataset, BELA matches the performance of models trained on embryologists’ manual scores. While not a replacement for preimplantation genetic testing for aneuploidy, BELA exemplifies how such models can streamline the embryo evaluation process.

Machine Learning Strategies to Tackle Data Challenges in Mass Spectrometry-Based Proteomics.

In computational proteomics, machine learning (ML) has emerged as a vital tool for enhancing data analysis. Despite significant advancements, the diversity of ML model architectures and the complexity of proteomics data present substantial challenges in the effective development and evaluation of these tools. Here, we highlight the necessity for high-quality, comprehensive data sets to train ML models and advocate for the standardization of data to support robust model development. We emphasize the instrumental role of key data sets like ProteomeTools and MassIVE-KB in advancing ML applications in proteomics and discuss the implications of data set size on model performance, highlighting that larger data sets typically yield more accurate models. To address data scarcity, we explore algorithmic strategies such as self-supervised pretraining and multitask learning. Ultimately, we hope that this discussion can serve as a call to action for the proteomics community to collaborate on data standardization and collection efforts, which are crucial for the sustainable advancement and refinement of ML methodologies in the field.

A multi-task deep learning approach for real-time view classification and quality assessment of echocardiographic images

High-quality standard views in two-dimensional echocardiography are essential for accurate cardiovascular disease diagnosis and treatment decisions. However, the quality of echocardiographic images is highly dependent on the practitioner’s experience. Ensuring timely quality control of echocardiographic images in the clinical setting remains a significant challenge. In this study, we aimed to propose new quality assessment criteria and develop a multi-task deep learning model for real-time multi-view classification and image quality assessment (six standard views and “others”). A total of 170,311 echocardiographic images collected between 2015 and 2022 were utilized to develop and evaluate the model. On the test set, the model achieved an overall classification accuracy of 97.8% (95%CI 97.7–98.0) and a mean absolute error of 6.54 (95%CI 6.43–6.66). A single-frame inference time of 2.8 ms was achieved, meeting real-time requirements. We also analyzed pre-stored images from three distinct groups of echocardiographers (junior, senior, and expert) to evaluate the clinical feasibility of the model. Our multi-task model can provide objective, reproducible, and clinically significant view quality assessment results for echocardiographic images, potentially optimizing the clinical image acquisition process and improving AI-assisted diagnosis accuracy.

Identification of genetic basis of brain imaging by group sparse multi-task learning leveraging summary statistics

Brain imaging genetics is an evolving neuroscience topic aiming to identify genetic variations related to neuroimaging measurements of interest. Traditional linear regression methods have shown success, but their reliance on individual-level imaging and genetic data limits their applicability. Herein, we proposed S-GsMTLR, a group sparse multi-task linear regression method designed to harness summary statistics from genome-wide association studies (GWAS) of neuroimaging quantitative traits. S-GsMTLR directly employs GWAS summary statistics, bypassing the requirement for raw imaging genetic data, and applies multivariate multi-task sparse learning to these univariate GWAS results. It amalgamates the strengths of conventional sparse learning methods, including sophisticated modeling techniques and efficient feature selection. Additionally, we implemented a rapid optimization strategy to alleviate computational burdens by identifying genetic variants associated with phenotypes of interest across the entire chromosome. We first evaluated S-GsMTLR using summary statistics derived from the Alzheimer's Disease Neuroimaging Initiative. The results were remarkably encouraging, demonstrating its comparability to conventional methods in modeling and identification of risk loci. Furthermore, our method was evaluated with two additional GWAS summary statistics datasets: One focused on white matter microstructures and the other on whole brain imaging phenotypes, where the original individual-level data was unavailable. The results not only highlighted S-GsMTLR's ability to pinpoint significant loci but also revealed intriguing structures within genetic variations and loci that went unnoticed by GWAS. These findings suggest that S-GsMTLR is a promising multivariate sparse learning method in brain imaging genetics. It eliminates the need for original individual-level imaging and genetic data while demonstrating commendable modeling and feature selection capabilities.

CRML-Net: Cross-Modal Reasoning and Multi-Task Learning Network for tooth image segmentation

Data from a single modality may suffer from noise, low contrast, or other imaging limitations that affect the model’s accuracy. Furthermore, due to the limited amount of data, most models trained on single-modality data tend to overfit the training set and perform poorly on out-of-domain data. Therefore, in this paper, we propose a network named Cross-Modal Reasoning and Multi-Task Learning Network (CRML-Net), which combines cross-modal reasoning and multi-task learning, aiming to leverage the complementary information between different modalities and tasks to enhance the model’s generalization ability and accuracy. Specifically, CRML-Net consists of two stages. In the first stage, our network extracts a new morphological information modality from the original image and then performs cross-modal fusion with the original modality image, aiming to leverage the morphological information to enhance the model’s robustness to out-of-domain datasets. In the second stage, based on the output of the previous stage, we introduce a multi-task learning mechanism, aiming to improve the model’s performance on unseen data by sharing surface detail information from auxiliary tasks. We validated our method on a publicly available tooth cone beam computed tomography dataset. Our evaluation demonstrates that our method outperforms state-of-the-art approaches.

Research on the Migration and Settlement Laws of Backflow Proppants after Fracturing Tight Sandstone

This article studies the migration and settlement laws of backflow proppants after fracturing tight sandstone. This paper proposes a fitting method based on a multi-task learning network to address the issue of interference from multiple physical parameters during the transport and settlement processes of proppants. This method can effectively handle multi-dimensional interference factors and fit the mapping logic of multiple engineering parameters to transport patterns through the continuous correction of multi-layer networks. We first introduce the characteristics of tight sandstone reservoirs and their important value in mining, as well as the status of current research on the migration and settlement laws of proppants at home and abroad. Based on this, we then deeply analyze the sedimentation rate model of proppants in tight sandstone backflow and the equilibrium height of proppants under multiple factors of interference while considering the distribution characteristics of proppants. In order to more accurately simulate the transport and settlement laws of proppants, this paper introduces a multi-task learning network. This network can comprehensively consider multi-dimensional parameters, learn the inherent laws of data through training, and achieve accurate fitting of the transport and settlement laws of proppants. This study trained and tested the model using actual production data, and the results showed that the proposed model can fit the input–output relationship well, thus effectively supporting the study of proppant transport and settlement laws.

Assessing the Capabilities of UV-NIR Spectroscopy for Predicting Macronutrients in Hydroponic Solutions with Single-Task and Multi-Task Learning

Macronutrients, including nitrogen (N), phosphorus (P), potassium (K), calcium (Ca), magnesium (Mg), and sulfur (S), are the most basic nutrient elements in the solution for the hydroponic system. However, the current management of hydroponic nutrient solutions usually depends on EC and pH sensors due to the lack of accurate specific macronutrient sensing equipment, which easily leads to nutritional imbalance for the cultivated plant. In this study, the UV-NIR absorption spectroscopy (200–1100 nm) was used to predict six macronutrients in hydroponic solutions; two kinds of single-task learning algorithms, including partial least squares (PLS) and least absolute shrinkage and selection operator (LASSO), and two kinds of multi-task learning algorithms, including dirty multi-task learning (DMTL) and robust multi-task learning (RMTL), were investigated to develop prediction models and assess capabilities of UV-NIR. The results showed that N and Ca could be quantitatively predicted by UV-NIR with the ratio of performance to deviation (RPD) more than 2, K could be qualitatively predicted (1.4 < RPD < 2), and P, Mg, and S could not be successfully predicted (RPD < 1.4); the RMTL algorithm outperformed others for predicting K and Ca benefit from the underlying task relationships with N; and predicting P, Mg, and S were identified as irrelevant (outlier) tasks. Our study provides a potential approach for predicting several macronutrients in hydroponic solutions with UV-NIR, especially using RMTL to improve model prediction ability.

Multi-Task Learning and Sparse Discriminant Canonical Correlation Analysis for Identification of Diagnosis-Specific Genotype-Phenotype Association.

The primary objective of imaging genetics research is to investigate the complex genotype-phenotype association for the disease under study. For example, to understand the impact of genetic variations over the brain functions and structure, the genotypic data such as single nucleotide polymorphism (SNP) is integrated with the phenotypic data such as imaging quantitative traits. The sparse models, based on canonical correlation analysis (CCA), are popular in this area to find the complex bi-multivariate genotype-phenotype association, as the number of features in genotypic and/or phenotypic data is significantly higher as compared to the number of samples. However, the sparse CCA based methods are, in general, unsupervised in nature, and fail to identify the diagnose-specific features those play an important role for the diagnosis and prognosis of the disease under study. In this regard, a new supervised model is proposed to study the complex genotype-phenotype association, by judiciously integrating the merits of CCA, linear discriminant analysis (LDA) and multi-task learning. The proposed model can identify the diagnose-specific as well as the diagnose-consistent features with significantly lower computational complexity. The performance of the proposed method, along with a comparison with the state-of-the-art methods, is evaluated on several synthetic data sets and one real imaging genetics data collected from Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. In the current study, the SNP as genetic data and resting state functional MRI ( fMRI) as imaging data are integrated to find the complex genotype-phenotype association. An important finding is that the proposed method has better correlation value, improved noise resistance and stability, and also has better feature selection ability. All the results illustrate the power and capability of the proposed method to find the diagnostic group-specific imaging genetic association, which may help to understand the neurodegenerative disorder in a more comprehensive way.

FluPMT: Prediction of Predominant Strains of Influenza A Viruses via Multi-Task Learning.

Seasonal influenza vaccines play a crucial role in saving numerous lives annually. However, the constant evolution of the influenza A virus necessitates frequent vaccine updates to ensure its ongoing effectiveness. The decision to develop a new vaccine strain is generally based on the assessment of the current predominant strains. Nevertheless, the process of vaccine production and distribution is very time-consuming, leaving a window for the emergence of new variants that could decrease vaccine effectiveness, so predictions of influenza A virus evolution can inform vaccine evaluation and selection. Hence, we present FluPMT, a novel sequence prediction model that applies an encoder-decoder architecture to predict the hemagglutinin (HA) protein sequence of the upcoming season's predominant strain by capturing the patterns of evolution of influenza A viruses. Specifically, we employ time series to model the evolution of influenza A viruses, and utilize attention mechanisms to explore dependencies among residues of sequences. Additionally, antigenic distance prediction based on graph network representation learning is incorporated into the sequence prediction as an auxiliary task through a multi-task learning framework. Experimental results on two influenza datasets highlight the exceptional predictive performance of FluPMT, offering valuable insights into virus evolutionary dynamics, as well as vaccine evaluation and production.

DualFaceNet: augmentation consistency for optimal facial landmark detection and face mask classification

<p>In an era where face masks are commonplace, facial recognition faces new challenges and opportunities. This study introduces DualFaceNet (DFN), a cutting-edge neural network that efficiently combines facial landmark detection with mask classification. Benefiting from multi-task learning (MTL) and enhanced with a unique consistency loss, DFN outperforms traditional single-task models. Tests using the reputable 300W dataset and a face mask dataset showcase DFN’s strengths: a significant reduction in landmark error to 5.42 and an increase in mask classification accuracy to 92.59%. These results highlight the potential of integrating MTL and custom loss functions in facial recognition. As face masks continue to be globally essential, DFN’s integrated approach offers a fresh perspective in facial recognition studies. Furthermore, DFN paves the way for adaptive facial recognition systems, emphasizing the adaptability needed in our current era.</p>

Facial Video-Based Non-Contact Stress Recognition Utilizing Multi-Task Learning With Peak Attention.

Negative emotional states, such as anxiety and depression, pose significant challenges in contemporary society, often stemming from the stress encountered in daily activities. Stress (state or level) recognition is a crucial prerequisite for effective stress management and intervention. Presently, wearable devices have been employed to capture physiological signals and analyze stress states. However, their constant skin contact can lead to discomfort and disturbance during prolonged monitoring. In this paper, a peak attention-based multitasking framework is presented for non-contact stress recognition. The framework extracts rPPG signals from RGB facial videos, utilizing them as inputs for a novel multi-task attentional convolutional neural network for stress recognition (MTASR). It incorporates peak detection and HR estimation as auxiliary tasks to facilitate stress recognition. By leveraging multi-task learning, MTASR can utilize information related to stress physiological responses, thereby enhancing feature extraction efficiency. For stress recognition, two binary classification tasks are applied: stress state recognition and stress level recognition. The model is validated on the UBFC-Phys public dataset and demonstrates an accuracy of 94.33% for stress state recognition and 83.83% for stress level recognition. The proposed method outperforms the dataset's baseline methods and other competing approaches.

Modeling High-Order Relationships: Brain-Inspired Hypergraph-Induced Multimodal-Multitask Framework for Semantic Comprehension.

Semantic comprehension aims to reasonably reproduce people's real intentions or thoughts, e.g., sentiment, humor, sarcasm, motivation, and offensiveness, from multiple modalities. It can be instantiated as a multimodal-oriented multitask classification issue and applied to scenarios, such as online public opinion supervision and political stance analysis. Previous methods generally employ multimodal learning alone to deal with varied modalities or solely exploit multitask learning to solve various tasks, a few to unify both into an integrated framework. Moreover, multimodal-multitask cooperative learning could inevitably encounter the challenges of modeling high-order relationships, i.e., intramodal, intermodal, and intertask relationships. Related research of brain sciences proves that the human brain possesses multimodal perception and multitask cognition for semantic comprehension via decomposing, associating, and synthesizing processes. Thus, establishing a brain-inspired semantic comprehension framework to bridge the gap between multimodal and multitask learning becomes the primary motivation of this work. Motivated by the superiority of the hypergraph in modeling high-order relations, in this article, we propose a hypergraph-induced multimodal-multitask (HIMM) network for semantic comprehension. HIMM incorporates monomodal, multimodal, and multitask hypergraph networks to, respectively, mimic the decomposing, associating, and synthesizing processes to tackle the intramodal, intermodal, and intertask relationships accordingly. Furthermore, temporal and spatial hypergraph constructions are designed to model the relationships in the modality with sequential and spatial structures, respectively. Also, we elaborate a hypergraph alternative updating algorithm to ensure that vertices aggregate to update hyperedges and hyperedges converge to update their connected vertices. Experiments on the dataset with two modalities and five tasks verify the effectiveness of HIMM on semantic comprehension.

Joint Sensing and Semantic Communications with Multi-Task Deep Learning

Joint Sensing and Semantic Communications with Multi-Task Deep Learning

A Multitask Dynamic Graph Attention Autoencoder for Imbalanced Multilabel Time Series Classification.

Graph learning is widely applied to process various complex data structures (e.g., time series) in different domains. Due to multidimensional observations and the requirement for accurate data representation, time series are usually represented in the form of multilabels. Accurately classifying multilabel time series can provide support for personalized predictions and risk assessments. It requires effectively capturing complex label relevance and overcoming imbalanced label distributions of multilabel time series. However, the existing methods are unable to model label relevance for multilabel time series or fail to fully exploit it. In addition, the existing multilabel classification balancing strategies suffer from limitations, such as disregarding label relevance, information loss, and sampling bias. This article proposes a dynamic graph attention autoencoder-based multitask (DGAAE-MT) learning framework for multilabel time series classification. It can fully and accurately model label relevance for each instance by using a dynamic graph attention-based graph autoencoder to improve multilabel classification accuracy. DGAAE-MT employs a dual-sampling strategy and cooperative training approach to improve the classification accuracy of low-frequency classes while maintaining the classification accuracy of high-frequency and mid-frequency classes. It avoids information loss and sampling bias. DGAAE-MT achieves a mean average precision (mAP) of 0.955 and an F1 score of 0.978 on a mixed medical time series dataset. It outperforms state-of-the-art works in the past two years.

Lightweight Multitask Learning for Robust JND Prediction Using Latent Space and Reconstructed Frames

Lightweight Multitask Learning for Robust JND Prediction Using Latent Space and Reconstructed Frames

A sound event detection support system for smart home based on “two-to-one” teacher–student learning

Sound event detection (SED) is a core technology in smart home projects that rely on detected sound events to trigger specific actions. SED systems face two major challenges: high labeling costs and complex acoustic environments. To reduce labeling costs, some semi-supervised systems extract both global and local features for classification. However, these methods treat global and local features equally, not accounting for their varying importance when recognizing different types of sound events. Furthermore, to address complex acoustic environments, some studies use multitask learning frameworks to introduce SED-related tasks as auxiliaries to improve detection performance. However, these methods fail to align tasks within the framework, leading to conflicting outputs that may limit system performance. To address these issues, in this paper we propose a “two-to-one” teacher-student learning based semi-supervised SED system. This system employs a gating mechanism to selectively enhance global and local features, improving adaptability to different types of sound events, and incorporates a cross-task alignment module to interact SED with related tasks, reducing the risk of performance degradation caused by conflicting outputs. Experimental results on two datasets demonstrate that our system achieves the best performance in all metrics, with EB-F1 scores of 48.1 % and 64.7 %, representing improvements of 15.3 % and 10.6 % over the baseline ConformerSED system, respectively. Our work offers an effective SED solution for smart home projects by providing a semi-supervised SED system that performs well while reducing labeling costs.