Multi-task Learning Research Articles

YOLO-FD: An accurate fish disease detection method based on multi-task learning

Fish diseases often exhibit high risks of contagion, resulting in substantial economic losses. Accurate assessment of fish disease severity during diagnosis using deep learning poses a considerable challenge. Currently, deep learning models mainly focus on single tasks in fish disease detection, such as classification, object detection and segmentation. However, the accurate assessment of fish disease severity requires the integration of multiple dimensions of information, which is beyond the capabilities of traditional single-task methods. Therefore, this paper proposes YOLO-FD, a multi-task learning network specifically designed for simultaneous detection and segmentation. YOLO-FD extends the YOLOv8 backbone by integrating a novel semantic segmentation branch dedicated to precisely segmenting infected areas in diseased fish, while retaining the original object detection branch for identifying infected fish. Weight uncertainty and PCGrad are employed to balance the weights of different losses and to optimize conflicting gradients during the training process. With only a negligible increase in network parameters, YOLO-FD, tested on our constructed Nocardiosis fish dataset, achieves a detection accuracy of 94.2% mAP50 and gets mIOU of 79.4%, showcasing a 0.5% improvement over the baseline YOLOv8 and surpassing the state-of-the-art semantic segmentation network Deeplabv3plus by 4%. Notably, compared to the adapted multi-task network YOLOP, YOLO-FD demonstrates substantial improvements, displaying a 13.7% increase in mAP50-95 and a 15.1% boost in mIOU. On the VOC2012 segmentation dataset, the proposed method exhibits a 3.2% increase in mAP50 and a 2.2% rise in mAP50-95 compared to the baseline. Furthermore, results of the ablation experiment validate the effectiveness and generalization of the proposed multi-task learning approach. Source code is available at https://github.com/feifei-Lee/YOLO-FD.

Biologically interpretable multi-task deep learning pipeline predicts molecular alterations, grade, and prognosis in glioma patients

Deep learning models have been developed for various predictions in glioma; yet, they were constrained by manual segmentation, task-specific design, or a lack of biological interpretation. Herein, we aimed to develop an end-to-end multi-task deep learning (MDL) pipeline that can simultaneously predict molecular alterations and histological grade (auxiliary tasks), as well as prognosis (primary task) in gliomas. Further, we aimed to provide the biological mechanisms underlying the model’s predictions. We collected multiscale data including baseline MRI images from 2776 glioma patients across two private (FAHZU and HPPH, n = 1931) and three public datasets (TCGA, n = 213; UCSF, n = 410; and EGD, n = 222). We trained and internally validated the MDL model using our private datasets, and externally validated it using the three public datasets. We used the model-predicted deep prognosis score (DPS) to stratify patients into low-DPS and high-DPS subtypes. Additionally, a radio-multiomics analysis was conducted to elucidate the biological basis of the DPS. In the external validation cohorts, the MDL model achieved average areas under the curve of 0.892–0.903, 0.710–0.894, and 0.850–0.879 for predicting IDH mutation status, 1p/19q co-deletion status, and tumor grade, respectively. Moreover, the MDL model yielded a C-index of 0.723 in the TCGA and 0.671 in the UCSF for the prediction of overall survival. The DPS exhibits significant correlations with activated oncogenic pathways, immune infiltration patterns, specific protein expression, DNA methylation, tumor mutation burden, and tumor-stroma ratio. Accordingly, our work presents an accurate and biologically meaningful tool for predicting molecular subtypes, tumor grade, and survival outcomes in gliomas, which provides personalized clinical decision-making in a global and non-invasive manner.

A multi-task learning prediction model for the endpoint of converter steelmaking based on genetic algorithm optimised

A multi-task learning prediction model for the endpoint of converter steelmaking based on genetic algorithm optimised

Gas permeability, diffusivity, and solubility in polymers: Simulation-experiment data fusion and multi-task machine learning

Machine learning (ML) models for predicting gas permeability through polymers have traditionally relied on experimental data. While these models exhibit robustness within familiar chemical domains, reliability wanes when applied to new spaces. To address this challenge, we present a multi-tiered multi-task learning framework empowered with advanced machine-crafted polymer fingerprinting algorithms and data fusion techniques. This framework combines scarce “high-fidelity” experimental data with abundant diverse “low-fidelity” simulation or synthetic data, resulting in predictive models that display a high level of generalizability across novel chemical spaces. Additionally, this multi-task scheme capitalizes on known physics and interrelated properties, such as gas diffusivity and solubility, both of which are closely tied to permeability. By amalgamating high throughput generated simulation data with available experimental data for gas permeability, diffusivity, and solubility for various gases, we construct multi-task deep learning models. These models can simultaneously predict all three properties for all gases under consideration, with markedly enhanced predictive accuracy, particularly compared to traditional models reliant solely on experimental data for a singular property. This strategy underscores the potential of coupling high-throughput classical simulations with data fusion methodologies to yield state-of-the-art property predictors, especially when experimental data for targeted properties is scarce.

Deep multimodal saliency parcellation of cerebellar pathways: Linking microstructure and individual function through explainable multitask learning.

Parcellation of human cerebellar pathways is essential for advancing our understanding of the human brain. Existing diffusion magnetic resonance imaging tractography parcellation methods have been successful in defining major cerebellar fibre tracts, while relying solely on fibre tract structure. However, each fibre tract may relay information related to multiple cognitive and motor functions of the cerebellum. Hence, it may be beneficial for parcellation to consider the potential importance of the fibre tracts for individual motor and cognitive functional performance measures. In this work, we propose a multimodal data-driven method for cerebellar pathway parcellation, which incorporates both measures of microstructure and connectivity, and measures of individual functional performance. Our method involves first training a multitask deep network to predict various cognitive and motor measures from a set of fibre tract structural features. The importance of each structural feature for predicting each functional measure is then computed, resulting in a set of structure-function saliency values that are clustered to parcellate cerebellar pathways. We refer to our method as Deep Multimodal Saliency Parcellation (DeepMSP), as it computes the saliency of structural measures for predicting cognitive and motor functional performance, with these saliencies being applied to the task of parcellation. Applying DeepMSP to a large-scale dataset from the Human Connectome Project Young Adult study (n = 1065), we found that it was feasible to identify multiple cerebellar pathway parcels with unique structure-function saliency patterns that were stable across training folds. We thoroughly experimented with all stages of the DeepMSP pipeline, including network selection, structure-function saliency representation, clustering algorithm, and cluster count. We found that a 1D convolutional neural network architecture and a transformer network architecture both performed comparably for the multitask prediction of endurance, strength, reading decoding, and vocabulary comprehension, with both architectures outperforming a fully connected network architecture. Quantitative experiments demonstrated that a proposed low-dimensional saliency representation with an explicit measure of motor versus cognitive category bias achieved the best parcellation results, while a parcel count of four was most successful according to standard cluster quality metrics. Our results suggested that motor and cognitive saliencies are distributed across the cerebellar white matter pathways. Inspection of the final k = 4 parcellation revealed that the highest-saliency parcel was most salient for the prediction of both motor and cognitive performance scores and included parts of the middle and superior cerebellar peduncles. Our proposed saliency-based parcellation framework, DeepMSP, enables multimodal, data-driven tractography parcellation. Through utilising both structural features and functional performance measures, this parcellation strategy may have the potential to enhance the study of structure-function relationships of the cerebellar pathways.

SAIPy: A Python package for single-station earthquake monitoring using deep learning

Seismology has witnessed significant advancements in recent years with the application of deep learning methods to address a broad range of problems. These techniques have demonstrated their remarkable ability to effectively extract statistical properties from extensive datasets, surpassing the capabilities of traditional approaches to an extent. In this study, we present SAIPy, an open-source Python package specifically developed for fast seismic data processing by implementing deep learning. SAIPy offers solutions for multiple seismological tasks, including earthquake signal detection, seismic phase picking, first motion polarity identification and magnitude estimation. We introduce upgraded versions of previously published models such as CREIME_RT capable of identifying earthquakes with an accuracy above 99.8% and a root mean squared error of 0.38 unit in magnitude estimation. These upgraded models outperform state-of-the-art approaches like the Vision Transformer network. SAIPy provides an API that simplifies the integration of these advanced models, including CREIME_RT, DynaPicker_v2, and PolarCAP, along with benchmark datasets. It also, to the best of our knowledge, introduces the first fully automated deep learning based pipeline to process continuous waveforms. The package has the potential to be used for real-time earthquake monitoring to enable timely actions to mitigate the impact of seismic events. Ongoing development efforts aim to further enhance SAIPy’s performance and incorporate additional features that enhance exploration efforts, and it also would be interesting to approach the retraining of the whole package as a multi-task learning problem. A detailed description of all functions is available in a supplementary document.

Deep multi-task learning based detection of correlated mental disorders using audio modality

The existence of correlation among mental disorders is a well-known phenomenon. Multi-task learning (MTL) has been reported to yield enhanced detection performance of a targeted mental disorder by leveraging its correlation with other related mental disorders, mainly in textual and visual modalities. The validation of the same on audio modality is yet to be explored. In this study, we explore homogeneous and heterogeneous MTL paradigms for detecting two correlated mental disorders, namely major depressive disorder (MDD) and post-traumatic stress disorder (PTSD), on a publicly available audio dataset. The detection of both disorders is interchangeably employed as an auxiliary task when the other is the main task. In addition, a few other tasks are employed as auxiliary tasks. The results show that both MTL paradigms, implemented using two considered deep-learning models, outperformed the corresponding single-task learning (STL). The best relative improvement in the detection performance of MDD and PTSD is found to be 29.9% and 28.8%, respectively. Furthermore, we analyzed the cross-corpus generalization of MTL using two distinct datasets that involve MDD/PTSD instances. The results indicate that the generalizability of MTL is significantly superior to that of STL. The best relative increment in the cross-corpus generalization performance of MDD and PTSD detection is found to be 25.0% and 56.5%, respectively.

Rethinking of BERT sentence embedding for text classification

Text classification is a fundamental task in NLP that is used in several real-life tasks and applications. Large pre-trained language models such as BERT achieve state-of-the-art performance in several NLP tasks including text classification tasks. Although BERT boosts text classification performance, the common way of using it for classification lacks many aspects of its advantages. This work rethinks the way of using BERT final layer and hidden layers embeddings by proposing different aggregation architectures for text classification tasks such as sentiment analysis and sarcasm detection. This research also proposes different approaches for using BERT as a feature extractor without fine-tuning whose performance surpasses its fine-tuning counterpart. It also proposes promising multi-task learning aggregation architectures to improve the performance of the related classification problems. The experiments of the different architectures show that freezing BERT can outperform fine-tuning it for sentiment analysis. The experiments also show that multi-task learning while freezing BERT boosts the performance of yet hard tasks such as sarcasm detection. The best-performing models achieved new state-of-the-art performance on the ArSarcasm-v2 dataset for Arabic sarcasm detection and sentiment analysis. For multi-task learning and freezing BERT, a new SOTA F1-score of 64.41 was achieved for the sarcasm detection with a 3.47% improvement and near SOTA FPN of 75.78 for the sentiment classification. For single-task learning, a new SOTA FPN of 75.26 was achieved for the sentiment with a 1.81% improvement.

KMPR-AEP: Knowledge-Enhanced Multi-task Parallelized Recommendation Algorithm Incorporating Attention-Embedded Propagation

Existing knowledge graph-based recommendation algorithms either use Knowledge Graphs (KGs) as auxiliary information or use KG prediction tasks as regular terms to constrain recommendation with multi-task learning. However, the first method, which introduces multi-hop neighbors to enhance item representations, also weakens the relationship information within individual triples to some extent. The second method ignores neighboring information and thus fails to capture the long-range connectivity between items, which leads to a lack of utilization of the structural information in the KG. In response to the limitations of existing recommendation algorithms that do not fully leverage the relational and structural information within KGs, we propose a novel method named Knowledge-Enhanced Multi-Task Parallelized Recommendation Algorithm Incorporating Attention-Embedded Propagation (KMPR-AEP). It employs a parallel approach to simultaneously incorporate KG as auxiliary information and a regular term. We take into account the multi-hop neighbor information of users and items as well as enhance the relations among individual triples, which considers both the structural and relational information of KG. Extensive experiments conducted on three real-world datasets for CTR prediction and Top-K recommendation scenarios demonstrate the superiority of our proposed method over the state-of-the-art. On average, our KMPR-AEP shows improvements of 2.44% in AUC, 3.49% in ACC, and 2.64% in F1.

TEMM: text-enhanced multi-interactive attention and multitask learning network for multimodal sentiment analysis

TEMM: text-enhanced multi-interactive attention and multitask learning network for multimodal sentiment analysis

Residual feature decomposition and multi-task learning-based variation-invariant face recognition

AbstractFacial identity is subject to two primary natural variations: time-dependent (TD) factors such as age, and time-independent (TID) factors including sex and race. This study aims to address a broader problem known as variation-invariant face recognition (VIFR) by exploring the question: “How can identity preservation be maximized in the presence of TD and TID variations?" While existing state-of-the-art (SOTA) methods focus on either age-invariant or race and sex-invariant FR, our approach introduces the first novel deep learning architecture utilizing multi-task learning to tackle VIFR, termed “multi-task learning-based variation-invariant face recognition (MTLVIFR)." We redefine FR by incorporating both TD and TID, decomposing faces into age (TD) and residual features (TID: sex, race, and identity). MTLVIFR outperforms existing methods by 2% in LFW and CALFW benchmarks, 1% in CALFW, and 5% in AgeDB (20 years of protocol) in terms of face verification score. Moreover, it achieves higher face identification scores compared to all SOTA methods. Open source code.

SS-CCDN: A semi-supervised pixel-wise concrete crack detection network using multi-task learning and memory information

Concrete cracks pose significant challenges to infrastructure maintenance and safety. Traditional methods for detecting cracks suffer from inefficiency and subjectivity. Deep learning has shown promise recently, yet it often requires extensive labeled data. A semi-supervised concrete crack detection network (SS-CCDN) is proposed to handle the aforementioned challenge. To be specific, a multi-task model that takes edge feature detection of cracks as an auxiliary task is established. This model is then applied to both student and teacher networks to realize semi-supervised detection. Furthermore, the feature fusion networks incorporate the innovative multi-head cosine non-local attention module to achieve comprehensive acquisition and implementation of feature information at different scales. Lastly, the memory module is integrated into the network to compare the commonalities and distinctions between input sample information and memory sample data, successfully and effectively detecting concrete crack areas. SS-CCDN outperforms other baselines in detection, according to experimental results on two benchmark datasets.

Adaptive multi-task ensemble framework for smart home automation

Smart home technology has evolved rapidly with an emphasis on indoor environmental control. It aims to automate the control of the indoor environment through sensors, regulators, and other technologies in order to improve comfort, energy saving, emission reduction, safety, and intelligence. However, there is still a research gap between existing research and real-world applications. Most studies focus on modeling individual regulators, neglecting the cooperation between elements that influence the experience of the occupants. This leads to several cooperation issues like inconsistent and overlapping control. In addition, few studies have focused on the performance degradation of the entire system over the life cycle. To bridge the above missing gap, this paper proposes an adaptive multi-task framework that combines graph neural networks, multi-task learning, and incremental learning to achieve better cooperation and synergy among multiple regulators. The framework has been successfully tested on a wide range of smart home automation systems, achieving a 5–15 % improvement in performance for each individual regulator. Meanwhile, the proposed framework also demonstrates significant advantages against performance degradation over the life cycle. It provides a feasible, user-friendly, consumer-oriented framework for indoor environment automation that assists the entire system to improve the user experience.

The Use of Cnn-Based Multitask Learning for Smart Motorcycle Helmet Design

The Use of Cnn-Based Multitask Learning for Smart Motorcycle Helmet Design

Aliasing Detection in Rendered Images via a Multi-Task Learning

As technology advances from simple 2D designs to intricate 3D environments, the demand for high-quality visuals in video games and interactive media necessitates robust image quality assessment (IQA) techniques. Traditional methods like PSNR and SSIM, reliant on reference images, struggle with the unique challenges of 3D rendered content, highlighting the need for specialized non-reference IQA approaches. This paper introduces a novel multi-task learning architecture that corrects and predicts aliasing artifacts simultaneously, enhancing predictive accuracy without reference images. It also incorporates temporal information to improve visual coherence and smoothness. An automated labeling pipeline developed using Unity ensures a stable and unbiased dataset for model training and evaluation. Our experiments demonstrate that this approach reliably detects aliasing across various complexities, achieving state-of-the-art performance. By addressing specific challenges in rendered image assessment and leveraging innovative learning techniques, our work advances IQA for video games and simulations, ensuring high visual quality.

Enhancing Coherence and Diversity in Multi-class Slogan Generation Systems

Many problems related to natural language processing are solved by neural networks and big data. Researchers have previously focused on single-task supervised goals with limited data management to train slogan classification. A multi-task learning framework is used to learn jointly across several tasks related to generating multi-class slogan types. This study proposes a multi-task model named slogan generative adversarial network systems (Slo-GAN) to enhance coherence and diversity in slogan generation, utilizing generative adversarial networks and recurrent neural networks (RNN). Slo-GAN generates a new text slogan-type corpus, and the training generalization process is improved. We explored active learning (AL) and meta-learning (ML) for dataset labeling efficiency. AL reduced annotations by 10% compared to ML but still needed about 70% of the full dataset for baseline performance. The whole framework of Slo-GAN is supervised and trained together on all of these tasks. The text with the higher reporting score level is filtered by Slo-GAN, and a classification accuracy of 87.2% is achieved. We leveraged relevant datasets to perform a cross-domain experiment, reinforcing our assertions regarding both the distinctiveness of our dataset and the challenges of adapting bilingual dialects to one another.

Understanding and Therapeutic Application of Immune Response in Major Histocompatibility Complex (MHC) Diversity Using Multimodal Artificial Intelligence

Human Leukocyte Antigen (HLA) is like a device that monitors the internal environment of the body. T lymphocytes immediately recognize the HLA molecules that are expressed on the surface of the cells of the different individual, attacking it defeats microorganisms that is one of the causes of rejection in organ transplants performed between people with unmatched HLA types. Over 2850 and 3580 different polymorphisms have been reported for HLA-A and HLA-B respectively, around the world. HLA genes are associated with the risk of developing a variety of diseases, including autoimmune diseases, and play an important role in pathological conditions. By using a deep learning method called multi-task learning to simultaneously predict the gene sequences of multiple HLA genes, it is possible to improve accuracy and shorten execution time. Some new systems use a model called convolutional neural network (CNNs) in deep learning, which uses neural networks consisting of many layers and can learn complex correlations between SNP information and HLA gene sequences based on reference data for HLA imputation, which serves as training data. The learned model can output predicted values of HLA gene sequences with high accuracy using SNP information as input. To investigate which part of the input information surrounding the HLA gene is used to make learning predictions, predictions were made using not only a small number of nearby SNP information but also many SNP information distributed over a wider area by visualizing the learning information of the model. While conventional methods are strong at learning using nearly SNP information and not good at learning using SNP information located at distant locations, some new systems are thought that prediction accuracy may have improved because this problem was overcome. HLA genes are involved in the onset of a variety of diseases and are attracting attention. As an important area from the perspective of elucidating pathological conditions and realizing personalized medicine. The applied multi-task learning to two different HLA imputation reference panels—a Japanese panel (n = 1118) and type I diabetes genetics consortium panel (n = 5122). Through 10-fold cross-validation on these panels, the multi-task learning achieved higher imputation accuracy than conventional methods, especially for imputing low-frequency and rare HLA alleles. The increased prediction accuracy of HLA gene sequences is expected to increase the reliability of HLA analysis, including integrated analysis between different racial populations, and is expected to greatly contribute to the identification of HLA gene sequences associated with diseases and further elucidation of pathological conditions.

Integration of Relation Filtering and Multi-Task Learning in GlobalPointer for Entity and Relation Extraction

The rise of knowledge graphs has been instrumental in advancing artificial intelligence (AI) research. Extracting entity and relation triples from unstructured text is crucial for the construction of knowledge graphs. However, Chinese text has a complex grammatical structure, which may lead to the problem of overlapping entities. Previous pipeline models have struggled to address such overlap problems effectively, while joint models require entity annotations for each predefined relation in the set, which results in redundant relations. In addition, the traditional models often lead to task imbalance by overlooking the differences between tasks. To tackle these challenges, this research proposes a global pointer network based on relation prediction and loss function improvement (GPRL) for joint extraction of entities and relations. Experimental evaluations on the publicly available Chinese datasets DuIE2.0 and CMeIE demonstrate that the GPRL model achieves a 1.2–26.1% improvement in F1 score compared with baseline models. Further, experiments of overlapping classification conducted on CMeIE have also verified the effectiveness of overlapping triad extraction and ablation experiments. The model is helpful in identifying entities and relations accurately and can reduce redundancy by leveraging relation filtering and the global pointer network. In addition, the incorporation of a multi-task learning framework balances the loss functions of multiple tasks and enhances task interactions.

CUSTOMER SEGMENTATION AND LIFETIME VALUE PREDICTION

Customer segmentation and lifetime value estimation are two areas of development today. Today, the business world works differently, so machine learning uses different algorithms and different models to increase business efficiency. The value of life measured by its results is the bond between the customer and the company, as well as the value of the customer for the company. The reason for choosing this project is to examine the predictive behavior of future personalization models. From the company's perspective, this model can be very effective in terms of customer relations. Both segmentation and LVP are good for strategy development in business. As the world continues to evolve with major changes in technology, it is only natural to do such a project. LVP has a lot of predictive power when evaluating the customer buying process and choosing the best services to offer to different customers. Current methods for estimating CLV involve developing a model that uses all customer-based inputs. This not only loses the granularity of the data, but also makes the marketing plan less effective from the customer’s perspective. This article focuses on the advantages of intelligently combining small models by focusing on separate models for different customers. This allows companies to effectively use customer data to make informed decisions. Using the data as an example, the proposed multi-project study where the learning experience is shared with relevant groups increases the CLV estimation accuracy compared to using a single model size with some minor modifications. Moreover, the same method reduces the standard deviation of the error when compared with a large sample. Most importantly, when the groups have different data to train, multiple learning models will outperform single-shot models, which is normally considered a more difficult task. These results show that multi-task learning can be more effective than existing business models and can be a better alternative to existing models.

A Hierarchical Multi-Task Learning Framework for Semantic Annotation in Tabular Data.

To optimize the utilization and analysis of tables, it is essential to recognize and understand their semantics comprehensively. This requirement is especially critical given that many tables lack explicit annotations, necessitating the identification of column types and inter-column relationships. Such identification can significantly augment data quality, streamline data integration, and support data analysis and mining. Current table annotation models often address each subtask independently, which may result in the neglect of constraints and contextual information, causing relational ambiguities and inference errors. To address this issue, we propose a unified multi-task learning framework capable of concurrently handling multiple tasks within a single model, including column named entity recognition, column type identification, and inter-column relationship detection. By integrating these tasks, the framework exploits their interrelations, facilitating the exchange of shallow features and the sharing of representations. Their cooperation enables each task to leverage insights from the others, thereby improving the performance of individual subtasks and enhancing the model's overall generalization capabilities. Notably, our model is designed to employ only the internal information of tabular data, avoiding reliance on external context or knowledge graphs. This design ensures robust performance even with limited input information. Extensive experiments demonstrate the superior performance of our model across various tasks, validating the effectiveness of unified multi-task learning framework in the recognition and comprehension of table semantics.