Multi-task Learning Method Research Articles

A Tree-Structured Multitask Model Architectures Recommendation System.

Neural networks with branched architectures, namely, tree-structured models, have been employed to jointly tackle multiple vision tasks in the context of multitask learning (MTL). Such tree-structured networks typically start with a number of shared layers, after which different tasks branch out into their own sequence of layers. Hence, the major challenge is to determine where to branch out for each task given a backbone model to optimize for both task accuracy and computation efficiency. To address the challenge, this article proposes a recommendation system that, given a set of tasks and a convolutional neural network-based backbone model, automatically suggests tree-structured multitask architectures that could achieve a high task performance while meeting a user-specified computation budget without performing model training. Extensive evaluations on popular MTL benchmarks show that the recommended architectures could achieve competitive task accuracy and computation efficiency compared with state-of-the-art MTL methods. Our tree-structured multitask model recommender is open-sourced and available at https://github.com/zhanglijun95/TreeMTL.

Unprejudiced Training Auxiliary Tasks Makes Primary Better: A Multitask Learning Perspective.

Human beings can leverage knowledge from relative tasks to improve learning on a primary task. Similarly, multitask learning (MTL) methods suggest using auxiliary tasks to enhance a neural network's performance on a specific primary task. However, previous methods often select auxiliary tasks carefully but treat them as secondary during training. The weights assigned to auxiliary losses are typically smaller than the primary loss weight, leading to insufficient training on auxiliary tasks and ultimately failing to support the main task effectively. To address this issue, we propose an uncertainty-based impartial learning method that ensures balanced training across all tasks. In addition, we consider both gradients and uncertainty information during backpropagation to further improve performance on the primary task. Extensive experiments show that our method achieves performance comparable to or better than state-of-the-art approaches. Moreover, our weighting strategy is effective and robust in enhancing the performance of the primary task regardless of the noise auxiliary tasks' pseudolabels.

Lightweight Multi-task Learning Method for System Log Anomaly Detection

Lightweight Multi-task Learning Method for System Log Anomaly Detection

Sparse and Expandable Network for Google's Pathways.

Recently, Google introduced Pathways as its next-generation AI architecture. Pathways must address three critical challenges: learning one general model for several continuous tasks, ensuring tasks can leverage each other without forgetting old tasks, and learning from multi-modal data such as images and audio. Additionally, Pathways must maintain sparsity in both learning and deployment. Current lifelong multi-task learning approaches are inadequate in addressing these challenges. To address these challenges, we propose SEN, a Sparse and Expandable Network. SEN is designed to handle multiple tasks concurrently by maintaining sparsity and enabling expansion when new tasks are introduced. The network leverages multi-modal data, integrating information from different sources while preventing interference between tasks. The proposed SEN model demonstrates significant improvements in multi-task learning, successfully managing task interference and forgetting. It effectively integrates data from various modalities and maintains efficiency through sparsity during both the learning and deployment phases. SEN offers a straightforward yet effective solution to the limitations of current lifelong multi-task learning methods. By addressing the challenges identified in the Pathways architecture, SEN provides a promising approach for developing AI systems capable of learning and adapting over time without sacrificing performance or efficiency.

Multi-task label noise learning for classification

Multi-task classification improves generalization performance via exploiting the correlations between tasks. However, most multi-task learning methods fail to recognize and filter noisy labels for the classification problems with label noises. To address this issue, this paper proposes a novel multi-task label noise learning method based on loss correction, called MTLNL. MTLNL introduces the class-wise denoising (CWD) method for loss decomposition and centroid estimation of the loss function in multi-task learning, and eliminates the impact of label noise by using label flipping rate. It also extends to the multi-task positive-unlabeled (PU) learning domain, which offers better flexibility and generalization performance. Moreover, Nesterov’s method is applied to accelerate the solution of the model. MTLNL is compared with other algorithms on five benchmark datasets, five image datasets, and a multi-task PU dataset to demonstrate its effectiveness.

MT-EpiPred: Multitask Learning for Prediction of Small-Molecule Epigenetic Modulators.

Epigenetic modulators play an increasingly crucial role in the treatment of various diseases. In this case, it is imperative to systematically investigate the activity of these agents and understand their influence on the entire epigenetic regulatory network rather than solely concentrate on individual targets. This work introduces MT-EpiPred, a multitask learning method capable of predicting the activity of compounds against 78 epigenetic targets. MT-EpiPred demonstrated outstanding performance, boasting an average auROC of 0.915 and the ability to handle few-shot targets. In comparison to the existing method, MT-EpiPred not only expands the target pool but also achieves superior predictive performance with the same data set. MT-EpiPred was then applied to predict the epigenetic target of a newly synthesized compound (1), where the molecular target was unknown. The method identified KDM4D as a potential target, which was subsequently validated through an in vitro enzyme inhibition assay, revealing an IC50 of 4.8 μM. The MT-EpiPred method has been implemented in the web server MT-EpiPred (http://epipred.com), providing free accessibility. In summary, this work presents a convenient and accurate tool for discovering novel small-molecule epigenetic modulators, particularly in the development of selective inhibitors and evaluating the impact of these inhibitors over a broad epigenetic network.

Analyzing the Emotional Characteristics of British and American Literature by Combining Semiotics and the Operational Model of Thinking

Abstract This paper first analyzes the emotional features of English and American literary works by combining semiotics of interactivity, language class emotional context and thinking operation model. Then, a B-Feature-BP text emotion feature construction model is constructed on the basis of the BRET model combined with the BP neural network, and the emotion features of English and American literary works are constructed by combining semiotics and the thought operation model. Then, based on the multi-task learning method in deep learning, a multi-task MT-GSU model is proposed to classify and recognize the emotional features of constructed text. Finally, the performance of constructing, classifying and recognizing the emotional features of English and American literary works in this paper is analyzed so as to analyze the emotional features of English and American literary works. The results show that the performance of the constructed emotional features is all greater than 0.8, and the emotional features of the characters, the environment, and the whole of the English and American literary works are above 0.87, and the classification time is between [0.338,0.721]s. The intensity of tendency of the characteristics of the emotions of the works of English and American literature is [0.68,0.78], the intensity of stability is [0.6,0.74], the intensity of profundity is [0.71,0.79], and the intensity of efficacy is [0.72,0.82]. This study has a positive impact on the appreciation and translation of English and American literary works.

Joint localization and classification of breast masses on ultrasound images using an auxiliary attention-based framework.

Multi-task learning (MTL) methods have been extensively employed for joint localization and classification of breast lesions on ultrasound images to assist in cancer diagnosis and personalized treatment. One typical paradigm in MTL is a shared trunk network architecture. However, such a model design may suffer information-sharing conflicts and only achieve suboptimal performance for individual tasks. Additionally, the model relies on fully-supervised learning methodologies, imposing heavy burdens on data annotation. In this study, we propose a novel joint localization and classification model based on attention mechanisms and a sequential semi-supervised learning strategy to address these challenges. Our proposed framework offers three primary advantages. First, a lesion-aware network with multiple attention modules is designed to improve model performance on lesion localization. An attention-based classifier explicitly establishes correlations between the two tasks, alleviating information-sharing conflicts while leveraging location information to assist in classification. Second, a two-stage sequential semi-supervised learning strategy is designed for model training to achieve optimal performance on both tasks and substantially reduces the need for data annotation. Third, the asymmetric and modular model architecture allows for the flexible interchangeability of individual components, rendering the model adaptable to various applications. Experimental results from two different breast ultrasound image datasets under varied conditions have demonstrated the effectiveness of the proposed method. Furthermore, we conduct comprehensive investigations into the impacts of various factors on model performance, gaining in-depth insights into the mechanism of our proposed framework. The code is available at https://github.com/comp-imaging-sci/lanet-bus.git.

A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning.

Multitask learning (MTL) is a powerful technique for jointly learning multiple tasks. However, it is difficult to achieve a tradeoff between tasks during iterative training, as some tasks may compete with each other. Existing methods manually design specific network models to mitigate task conflicts, but they require considerable manual effort and prior knowledge about task relationships to tune the model so as to obtain the best performance for each task. Moreover, few works have offered formal descriptions of task conflicts and theoretical explanations for the cause of task conflict problems. In this article, we provide a formal description of task conflicts that are caused by the gradient interference problem of tasks. To alleviate this issue, we propose a novel model-agnostic approach to mitigate gradient interference (MAMG) by designing a gradient clipping rule that directly modifies the interfering components on the gradient interfering direction. Specifically, MAMG is model-agnostic and thus it can be applied to a large number of multitask models. We also theoretically prove the convergence of MAMG and its superiority to existing MTL methods. We evaluate our method on a variety of real-world large datasets, and extensive experimental results confirm that MAMG can outperform some state-of-the-art algorithms on different types of tasks and can be easily applied to various methods.

The Effect of Grouping Output Parameters by Quality Characteristics on the Predictive Performance of Artificial Neural Networks in Injection Molding Process

In this study, a multi-input, multi-output-based artificial neural network (ANN) was constructed by classifying output parameters into different groups, considering the physical meanings and characteristics of product quality factors in the injection molding process. Injection molding experiments were conducted for bowl products, and a dataset was established. Based on this dataset, an ANN model was developed to predict the quality of molded products. The input parameters included melt temperature, mold temperature, packing pressure, packing time, and cooling time. The output parameters included mass, diameter, and height of the molded product. The output parameters were divided into two cases. In one case, diameter, and height, representing length, were grouped together, while mass was organized into a separate group. In the other case, mass, diameter and height were separated individually and applied to the ANN. A multi-task learning method was used to group the output parameters. The performance of the two constructed multi-task learning-based ANNs was compared with that of the conventional ANN where the output parameters were not separated and applied to a single layer. The comparative results showed that the multi-task learning architecture, which grouped the output parameters considering the physical meaning and characteristics of the quality of molded products, exhibited an improved prediction performance of about 32.8% based on the RMSE values.

High-Performance Detection-Based Tracker for Multiple Object Tracking in UAVs

As a result of increasing urbanization, traffic monitoring in cities has become a challenging task. The use of Unmanned Aerial Vehicles (UAVs) provides an attractive solution to this problem. Multi-Object Tracking (MOT) for UAVs is a key technology to fulfill this task. Traditional detection-based-tracking (DBT) methods begin by employing an object detector to retrieve targets in each image and then track them based on a matching algorithm. Recently, the popular multi-task learning methods have been dominating this area, since they can detect targets and extract Re-Identification (Re-ID) features in a computationally efficient way. However, the detection task and the tracking task have conflicting requirements on image features, leading to the poor performance of the joint learning model compared to separate detection and tracking methods. The problem is more severe when it comes to UAV images due to the presence of irregular motion of a large number of small targets. In this paper, we propose using a balanced Joint Detection and Re-ID learning (JDR) network to address the MOT problem in UAV vision. To better handle the non-uniform motion of objects in UAV videos, the Set-Membership Filter is applied, which describes object state as a bounded set. An appearance-matching cascade is then proposed based on the target state set. Furthermore, a Motion-Mutation module is designed to address the challenges posed by the abrupt motion of UAV. Extensive experiments on the VisDrone2019-MOT dataset certify that our proposed model, referred to as SMFMOT, outperforms the state-of-the-art models by a wide margin and achieves superior performance in the MOT tasks in UAV videos.

Iterative multitask learning and inference from seismic images

SUMMARY Seismic interpretation aims to extract quantitative and interpretable attributes from a seismic image produced using some migration method to inform characteristics of a subsurface reservoir or target of interest. Current paradigms for computing seismic attributes mostly rely on single-task algorithms. We develop an iterative, multitask machine learning method to learn and infer multiple attributes from a seismic image. This method is composed of two stages: a multitask inference stage and a multimodal, multitask refinement stage. The basic mechanism of this method is that we train a multitask inference neural network to estimate a set of attributes, including a relative geological time volume, a denoised higher-resolution seismic image and multiple fault attributes (including probability, dip and strike), from a low-resolution, noisy seismic image; then we input the inferred attributes to a multitask refinement NN to enhance the raw inference results iteratively. The two multitask neural networks are trained separately based on synthetic seismic images and associated labels generated by geological modelling. Applications of this multitask learning and inference method to synthetic and field seismic images show that our method can improve the structural consistency among output seismic attributes compared with single-task neural networks, leading to more reliable automatic interpretation and subsurface characterization.

RIRGAN: An end-to-end lightweight multi-task learning method for brain MRI super-resolution and denoising

A common problem in the field of deep-learning-based low-level vision medical images is that most of the research is based on single task learning (STL), which is dedicated to solving one of the situations of low resolution or high noise. Our motivation is to design a model that can perform both SR and DN tasks simultaneously, in order to cope with the actual situation of low resolution and high noise in low-level vision medical images. By improving the existing single image super-resolution (SISR) network and introducing the idea of multi-task learning (MTL), we propose an end-to-end lightweight MTL generative adversarial network (GAN) based network using residual-in-residual-blocks (RIR-Blocks) for feature extraction, RIRGAN, which can concurrently accomplish super-resolution (SR) and denoising (DN) tasks. The generator in RIRGAN is composed of several residual groups with a long skip connection (LSC), which can help form a very deep network and enable the network to focus on learning high-frequency (HF) information. The introduction of a discriminator based on relativistic average discriminator (RaD) greatly improves the discriminator’s ability and makes the generated image have more realistic details. Meanwhile, the use of hybrid loss function not only ensures that RIRGAN has the ability of MTL, but also enables RIRGAN to give a more balanced attention between quantitative evaluation of metrics and qualitative evaluation of human vision. The experimental results show that the quality of the restoration image of RIRGAN is superior to the SR and DN methods based on STL in both subjective perception and objective evaluation metrics when processing medical images with low-level vision. Our RIRGAN is more in line with the practical requirements of medical practice.

Sequence signal reconstruction based multi-task deep learning for sleep staging on single-channel EEG

The temporal context information between sleep stage sequence contains sleep transition rules, which is important for improving sleep staging performance. Existing multi-task learning methods reconstruct EEG signals from one sleep stage, ignoring the importance of sequential temporal context in capturing long-term dependencies to enhance representation learning. To address these issues, we propose a multi-task deep learning model to jointly reconstruct sequence signal and segment time series. The model enhances the ability of time series segmentation task to capture sequential temporal context and improves the performance of single-channel EEG by optimizing the common encoder for sequence signal reconstruction task. In addition, we design a one-dimensional channel attention module to enhance the feature representation extracted for sleep sequence signal. The experimental results on four datasets show that the multi-task deep learning model can improve the generalization using sequence signal reconstruction. Compared with other state-of-the-art methods, the method proposed in this study obtained competitive performance in terms of metrics such as accuracy, which is 85.6% on the 2013 version of Sleep-EDF Database Expanded, 83.4% on the 2018 version of Sleep-EDF Database Expanded, 85.6% on Sleep Heart Health Study, and 77.4% on CAP Sleep Database.

ADMET property prediction via multi-task graph learning under adaptive auxiliary task selection

It is a critical step in lead optimization to evaluate the absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties of drug-like compounds. Classical single-task learning (STL) has effectively predicted individual ADMET endpoints with abundant labels. Conversely, multi-task learning (MTL) can predict multiple ADMET endpoints with fewer labels, but ensuring task synergy and highlighting key molecular substructures remain challenges. To tackle these issues, this work elaborates a multi-task graph learning framework for predicting multiple ADMET properties of drug-like small molecules (MTGL-ADMET) by holding a new paradigm of MTL, "one primary, multiple auxiliaries." It first adeptly combines status theory with maximum flow for auxiliary task selection. The subsequent phase introduces a primary-task-centric MTL model with integrated modules. MTGL-ADMET not only outstrips existing STL and MTL methods but also offers a transparent lens into crucial molecular substructures. It is anticipated that this work can promote lead compound finding and optimization in drug discovery.

DSMM: A dual stance-aware multi-task model for rumour veracity on social networks

Rumour veracity on social networks aims to determine the authenticity of a claim based on the conversation graph. Current works typically employ graph neural networks (GNNs) to model the conversation graph and leverage the users’ stance classification on the claim (referred to as global stance) to enhance rumour veracity with multi-task learning methods. However, due to the over-smoothing problem, the layers of GNNs are usually very shallow. In Consequence, the graph nodes with limited receptive fields cannot perceive the semantic information of the claim node. This results in GNNs being sub-optimal for global stance learning. To address this issue, we present a novel dual stance-aware multi-task model, DSMM, to improve the rumour veracity results. Firstly, we design a position-aware semantic compensation method, which relies on the relative topological positions between the claim node and comment nodes to compensate graph nodes with different degrees of claim semantics, thus enhancing the awareness of graph nodes for claim semantics. In addition, we propose using local stance, the attitude tendency of comment nodes towards their predecessor nodes, to model local interactions of the conversation graph for the first time. Since the local stance is a fine-grained interaction representation of the conversation graph, it can complement the global stance and enhance the model’s understanding for conversation graphs. Further, we introduce the directed attention network to simulate the directedness of local stances and differentiate the importance of different local stances simultaneously. Finally, we use the cross-attention technique to fuse information from the above two stance classifications into the rumour veracity module to enhance model performance further. Experimental results on two publicly available competition datasets demonstrate the effectiveness of our approach.

A novel multi-task learning method for evaluating short-term voltage stability with incomplete PMU measurements

Methods for short-term voltage stability (STVS) assessment based on phasor measurement unit (PMU) data have become quite abundant. However, most of them are challenged to deal with the commonly encountered incomplete PMU data. Existing models for STVS assessment of incomplete PMU data mostly have the disadvantages of high computational complexity and large impact from the level of data missing. This paper proposes a multi-task learning method that performs stability assessment and missing data completion in parallel. The Transformer Encoder is used as the shared feature extractor, a GRU structure is used to output complete PMU data sequences, and a BP neural network is used to output stability assessment results. The loss function used for the missing completion task considers the accuracy in both the shape and time domains. A comparison with relevant models shows that the model proposed in this paper achieves the best performance in all indicators and is minimally affected by the observation windows and the level of data loss, demonstrating good robustness.

Multitask Learning From Augmented Auxiliary Data for Improving Speech Emotion Recognition

Despite the recent progress in speech emotion recognition (SER), state-of-the-art systems lack generalisation across different conditions. A key underlying reason for poor generalisation is the scarcity of emotion datasets, which is a significant roadblock to designing robust machine learning (ML) models. Recent works in SER focus on utilising multitask learning (MTL) methods to improve generalisation by learning shared representations. However, most of these studies propose MTL solutions with the requirement of meta labels for auxiliary tasks, which limits the training of SER systems. This paper proposes an MTL framework (MTL-AUG) that learns generalised representations from augmented data. We utilise augmentation-type classification and unsupervised reconstruction as auxiliary tasks, which allow training SER systems on augmented data without requiring any meta labels for auxiliary tasks. The semi-supervised nature of MTL-AUG allows for the exploitation of the abundant unlabelled data to further boost the performance of SER. We comprehensively evaluate the proposed framework in the following settings: (1) within corpus, (2) cross-corpus and cross-language, (3) noisy speech, (4) and adversarial attacks. Our evaluations using the widely used IEMOCAP, MSP-IMPROV, and EMODB datasets show improved results compared to existing state-of-the-art methods.

Deep multitask learning with label interdependency distillation for multicriteria street-level image classification

Multitask learning (MTL) aims at beneficial joint solving of multiple prediction problems by sharing information across different tasks. However, without adequate consideration of interdependencies, MTL models are prone to miss valuable information. In this paper, we introduce a novel deep MTL architecture that specifically encodes cross-task interdependencies within the setting of multiple image classification problems. Based on task-wise interim class label probability predictions by an intermediately supervised hard parameter sharing convolutional neural network, interdependencies are inferred in two ways: i) by directly stacking label probability sequences to the image feature vector (i.e., multitask stacking), and ii) by passing probability sequences to gated recurrent unit-based recurrent neural networks to explicitly learn cross-task interdependency representations and stacking those to the image feature vector (i.e., interdependency representation learning). The proposed MTL architecture is applied as a tool for generic multi-criteria building characterization using street-level imagery related to risk assessments toward multiple natural hazards. Experimental results for classifying buildings according to five vulnerability-related target variables (i.e., five learning tasks), namely height, lateral load-resisting system material, seismic building structural type, roof shape, and block position are obtained for the Chilean capital Santiago de Chile. Our MTL methods with cross-task label interdependency modeling consistently outperform single task learning (STL) and classical hard parameter sharing MTL alike. Even when starting already from high classification accuracy levels, estimated generalization capabilities can be further improved by considerable margins of accumulated task-specific residuals beyond +6% κ. Thereby, the combination of multitask stacking and interdependency representation learning attains the highest accuracy estimates for the addressed task and data setting (up to cross-task accuracy mean values of 88.43% overall accuracy and 84.49% κ). From an efficiency perspective, the proposed MTL methods turn out to be substantially favorable compared to STL in terms of training time consumption.

Multi-task-based spatiotemporal generative inference network: A novel framework for predicting the highway traffic speed

Accurately predicting the highway traffic speed can reduce traffic accidents and transit time, and it also provides valuable reference data for traffic control in advance. Three essential elements should be considered in highway traffic speed prediction: (1) dynamic spatiotemporal correlation, (2) prediction error propagation elimination, and (3) traffic speed heterogeneity on the highway network. A multi-task-based spatiotemporal generative inference network (MT-STGIN) is proposed to address the above challenges. 11https://doi.org/10.24433/CO.8500259.v1. MT-STGIN consists of three parts: an encoder based on spatiotemporal correlation extraction block (ST-Block), a decoder based on masked ST-Block and generative inference, and multi-task learning. The encoder is first used to extract the dynamic spatiotemporal correlation of the highway network. The decoder concentrates on correlating historical and - target sequences and generates the target hidden outputs rather than a dynamic step-by-step decoding way. Finally, a multi-task learning method is used to predict traffic speed on different types of road segments because of heterogeneity and shares the underlying network parameters. The evaluation experiments use the monitoring data of the highway in Yinchuan City, Ningxia Province, China. The experimental results demonstrate that the performance of the proposed prediction model is better than that of the baselines, and it can efficiently solve the problem of long-term highway speed prediction.