Multi-task Learning Approach Research Articles

Multi-task deep learning approach for sound event recognition and tracking

Multi-task deep learning approach for sound event recognition and tracking

A Latent Variable-Based Multitask Learning Approach for Degradation Modeling of Machines With Dependency and Heterogeneity

A Latent Variable-Based Multitask Learning Approach for Degradation Modeling of Machines With Dependency and Heterogeneity

Multi-task Adaptive Resolution Network for Lymph Node Metastasis Diagnosis from Whole Slide Images of Colorectal Cancer.

Automated detection of lymph node metastasis (LNM) holds great potential to alleviate the workload of doctors and reduce misinterpretations. Despite the practical successes achieved, effectively addressing the highly complex and heterogeneous tumor microenvironment remains an open and challenging problem, especially when tumor subtypes intermingle and are difficult to delineate. In this paper, we propose a multi-task adaptive resolution network, named MAR-Net, for LNM detection and subtyping in complex mixed-type cancers. Specifically, we construct a resolution-aware module to mine heterogeneous diagnostic information, which exploits the multi-scale pyramid information and adaptively combines multi-resolution structured features for comprehensive representation. Additionally, we adopt a multi-task learning approach that simultaneously addresses LNM detection and subtyping, reducing model instability during optimization and improving performance across both tasks. More importantly, to rectify the potential misclassification of tumor subtypes, we elaborately design a hierarchical subtying refinement (HSR) algorithm that leverages a generic segmentation model informed by pathologists' prior knowledge. Evaluations have been conducted on three private and one public cancer datasets (554 WSIs, 4.8 million patches). Our experimental results demonstrate that the proposed method consistently achieves superior performance compared to the state-of-the-art methods, achieving 0.5% to 3.2% higher AUC in LNM detection and 3.8% to 4.4% higher AUC in LNM subtyping.

Dysarthric Speech Recognition Using Pseudo-Labeling, Self-Supervised Feature Learning, and a Joint Multi-Task Learning Approach

Dysarthric Speech Recognition Using Pseudo-Labeling, Self-Supervised Feature Learning, and a Joint Multi-Task Learning Approach

Sparse and Expandable Network for Google's Pathways.

Recently, Google introduced Pathways as its next-generation AI architecture. Pathways must address three critical challenges: learning one general model for several continuous tasks, ensuring tasks can leverage each other without forgetting old tasks, and learning from multi-modal data such as images and audio. Additionally, Pathways must maintain sparsity in both learning and deployment. Current lifelong multi-task learning approaches are inadequate in addressing these challenges. To address these challenges, we propose SEN, a Sparse and Expandable Network. SEN is designed to handle multiple tasks concurrently by maintaining sparsity and enabling expansion when new tasks are introduced. The network leverages multi-modal data, integrating information from different sources while preventing interference between tasks. The proposed SEN model demonstrates significant improvements in multi-task learning, successfully managing task interference and forgetting. It effectively integrates data from various modalities and maintains efficiency through sparsity during both the learning and deployment phases. SEN offers a straightforward yet effective solution to the limitations of current lifelong multi-task learning methods. By addressing the challenges identified in the Pathways architecture, SEN provides a promising approach for developing AI systems capable of learning and adapting over time without sacrificing performance or efficiency.

A Framework for Single-Panicle Litchi Flower Counting by Regression with Multitask Learning

The number of flowers is essential for evaluating the growth status of litchi trees and enables researchers to estimate flowering rates and conduct various phenotypic studies, particularly focusing on the information of individual panicles. However, manual counting remains the primary method for quantifying flowers, and there has been insufficient emphasis on the advancement of reliable deep learning methods for estimation and their integration into research. Furthermore, the current density map-based methods are susceptible to background interference. To tackle the challenges of accurately quantifying small and dense male litchi flowers, a framework counting the flowers in panicles is proposed. Firstly, an existing effective algorithm YOLACT++ is utilized to segment individual panicles from images. Secondly, a novel algorithm FlowerNet based on density map regression is proposed to accurately count flowers in each panicle. By employing a multitask learning approach, FlowerNet effectively captures both foreground and background information, thereby overcoming interference from non-target areas during pixel-level regression tasks. It achieves a mean absolute error of 47.71 and a root mean squared error of 61.78 on the flower dataset constructed. Additionally, a regression equation is established using a dataset of inflorescences to examine the application of the algorithm for flower counting. It captures the relationship between the predicted number of flowers by FlowerNet and the manually counted number, resulting in a determination coefficient ( R 2 ) of 0.81. The proposed algorithm shows promise for automated estimation of litchi flowering quantity and can serve as a valuable reference for litchi orchard management during flowering period.

Multitask feature selection within structural datasets

Abstract Population-based structural health monitoring (PBSHM) systems use data from multiple structures to make inferences of health states. An area of PBSHM that has recently been recognized for potential development is the use of multitask learning (MTL) algorithms that differ from traditional single-task learning. This study presents an application of the MTL approach, Joint Feature Selection with LASSO, to provide automatic feature selection. The algorithm is applied to two structural datasets. The first dataset covers a binary classification between the port and starboard side of an aircraft tailplane, for samples from two aircraft of the same model. The second dataset covers normal and damaged conditions for pre- and postrepair of the same aircraft wing. Both case studies demonstrate that the MTL results are interpretable, highlighting features that relate to structural differences by considering the patterns shared between tasks. This is opposed to single-task learning, which improved accuracy at the cost of interpretability and selected features, which failed to generalize in previously unobserved experiments.

STID-Prompt: Prompt learning for sentiment-topic-importance detection in financial news

With the development of the Internet and the financial industry, the analysis and judgement of financial news has become increasingly important. Common tasks in this area include sentiment analysis, topic classification, and importance judgement. Existing approaches use fine-tuning paradigms to address these tasks. However, the lack of labeled data in the field of financial news poses a small sample learning problem. The new paradigm represented by prompt learning provides a new way and means to improve the performance of small-sample classification. In addition, existing methods do not focus on research on sentiment, topic, and importance simultaneously, which is also an important factor in evaluating financial news. In practical applications, joint judgment of sentiment, topic, and importance is more effective. Finally, these methods require different tuning for each specific task. They do not consider relationships between individual tasks and cannot utilize information across tasks. This limits their application in complex situations. In this work, we propose a prompt-based financial news classification model (STID-Prompt) to address these issues. For the first two problems, we solve the sentiment analysis task, the topic classification task, the importance judgment task, and the 〈sentiment, topic, importance〉 classification task by designing complex prompt templates. For the third problem, we propose a unique prompt-based joint multi-task learning approach. It learns knowledge from multiple tasks and integrates it into the target task, and the multi-task learning approach further improves the performance of the model. Experimental results show the effectiveness of our approach even with less training data.

An adaptive deep multi-task learning approach for citywide travel time collaborative estimation

The provision of accurate travel time information holds paramount significance for optimizing traffic operations and management. However, existing approaches typically address multiple travel time learning tasks independently, neglecting the correlation of tasks and necessitating the construction of separate models for each task. As the scale of the road network expands, this practice becomes increasingly infeasible. To bridge this gap, this paper presents an adaptive deep multi-task learning model to solve multiple travel time estimation tasks collaboratively from a citywide perspective. The novel model seamlessly integrates geographical, environmental, and spatial-correlated features, capturing the complex non-linear characteristics of travel time in the urban road network through a deep network learning feature representations layer by layer from the input data. Through the paradigm of deep multi-task collaborative learning, the knowledge learned from one task serves as an inductive bias for other tasks, thereby facilitating the learning of related tasks. By modeling the uncertainty of travel time with a probabilistic model, the weights of jointly learned tasks are adaptively fine-tuned in a principled way. Additionally, a novel algorithm is devised to fully leverage the information embedded within the data with partial information. The proposed methodology is evaluated through a case study on the citywide road network in Beijing, China. Empirical results of extensive experiments demonstrate the adeptness of the proposed model in effectively harnessing information transferred from parallel tasks, adaptively fine-tuning the weights of joint learning tasks, and proficiently exploiting data characterized by incomplete labels, therefore leading to superior performance compared to competing methods. The average estimation error of the four jointly learned travel time estimation tasks is about 28.48%, yielding an improvement of over 4% compared to single-task learning models. The results verify the effectiveness and robustness of the proposed model for travel time estimation in the urban road network.

Combining a multi-feature neural network with multi-task learning for emergency calls severity prediction

In emergency call centers, operators are required to analyze and prioritize emergency situations prior to any intervention. This allows the team to deploy resources efficiently if needed, and thereby provide the optimal assistance to the victims. The automation of such an analysis remains challenging, given the unpredictable nature of the calls. Therefore, in this study, we describe our attempt in improving an emergency calls processing system’s accuracy in the classification of an emergency’s severity, based on transcriptions of the caller’s speech. Specifically, we first extend the baseline classifier to include additional feature extractors of different modalities of data. These features include detected emotions, time-based features, and the victim’s personal information. Second, we experiment with a multi-task learning approach, in which we attempt to detect the nature of the emergency on the one hand, and improve the severity classification score on the other hand. Additional improvements include the use of a larger dataset and an explainability study of the classifier’s decision-making process. Our best model was able to predict 833 emergency calls’ severity with a 71.27% accuracy, a 5.33% improvement over the baseline model. Moreover, we extended our tool with additional modules that can prove to be useful when handling emergency calls.

A Deep Learning-Based Multidimensional Aesthetic Quality Assessment Method for Mobile Game Images

Mobile games have played an increasingly significant role in people's leisure lives in recent years, thanks to the fast expansion of the gaming industry and the widespread use of mobile devices. The aesthetic quality of game pictures is a very important factor that attracts users' interest. However, evaluating the aesthetic quality of mobile game pictures is difficult since the painting styles of games vary greatly and the evaluation criteria are also diversified. In this paper, we propose a multi-task deep learning based method, which is able to predict the aesthetic quality of mobile game images in multiple dimensions (MGQA). The proposed model consists of two modules, a feature extraction module and a quality regression module. We extract quality-aware features from intermediate layers of the deep convolution neural network (CNN) and then incorporate them into the final feature representation in the feature extraction module, allowing the model to fully use visual information from low to high levels. The quality regression module uses fully connected (FC) layers to map quality-aware features into quality scores across multiple dimensions. The multi-dimensional aesthetic quality scores are trained using a multi-task learning approach, in which quality-aware features are shared across multiple dimensional quality prediction tasks. Finally, several key factors which help the proposed model perform better are analyzed. The experimental results indicate that our proposed method not only achieves the greatest performance on mobile game images, but also is applicable to natural scene images.

Explaining legal judgments: A multitask learning framework for enhancing factual consistency in rationale generation

The explanation of legal judgments is crucial for the transparency, fairness, and trustworthiness, aiming to provide rationales for decision-making. While previous works have focused on improving the accuracy of legal judgment prediction, the lack of explainability seriously limits the practical application of these methods. Many researchers have dedicated effort to extracting or generating rationales as explanations for legal judgments, ignoring the factual consistency of these rationales. Inconsistencies between rationales and fact descriptions severely impact their applicability. To address these issues, we investigate Event Chain – ordered sequences of events related to criminal behavior – as an intermediate representation in the fact descriptions, to enhance the representation of causal relationships among events, and focus on crucial events within the fact description. Specifically, we propose a multi-task learning approach, dubbed LegalMind, that introduces Event Chain as an auxiliary task and jointly models Event Chain and rationale in a unified decoder, to improve factual consistency in rationale generation. The experiment results show that our model outperforms the state-of-the-art methods, achieving improvements of 7.65% and 6.65% in AVG-BLEU on the CJO dataset and LAIC2021 dataset respectively compared to BART-C3VG. Furthermore, in comparison to BART-C3VG, our model demonstrated superior performance with increases of 6.17% and 8.33% in factual consistency on the CJO and LAIC2021 datasets, respectively.

Emotion sensitive analysis of learners’ cognitive state using deep learning

<p class="IndexTerms">The assessment of the state of mind of a student has traditionally been a troublesome task. The advances in deep learning have given analysts new opportunities to try and do therefore. Most state of mind methods focus principally on attention, failing to account for the significance of human emotions. Emotions are significant in laptop vision and a good deal of analysis is conducted exploitation human feelings. Our objective is to propose an emotion-sensitive analysis of individuals’ mental state, specifically focusing on students’ attention levels. This analysis will be carried out in a non-intrusive manner by detecting both head posture and emotions. To achieve this, we employ a multi-task learning approach that utilizes convolutional neural networks (CNNs). These networks are capable of simultaneously identifying facial expressions, locating facial landmarks, and estimating head position, all in real-time. Face alignment is additional assessed by estimating the pinnacle position and face alignment. The estimation of the pinnacle cause and alignment of the face is additional employed by the trainer to live the learner’s span. Experimental results show that the technique will accurately verify students’ emotions with a ninety-four accuracy rate.</p>

User-centric recommendations on energy-efficient appliances in smart grids: A Multi-task learning approach

Deploying energy-efficient appliances is one of the most effective ways to save energy bills for residents. However, the existing recommender systems for energy-efficient appliances passively rely on energy consumption patterns without the knowledge of users’ true needs. This paper proposes a user-centric energy-efficient appliance personalized recommender system (EEA-PRS) based on information collected from load monitoring platforms and e-commerce websites. The proposed system is built in a novel multi-task learning approach to collaboratively infer user's preference on: (1) common types of appliances that appear in historical data; (2) energy-efficient models of common appliances; and (3) types of appliances that are novel to the users. The proposed system provides supervisory recommendation services with user feedback preferences on appliances as data labeling, which enables closed-loop evaluation to adhere to users’ needs and interests. Simulation studies with comparative analysis have been conducted to validate its leading recommendation performance in terms of conforming to user preferences.

Beyond the single-outcome approach: A comparison of outcome-wide analysis methods for exposome research

Outcome-wide analysis can offer several benefits, including increased power to detect weak signals and the ability to identify exposures with multiple effects on health, which may be good targets for preventive measures. Recently, advanced statistical multivariate techniques for outcome-wide analysis have been developed, but they have been rarely applied to exposome analysis. In this work, we provide an overview of a selection of methods that are well-suited for outcome-wide exposome analysis and are implemented in the R statistical software. Our work brings together six different methods presenting innovative solutions for typical problems arising from outcome-wide approaches in the context of the exposome, including dependencies among outcomes, high dimensionality, mixed-type outcomes, missing data records, and confounding effects. The identified methods can be grouped into four main categories: regularized multivariate regression techniques, multi-task learning approaches, dimensionality reduction approaches, and bayesian extensions of the multivariate regression framework. Here, we compare each technique presenting its main rationale, strengths, and limitations, and provide codes and guidelines for their application to exposome data. Additionally, we apply all selected methods to a real exposome dataset from the Human Early-Life Exposome (HELIX) project, demonstrating their suitability for exposome research. Although the choice of the best method will always depend on the challenges to be faced in each application, for an exposome-like analysis we find dimensionality reduction and bayesian methods such as reduced rank regression (RRR) or multivariate bayesian shrinkage priors (MBSP) particularly useful, given their ability to deal with critical issues such as collinearity, high-dimensionality, missing data or quantification of uncertainty.

A Deep Multi-Task Learning Approach for Bioelectrical Signal Analysis

Deep learning is a promising technique for bioelectrical signal analysis, as it can automatically discover hidden features from raw data without substantial domain knowledge. However, training a deep neural network requires a vast amount of labeled samples. Additionally, a well-trained model may be sensitive to the study object, and its performance may deteriorate sharply when transferred to other study objects. We propose a deep multi-task learning approach for bioelectrical signal analysis to address these issues. Explicitly, we define two distinct scenarios, the consistent source-target scenario and the inconsistent source-target scenario based on the motivation and purpose of the tasks. For each scenario, we present methods to decompose the original task and dataset into multiple subtasks and sub-datasets. Correspondingly, we design the generic deep parameter-sharing neural networks to solve the multi-task learning problem and illustrate the details of implementation with one-dimension convolutional neural networks (1D CNN), vanilla recurrent neural networks (RNN), recurrent neural networks with long short-term memory units (LSTM), and recurrent neural networks with gated recurrent units (GRU). In these two scenarios, we conducted extensive experiments on four electrocardiogram (ECG) databases. The results demonstrate the benefits of our approach, showing that our proposed method can improve the accuracy of ECG data analysis (up to 5.2%) in the MIT-BIH arrhythmia database.

A novel spatiotemporal 3D CNN framework with multi-task learning for efficient structural damage detection

In recent years, convolutional neural networks (CNNs) have demonstrated promising results in detecting structural damage. However, their architectures often overlook spatial and temporal effects simultaneously. This limitation can result in the loss of valuable information and an incapability to fully capture the complexity of the data, ultimately leading to reduced accuracy and suboptimal performance. This study proposes an intuitive three-dimensional CNN architecture that takes into account vibration history along with sensor spatial relations based on their relative positions. Furthermore, a multi-task learning (MTL) approach is suggested, which is a powerful approach for performing multiple tasks with a single network. The proposed 3D CNN method has been employed to detect single and double damage cases in an experimental steel frame through conventional classification alongside the transfer learning (TL). Moreover, MTL is used to detect single and double damage scenarios with a single unified network, which evaluates damage presence in separate tasks. The 3D CNN fulfilled state-of-the-art performance and 100% accuracy in detecting structural damage in almost all experiments. Additionally, the MTL model achieved promising results even in the presence of severe imbalanced classes of data. Furthermore, it was observed that the utilization of TL resulted in a notable reduction of computation time by 68% and the number of trainable parameters by 90% with the same level of accuracy in double-damage cases.

Multi-Task Learning of Scanning Electron Microscopy and Synthetic Thermal Tomography Images for Detection of Defects in Additively Manufactured Metals.

One of the key challenges in laser powder bed fusion (LPBF) additive manufacturing of metals is the appearance of microscopic pores in 3D-printed metallic structures. Quality control in LPBF can be accomplished with non-destructive imaging of the actual 3D-printed structures. Thermal tomography (TT) is a promising non-contact, non-destructive imaging method, which allows for the visualization of subsurface defects in arbitrary-sized metallic structures. However, because imaging is based on heat diffusion, TT images suffer from blurring, which increases with depth. We have been investigating the enhancement of TT imaging capability using machine learning. In this work, we introduce a novel multi-task learning (MTL) approach, which simultaneously performs the classification of synthetic TT images, and segmentation of experimental scanning electron microscopy (SEM) images. Synthetic TT images are obtained from computer simulations of metallic structures with subsurface elliptical-shaped defects, while experimental SEM images are obtained from imaging of LPBF-printed stainless-steel coupons. MTL network is implemented as a shared U-net encoder between the classification and the segmentation tasks. Results of this study show that the MTL network performs better in both the classification of synthetic TT images and the segmentation of SEM images tasks, as compared to the conventional approach when the individual tasks are performed independently of each other.

A syntax-guided multi-task learning approach for Turducken-style code generation

A syntax-guided multi-task learning approach for Turducken-style code generation

Optimizing fake news detection for Arabic context: A multitask learning approach with transformers and an enhanced Nutcracker Optimization Algorithm

The rapid proliferation of news and posts across social media platforms has spawned a concerning wave of misinformation. Disseminating false information and news significantly threatens public health and safety. To address this critical issue, we present an innovative disinformation detection framework, leveraging the power of multi-task learning (MTL) and meta-heuristic techniques. Our framework harnesses the potential of an MTL approach and a state-of-the-art pre-trained Transformer-based model, enabling the extraction of comprehensive contextual features from Arabic social media posts. These contextual are input to an advanced feature selection (FS) model utilizing a modified Nutcracker Optimization Algorithm. Through extensive evaluation of diverse datasets of Arabic social media posts, our proposed framework achieves remarkable results. Notably, our framework attains an accuracy rate of 87% and 69% for binary and multi-classification, respectively. In addition, the developed method outperforms all compared algorithms. Our findings demonstrate the potency of our disinformation detection framework, serving as a robust tool in the battle against misinformation spread. By shedding light on the truth amidst the vast social media content, we can safeguard public health and empower individuals with reliable information.