Multi-task Learning Research Articles

Understanding Public Opinion towards ESG and Green Finance with the Use of Explainable Artificial Intelligence

This study leverages explainable artificial intelligence (XAI) techniques to analyze public sentiment towards Environmental, Social, and Governance (ESG) factors, climate change, and green finance. It does so by developing a novel multi-task learning framework combining aspect-based sentiment analysis, co-reference resolution, and contrastive learning to extract nuanced insights from a large corpus of social media data. Our approach integrates state-of-the-art models, including the SenticNet API, for sentiment analysis and implements multiple XAI methods such as LIME, SHAP, and Permutation Importance to enhance interpretability. Results reveal predominantly positive sentiment towards environmental topics, with notable variations across ESG categories. The contrastive learning visualization demonstrates clear sentiment clustering while highlighting areas of uncertainty. This research contributes to the field by providing an interpretable, trustworthy AI system for ESG sentiment analysis, offering valuable insights for policymakers and business stakeholders navigating the complex landscape of sustainable finance and climate action. The methodology proposed in this paper advances the current state of AI in ESG and green finance in several ways. By combining aspect-based sentiment analysis, co-reference resolution, and contrastive learning, our approach provides a more comprehensive understanding of public sentiment towards ESG factors than traditional methods. The integration of multiple XAI techniques (LIME, SHAP, and Permutation Importance) offers a transparent view of the subtlety of the model’s decision-making process, which is crucial for building trust in AI-driven ESG assessments. Our approach enables a more accurate representation of public opinion, essential for informed decision-making in sustainable finance. This paper paves the way for more transparent and explainable AI applications in critical domains like ESG.

Augmenting biomedical named entity recognition with general-domain resources

ObjectiveTraining a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle those challenges through transfer learning from easily accessible resources with fewer concept overlaps with biomedical datasets. MethodsWe proposed GERBERA, a simple-yet-effective method that utilized general-domain NER datasets for training. We performed multi-task learning to train a pre-trained biomedical language model with both the target BioNER dataset and the general-domain dataset. Subsequently, we fine-tuned the models specifically for the BioNER dataset. ResultsWe systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances. Despite using fewer biomedical resources, our models demonstrated superior performance compared to baseline models trained with additional BioNER datasets. Specifically, our models consistently outperformed the baseline models in six out of eight entity types, achieving an average improvement of 0.9% over the best baseline performance across eight entities. Our method was especially effective in amplifying performance on BioNER datasets characterized by limited data, with a 4.7% improvement in F1 scores on the JNLPBA-RNA dataset. ConclusionThis study introduces a new training method that leverages cost-effective general-domain NER datasets to augment BioNER models. This approach significantly improves BioNER model performance, making it a valuable asset for scenarios with scarce or costly biomedical datasets. We make data, codes, and models publicly available via https://github.com/qingyu-qc/bioner_gerbera.

Knowledge enhanced multi-task learning for simultaneous optimization of human parsing and pose estimation

Human parsing and pose estimation are both closely related to the human body structure, and it is advantageous to deal with both together in order to assist each other’s learning. Nevertheless, the two tasks have their own unique characteristics, and it is therefore difficult to efficiently and simultaneously obtain a good performance for both tasks in one single multi-task learning (MTL) network. This paper proposes a new MTL network (named “KDNet”) to enhance the communication between human parsing and pose estimation through knowledge distillation so as to improve the simultaneous learning of both tasks. In the proposed KDNet, a consistent representation module is used to enhance the related information sharing between the tasks, and a single-task model-based loss weighting method is developed to balance the loss levels. The proposed KDNet has been verified across three benchmark datasets, and in comparison to other state-of-the-art methods it achieves substantial performance gains without increasing model parameters during inference. The codes are shared at https://github.com/Yhdian/KDNet.git.

CoMaSa:Context Multi-aware Self-attention for emotional response generation

Dialogue generation is an important research direction in natural language generation, and generating utterances with appropriate contextual emotion is a challenging task. The previous work incorporates commonsense knowledge as an auxiliary information source by simply concatenating or adding it to the embeddings of the dialogue context. They did not consider updating the commonsense knowledge from the perspective of context semantic information or selecting context from the perspective of commonsense knowledge semantic information. At the same time, they did not extract emotional information from dialogue context and commonsense knowledge to enhance the embeddings of the dialogue text. These defects may lead to the dialogue model’s inability to judge the semantic consistency between context and generated utterance, and result in the generation of utterance unassociated with the dialogue context and the corresponding context emotion being untrue. Our goal is to build an emotional dialogue generation model that can deeply integrate commonsense knowledge with dialogue context and achieve bidirectional semantic interaction and enhancement. Therefore, in this paper, we propose a context multi-aware self-attention emotion dialogue generation model that models from multiple aspects such as the semantic information, emotional information of context and external knowledge, it effectively alleviates the problems of generating utterance unrelated to the context and disordered emotional expressions in utterance. First, we adopt the transformer encoder as the basic framework and carefully construct a context-aware encoder capable of deeply extracting semantic information. Specifically, we ingeniously integrate different levels of positional information to simulate the natural flow of conversation between speakers. Then, we effectively integrate these positional information with the basic word embedding vectors, and enrich the dialogue’s embedding representation. Lastly, by introducing the attention mechanism, the model can precisely capture and extract context information closely related to the current situation and achieve a deep understanding and accurate grasp of the dialogue content. Second, we consider that the relationship between context and commonsense knowledge is not just a unidirectional selection but a more intimate bidirectional interaction. Therefore, we designed a bidirectional interaction attention component for context and external knowledge. Specifically, we use the method of iterative similarity matrices to calculate the bidirectional similarity relationship between context and external knowledge. Then, using the attention algorithm, we calculate the connectivity from context to commonsense knowledge and from commonsense knowledge to context. Finally, by combining the semantic embeddings of both aspects, we achieve bidirectional updating of context and external knowledge. Third, we establish an emotion-aware decoder based on multi-task learning, which includes an emotion classifier and an utterance generator. In the emotion classifier, we establish an auxiliary task for emotion recognition to help the model understand the process of emotional transition. We use the output of the emotion recognition model as an additional supervisory signal to better control the generation of emotional responses. In the utterance generator, we integrate the output of the emotion classifier with the interaction vector from the bidirectional interaction attention component. This fusion allows us to generate utterance that aligns with the emotional context. Our experiments on DailyDialog and ESConv datasets show that CoMaSa outperforms the baselines in terms of Perplexity, Distinct-1, Distinct-2, and human evaluation. The experiment demonstrates the effectiveness of our Context Multi-aware Self-attention model in the task of emotion generation.

Online parallel multi-task relationship learning via alternating direction method of multipliers

Online multi-task learning (OMTL) enhances streaming data processing by leveraging the inherent relations among multiple tasks. It can be described as an optimization problem in which a single loss function is defined for multiple tasks. Existing gradient-descent-based methods for this problem might suffer from gradient vanishing and poor conditioning issues. Furthermore, the centralized setting hinders their application to online parallel optimization, which is vital to big data analytics. Therefore, this study proposes a novel OMTL framework based on the alternating direction multiplier method (ADMM), a recent breakthrough in optimization suitable for the distributed computing environment because of its decomposable and easy-to-implement nature. The relations among multiple tasks are modeled dynamically to fit the constant changes in an online scenario. In a classical distributed computing architecture with a central server, the proposed OMTL algorithm with the ADMM optimizer outperforms SGD-based approaches in terms of accuracy and efficiency. Because the central server might become a bottleneck when the data scale grows, we further tailor the algorithm to a decentralized setting, so that each node can work by only exchanging information with local neighbors. Experimental results on a synthetic and several real-world datasets demonstrate the efficiency of our methods.

Multi-swarm multi-tasking ensemble learning for multi-energy demand prediction

The increasing uptake of renewable energy sources leads to more uncertainties of energy scheduling, resulting in more difficulties of energy demand management in smart grid. Accurate forecasting plays a significant role in energy management, and in some cases it may involve predictions of different consumers or geographic regions. Traditionally, these prediction problems have been solved independently, ignoring the potential shared knowledge among them that may help facilitate the overall performance. In this manuscript, we propose a multi-swarm multi-tasking ensemble learning (MSMTEL) framework for solving the energy demand forecasting across multiple cities. The proposed method comprises single-task pretraining, multi-task optimization (MTO), and ensemble learning (EL). For each prediction task, several subtasks are generated based on predefined parameters in the forecasting model. Each subtask utilizes an independent deep neural network (DNN) as the predictor, which is pretrained individually. Subsequently, we modify the dynamic multi-swarm particle swarm optimization (DMS-PSO) as a customized multi-swarm PSO (CMS-PSO) algorithm for implementing MTO. Each subswarm in CMS-PSO focuses on finding the optimal model knowledge (pretrained DNN weights and biases) of source tasks (all subtasks) to be reused by the target subtask. Finally, instead of choosing the best forecasting among the subtasks, results of subtasks over the same energy prediction problem are combined to yield the EL result, wherein weight coefficients determining the contributions of subtasks to EL are optimized by PSO. MSMTEL is evaluated against single-task learning (STL) and MTO to demonstrate its superior performance.

Personalised soft prompt tuning in pre-trained language models: Bridging multitask transfer learning and crowdsourcing learning

Soft prompt tuning significantly enhances the performance of pre-trained language models, especially in complex tasks where abundant annotated data is available. Crowdsourcing provides a cost-effective means of obtaining large-scale annotations; however, it can hinder the effectiveness of soft prompt tuning due to varying annotation criteria among different annotators, introducing data noise and degrading performance. To address this issue, we conceptualise annotations from each annotator as a subtask and frame crowdsourcing learning as multitask transfer learning. We propose a novel soft prompt tuning method utilising personalised prompts designed to capture the principles of individual annotators through a knowledge distillation approach. To validate our hypothesis, we apply our method across four benchmark datasets in two specific crowdsourcing tasks: crowdsourced named entity recognition (CNER) and crowdsourced relation extraction (CRE). Our personalised soft prompt method shows significant improvements, with average increases of 8.96% in CNER and 14.44% in CRE compared to the standard soft prompt tuning method, while also achieving competitive results against state-of-the-art crowdsourcing methods.

Knowledge-informed FIR-based cross-category filtering framework for interpretable machinery fault diagnosis under small samples

Relying on sufficient training data, the existing fault diagnosis methods rarely focus on the methodological interpretability and the data scarcity in real industrial scenarios simultaneously. Motivated by this issue, we deeply reexamined the intrinsic characteristics of fault signals and the guiding significance of classical signal-processing methods for feature enhancement. From the perspective of multiscale modes, this study tailors multiple learnable knowledge-informed finite impulse response (FIR) filtering kernels to extract sensitive modes for explainable feature enhancement. On this foundation, a knowledge-informed FIR-based cross-category filtering (FIR-CCF) framework is further proposed for interpretable small-sample fault diagnosis. With the consideration of the mode complexity, a cross-category filtering strategy is explored to further enhance feature expressions for identifying single state. To be special, this strategy divides a multi-class recognition process into multiple two-class recognition task. A multi-task learning is then presented where multiple binary-class base learners (BCBLearners) that consists of a feature extractor and a two-class classifier is established to seek discriminate mode features for each type of state. Eventually, all feature extractors are fixed and a multi-class classifier is established and to fuse all mode features for high-precision multi-class identification via ensemble learning. As a variant of signal-processing-collaborated deep learning frameworks, the FIR-CCF method fully exploits the strengths of signal-processing methods in interpretability and feature extraction. Three experimental cases highlight the superiority and significant improvement of the FIR-CCF framework over other five state-of-the-art diagnosis methods and three ablation models. Specially, extensive visualization is implemented to place in-depth insight into how the FIR-CCF framework works. It can be also foreseen that the signal-processing-collaborated deep learning framework shows enormous potential in interpretable fault diagnosis for knowledge-informed artificial intelligence. Related source codes will be available at: https://github.com/BITS/FIR-CCF-main.

Enhancing microgrid forecasting accuracy with SAQ-MTCLSTM: A self-adjusting quantized multi-task ConvLSTM for optimized solar power and load demand predictions

Accurate forecasting of solar power output and load demand is critical for the efficient operation and management of isolated microgrids, where reliability and sustainability are paramount. Traditional methods often struggle with data scarcity, limitations in capturing intricate temporal dynamics, and lack of scalability. This research introduces a novel multi-task learning (MTL) model, the Self-Aware Quantized Multi-Task ConvLSTM (SAQ-MTCLSTM), which addresses these challenges by jointly forecasting solar power and load demand while leveraging shared representations across these interdependent time series. The SAQ-MTCLSTM incorporates a sophisticated architecture that combines convolutional and LSTM layers with self-aware quantization to enhance computational efficiency and model adaptability. This allows for effective knowledge transfer, improved data utilization, and the capture of intricate temporal patterns. Evaluated on a real-world isolated microgrid dataset from the Micro Reseau Mafate research project, the model demonstrates significant improvements in forecasting accuracy, with MSE values of 0.0021 for solar power output and 0.0037 for load demand forecasting, compared to single-task and other advanced models. Extensive experiments highlight the impact of data scarcity, seasonality patterns, and microgrid topology on forecasting performance. Such forecasting is essential to optimize the integration and utilization of renewable resources, enhancing operational stability and reducing dependency on external energy supplies.

Advancing streamflow prediction in data-scarce regions through vegetation-constrained distributed hybrid ecohydrological models

Hybrid models that combine deep learning with physical principles have recently shown significant promise in improving streamflow prediction in data-scarce regions, achieving generally greater accuracy and reliability than either method alone. However, existing hybrid hydrological models underutilize the extensive and readily available remote sensing data on vegetation, which plays a critical role in hydrologic processes. Incorporating vegetation information can better constrain model processes by accounting for the interactions and feedbacks between vegetation and hydrology, thereby enabling more robust predictions. In this context, we present a novel distributed hybrid ecohydrological model that integrates dynamic vegetation processes into a distributed hybrid hydrological model, capable of simultaneously simulating leaf area index (LAI) and streamflow through multi-task learning. Our results indicate that in scenarios with limited available streamflow data, the combined constraint of LAI spatiotemporal patterns and streamflow significantly enhances the model’s ability to generalize streamflow simulation across space and time compared to models trained solely with either LAI or streamflow. Even with the increased availability of streamflow data, models utilizing both LAI and streamflow continue to exhibit superior robustness in streamflow prediction. The distributed ecohydrological model, which employs multi-process coupling and multi-source data constraints, accurately simulates the target variables (i.e., streamflow and LAI) while reliably capturing dynamic changes in other ecohydrological processes, such as evapotranspiration. This capability highlights its potential for multi-process diagnostics. Overall, our approach offers a new perspective for streamflow prediction in data-scarce regions and represents an important step towards integrating observational constraints from multiple subsystems into hybrid Earth system modeling

MT-CrackNet:A multi-task deep learning framework for automatic in-situ fatigue micro-crack detection and quantification

Characterizing fatigue micro-cracks is crucial for understanding the mechanisms and behaviors of material damage. In-situ fatigue testing is an essential method for observing the evolution of fatigue micro-cracks; however, the process often requires significant time, making the measurement of micro-cracks a tedious task. This paper introduces a multi-task deep learning framework called MT-CrackNet, which enables automatic detection and quantification of in-situ fatigue micro-cracks. The framework is capable of recognizing or segmenting multiple tasks such as micro-cracks, text, and scales simultaneously, and its effectiveness is not limited by the magnification of in-situ images. By integrating attention mechanisms and multi-scale strategies, the model enhances its ability to handle long-range dependencies and preserve detail information, accurately identifying and measuring the length of micro-cracks. The effectiveness of the proposed MT-CrackNet is validated through three in-situ fatigue micro-crack propagation experiments.

Efficient Multitask Reinforcement Learning Without Performance Loss.

We propose an iterative sparse Bayesian policy optimization (ISBPO) scheme as an efficient multitask reinforcement learning (RL) method for industrial control applications that require both high performance and cost-effective implementation. Under continual learning scenarios in which multiple control tasks are sequentially learned, the proposed ISBPO scheme preserves the previously learned knowledge without performance loss (PL), enables efficient resource use, and improves the sample efficiency of learning new tasks. Specifically, the proposed ISBPO scheme continually adds new tasks to a single policy neural network while completely preserving the control performance of previously learned tasks through an iterative pruning method. To create a free-weight space for adding new tasks, each task is learned through a pruning-aware policy optimization method called the sparse Bayesian policy optimization (SBPO), which ensures efficient allocation of limited policy network resources for multiple tasks. Furthermore, the weights allocated to the previous tasks are shared and reused in new task learning, thereby improving sample efficiency and the performance of new task learning. Simulations and practical experiments demonstrate that the proposed ISBPO scheme is highly suitable for sequentially learning multiple tasks in terms of performance conservation, efficient resource use, and sample efficiency.

Low-complexity multi task learning for joint acoustic scenes classification and sound events detection

Low-complexity multi task learning for joint acoustic scenes classification and sound events detection

DeepMarkerNet: Leveraging supervision from the Duchenne Marker for spontaneous smile recognition

Distinguishing between spontaneous and posed smiles from videos poses a significant challenge in pattern classification literature. Researchers have developed feature-based and deep learning-based solutions for this problem. To this end, deep learning outperforms feature-based methods. However, certain aspects of feature-based methods could improve deep learning methods. For example, previous research has shown that Duchenne Marker (or D-Marker) features from the face play a vital role in spontaneous smiles, which can be useful to improve deep learning performances. In this study, we propose a deep learning solution that leverages D-Marker features to improve performance further. Our multi-task learning framework, named DeepMarkerNet, integrates a transformer network with the utilization of facial D-Markers for accurate smile classification. Unlike past methods, our approach simultaneously predicts the class of the smile and associated facial D-Markers using two different feed-forward neural networks, thus creating a symbiotic relationship that enriches the learning process. The novelty of our approach lies in incorporating supervisory signals from the pre-calculated D-Markers (instead of as input in previous works), harmonizing the loss functions through a weighted average. In this way, our training utilizes the benefits of D-Markers, but the inference does not require computing the D-Marker. We validate our model’s effectiveness on four well-known smile datasets: UvA-NEMO, BBC, MMI facial expression, and SPOS datasets, and achieve state-of-the-art results.

Table Transformers for imputing textual attributes

Missing data in tabular dataset is a common issue as the performance of downstream tasks usually depends on the completeness of the training dataset. Previous missing data imputation methods focus on numeric and categorical columns, but we propose a novel end-to-end approach called Table Transformers for Imputing Textual Attributes (TTITA) based on the transformer to impute unstructured textual columns using other columns in the table. We conduct extensive experiments on three datasets, and our approach shows competitive performance outperforming baseline models such as recurrent neural networks and Llama2. The performance improvement is more significant when the target sequence has a longer length. Additionally, we incorporate multi-task learning to simultaneously impute for heterogeneous columns, boosting the performance for text imputation. We also qualitatively compare with ChatGPT for realistic applications.

Spatio-temporal Dual Graph Neural Networks for Travel Time Estimation

Travel time estimation is one of the core tasks for the development of intelligent transportation systems. Most previous works model the road segments or intersections separately by learning their spatio-temporal characteristics to estimate travel time. However, due to the continuous alternations of the road segments and intersections in a path, the dynamic features are supposed to be coupled and interactive. Therefore, modeling one of them limits further improvement in accuracy of estimating travel time. To address the above problems, a novel graph-based deep learning framework for travel time estimation is proposed in this article, namely, Spatio-temporal Dual Graph Neural Networks (STDGNN). Specifically, we first establish the node-wise and edge-wise graphs to, respectively, characterize the adjacency relations of intersections and that of road segments. To extract the joint spatio-temporal correlations of the intersections and road segments, we adopt the spatio-temporal dual graph learning approach that incorporates multiple spatial-temporal dual graph learning modules with multi-scale network architectures for capturing multi-level spatial-temporal information from the dual graph. Finally, we employ the multi-task learning approach to estimate the travel time of a given whole route, each road segment and intersection simultaneously. We conduct extensive experiments to evaluate our proposed model on three real-world trajectory datasets, and the experimental results show that STDGNN significantly outperforms several state-of-art baselines.

A preliminary investigation on using multi-task learning to predict change performance in code reviews

A preliminary investigation on using multi-task learning to predict change performance in code reviews

A multi-view feature representation for predicting drugs combination synergy based on ensemble and multi-task attention models

This paper proposes a novel multi-view ensemble predictor model that is designed to address the challenge of determining synergistic drug combinations by predicting both the synergy score value values and synergy class label of drug combinations with cancer cell lines. The proposed methodology involves representing drug features through four distinct views: Simplified Molecular-Input Line-Entry System (SMILES) features, molecular graph features, fingerprint features, and drug-target features. On the other hand, cell line features are captured through four views: gene expression features, copy number features, mutation features, and proteomics features. To prevent overfitting of the model, two techniques are employed. First, each view feature of a drug is paired with each corresponding cell line view and input into a multi-task attention deep learning model. This multi-task model is trained to simultaneously predict both the synergy score value and synergy class label. This process results in sixteen input view features being fed into the multi-task model, producing sixteen prediction values. Subsequently, these prediction values are utilized as inputs for an ensemble model, which outputs the final prediction value. The ‘MVME’ model is assessed using the O’Neil dataset, which includes 38 distinct drugs combined across 39 distinct cancer cell lines to output 22,737 drug combination pairs. For the synergy score value, the proposed model scores a mean square error (MSE) of 206.57, a root mean square error (RMSE) of 14.30, and a Pearson score of 0.76. For the synergy class label, the model scores 0.90 for accuracy, 0.96 for precision, 0.57 for kappa, 0.96 for the area under the ROC curve (ROC-AUC), and 0.88 for the area under the precision-recall curve (PR-AUC).

The use of multi-task learning in cybersecurity applications: a systematic literature review

The use of multi-task learning in cybersecurity applications: a systematic literature review

Leveraging multi-output modelling for CIELAB using colour difference formula towards sustainable textile dyeing

Textile dyeing requires optimizing combinations of ingredients and process parameters to achieve target colour properties. Modelling the complex relationships between these factors and the resulting colour is challenging. In this case, a physics-informed approach for multi-output regression to model CIELAB colour values from dyeing ingredient and process inputs is proposed. Leveraging attention mechanisms and multi-task learning, the model outperforms baseline methods at predicting multiple colour outputs jointly. Specifically, the Transformer model’s attention mechanism captures the complex interactions between dyeing ingredients and process parameters, while the multi-task learning framework exploits the intrinsic correlations among the L*, a*, and b* dimensions of the CIELAB colour space. In addition, the incorporation of physical knowledge through a physics-informed loss function integrates the CMC colour difference formula. This loss function, along with the attention mechanisms, enables the model to learn the nuanced relationships between the dyeing process variables and the final colour output, thereby improving the overall prediction accuracy. This reduces trial-and-error costs and resource waste, contributing to environmental sustainability by minimizing water and energy consumption and chemical emissions.