Multi-task Learning Research Articles

Multi-task deep learning for quantifying methane emissions from 2-D plume imagery with Low Signal-to-Noise Ratio

ABSTRACT Methane is the second most significant greenhouse gas after carbon dioxide. As global warming intensifies, the quantification of methane point sources is becoming increasingly crucial. However, retrieving high-quality methane signals from remote sensing data remains challenging due to various factors, including surface reflectance, atmospheric interference, sensor noise, wind speed, and sensor sensitivity. In real-world scenarios, methane remote sensing quantification often encounters unfavourable conditions that lead to low signal-to-noise ratio (SNR) signals, resulting in reduced quantification accuracy. To address these challenges, we introduce a novel multi-task learning-based approach. Specifically, we incorporate a denoising auxiliary task into the quantification network by introducing an additional denoising branch that recovers clean plume column concentration maps. The network is trained with supervision using both denoising and quantification losses, which enables it to acquire robust feature representations to noise and benefits the quantification of low SNR images. During the inference phase, the denoising branch is removed, resulting in an efficient and robust single quantification network with reduced inferring time, supporting onboard computation. We construct a low SNR database based on the AVIRIS-NG sensor and evaluate the generalization ability of our method. DQNet achieves the highest RMSE, MAPE, and R of 16.478 kg/h, 15.296%, and 95.167% respectively on the synthetic dataset. Across the entire range of SNR, DQNet exhibits a greater relative advantage as SNR decreases. However, our approach is not limited to AVIRIS-NG and can be easily extended to various multispectral and hyperspectral satellites.

Integrated Hybrid Modelling and Surrogate Model-Based Operation Optimization of Fluid Catalytic Cracking Process

Fluid Catalytic Cracking (FCC) is one of the most important conversion processes in oil refineries, widely used to convert high-boiling, high-molecular-weight hydrocarbon components from crude oil into more valuable products like gasoline and diesel. Advanced simulation and optimization technologies are critical for improving the operational efficiency and economic performance of the FCC process. First-principles-based simulators rely on parameter estimation and are computationally intensive, making them unsuitable for online optimization. In recent years, with the development of deep learning, data-driven models have made significant progress in FCC modeling. However, due to their black-box nature and difficulty with extrapolation, they are rarely used for optimization. To bridge this gap, we propose an integrated framework that combines hybrid modeling and surrogate model-based optimization. This approach combines plant and simulation data to train a multi-task learning prediction model, which then serves as a surrogate for operational optimization. Validated on a large-scale FCC unit in southern China, the model predicts product yields with an error margin of under 4.84% for all products. Following optimization, yields of LNG, gasoline, and diesel rose by an average of 0.10 wt%, 1.58 wt%, and 1.05 wt%, respectively, resulting in a 3.67% increase in product revenues. This highlights the substantial potential of this framework for industrial applications.

Signal-guided multitask learning for myocardial infarction classification using images of electrocardiogram.

The diagnosis of myocardial infarction (MI) needs to be swift and accurate, but definitively diagnosing it based on the first test encountered in clinical practice, the electrocardiogram (ECG), is not an easy task. The purpose of the study is to develop a deep learning (DL) algorithm using multitask learning method to differentiate patients experiencing MI from those without coronary artery disease using image-based ECG data. A DL model was developed based on 11,227 ECG images. We developed a new ECG interpretation algorithm through signal-guided multitask learning, building on a previously published single-task algorithm. The utility of this model was evaluated by testing 51 physicians in interpreting ECGs with and without the assistance of the DL model. The proposed model demonstrated superior performance, achieving 90.56% accuracy, 83.82% sensitivity, 93.02% specificity, 81.44% precision, and an F1 score of 82.61% in discriminating MI ECG. Overall, the median accuracy of ECG interpretation improved from 62% to 68% with the DL algorithm. Trainees and specialists in internal medicine experienced significant accuracy increases (60% to 66% for trainees, 72% to 80% for specialists). In the MI group, NSTEMI accuracy was notably lower than STEMI (33% vs. 80%, p < 0.001), but the DL algorithm improved interpretative capabilities in both NSTEMI and STEMI. Signal-guided multitask DL algorithm demonstrated superior performance compared with previous single-task algorithm. The DL algorithm supports the physicians' decision discriminating MI ECGs from non-MI ECGs. The improvement was consistent in subgroups of STEMI and NSTEMI.

Improved patient identification by incorporating symptom severity in deep learning using neuroanatomic images in first episode schizophrenia.

Brain alterations associated with illness severity in schizophrenia remain poorly understood. Establishing linkages between imaging biomarkers and symptom expression may enhance mechanistic understanding of acute psychotic illness. Constructing models using MRI and clinical features together to maximize model validity may be particularly useful for these purposes. A multi-task deep learning model for standard case/control recognition incorporated with psychosis symptom severity regression was constructed with anatomic MRI collected from 286 patients with drug-naïve first-episode schizophrenia and 330 healthy controls from two datasets, and validated with an independent dataset including 40 first-episode schizophrenia. To evaluate the contribution of regression to the case/control recognition, a single-task classification model was constructed. Performance of unprocessed anatomical images and of predefined imaging features obtained using voxel-based morphometry (VBM) and surface-based morphometry (SBM), were examined and compared. Brain regions contributing to the symptom severity regression and illness identification were identified. Models developed with unprocessed images achieved greater group separation than either VBM or SBM measurements, differentiating schizophrenia patients from healthy controls with a balanced accuracy of 83.0% with sensitivity = 76.1% and specificity = 89.0%. The multi-task model also showed superior performance to single-task classification model without considering clinical symptoms. These findings showed high replication in the site-split validation and external validation analyses. Measurements in parietal, occipital and medial frontal cortex and bilateral cerebellum had the greatest contribution to the multi-task model. Incorporating illness severity regression in pattern recognition algorithms, our study developed an MRI-based model that was of high diagnostic value in acutely ill schizophrenia patients, highlighting clinical relevance of the model.

Multi-task Bayesian model combining FDG-PET/CT imaging and clinical data for interpretable high-grade prostate cancer prognosis.

We propose a fully automatic multi-task Bayesian model, named Bayesian Sequential Network(BSN), for predicting high-grade(Gleason 8) prostate cancer(PCa) prognosis using pre-prostatectomy FDG-PET/CT images and clinical data. BSN performs one classification task and five survival tasks: predicting lymph node invasion (LNI), biochemical recurrence-free survival (BCR-FS), metastasis-free survival, definitive androgen deprivation therapy-free survival, castration-resistant PCa-free survival, and PCa-specific survival(PCSS). Experiments are conducted using a dataset of 295patients. BSN outperforms widely used nomograms on all tasks except PCSS, leveraging multi-task learning and imaging data. BSN also provides automated prostate segmentation, uncertainty quantification, personalized feature-based explanations, and introduces dynamic predictions, a novel approach that relies on short-term outcomes to refine long-term prognosis. Overall, BSN shows great promise in its ability to exploit imaging and clinicopathological data to predict poor outcome patients that need treatment intensification with loco-regional or systemic adjuvant therapy for high-risk PCa.

A multi-task learning based line parameter identification method for medium-voltage distribution network

Accurate line parameters are critical for and dispatch in distribution systems. External operating condition variations affect line parameters, reducing the accuracy of state estimation and power flow calculations. While many methods have been proposed and obtained results rather acceptable, there is room for improvement as they don’t fully consider line connections in known topologies. Furthermore, inaccuracies in measurement devices and data acquisition systems can introduce noise and outliers, impacting the reliability of parameter identification. To address these challenges, we propose a line parameter identification method based on Graph Attention Networks and Multi-gate Mixture-of-Experts. The topological structure of the power grid and the capabilities of modern data acquisition equipment are utilized to capture. We also introduce a multi-task learning framework to enable joint training of parameter identification across different branches, thereby enhancing computational efficiency and accuracy. Experiments show that the GAT-MMoE model outperforms traditional methods, with notable improvements in both accuracy and robustness.

Research on Medical Text Parsing Method Based on BiGRU-BiLSTM Multi-Task Learning

With the development of technology, the popularity of online medical treatment is becoming more and more widespread. However, the accuracy and credibility of online medical treatment are affected by model design and semantic understanding. In particular, there are still some problems in the accurate understanding of complex structured texts, which affects the accuracy of judging users’ intentions and needs. Therefore, this paper proposes a new method for medical text parsing, which realizes core tasks such as named entity recognition, intention recognition, and slot filling through a multi-task learning framework; uses BERT to obtain contextual semantic information; and combines BiGRU and BiLSTM neural networks, and uses CRF to realize sequence annotation and DPCNN to realize classification prediction. Thus, the task of entity recognition and intention recognition can be accomplished. On this basis, this paper builds a multi-task learning model based on BiGRU-BiLSTM, and uses CBLUE and CMID databases to verify the method. The verification results show that the accuracy of named entity recognition and intention recognition reaches 86% and 89%, respectively, which improves the performance of various tasks. The ability of the model to process complex text is enhanced. This method can improve the text generalization ability and improve the accuracy of online medical intelligent dialogue when it is used to analyze medical texts.

Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy

Colonoscopy screening is the gold standard procedure for assessing abnormalities in the colon and rectum, such as ulcers and cancerous polyps. Measuring the abnormal mucosal area and its 3D reconstruction can help quantify the surveyed area and objectively evaluate disease burden. However, due to the complex topology of these organs and variable physical conditions, for example, lighting, large homogeneous texture, and image modality estimating distance from the camera (aka depth) is highly challenging. Moreover, most colonoscopic video acquisition is monocular, making the depth estimation a non-trivial problem. While methods in computer vision for depth estimation have been proposed and advanced on natural scene datasets, the efficacy of these techniques has not been widely quantified on colonoscopy datasets. As the colonic mucosa has several low-texture regions that are not well pronounced, learning representations from an auxiliary task can improve salient feature extraction, allowing estimation of accurate camera depths. In this work, we propose to develop a novel multi-task learning (MTL) approach with a shared encoder and two decoders, namely a surface normal decoder and a depth estimator decoder. Our depth estimator incorporates attention mechanisms to enhance global context awareness. We leverage the surface normal prediction to improve geometric feature extraction. Also, we apply a cross-task consistency loss among the two geometrically related tasks, surface normal and camera depth. We demonstrate an improvement of 15.75% on relative error and 10.7% improvement on δ1.25 accuracy over the most accurate baseline state-of-the-art Big-to-Small (BTS) approach. All experiments are conducted on a recently released C3VD dataset, and thus, we provide a first benchmark of state-of-the-art methods on this dataset.

HirMTL: Hierarchical Multi-Task Learning for dense scene understanding

In the realm of artificial intelligence, simultaneous multi-task learning is crucial, particularly for dense scene understanding. To address this, we introduce HirMTL, a novel hierarchical multi-task learning framework designed to enhance dense scene analysis. HirMTL is adept at facilitating interaction at the scale level, ensuring task-adaptive multi-scale feature fusion, and fostering task-level feature interchange. It leverages the inherent correlations between tasks to create a synergistic learning environment. Initially, HirMTL enables concurrent sharing and fine-tuning of features at the single-scale level. This is further extended by the Task-Adaptive Fusion module (TAF), which intelligently blends features across scales, specifically attuned to each task’s unique requirements. Complementing this, the Asymmetric Information Comparison Module (AICM) skillfully differentiates and processes both shared and unique features, significantly refining task-specific performance with enhanced accuracy. Our extensive experiments on various dense prediction tasks validate HirMTL’s exceptional capabilities, showcasing its superiority over existing multi-task learning models and underscoring the benefits of its hierarchical approach.

Short-Term Load Forecasting of an Integrated Energy System Based on STL-CPLE with Multitask Learning

Short-Term Load Forecasting of an Integrated Energy System Based on STL-CPLE with Multitask Learning

Multi-Focus Image Fusion Algorithm Based on Multi-Task Learning and PS-ViT

Multi-Focus Image Fusion Algorithm Based on Multi-Task Learning and PS-ViT

Dynamic Energy Management and Control of Networked Microgrids based on Load to Grid Services and Incentive-Based Demand Response Programs: A Multi-Agent Deep Reinforcement Learning Approach

This study has presented the energy management paradigm in a networked microgrid structure based on L2G services and considering incentive-based load response (IBDR) programs and energy market requirements to reduce operating costs, control and restore voltage and frequency index, providing the benefits of subscribers and distribution system operators. In this study, multi-objective functions such as optimal operation based on IBDR structure and energy market requirements, risk assessment, and L2G service approach are configured in the framework of central and local controllers. Optimal operation and risk assessment are analyzed by a multi-task learning algorithm based on multi-objective function and L2G service policies are evaluated based on multi-agent deep reinforcement learning. Control policies are sent by the communication system to the components affecting the optimal power distribution as well as the voltage and frequency controllers. L2G services have been evaluated in different scenarios such as plug-and-play operating conditions, load fluctuations, and operating in island mode. The results of optimal operation based on L2G services show that the IBDR program implementation reduces the total operation cost by 21%. Also, the total operating cost of the proposed framework is 13.97% less than the RL method and 27.8% less than the ANN method.

Multi-point temperature or humidity prediction for office building indoor environment based on CGC-BiLSTM deep neural network

The aim of this study is to predict the temperature or humidity changes at multiple relevant points in a building using a deep neural network architecture with multi-task learning to provide more reference information for the design and optimal operation of heating and ventilation systems. For this purpose, traditional multi-task prediction algorithm architecture is combined with Customized Gate Control and other neural networks to build deep neural network architectures for indoor environments with multi-point temperature or humidity prediction tasks. To test the prediction effectiveness of the architecture, a task of predicting temperature or humidity 24h in advance was designed on a real office building indoor environment dataset, and the prediction results were compared with other single-task and multi-task prediction models. Two experimental conditions were designed for this study, one using the complete training set and the other reducing the training set at a certain point. Through the final prediction results, it is found that the multi-task prediction architecture used in this paper shows better or nearly optimal results compared to other prediction models under both working conditions. This study provides some reference value for the application of multi-task prediction algorithms to the task of predicting indoor environments in buildings.

Accelerating Communication-Efficient Federated Multi-Task Learning With Personalization and Fairness

Accelerating Communication-Efficient Federated Multi-Task Learning With Personalization and Fairness

Hierarchical multi-task learning with self-supervised auxiliary task

Hierarchical multi-task learning with self-supervised auxiliary task

Fine-grained recognition algorithm of crop pests based on cross-layer bilinear aggregation and multi-task learning

Fine-grained recognition of crop pests is a crucial concern in the field of agriculture, as the accuracy of recognition and generalization ability directly affect the yield and quality of crops. Aiming at the characteristics of crop pests with a wide variety of species, small inter-class and large intra-class differences in external morphology, as well as the problems of uneven sample distribution and noisy labels in fine-grained image datasets under complex environments, we propose a fine-grained recognition model of crop pests (MT-MACLBPHSNet) based on cross-layer bilinear aggregation and multi-task learning, which consists of three key modules: the backbone network module, the cross-layer bilinear aggregation module, and the multi-task learning module. A new union loss function is designed in the primary task of the multi-task learning module, which is used to alleviate the two problems existing in the model training fine-grained image datasets. The experimental results show that the model effectively balances the model complexity and recognition accuracy in a comparative analysis with several existing excellent network models on the IP102-CP13 dataset, with the recognition accuracy reaching 75.37%, which is 7.06% higher than the Baseline model, and the F1-score reaching 67.06%. Additionally, the generalization of the model is also verified on the IP102-VP16 dataset, and the model outperforms most of the models in terms of recognition accuracy and generalization ability, which can provide an effective reference for fine-grained recognition of crop pests.

A multi-task spatio-temporal fusion network for offshore wind power ramp events forecasting

With the accelerated development of offshore wind farms, the impact of Wind Power Ramp Events (WPRE) on power systems has become more pronounced. Accurate prediction of WPRE is essential for mitigating their adverse effects on the grid. However, current studies on offshore WPRE are limited, especially regarding their spatio-temporal correlations and variability across large wind farm clusters. To address this, a novel forecasting approach using a multi-task spatio-temporal fusion network is presented. This method improves WPRE prediction by integrating spatial and temporal data and utilizing multitask learning to characterize ramp features. Firstly, an index system for WPRE characterization is developed, including metrics such as power change rate and duration. Based on this, an X-means classification method for WPRE is established. Secondly, a spatio-temporal encoder combining graph convolutional networks and temporal attention mechanisms has been proposed to model the spatiotemporal dependencies of meteorological changes in offshore wind farms. Finally, a multitask learning framework has been introduced to predict WPRE characteristic indices and occurrence probabilities. This approach enhances feature extraction efficiency through the joint feedback of multiple related tasks, thereby improving prediction accuracy. A case study using data from five offshore wind farms in Eastern China validates the efficacy of the proposed method.

Leveraging multiplexed metasurfaces for multi-task learning with all-optical diffractive processors

Abstract Diffractive Neural Networks (DNNs) leverage the power of light to enhance computational performance in machine learning, offering a pathway to high-speed, low-energy, and large-scale neural information processing. However, most existing DNN architectures are optimized for single tasks and thus lack the flexibility required for the simultaneous execution of multiple tasks within a unified artificial intelligence platform. In this work, we utilize the polarization and wavelength degrees of freedom of light to achieve optical multi-task identification using the MNIST, FMNIST, and KMNIST datasets. Employing bilayer cascaded metasurfaces, we construct dual-channel DNNs capable of simultaneously classifying two tasks, using polarization and wavelength multiplexing schemes through a meta-atom library. Numerical evaluations demonstrate performance accuracies comparable to those of individually trained single-channel, single-task DNNs. Extending this approach to three-task parallel recognition reveals an expected performance decline yet maintains satisfactory classification accuracies of greater than 80 % for all tasks. We further introduce a novel end-to-end joint optimization framework to redesign the three-task classifier, demonstrating substantial improvements over the meta-atom library design and offering the potential for future multi-channel DNN designs. Our study could pave the way for the development of ultrathin, high-speed, and high-throughput optical neural computing systems.

Real-Time Forest Fire Detection with Lightweight CNN Using Hierarchical Multi-Task Knowledge Distillation

Forest fires pose a significant threat to ecosystems, property, and human life, making their early and accurate detection crucial for effective intervention. This study presents a novel, lightweight approach to real-time forest fire detection that is optimized for resource-constrained devices like drones. The method integrates multi-task knowledge distillation, transferring knowledge from a high-performance DenseNet201 teacher model that was trained on a hierarchically structured wildfire dataset. The dataset comprised primary classes (fire vs. non-fire) and detailed subclasses that account for confounding elements such as smoke, fog, and reflections. The novelty of this approach lies in leveraging knowledge distillation to transfer the deeper insights learned by the DenseNet201 teacher model—specifically, the auxiliary task of recognizing the confounding elements responsible for false positives—into a lightweight student model, enabling it to achieve a similar robustness without the need for complex architectures. Using this distilled knowledge, we trained a MobileNetV3-based student model, which was designed to operate efficiently in real-time while maintaining a low computational overhead. To address the challenge of false positives caused by visually similar non-fire elements, we introduced the Confounding Element Specificity (CES) metric. This novel metric, made possible by the hierarchical structure of the wildfire dataset, is unique in its focus on evaluating how well the model distinguishes actual fires from the confounding elements that typically result in false positives within the negative class. The proposed approach outperformed the baseline methods—including single-task learning and direct multi-task learning—achieving a primary accuracy of 93.36%, an F1-score of 91.57%, and a higher MES score, demonstrating its enhanced robustness and reliability in diverse environmental conditions. This work bridges the gap between advanced deep learning techniques and practical, scalable solutions for environmental monitoring. Future research will focus on integrating multi-modal data and developing adaptive distillation techniques to further enhance the model’s performance in real-time applications.

Interpretable prediction of sinter particle size distribution based on multi-task learning and bidirectional gated recurrent unit

The particle size distribution of sintered ore is the key indicator reflecting the quality of sintered ore products. Aiming at the problems of low accuracy and lack of interpretability of the existing sinter ore particle size distribution prediction models, this paper proposes a multi-task learning prediction model that uses the bidirectional gated recurrent unit as an information sharing layer to predict the particle size distribution of sinter ore. Firstly, features with high correlation with different particle size ranges (&lt;5 mm; &lt;10 mm) are selected as model inputs by the feature selection method. Secondly, a bidirectional gated recurrent unit is used as the information-sharing layer to construct a multi-task learning model, and the information sharing between two tasks is used to obtain their coupling information to consider the correlation between different particle sizes and improve the prediction accuracy of the model. Additionally, the Shapely interpretation method is introduced to analyse the interpretability of the important features of the model to enhance the predictive model interpretability. Finally, the model proposed in this article is compared with the rest of the combined models, and the results show that the proposed model has high prediction accuracy. This method provides a new direction for the high-quality production of sintered ore.