Multi-task Training Research Articles

Learning to explain is a good biomedical few-shot learner.

Significant progress has been achieved in biomedical text mining using deep learning methods, which rely heavily on large amounts of high-quality data annotated by human experts. However, the reality is that obtaining high-quality annotated data is extremely challenging due to data scarcity (e.g. rare or new diseases), data privacy and security concerns, and the high cost of data annotation. Additionally, nearly all researches focus on predicting labels without providing corresponding explanations. Therefore, in this paper, we investigate a more realistic scenario, biomedical few-shot learning, and explore the impact of interpretability on biomedical few-shot learning. We present LetEx-Learning to explain-a novel multi-task generative approach that leverages reasoning explanations from large language models (LLMs) to enhance the inductive reasoning ability of few-shot learning. Our approach includes (1) collecting high-quality explanations by devising a suite of complete workflow based on LLMs through CoT prompting and self-training strategies, (2) converting various biomedical NLP tasks into a text-to-text generation task in a unified manner, where collected explanations serve as additional supervision between text-label pairs by multi-task training. Experiments are conducted on three few-shot settings across six biomedical benchmark datasets. The results show that learning to explain improves the performances of diverse biomedical NLP tasks in low-resource scenario, outperforming strong baseline models significantly by up to 6.41%. Notably, the proposed method makes the 220M LetEx perform superior reasoning explanation ability against LLMs. Our source code and data are available at https://github.com/cpmss521/LetEx.

Highly resilient aerogel/sponge nested structures assisted multimodal tactile sensory system for robotic embodied perception

Large language models (LLMs) exhibit outstanding reasoning abilities when robots perform tasks, but are limited to the passively receiving information and lack multi-feature perception of realistic objects. Humans obtain complementary and coupled information from interactions with objects through the skin. Here, a multimodal tactile sensory system is reported to assist the LLM-equipped robots in actively perceiving and understanding multiple features of objects including material, compliance and thermal insulation. A multi-parametric tactile sensing array, where each sensor in the array comprises of a triboelectric unit, a decoupling pressure-temperature dual-mode sensing unit and a heating unit, is constructed to acquire the multiparametric tactile signals. The pressure-temperature sensing unit is fabricated from the developed aerogel/sponge nested composite structures, which exhibit stable compression resilience under the compression strain of over 90 %, impressive fatigue resistance with stable responses after 10,000 compression cycles, and linear thermoelectric responses over wide temperature gradients. The multi-parametric tactile signals are analyzed using Mamba time series network model, resulting in the high-precision clustering of multiple features (up to 98.3 %). Supported by a multimodal interactive perception framework, the LLM realizes multi-feature understanding of objects. With large-scale sample datasets and multi-task training, the LLM-equipped robots powered by the multimodal tactile sensory system are expected to realize embodied intelligence.

Simple Debiased Contrastive Learning for Sequential Recommendation

Recently, contrastive learning has been shown to be effective in improving sequential recommendation, alleviating noise and long-tail issues caused by real-world noise and data sparsity, and enabling models to learn high-quality user representations. The data augmentation approach of contrastive learning mainly includes randomly changing the sequence data and adding randomly perturbations, and by pulling the distance between the original sample and the positive sample and pushing the distance between the original sample and the negative sample to enhance alignment and to optimize the uniformity of representation space. However, these approaches may use false positive/negative samples, learn biased user representations, and compromise the uniformity of the representation space, leading to second-best recommendation results. To solve these issues, we propose a generic Simple Debiased Contrastive Learning for Sequential Recommendation framework (SimDCL), to alleviate these biased samples, optimize the uniformity of the representation space, and enhance anti-noise ability. Specifically, we augment the samples based on noise, and optimize the uniformity of the representation space through gradient algorithms, and design an object filtering approach to punish false positive and false negative samples in user interaction sequences. Finally, we combine contrastive learning and sequential recommendation for multi-task collaborative training to improve the model’s anti-noise capability and the quality of recommendation. The empirical studies on four benchmark datasets prove the superiority of our approach, and the code and datasets are available at URL.

Language-aware multiple datasets detection pretraining for DETRs

Pretraining on large-scale datasets can boost the performance of object detectors while the annotated datasets for object detection are hard to scale up due to the high labor cost. What we possess are numerous isolated filed-specific datasets, thus, it is appealing to jointly pretrain models across aggregation of datasets to enhance data volume and diversity. In this paper, we propose a strong framework for utilizing Multiple datasets to pretrain DETR-like detectors, termed METR, without the need for manual label spaces integration. It converts the typical multi-classification in object detection into binary classification by introducing a pre-trained language model. Specifically, we design a category extraction module for extracting potential categories involved in an image and assign these categories into different queries by language embeddings. Each query is only responsible for predicting a class-specific object. Besides, to adapt our novel detection paradigm, we propose a Class-wise Bipartite Matching strategy that limits the ground truths to match queries assigned to the same category. Extensive experiments demonstrate that METR achieves extraordinary results on either multi-task joint training or the pretrain & finetune paradigm. Notably, our pre-trained models have high flexible transferability and increase the performance upon various DETR-like detectors on COCO val2017 benchmark. Our code is publicly available at: https://github.com/isbrycee/METR.

Automatic depression prediction via cross-modal attention-based multi-modal fusion in social networks

Depression has become a worldwide public health concern in recent years. The online social network can be served as a tool to predict latent depressed users. However, existing studies primarily use machine learning or deep learning methods on multi-modal information with simple concatenation, resulting in unsatisfactory prediction. Additionally, the lack of interpretability in extracted features is a challenge. Furthermore, the majority of these studies primarily concentrate on English social networks. To address these issues, we propose a novel unified multi-tasking training model named CrossAMF (Cross-modal Attention-based Multi-modal Fusion). CrossAMF extracts crucial multi-modal features with explainability and effectively fuses representations of different modalities. Our approach employs a sentiment filtering-based XLNet-CNN-BiGRU model and Reduce-VGGNet to extract textual and image features, respectively. We also introduce innovative user behavior metrics previously unexplored. We collect a dataset of Chinese depressed users from Sina microblog with their texts and images, including 5692 depressed users and 9121 normal users. Experimental results demonstrate the significance of each modality to our task, with the combination yielding the best performance. CrossAMF outperforms other state-of-the-art methods with 0.9097 in accuracy and 0.8915 in F1, showing the potential of application in screening out early depression automatically.

Multi-task learning for hand heat trace time estimation and identity recognition

The thermal imager can capture the invisible thermal trace, and the potential information contained in the thermal trace can be extracted through the identification and analysis of the infrared thermal trace, such as the identity information and time information of the trace. This technology has great application value in criminal investigation, military and other fields. However, with the passage of time, the heat traces will gradually fade, and the intra-class distances will gradually increase. For hand heat trace, the trace may exhibit significant variations at different time points or in different states. This will pose challenges for extracting potential information and present significant challenges for model recognition. Therefore, we propose a multi task framework using deep convolutional neural networks (MTLHand) to jointly handle the two tasks of heat trace identification and heat trace departure time estimation. It can learn invariant representations of thermal trace identities, better capturing the dynamic changes in palmprint data, to enhance the accuracy of identity recognition. Specifically, soft threshold is inserted as nonlinear transformation layers into an improved ResNet to extract deep mixed hand trace features. The mixed features are decomposed into identity related features and time related features through spatial attention and channel attention. Use the multi task training method to carry out the corresponding learning tasks. In addition, we collected and constructed an infrared hand heat trace dataset containing labels such as departure time, gender, hand posture, identity category, etc., to promote the development of heat trace research. Experiments on this dataset show that our method is superior to other deep learning methods for the specific task of infrared heat trace recognition, and the identification accuracy can reach 83.48 %. In the trace leaving time estimation task, the recognition error rate is 18 % if the residual value between the estimated time and the real time is less than 60 s, and 3.55 % if the residual value is less than 120 s.

EMPDTA: An End-to-End Multimodal Representation Learning Framework with Pocket Online Detection for Drug-Target Affinity Prediction.

Accurately predicting drug-target interactions is a critical yet challenging task in drug discovery. Traditionally, pocket detection and drug-target affinity prediction have been treated as separate aspects of drug-target interaction, with few methods combining these tasks within a unified deep learning system to accelerate drug development. In this study, we propose EMPDTA, an end-to-end framework that integrates protein pocket prediction and drug-target affinity prediction to provide a comprehensive understanding of drug-target interactions. The EMPDTA framework consists of three main modules: pocket online detection, multimodal representation learning for affinity prediction, and multi-task joint training. The performance and potential of the proposed framework have been validated across diverse benchmark datasets, achieving robust results in both tasks. Furthermore, the visualization results of the predicted pockets demonstrate accurate pocket detection, confirming the effectiveness of our framework.

Stimulating prefrontal cortex facilitates training transfer by increasing representational overlap.

A recent hypothesis characterizes difficulties in multitasking as being the price humans pay for our ability to generalize learning across tasks. The mitigation of these costs through training has been associated with reduced overlap of constituent task representations within frontal, parietal, and subcortical regions. Transcranial direct current stimulation, which can modulate functional brain activity, has shown promise in generalizing performance gains when combined with multitasking training. However, the relationship between combined transcranial direct current stimulation and training protocols with task-associated representational overlap in the brain remains unexplored. Here, we paired prefrontal cortex transcranial direct current stimulation with multitasking training in 178 individuals and collected functional magnetic resonance imaging data pre- and post-training. We found that 1mA transcranial direct current stimulation applied to the prefrontal cortex paired with multitasking training enhanced training transfer to spatial attention, as assessed via a visual search task. Using machine learning to assess the overlap of neural activity related to the training task in task-relevant brain regions, we found that visual search gains were predicted by changes in classification accuracy in frontal, parietal, and cerebellar regions for participants that received left prefrontal cortex stimulation. These findings demonstrate that prefrontal cortex transcranial direct current stimulation may interact with training-related changes to task representations, facilitating the generalization of learning.

Effects of one-week bilateral cerebellar iTBS on resting-state functional brain network and multi-task attentional performance in healthy individuals: A randomized, sham-controlled trial

Background:Cerebellar intermittent theta burst stimulation (iTBS) modulates the excitability of the cerebral cortex and may enhance attentional performance. To date, few studies have conducted iTBS on healthy subjects for one week and used electroencephalography (EEG) to investigate the effect of multiple stimulation sessions on resting-state functional brain networks and the daily stimulation effect on attentional performance. Methods:16 healthy subjects participated in a one-week experiment, receiving bilateral cerebellar iTBS or sham stimulation and engaging in multi-task attentional training. The primary measures were the one-week attentional performance and pre- and post-experiment resting-state EEG activities. Amplitude Envelope Correlation (AEC) was used to construct the functional connectivity in the eye-open (EO) and eye-closed (EC) phases. Results:At least three sessions of iTBS were required to enhance multi-task performance significantly, whereas only one or two sessions failed to elicit the improvement. Compared with the control group, iTBS induced significant changes in PSD, AEC functional connectivity, and AEC network properties during the EO phase, while it had little effect during the EC phase. During the EO phase, the network property changes of the iTBS subject were correlated with improved attentional performance. Conclusion:The multi-task performance requires multiple stimulations to enhance. iTBS affects the resting-state alpha band brain activities during the EO rather than the EC phase. The AEC network properties may serve as a biomarker to assess the attentional potential of healthy subjects.

Innovative methods for microplastic characterization and detection: Deep learning supported by photoacoustic imaging and automated pre-processing data

Plastic products' widespread applications and their non-biodegradable nature have resulted in the continuous accumulation of microplastic waste, emerging as a significant component of ecological environmental issues. In the field of microplastic detection, the intricate morphology poses challenges in achieving rapid visual characterization of microplastics. In this study, photoacoustic imaging technology is initially employed to capture high-resolution images of diverse microplastic samples. To address the limited dataset issue, an automated data processing pipeline is designed to obtain sample masks while effectively expanding the dataset size. Additionally, we propose Vqdp2, a generative deep learning model with multiple proxy tasks, for predicting six forms of microplastics data. By simultaneously constraining model parameters through two training modes, outstanding morphological category representations are achieved. The results demonstrate Vqdp2's excellent performance in classification accuracy and feature extraction by leveraging the advantages of multi-task training. This research is expected to be attractive for the detection classification and visual characterization of microplastics.

Creating a model of a training task set for a multitask practical training in the courses “Informatics” and “Information technologies in professional activity”

According to the federal state educational standards, the curriculum for socio-economic and humanitarian students of secondary vocational education (SVE) and higher education (HE) includes disciplines that provide mastery of competencies in the field of information and communication technologies. Along with the course of informatics, they include disciplines of the block “Information technologies in professional activity”: “Information Technologies in Court Activity” (SVE), “Information Technologies in Professional Activity” (for several specialties of SVE), “Information and Communication Technologies in Judicial Activity” (HE), “Information Technologies in Management” (HE), “Information Technologies in Translation Studies” (HE) and several other disciplines.Based on the analysis of scientific literature and an array of manuals — workshops provided to students and teachers by leading Russian electronic library systems (ELS), as well as own pedagogical work in teaching informatics and the block of academic courses “Information Technologies in Professional Activity”, the author proposes a model of training tasks set to be solved during the study of these courses. The relevance of the article subject is conditioned by the necessity to effectively use the time allocated by the curricula of socio-economic and humanitarian training areas of vocational education and higher education for the study of informatics and information technologies in professional activities. The novelty of the results presented by the author is confirmed by the absence of articles and reports on the topic in the Russian Science Citation Index (RSCI) database

Electrical Activity Changes and Neurovascular Unit Markers in the Brains of Patients after Cardiac Surgery: Effects of Multi-Task Cognitive Training.

There is growing interest in finding methods to enhance cognitive function and comprehend the neurophysiological mechanisms that underlie these improvements. It is assumed that non-pharmacological interventions have better results in cognitive recovery. The aim of this study was to assess the effect of multi-task cognitive training (MTT) on electroencephalographic (EEG) changes and markers of the neurovascular unit in patients undergoing coronary artery bypass grafting (CABG). This prospective cohort study involved 62 CABG patients aged 45-75 years, 30 of whom underwent a 5-7-day MTT course. The groups of patients were comparable with respect to baseline clinical and anamnestic characteristics. An EEG study was performed before surgery and 11-12 days after CABG. Markers of the neurovascular unit (S100β, NSE, and BDNF) were examined at three time points: before surgery, within the first 24 h after surgery, and 11-12 days after CABG. Patients without training demonstrated higher relative theta power changes compared to the MTT patients. The course of MTT was associated with low plasma S100β concentration but high BDNF levels at the end of the training course. The theta activity changes and the markers of the neurovascular unit (S100β, BDNF) indicated that the severity of brain damage in cardiac surgery patients after a short course of MTT was slightly reduced. Electrical brain activity indicators and vascular markers can be informative for monitoring the process of cognitive rehabilitation in cardiac surgery patients.

Neurochemical Predictors of Generalized Learning Induced by Brain Stimulation and Training.

Methods of cognitive enhancement for humans are most impactful when they generalize across tasks. However, the extent to which such "transfer" is possible via interventions is widely debated. In addition, the contribution of excitatory and inhibitory processes to such transfer is unknown. Here, in a large-scale neuroimaging individual differences study with humans (both sexes), we paired multitasking training and noninvasive brain stimulation (transcranial direct current stimulation, tDCS) over multiple days and assessed performance across a range of paradigms. In addition, we varied tDCS dosage (1.0 and 2.0 mA), electrode montage (left or right prefrontal regions), and training task (multitasking vs a control task) and assessed GABA and glutamate concentrations via ultrahigh field 7T magnetic resonance spectroscopy. Generalized benefits were observed in spatial attention, indexed by visual search performance, when multitasking training was combined with 1.0 mA stimulation targeting either the left or right prefrontal cortex (PFC). This transfer effect persisted for ∼30 d post intervention. Critically, the transferred benefits associated with right prefrontal tDCS were predicted by pretraining concentrations of glutamate in the PFC. Thus, the effects of this combined stimulation and training protocol appear to be linked predominantly to excitatory brain processes.

CAR-Transformer: Cross-Attention Reinforcement Transformer for Cross-Lingual Summarization

Cross-Lingual Summarization (CLS) involves generating a summary for a given document in another language. Most of the existing approaches adopt multi-task training and knowledge distillation, which increases the training cost and improves the performance of CLS tasks intuitively but unexplainably. In this work, we propose Cross-Attention Reinforcement (CAR) module and incorporate the module into the transformer backbone to formulate the CAR-Transformer. The CAR module formulates a pseudo summarization policy parameterized by the cross-attention weights reinforced by the ground-truth monolingual summary without introducing extra model parameters. Our approach demonstrates more consistent improvement across CLS tasks compared to traditional multi-task training methods and outperforms the fine-tuned vanilla mBART by 3.67 and the best-performing multi-task training approach by 1.48 in ROUGE-L F1 score on the WikiLingua Korean-to-English CLS task.

MorphVAE: Advancing Morphological Design of Voxel-Based Soft Robots with Variational Autoencoders

Soft robot design is an intricate field with unique challenges due to its complex and vast search space. In the past literature, evolutionary computation algorithms, including novel probabilistic generative models (PGMs), have shown potential in this realm. However, these methods are sample inefficient and predominantly focus on rigid robots in locomotion tasks, which limit their performance and application in robot design automation. In this work, we propose MorphVAE, an innovative PGM that incorporates a multi-task training scheme and a meticulously crafted sampling technique termed ``continuous natural selection'', aimed at bolstering sample efficiency. This method empowers us to gain insights from assessed samples across diverse tasks and temporal evolutionary stages, while simultaneously maintaining a delicate balance between optimization efficiency and biodiversity. Through extensive experiments in various locomotion and manipulation tasks, we substantiate the efficiency of MorphVAE in generating high-performing and diverse designs, surpassing the performance of competitive baselines.

Learning Time Slot Preferences via Mobility Tree for Next POI Recommendation

Next Point-of-Interests (POIs) recommendation task aims to provide a dynamic ranking of POIs based on users' current check-in trajectories. The recommendation performance of this task is contingent upon a comprehensive understanding of users' personalized behavioral patterns through Location-based Social Networks (LBSNs) data. While prior studies have adeptly captured sequential patterns and transitional relationships within users' check-in trajectories, a noticeable gap persists in devising a mechanism for discerning specialized behavioral patterns during distinct time slots, such as noon, afternoon, or evening. In this paper, we introduce an innovative data structure termed the ``Mobility Tree'', tailored for hierarchically describing users' check-in records. The Mobility Tree encompasses multi-granularity time slot nodes to learn user preferences across varying temporal periods. Meanwhile, we propose the Mobility Tree Network (MTNet), a multitask framework for personalized preference learning based on Mobility Trees. We develop a four-step node interaction operation to propagate feature information from the leaf nodes to the root node. Additionally, we adopt a multitask training strategy to push the model towards learning a robust representation. The comprehensive experimental results demonstrate the superiority of MTNet over eleven state-of-the-art next POI recommendation models across three real-world LBSN datasets, substantiating the efficacy of time slot preference learning facilitated by Mobility Tree.

Model and Empirical Study on Multi-tasking Learning for Human Fall Detection

Many fall detection systems are being used to provide real-time responses to fall occurrences. Automated fall detection is challenging because it requires very high accuracy to be clinically acceptable. Recent research has tried to improve sensitivity while reducing the high rate of false positives. Nevertheless, there are still limitations in terms of having efficient learning approaches and proper datasets to train. To reduce false alarms, one approach is to add more nonfall data as negative samples to train the deep learning model. However, this approach increases class imbalance in the training set. To tackle this problem, we propose a multi-task deep learning approach that divides datasets into multiple training sets for multiple tasks. We prove this approach gives better results than a single-task model trained on all datasets. Many experiments are conducted to find the best combination of tasks for multi-task model training for fall detection.

Regional choroidal thickness estimation from color fundus images based on convolutional neural networks

PurposeThis study aims to estimate the regional choroidal thickness from color fundus images from convolutional neural networks in different network structures and task learning models. Method1276 color fundus photos and their corresponding choroidal thickness values from healthy subjects were obtained from the Topcon DRI Triton optical coherence tomography machine. Initially, ten commonly used convolutional neural networks were deployed to identify the most accurate model, which was subsequently selected for further training. This selected model was then employed in combination with single-, multiple-, and auxiliary-task training models to predict the average and sub-region choroidal thickness in both ETDRS (Early Treatment Diabetic Retinopathy Study) grids and 100-grid subregions. The values of mean absolute error and coefficient of determination (R2) were involved to evaluate the models' performance. ResultsEfficientnet-b0 network outperformed other networks with the lowest mean absolute error value (25.61 μm) and highest R2 (0.7817) in average choroidal thickness. Incorporating diopter spherical, anterior chamber depth, and lens thickness as auxiliary tasks improved predicted accuracy (p-value = 6.39×10−44, 2.72×10−38, 1.15×10−36 respectively). For ETDRS regional choroidal thickness estimation, multi-task model achieved better results than single task model (lowest mean absolute error = 31.10 μm vs. 33.20 μm). The multi-task training also can simultaneously predict the choroidal thickness of 100 grids with a minimum mean absolute error of 33.86 μm. ConclusionsEfficientnet-b0, in combination with multi-task and auxiliary task models, achieve high accuracy in estimating average and regional macular choroidal thickness directly from color fundus photographs.

One model to unite them all: Personalized federated learning of multi-contrast MRI synthesis

Curation of large, diverse MRI datasets via multi-institutional collaborations can help improve learning of generalizable synthesis models that reliably translate source- onto target-contrast images. To facilitate collaborations, federated learning (FL) adopts decentralized model training while mitigating privacy concerns by avoiding sharing of imaging data. However, conventional FL methods can be impaired by the inherent heterogeneity in the data distribution, with domain shifts evident within and across imaging sites. Here we introduce the first personalized FL method for MRI Synthesis (pFLSynth) that improves reliability against data heterogeneity via model specialization to individual sites and synthesis tasks (i.e., source-target contrasts). To do this, pFLSynth leverages an adversarial model equipped with novel personalization blocks that control the statistics of generated feature maps across the spatial/channel dimensions, given latent variables specific to sites and tasks. To further promote communication efficiency and site specialization, partial network aggregation is employed over later generator stages while earlier generator stages and the discriminator are trained locally. As such, pFLSynth enables multi-task training of multi-site synthesis models with high generalization performance across sites and tasks. Comprehensive experiments demonstrate the superior performance and reliability of pFLSynth in MRI synthesis against prior federated methods.

Rapid screening of chemicals for their potential to cause specific toxidromes

Toxidromes constitute patterns of symptoms and signs caused by specific toxic effects that guide emergency treatments. Computational identification of chemicals that cause different toxidromes allows us to rapidly screen novel compounds and compound classes as to their potential toxicity. The aim of the current study was to create a computational toolset that can map chemicals to their potential toxidromes. Hence, we evaluated the performance of a state-of-the-art deep learning method—the recently developed communicative message passing neural network (CMPNN)—for its ability to overcome the use of small datasets for training deep learning models. Our results indicated that multi-task training—a technique known for its ability to use multiple small datasets to train conventional deep neural networks—works equally well with CMPNN. We also showed that CMPNN-based ensemble learning results in more reliable predictions than those obtained using a single CMPNN model. In addition, we showed that the standard deviations of individual model predictions from an ensemble of CMPNN models correlated with the errors of ensemble predictions and could be used to estimate the reliability of ensemble predictions. For toxidromes that do not have well-defined molecular mechanisms or sufficient data to train a deep learning model, we used the similarity ensemble approach to develop molecular structural similarity-based toxidrome models. We made the toolset developed in this study publicly accessible via a web user interface at https://toxidrome.bhsai.org/.