Adversarial Multi-task Learning Research Articles

Missing data imputation and classification of small sample missing time series data based on gradient penalized adversarial multi-task learning

Missing data imputation and classification of small sample missing time series data based on gradient penalized adversarial multi-task learning

MCA-NER: Multi-Contextualized Adversarial-Based Attentional Deep Neural Network for Named Entity Recognition

Multi-contextualized representations learning is vital for named entity recognition (NER), which is a fundamental task for effectively extracting structured information from unstructured text, and forming knowledge bases. This task is particularly challenging when dealing with Chinese text given the absence of evident word boundaries. Chinese word segmentation (CWS) can be leveraged to recognize word boundaries, but named entities often encompass multiple segmented words, making it crucial to use boundary information to correctly recognize and distinguish the relationships between these words. In this paper, we propose MCA-NER, a multi-contextualized adversarial-based attentional deep learning approach for Chinese NER, which combines CWS and part-of-speech (POS) tagging information with the classic BiLSTM-CRF NER model, using adversarial multi-task learning. The model incorporates several self-attention components for adversarial and multi-task learning, effectively synthesizing task-specific and common information attribution while improving performance across all three tasks. Experimental results on the three datasets provide compelling evidence that supports the effectiveness and performance of our model.

Adversarial Multi-task Learning for Efficient Chinese Named Entity Recognition

Named entity recognition (NER) is a fundamental task for information extraction applications. NER is challenging because of semantic ambiguities in academic literature, especially for non-Latin languages. Besides word semantic information, recognizing Chinese named entities needs to consider word boundary information, as words contained in Chinese texts are not separated with spaces. Leveraging word boundary information could help to determine entity boundaries and thus improve entity recognition performance. In this article, we propose to combine word boundary information and semantic information for named entity recognition based on multi-task adversarial learning. Specifically, we learn commonly shared boundary information of entities from multiple kinds of tasks, including Chinese word segmentation (CWS), part-of-speech (POS) tagging, and entity recognition, with adversarial learning. We learn task-specific semantic information of words from these tasks and combine the learned boundary information with the semantic information to improve entity recognition with multi-task learning. We then propose a compression method based on improved clustering to accelerate the proposed model. We conduct extensive experiments on four public benchmark datasets and two private datasets, compared with state-of-the-art baseline models, and the experimental results demonstrate that our model achieves considerable performance improvements on various evaluation datasets.

Smart contracts vulnerability detection model based on adversarial multi-task learning

Vulnerability detection is important for smart contracts because of their immutable and irreversible features. In this work, a new detection method based on adversarial multi-task learning is proposed to improve the accuracy of existing vulnerability detection methods, which is based on the multi-task learning framework, including a shared part and a task-specific part. We optimize the multi-task learning frameworks and propose the mixed parameter sharing method to make each task not only maintain its uniqueness, but also share features with other tasks, which helps solve the problem that the hard parameter sharing method cannot constrain the underlying shared layer and improve the quality of extracted features. In addition, we introduce adversarial transfer learning to reduce noise pollution caused by the private feature and interference between the general feature and the private feature. We experimented on datasets obtained from our previous work, and the experimental results prove that our proposed model can judge whether there are vulnerabilities in smart contracts and then identify their types. Additionally, the results also show that our model effectively improves detection accuracy and has an advantage in performance over representative methods.

Domain and Patient Adversarial Multi-Task Learning for Arrhythmia Classification.

Manual screening of electrocardiograms (ECGs) for heart arrhythmias by clinicians is time-consuming and labor-intensive. A machine learning model for the automated diagnosis of heart arrhythmia from ECG signals can facilitate improved diagnosis, greater accessibility and earlier intervention for patients. The potential of such models is limited however by the small size of clinical datasets available for training. Methods that can be trained with multiple datasets to classify heart arrhythmia are needed to overcome this problem.In this paper, we propose using adversarial multi-task learning (AMTL) to extract domain and patient invariant features from two electrocardiogram databases. We further investigated the influence of beat segmentation location and beat normalization on domain invariance. Our proposed methods were tested on the MIT-BIH Arrhythmia and the St Petersburg INCART 12-lead Arrhythmia Databases. The domain adversarial models achieved a higher accuracy and average F1 score than their counterparts without domain adversarial learning. In particular, the patient and domain adversarial model improved the F1 scores on the two tested databases from 70% and 74% to 77% each.Clinical Relevance-This establishes that adversarial multitask learning with multiple datasets and multiple adversarial tasks can improve the F1 score of arrhythmia classification.

Transfer knowledge for punctuation prediction via adversarial training

Previous studies demonstrate that part-of-speech (POS) tags are helpful for punctuation restoration tasks. However, extra computation cost will be needed during decoding, due to POS tags are provided by an external POS tagger. This paper proposes to transfer knowledge via adversarial training and orthogonality constraints to fill in the gap. Adversarial multi-task learning is introduced to learn task invariant knowledge from the extra POS tagging task for a punctuation prediction task. Furthermore, orthogonality constraints are used to make private and shared features dissimilar. Only the punctuation predicting task is used during decoding. So extra computation is not needed. Experiments are conducted on IWSLT2011 datasets. The results show that the punctuation predicting models trained with adversarial learning obtain performance gains over the baseline models on test sets. The results also demonstrate that the models trained with orthogonality constraints further obtain performance improvement.

Design of Efficient Speech Emotion Recognition Based on Multi Task Learning

Speech emotion recognition technology includes feature extraction and classifier construction. However, the recognition efficiency is reduced due to noise interference and gender differences. To solve this problem, this paper used two multi-task learning models based on adversarial multi-task learning(ASP-MTL). The first model took emotion recognition as the main task and noise recognition as the auxiliary task, and removed the noise part identified by the auxiliary task. After identifying the non-noise part, the second model was constructed. The second model took emotion recognition as the main task and gender classification as the auxiliary task. These two multi-task learning models can not only can use shared information to learn the relationship between different tasks, but also can identify specific tasks. This paper used Audio/Visual Emotion Challenge (AVEC) database and AFEW6.0 database,which were recorded in the field environment. Considering the problem of data imbalance between datasets, the data balance operation was carried out on the data sets in the process of data preprocessing. The paper shows an increase of around 10% in terms of accuracy and F1 score with the recent works using AVEC database and AFEW6.0 datasets, which proved that this paper has made a great progress in SER.

Multi-label emotion classification based on adversarial multi-task learning

In this paper, we focus on the task of multi-label emotion classification and aim to tackle two problems of this task. First, few studies try to exploit the correlation among different emotions, which motivates us to introduce the task-specific information into the shared hidden layer. Second, the public multi-label emotion datasets for low-resource languages are limited. To overcome these problems, we propose a novel multi-task multi-label emotion classification. Our approach consists of three components: general representation module, emotion representation module and adversarial classifier. The model applies emotion descriptors to incorporate the correlation among different emotions, and then uses adversarial training to prevent too much emotion-relevant information from being injected into the shared layer. Extensive experiments demonstrate that our approach outperforms the state-of-the-art baselines across a variety of evaluation metrics, achieving macro-average F1 scores of 50.21%, 41.33% and 40.24% on the Chinese, English, and Indonesian datasets, respectively. In addition, an Indonesian dataset and an English one containing 4207 and 26,019 samples respectively, are constructed for the multi-label emotion classification task. The datasets will be publicly available and we believe this can support the future study of Indonesian multi-label emotion recognition resources, which are limited for the related research fields now. We make our codes and resources in this work publicly available on: https://github.com/GKLMIP/MLEC-AML.

Adversarial multi-task deep learning for signer-independent feature representation

Adversarial multi-task deep learning for signer-independent feature representation

Writer adaptation for E2E Arabic online handwriting recognition via adversarial multi task learning

The importance of online handwriting recognition has been rapidly increasing over recent years due to the rapid technological advances in handheld devices and communication software with handwriting interfaces. Deep learning end-to-end (E2E) models have provided high recognition rates as part of online handwriting recognition systems. However, attaining even higher performance levels requires supplementing these models with adaptation techniques that cater to individual penmanship. This study proposes a writer adaptation technique for Arabic online handwriting recognition systems that employs adversarial Multi-Task Learning (MTL). Adversarial training and MTL modify the deep-features distribution of the Writer Dependent (WD) model, leading its output to closely resemble that of the Writer Independent (WI) model. The design of the proposed method entails two tasks: label classification (primary task) and model features discrimination (secondary task). Our method was designed to jointly optimize both sub-networks. The proposed technique was tested against the E2E Connectionist Temporal Classification (CTC) based model, a combination of both Convolutional Neural Networks (CNNs) and Bidirectional Long Short-term Memory (BiLSTM). The proposed models were trained and evaluated against two large datasets (the Online-KHATT and CHAW). In supervised adaptation, it achieved an absolute Character Error Rate (CER) of up to 1.83% and an absolute Word Error Rate (WER) reduction of 11.71% over the WI model. Additionally, supervised adaptation achieved an absolute CER of up to 0.84% and an absolute WER reduction of 6.77% over the fine-tuned model. In unsupervised adaptation, the proposed method achieved an absolute CER of up to 0.5% absolute and an absolute WER reduction of 1.74% absolute (WER) reduction over the WI. Our experimental results indicate that our proposed supervised writer adaptation can achieve significant improvements in recognition accuracy compared with the baseline models: WI and fine-tuned models.

Adversarial multi-task learning with inverse mapping for speech enhancement

Adversarial Multi-Task Learning (AMTL) has demonstrated its promising capability of information capturing and representation learning, however, is hardly explored in speech enhancement. In this paper, we propose a novel adversarial multi-task learning with inverse mapping method for speech enhancement. Our method focuses on enhancing the generator’s capability of speech information capturing and representation learning. To implement this method, two extra networks (namely P and Q) are developed to establish the inverse mapping from the generated distribution to the input data domains. Correspondingly, two new loss functions (i.e., latent loss and equilibrium loss) are proposed for the inverse mapping learning and the enhancement model training with the original adversarial loss. Our method obtains the state-of-the-art performance in terms of speech quality (PESQ=2.93, CVOL=3.55). For speech intelligibility, our method can also obtain competitive performance (STOI=0.947). The experimental results demonstrate that our method can effectively improve speech representation learning and speech enhancement performance.

Knowledge augmented transformer for adversarial multidomain multiclassification multimodal fake news detection

The spread of disinformation and fake news on social platforms has an unfavorable impact on social harmony and stability. The timely and accurate identification of fake news might help restrain the propagation of fake news and mitigate its influence on society. In this paper, we propose a novel multimodal fake news detection framework: the Knowledge Augmented Transformer for adversarial Multidomain multiclassification multimodal Fake news detection framework (KATMF). In contrast to most of the existing studies, which ignore the differences among news articles from different domains in terms of the feature distribution, the KATMF employs a multimodal adversarial multitask learning module to capture these differences. Moreover, because social media news entities generally lack sufficient background knowledge, to enrich news with knowledge information in a homogeneous embedding space, we use the Knowledge Augmented Transformer (KAT) to selectively encode the information of entities from an external knowledge source into the representation of news. We evaluate our approach on a large-scale real-world dataset, and the experimental results demonstrate that our proposed model outperforms state-of-the-art fake news detection methods.

Cross domains adversarial learning for Chinese named entity recognition for online medical consultation

Deep learning methods have been applied to Chinese named entity recognition for the online medical consultation. They require a large number of marked samples. However, no such database is available at present. This paper begins with constructing a larger labelled Chinese texts database for the online medical consultation. Second, a basic framework unit is proposed, which is pre-trained by the transfer learning from both Bidirectional language model and Mask language model trained on the larger unlabelled data. Finally, cross domains adversarial learning (CDAL) for Chinese named entity recognition is proposed to further improve the performance, which not only uses the pre-trained basic framework unit, but also uses the adversarial multi-task learning on both electronic medical record texts and online medical consultation texts. Experimental results validate the effectiveness of CDAL.

Multitask deep learning for native language identification

Identifying the native language of a person by their text written in English (L1 identification) plays an important role in such tasks as authorship profiling and identification. With the current proliferation of misinformation in social media, these methods are especially topical. Most studies in this field have focused on the development of supervised classification algorithms, that are trained on a single L1 dataset. Although multiple labeled datasets are available for L1 identification, they contain texts authored by speakers of different languages and do not completely overlap. Current approaches achieve high accuracy on available datasets, but this is attained by training an individual classifier for each dataset. Studies show that joint training of multiple classifiers on different datasets can result in sharing information between the classifiers, leading to an increase in the accuracy of both tasks. In this study, we develop a novel deep neural network (DNN) architecture for L1 classification; it is based on an adversarial multitask learning method that integrates shared knowledge from multiple L1 datasets. We propose several variants of the architecture and rigorously evaluate their performance on multiple datasets. Our results indicate the proposed multitask architecture is more efficient in terms of classification accuracy than previously proposed methods.

Adversarial Multi-Task Learning for Robust End-to-End ECG-based Heartbeat Classification.

In clinical practice, heart arrhythmias are manually diagnosed by a doctor, which is a time-consuming process. Furthermore, this process is error-prone due to noise from the recording equipment and biological non-idealities of patients. Thus, an automated arrhythmia classifier would be time and cost-effective as well as offer better generalization across patients. In this paper, we propose an adversarial multitask learning method to improve the generalization of heartbeat arrythmia classification. We built an end-to-end deep neural network (DNN) system consisting of three sub-networks; a generator, a heartbeat-type discriminator, and a subject (or patient) discriminator. Each of these two discriminators had its own loss function to control its impact. The generator was "friendly" to the heartbeat-type discrimination task by minimizing its loss function and "hostile" to the subject discrimination task by maximizing its loss function. The network was trained using raw ECG signals to discriminate between five types of heartbeats - normal heartbeats, right bundle branch blocks (RBBB), premature ventricular contractions (PVC), paced beats (PB) and fusion of ventricular and normal beats (FVN). The method was tested with the MIT-BIH arrhythmia dataset and achieved a 17% reduction in classification error compared to a baseline using a fully-connected DNN classifier.Clinical Relevance-This work validates that it is possible to develop a subject-independent automated heart arrhythmia detection system to assist clinicians in the diagnosis process.

Regularization of Deep Neural Networks for EEG Seizure Detection to Mitigate Overfitting.

Seizure detection is a major goal for simplifying the workflow of clinicians working on EEG records. Current algorithms can only detect seizures effectively for patients already presented to the classifier. These algorithms are hard to generalize outside the initial training set without proper regularization and fail to capture seizures from the larger population. We proposed a data processing pipeline for seizure detection on an intra-patient dataset from the world's largest public EEG seizure corpus. We created spatially and session invariant features by forcing our networks to rely less on exact combinations of channels and signal amplitudes, but instead to learn dependencies towards seizure detection. For comparison, the baseline results without any additional regularization on a deep learning model achieved an F1 score of 0.544. By using random rearrangements of channels on each minibatch to force the network to generalize to other combinations of channels, we increased the F1 score to 0.629. By using random rescale of the data within a small range, we further increased the F1 score to 0.651 for our best model. Additionally, we applied adversarial multi-task learning and achieved similar results. We observed that session and patient specific dependencies were causing overfitting of deep neural networks, and the most overfitting models learnt features specific only to the EEG data presented. Thus, we created networks with regularization that the deep learning did not learn patient and session-specific features. We are the first to use random rearrangement, random rescale, and adversarial multitask learning to regularize intra-patient seizure detection and have increased sensitivity to 0.86 comparing to baseline study.

Scientific Literature based Big Data Analysis for Technology Insight

The increasingly massive amount and open access of literature provide a data foundation for technology insight based on big data analysis. This paper proposed a new technology insight framework -Technology Dependency Graph (TDG). Firstly, an adversarial multitask learning and distantly-supervised learning were applied to extract the technology entity and dependency relation with limited labeled sample. Then, a TDG was constructed with the entities as vertices and the dependency relations as edges. A TDG contains rich and valuable semantic information which represents the support, contribution or relying on relationship between technologies. At the same time, the social network properties of TDG allow researchers to analyze and mine hot topics, key technologies, and technology architecture by using network theories, methods and tools. In the case study, the TDG of DSSC (dye-sensitized solar cell) was constructed. Furthermore, the technology dependency architecture for DSSC was constructed according to a spanning tree out of the TDG, which provides a global perspective for the research of DSSC.