Text Augmentation Research Articles

Syntactically Coherent Text Augmentation for Sequence Classification

In this article, we address the problem of data scarcity for the sequence classification tasks. We propose AugmentGAN, a simple-yet-effective generative adversarial network-based text augmentation model, which ensures syntactic coherency in the newly generated samples. Given an input with a label, AugmentGAN aims to generate a semantically similar sequence that follows the syntactic structure of the original sample. Exhaustive task-based evaluation is conducted to show the efficacy of AugmentGAN—we employ 12 different datasets across five classification tasks, i.e., sentiment analysis, emotion recognition, sarcasm detection, intent classification, and spam detection. We observe that, compared to the existing text augmentation techniques, AugmentGAN yields an improved performance across datasets for all the tasks. AugmentGAN also turns out to be effective for multiple languages, i.e., English, Hindi, and Bengali.

Learning policy scheduling for text augmentation

When training deep learning models, data augmentation is an important technique to improve the performance and alleviate overfitting. In natural language processing (NLP), existing augmentation methods often use fixed strategies. However, it might be preferred to use different augmentation policies in different stage of training, and different datasets may require different augmentation policies. In this paper, we take dynamic policy scheduling into consideration. We design a search space over augmentation policies by integrating several common augmentation operations. Then, we adopt a population based training method to search the best augmentation schedule. We conduct extensive experiments on five text classification and two machine translation tasks. The results show that the optimized dynamic augmentation schedules achieve significant improvements against previous methods.

Comparison of data augmentation methods for legal document classification

Sorting out the legal documents by their subject matter is an essential and time-consuming task due to the large amount of data. Many machine learning-based text categorization methods exist, which can resolve this problem. However, these algorithms can not perform well if they do not have enough training data for every category. Text augmentation can resolve this problem. Data augmentation is a widely used technique in machine learning applications, especially in computer vision. Textual data has different characteristics than images, so different solutions must be applied when the need for data augmentation arises. However, the type and different characteristics of the textual data or the task itself may reduce the number of methods that could be applied in a certain scenario. This paper focuses on text augmentation methods that could be applied to legal documents when classifying them into specific groups of subject matters.

COVID-19 information retrieval with deep-learning based semantic search, question answering, and abstractive summarization

The COVID-19 global pandemic has resulted in international efforts to understand, track, and mitigate the disease, yielding a significant corpus of COVID-19 and SARS-CoV-2-related publications across scientific disciplines. Throughout 2020, over 400,000 coronavirus-related publications have been collected through the COVID-19 Open Research Dataset. Here, we present CO-Search, a semantic, multi-stage, search engine designed to handle complex queries over the COVID-19 literature, potentially aiding overburdened health workers in finding scientific answers and avoiding misinformation during a time of crisis. CO-Search is built from two sequential parts: a hybrid semantic-keyword retriever, which takes an input query and returns a sorted list of the 1000 most relevant documents, and a re-ranker, which further orders them by relevance. The retriever is composed of a deep learning model (Siamese-BERT) that encodes query-level meaning, along with two keyword-based models (BM25, TF-IDF) that emphasize the most important words of a query. The re-ranker assigns a relevance score to each document, computed from the outputs of (1) a question–answering module which gauges how much each document answers the query, and (2) an abstractive summarization module which determines how well a query matches a generated summary of the document. To account for the relatively limited dataset, we develop a text augmentation technique which splits the documents into pairs of paragraphs and the citations contained in them, creating millions of (citation title, paragraph) tuples for training the retriever. We evaluate our system (http://einstein.ai/covid) on the data of the TREC-COVID information retrieval challenge, obtaining strong performance across multiple key information retrieval metrics.

Syntactic- and morphology-based text augmentation framework for Arabic sentiment analysis.

Arabic language is a challenging language for automatic processing. This is due to several intrinsic reasons such as Arabic multi-dialects, ambiguous syntax, syntactical flexibility and diacritics. Machine learning and deep learning frameworks require big datasets for training to ensure accurate predictions. This leads to another challenge faced by researches using Arabic text; as Arabic textual datasets of high quality are still scarce. In this paper, an intelligent framework for expanding or augmenting Arabic sentences is presented. The sentences were initially labelled by human annotators for sentiment analysis. The novel approach presented in this work relies on the rich morphology of Arabic, synonymy lists, syntactical or grammatical rules, and negation rules to generate new sentences from the seed sentences with their proper labels. Most augmentation techniques target image or video data. This study is the first work to target text augmentation for Arabic language. Using this framework, we were able to increase the size of the initial seed datasets by 10 folds. Experiments that assess the impact of this augmentation on sentiment analysis showed a 42% average increase in accuracy, due to the reliability and the high quality of the rules used to build this framework.

음성 인식과 텍스트 확장 알고리즘을 이용한 인물 성향 분석에 관한 연구 - 영화 내 주연 배우의 성향 중심으로

음성 인식과 텍스트 확장 알고리즘을 이용한 인물 성향 분석에 관한 연구 - 영화 내 주연 배우의 성향 중심으로

Iterative Translation-Based Data Augmentation Method for Text Classification Tasks

A well-known limitation of existing rule-based text augmentation is that it cannot be applied to other languages because it depends on grammatical and structural characteristics. Moreover, most text Generative Adversarial Networks (GAN) are unstable in training due to inefficient generator optimization and rely on maximum likelihood pre-training. This paper addresses the above problems by proposing a novel augmentation method with a Sentence Generator (SG) and Sentence Discriminator (SD) for Iterative Translation-based Data Augmentation (ITDA). This paper makes three original contributions. First, the ITDA SG is designed to provide universal multiple-language support by generating comprehensive augmented sentences through serial and parallel iterations of an existing translator, such as Google Translate. Second, given that the quality of the generated sentences varies depending on the translation combination or the type of sentence, the ITDA addresses this issue using a discriminator to achieve sentence augmentation, which can select high-quality augmented data using a text classifier. Third, the ITDA can perform sentence augmentation for 109 different languages using discriminators based on text classifiers trained for a specific language or type of data set. Extensive experiments are conducted to evaluate the efficacy of the ITDA using a Convolutional Neural Network (CNN), Bidirectional Long Short-Term Memory (BiLSTM), CNN-BiLSTM, and self-attention. The results demonstrate that when the ITDA is applied to 480 sentence classification tasks, the average accuracy increases by 4.24%.

Text Augmentation Using BERT for Image Captioning

Image captioning is an important task for improving human-computer interaction as well as for a deeper understanding of the mechanisms underlying the image description by human. In recent years, this research field has rapidly developed and a number of impressive results have been achieved. The typical models are based on a neural networks, including convolutional ones for encoding images and recurrent ones for decoding them into text. More than that, attention mechanism and transformers are actively used for boosting performance. However, even the best models have a limit in their quality with a lack of data. In order to generate a variety of descriptions of objects in different situations you need a large training set. The current commonly used datasets although rather large in terms of number of images are quite small in terms of the number of different captions per one image. We expanded the training dataset using text augmentation methods. Methods include augmentation with synonyms as a baseline and the state-of-the-art language model called Bidirectional Encoder Representations from Transformers (BERT). As a result, models that were trained on a datasets augmented show better results than that models trained on a dataset without augmentation.

3rd ACM SIGSPATIAL Workshop on Analytics for Local Events and News (LENS 2019)

The 3rd ACM SIGSPATIAL Workshop on Analytics for Local Events and News (LENS 2019) was held in conjunction with the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL 2019). The workshop is intended to bring together experts in the workshop scope to exchange ideas on opportunities, challenges and cutting-edge techniques for local events and news analytics. The workshop has attracted ten submissions, all full-length research papers. Six submissions were accepted for publications; with 60% acceptance rate. The papers are reviewed by 7 program committee members, where each paper is assigned to two reviewers. Total 8 attendants registered for the workshop. The actual number of attendants along the day fluctuated from 8 to 12, with an average of 9 attendants maintained almost all the time. The paper titled DeepSpot: Understanding Online Opinion Spam by Text Augmentation using Sentiment Encoder-Decoder Networks , by Avik Nayak, Haiquan Chen, Xiaojun Ruan, and Jinsong Ouyang was awarded the best paper award. The paper titled Scalable Community Detection over Geo-Social Network , by Xiuwen Zheng, Qiyu Liu, and Amarnath Gupta was awarded the best presentation award. The workshop has achieved its goal in bringing up together experts and conducted a set of productive discussions and talks.

Triggering dialogic activities across networks

Our study originates in exploratory interventions aimed at engaging students with relevant practices to support learning both in and about digital environments. Our students come from a wide range of professional contexts all over the world. Most are involved in teaching or supporting teaching. What they learn on our programme is carried into their practice - and vice versa. Their introductory course explores a range of environments along with critical literature, with a strong theoretical emphasis. The authors both have an interest in dialogic approaches to education, and we share findings on activities designed and tutored by each of us. A webquest and an online text augmentation exercise were both found to promote student creativity, dialogue and learning. While we had respectable pedagogical reasons for these activities, our subsequent reflections and conversations about them suggested that they were worthy of further theoretical analysis. Our students demonstrated considerable use of existing networks while at the same time apparently generating new networks that would sustain them throughout a programme of study and beyond. Drawing on Wegerif's (2013) notion of dialogic and its Bakhtinian influences, we attempt to distinguish and name features of networks likely to trigger dialogic exchanges that foster learning. We have isolated examples from each practice of particularly ‘interanimating' sections of dialogue and created a thick description of them, including their antecedents and consequences. Our examples illustrate that both practices raised questions about purpose, offered opportunities for showcasing knowledge and connections, led to sharing and making practices visible, and were taken forward to new contexts. The findings are not all positive; we have also identified concerns about exclusion or inadequacy from those who feel unable to participate fully, but even then there is potential to turn around difficult situations. This study might have practical application for learning designers but should also be of theoretical interest for research into newer forms of academic literacy.

New and improved: The role of text augmentation and the application of response interpretation standards (coding schemes) in a final iteration of birth defects warnings development

Several birth defects warning symbols identified as most successful in an earlier study (Mayhorn and Goldsworthy, 2007) were further modified and then evaluated within a nationally distributed field trial (n = 2773). The purpose for the current research was to determine whether symbol warning components could be improved further, whether the addition of text enhanced comprehension uniformly across symbols, and whether results varied by the application of different interpretation standards (coding schemes). A total of 11 warning labels were examined: four new symbols plus the existing baseline symbol, each in versions with and without text, plus a text-only condition. Participant interpretation accuracy and preferences were assessed during face-to-face interview sessions. For symbol-only conditions, several candidate symbols outperformed the existing symbol, one substantially so. The effect of adding text to symbols varied significantly by symbol. Symbol plus text and text-only conditions performed equivalently, generally exceeded symbol-only conditions, and often surpassed the American National Standards Institute benchmark of 85% accurate interpretation. The research effort has identified a teratogen symbol and warning that outperforms the one currently in use. The effort has also identified important pragmatic and conceptual issues that should inform future work to improve medication labeling and other hazard communication.