Piece Of Text Research Articles

Multi-Self-Attention for Aspect Category Detection and Biomedical Multilabel Text Classification with BERT

Multilabel classification is one of the most challenging tasks in natural language processing, posing greater technical difficulties than single-label classification. At the same time, multilabel classification has more natural applications. For individual labels, the whole piece of text has different focuses or component distributions, which require full use of local information of the sentence. As a widely adopted mechanism in natural language processing, attention becomes a natural choice for the issue. This paper proposes a multilayer self-attention model to deal with aspect category and word attention at different granularities. Combined with the BERT pretraining model, it achieves competitive performance in aspect category detection and electronic medical records’ classification.

Linguistic changes in the transition from summaries to abstracts: The case of the Journal of Experimental Medicine

Linguistic changes in the transition from summaries to abstracts: The case of the <scp> <i>Journal of Experimental Medicine</i> </scp>

3x3x6 – 9 Sq.m. and 6 Surveillance Cameras

The title of Shu Lea Cheang’s 3x3x6 which represented Taiwan at Venice Biennale 2019 derives from the 21st century high-security prison cell measured in 9 square meter and equipped with 6 surveillance cameras. As an immersive installation, 3x3x6 is comprised of multiple interfaces to reflect on the construction of sexual subjectivity by technologies of confinement and control, from physical incarceration to the omnipresent surveillance systems of contemporary society, from Jeremy Bentham’s panopticon conceptualized in 1791 to China’s Sharp Eyes that boasts 200 million surveillance cameras with facial recognition capacity for its 1.4 billion population. By employing strategic and technical interventions, 3x3x6 investigates 10 criminal cases in which the prisoners across time and space are incarcerated for sexual provocation and gender affirmation. The exhibition constructs collective counter-accounts of sexuality where trans punk fiction, queer, and anti-colonial imaginations hacks the operating system of the history of sexual subjection. This Image and Text piece intersperses images from the exhibition with handout texts written by curator Paul B. Preciado (against a grey background), as well as an interview between special section co-editor Paula Gardner and the artist that brings the extraordinary exhibition into further conversation with feminist technoscience scholarship. The project website is available at https://3x3x6-v2.webflow.io/.

Quantum Probability-inspired Graph Attention Network for Modeling Complex Text Interaction

Inspired by quantum-like phenomena in human language understanding, recent studies propose quantum probability-inspired neural networks to model natural language by treating words as superposition states and a sentence as a mixed state. However, many complex natural language processing tasks (e.g., emotion–cause pair extraction or joint dialog act recognition and sentiment classification) require modeling the complex and graphical interaction of multiple text pieces (e.g., multiple clauses in a document or multiple utterances in a dialog). The existing quantum probability-inspired neural networks only encode sequential interaction of a sequence of words, but cannot model the complex interaction of text pieces. To generalize the quantum framework from modeling word sequence to modeling complex and graphical text interaction, we propose a Quantum Probability-inspired Graph Attention NeTwork (QPGAT) by combining quantum probability and graph attention mechanism in a unified framework. Specifically, a text interaction graph is firstly constructed to describe the complex interaction of text pieces. Then QPGAT models each text node as a particle in a superposition state and each node’s neighborhood in the graph as a mixed system in a mixed state to learn interaction-aware text node representations. We apply QPGAT to the two important and complex NLP tasks, emotion–cause pair extraction and joint dialog act recognition and sentiment classification. Experiment results show that QPGAT is competitive compared with the state-of-the-art methods on the two complex NLP tasks, demonstrating the effectiveness of QPGAT. Moreover, QPGAT can also provide a reasonable post-hoc explanation about the model decision process for emotion–cause pair extraction.

Semantic Place Prediction With User Attribute in Social Media

In our daily life, we prefer to describe the location of a person with semantic words such as “travel,” “cinema” rather than geographic terminology such as GPS coordinates ($32^{\circ }24^{^{\prime }}12^{^{\prime \prime }}$32∘24'12''N, $2^{\circ }90^{^{\prime }}26.5^{^{\prime \prime }}$2∘90'26.5''E). This is because semantic places are easier to understand compared with precise geographic location. Existing methods only take text-image pairs into consideration, neglecting the user attribute, which is essential in semantic place prediction. In this work, we explore the impact of user attribute and propose user attribute included multimodal fusion models for semantic place prediction. Specifically, due to the complementarity of different modalities, multimodal fusion networks can adaptively learn the relationship between different modalities to fully use representational information from each modality. Besides, we propose a multi-image processing module to solve the scenario that a piece of Microblog text is attached to multiple images. Extensive experiments on our collected dataset verified the effectiveness of the proposed approach.

Perspectives on written L1 in Swedish CLIL education

This article presents a longitudinal investigation of texts written by students in upper secondary schools in Sweden. The texts are collected at three different schools implementing Content and Language Integrated Learning, CLIL, where school subjects are taught in a second or foreign language, L2, in this case English. CLIL research with an L1 focus in the Swedish context is rare. The present study explores and compares vocabulary use in texts written in L1 Swedish by students attending schools where English is used as the medium of instruction to various degrees, thereby representing diverse CLIL models. One school uses English in practically all subjects except in language arts (subject area of Swedish and optional German/French/Spanish). The other two schools use L2 English in some lessons but not all, thus representing other CLIL models. The data comprises 306 pieces of texts that were analysed using quantitative and corpus linguistic methods to examine the vocabulary use including linguistic variables connected to academic writing. The texts were written at four different occasions during a period of three years, Results indicate that the L1 vocabulary use concerning specific word variables show no substantial diversifications between the three CLIL schools despite the dissimilar exposure to L2 English and L1 use. The impact of L2 on students’ L1 is sometimes raised as an apprehension against CLIL education in Sweden. The results regarding productive written academic vocabulary of the present study indicate that there are no grounds for such concerns.

Extracting Data at Scale: Machine learning at the Natural History Museum

The Natural History Museum holds over 80 million specimens and 300 million pages of scientific text. This information is a vital research tool to help solve the most important challenge humans face over the coming years – mapping a sustainable future for ourselves and the ecosystems on which we depend. Digitising these collections and providing the data in a structured, computable form is a mammoth challenge. As of 2020, less than 15% of available specimen information currently residing on specimen labels or physical registers is digitised and publicly available (Walton et al. 2020). Machine learning applications can deliver a step-change in our activities’ scope, scale, and speed (Borsch et al. 2020). As part of SYNTHESYS+, the Natural History Museum is leading on the development of a cloud-based workflow platform for natural science specimens, the Specimen Data Refinery (SDR) (Smith et al. 2019). The SDR will provide a series of Machine Learning (ML) models, ranging from semantic segmentation to identify regions of interest on labels, to natural language processing to extract locality and taxonomic text entities from the labels, and image analysis to identify specimen traits and collection quality metrics. Each ML task is atomic, with users of the SDR selecting which model would best extract data from their digitised specimen images, allowing the workflows to be used in different institutions worldwide. It also solves one of the key problems in developing ML-based applications: the rapidity at which models become obsolete. New ML models can be introduced into the workflow, with incremental changes to improve processing, without interruption or refactoring of the pipeline. Alongside specimens, digitised images of pages of scientific literature provide another vital source of data. Functional traits mediate the interactions between plant species and their environment and play roles in determining species’ range size and threatened status. Such information is contained within the taxonomic descriptions of species and a natural language processing library has been developed to locate and extract plant functional traits from these texts (Hoehndorf et al. 2016). The ML models allow complex interrelationships between taxa and trait entities to be inferred based on the grammatical structure of sentences, improving the accuracy and extent of data point extraction. These two projects, like many other applications of ML in natural history collections, are focused on the extraction of visible information, for example, a piece of text or a measurable trait. Given the image of the specimen or page, a person would be able to extract the self-same information. However, ML excels in pattern matching and inferring unknown characters from an entire corpus. At the museum, we have started exploring this space, with our voyagerAI project for identifying specimens collected on historical expeditions of scientific discovery (e.g., the voyages of the Beagle and Challenger). This process fills in the gaps in specimen provenance and identifies 'lost' specimens collected by some of the most famous names in biodiversity history. Developing new applications of ML to uncover scientific meaning and tell the narratives of our collections, will be at the forefront of our scientific innovation in the coming years. This presentation will give an overview of these projects, and our future plans for using ML to extract data at scale within the Natural History Museum.

GAVIN

Annotation is an effective reading strategy people often undertake while interacting with digital text. It involves highlighting pieces of text and making notes about them. Annotating while reading in a desktop environment is considered trivial but, in a mobile setting where people read while hand-holding devices, the task of highlighting and typing notes on a mobile display is challenging. In this article, we introduce GAVIN, a gaze-assisted voice note-taking application, which enables readers to seamlessly take voice notes on digital documents by implicitly anchoring them to text passages. We first conducted a contextual enquiry focusing on participants’ note-taking practices on digital documents. Using these findings, we propose a method which leverages eye-tracking and machine learning techniques to annotate voice notes with reference text passages. To evaluate our approach, we recruited 32 participants performing voice note-taking. Following, we trained a classifier on the data collected to predict text passage where participants made voice notes. Lastly, we employed the classifier to built GAVIN and conducted a user study to demonstrate the feasibility of the system. This research demonstrates the feasibility of using gaze as a resource for implicit anchoring of voice notes, enabling the design of systems that allow users to record voice notes with minimal effort and high accuracy.

CapsTM: capsule network for Chinese medical text matching

BackgroundText Matching (TM) is a fundamental task of natural language processing widely used in many application systems such as information retrieval, automatic question answering, machine translation, dialogue system, reading comprehension, etc. In recent years, a large number of deep learning neural networks have been applied to TM, and have refreshed benchmarks of TM repeatedly. Among the deep learning neural networks, convolutional neural network (CNN) is one of the most popular networks, which suffers from difficulties in dealing with small samples and keeping relative structures of features. In this paper, we propose a novel deep learning architecture based on capsule network for TM, called CapsTM, where capsule network is a new type of neural network architecture proposed to address some of the short comings of CNN and shows great potential in many tasks.MethodsCapsTM is a five-layer neural network, including an input layer, a representation layer, an aggregation layer, a capsule layer and a prediction layer. In CapsTM, two pieces of text are first individually converted into sequences of embeddings and are further transformed by a highway network in the input layer. Then, Bidirectional Long Short-Term Memory (BiLSTM) is used to represent each piece of text and attention-based interaction matrix is used to represent interactive information of the two pieces of text in the representation layer. Subsequently, the two kinds of representations are fused together by BiLSTM in the aggregation layer, and are further represented with capsules (vectors) in the capsule layer. Finally, the prediction layer is a connected network used for classification. CapsTM is an extension of ESIM by adding a capsule layer before the prediction layer.ResultsWe construct a corpus of Chinese medical question matching, which contains 36,360 question pairs. This corpus is randomly split into three parts: a training set of 32,360 question pairs, a development set of 2000 question pairs and a test set of 2000 question pairs. On this corpus, we conduct a series of experiments to evaluate the proposed CapsTM and compare it with other state-of-the-art methods. CapsTM achieves the highest F-score of 0.8666.ConclusionThe experimental results demonstrate that CapsTM is effective for Chinese medical question matching and outperforms other state-of-the-art methods for comparison.

Autoencoder for Semisupervised Multiple Emotion Detection of Conversation Transcripts

Textual emotion detection is a challenge in computational linguistics and affective computing study as it involves the discovery of all associated emotions expressed within a given piece of text. It becomes an even more difficult problem when applied to conversation transcripts, as we need to model the spoken utterances between speakers, keeping in mind the context of the entire conversation. In this paper, we propose a semisupervised multilabel method of predicting emotions from conversation transcripts. The corpus contains conversational quotes extracted from movies. A small number of them are annotated, while the rest are used for unsupervised training. We use the word2vec word-embedding method to build an emotion lexicon from the corpus and to embed the utterances into vector representations. A deep-learning autoencoder is then used to discover the underlying structure of the unsupervised data. We fine-tune the learned model on labeled training data, and measure its performance on a test set. The experiment result suggests that the method is effective and is only slightly behind human annotators.

SMART APPROACH TO RECOGNIZE PUBLIC GRIEVANCE FROM MICROBLOGS

The foremost priority for the government of any country is to resolve the citizen's problems. Indian government provides various ICT (Information Communication Technology) based plat-forms such as website and mobile application for the Indian citizens to lodge their grievance against ministries and departments. Due to the easy access and transparency of social media, people prefer to raise their voices there. The dissatisfaction and anger that they express through microblogging is also a kind of complaint. This paper discusses the various categories of public grievance, their importance and its redressal mechanism in India. The main aim of this research is to take these social media complaints into consideration and solve them rapidly with the help of Artificial Intelligence where, it’s one branch Natural Language Processing, processes the text to retrieve the information from the piece of text and another branch Machine Learning helps to know the priority and severity of the grievance.

Semi-supervised Thai Sentence Segmentation Using Local and Distant Word Representations

A sentence is typically treated as the minimal syntactic unit used for extracting valuable information from a longer piece of text. However, in written Thai, there are no explicit sentence markers. We proposed a deep learning model for the task of sentence segmentation that includes three main contributions. First, we integrate n-gram embedding as a local representation to capture word groups near sentence boundaries. Second, to focus on the keywords of dependent clauses, we combine the model with a distant representation obtained from self-attention modules. Finally, due to the scarcity of labeled data, for which annotation is difficult and time-consuming, we also investigate and adapt Cross-View Training (CVT) as a semi-supervised learning technique, allowing us to utilize unlabeled data to improve the model representations. In the Thai sentence segmentation experiments, our model reduced the relative error by 7.4% and 10.5% compared with the baseline models on the Orchid and UGWC datasets, respectively. We also applied our model to the task of pronunciation recovery on the IWSLT English dataset. Our model outperformed the prior sequence tagging models, achieving a relative error reduction of 2.5%. Ablation studies revealed that utilizing n-gram presentations was the main contributing factor for Thai, while the semi-supervised training helped the most for English.

From clause to discourse marker: on the development of comment clauses

This paper proposes an alternative account for the development of comment clauses (e.g. I think, I mean, I admit), which differs from previous accounts centring on grammaticalization, pragmaticalization, or lexicalization. Based on the framework of Discourse Grammar and building on Heine et al. (2021a, b), it is argued that their development involves a stage of cooptation, whereby a text piece is transferred from the sentence level to the metatextual level of discourse processing thus acquiring new grammatical properties, viz. independence from the host clause in terms of meaning, syntax and prosody, metatextual function, and positional freedom. All these changes are hard to reconcile with grammaticalization. At the same time, however, grammaticalization does play a role in the process once cooptation has taken place, affecting mainly the internal form of coopted expressions.

The functions and evolution of xob ‘well’ in Persian: A thetical analysis

This study aims to explore the functions of the Persian xob ‘well’ and its evolution mechanisms. The Discourse Grammar model was mainly employed to analyze the occurrences of xob in the Bijankhān corpus. The data analysis revealed that xob is significantly frequent, especially in semi-formal and informal spoken genres. It also demonstrated that xob has risen from the adjective/adverb xub with several propositional meanings most of which converge on ‘good’. Having evolved from this meaning, the core procedural meaning of xob, i.e., ‘engaging in (positive) epistemic/affective consideration prefatory to continuation’, has developed metatextual functions in the three components of the situation of discourse. To express speaker's attitude and speaker-hearer interaction, xob mainly serves the functions of direct response (signaling acceptance, sufficiency, etc.), indirect response (signaling qualification, concession, etc.), and involvement prompting (signaling attention getting, backchannelling, and soliciting consensus). Meeting the needs of text organization, xob fulfills the dual purposes of framing the text pieces and transition between topics and events. The study also highlights the contribution of cooptation and late grammaticalization to the rise of the discourse markers xob and xeyli/besyār xob ‘very well’. Overall, these findings illustrate the potential of Discourse Grammar to address theticals such as discourse markers.

GDPNet: Refining Latent Multi-View Graph for Relation Extraction

Relation Extraction (RE) is to predict the relation type of two entities that are mentioned in a piece of text, e.g., a sentence or a dialogue. When the given text is long, it is challenging to identify indicative words for the relation prediction. Recent advances on RE task are from BERT-based sequence modeling and graph-based modeling of relationships among the tokens in the sequence. In this paper, we propose to construct a latent multi-view graph to capture various possible relationships among tokens. We then refine this graph to select important words for relation prediction. Finally, the representation of the refined graph and the BERT-based sequence representation are concatenated for relation extraction. Specifically, in our proposed GDPNet (Gaussian Dynamic Time Warping Pooling Net), we utilize Gaussian Graph Generator (GGG) to generate edges of the multi-view graph. The graph is then refined by Dynamic Time Warping Pooling (DTWPool). On DialogRE and TACRED, we show that GDPNet achieves the best performance on dialogue-level RE, and comparable performance with the state-of-the-arts on sentence-level RE. Our code is available at https://github.com/XueFuzhao/GDPNet.

Distribution Matching for Rationalization

The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks. By definition, rationales represent key text pieces used for prediction and thus should have similar classification feature distribution compared to the original input text. However, previous methods mainly focused on maximizing the mutual information between rationales and labels while neglecting the relationship between rationales and input text. To address this issue, we propose a novel rationalization method that matches the distributions of rationales and input text in both the feature space and output space. Empirically, the proposed distribution matching approach consistently outperforms previous methods by a large margin. Our data and code are available.

MARTA: Leveraging Human Rationales for Explainable Text Classification

Explainability is a key requirement for text classification in many application domains ranging from sentiment analysis to medical diagnosis or legal reviews. Existing methods often rely on "attention" mechanisms for explaining classification results by estimating the relative importance of input units. However, recent studies have shown that such mechanisms tend to mis-identify irrelevant input units in their explanation. In this work, we propose a hybrid human-AI approach that incorporates human rationales into attention-based text classification models to improve the explainability of classification results. Specifically, we ask workers to provide rationales for their annotation by selecting relevant pieces of text. We introduce MARTA, a Bayesian framework that jointly learns an attention-based model and the reliability of workers while injecting human rationales into model training. We derive a principled optimization algorithm based on variational inference with efficient updating rules for learning MARTA parameters. Extensive validation on real-world datasets shows that our framework significantly improves the state of the art both in terms of classification explainability and accuracy.

Computer Application for Assessing Subjective Answers using AI

Finding the similarity and relatability between two pieces of text is often used in multiple places today. Search Engines, recommendations, web indexes, etc., make use of the similarity between two pieces of data to come to various different conclusions. However, no reliable system has been developed so far that takes into account the similarity between a student’s answer and the teacher’s model answer to give the most accurate and precise marks to the student. The major reason lies in the fact that such a system must be built on multiple parameters and hence the complexity of the system increases. However, on close examination, it could be found that there are various methods that work towards helping in the formulation of the final scores secured by the student based on the similarity between the words of the student answer and model answer as well as the similarity between the meaning of then sentences formed by the student and the model answers. These methods have been reviewed and the most efficiently working ones have been put forward in this proposed system.

Reconstructing spatial information diffusion networks with heterogeneous agents and text contents

AbstractIt is important to reconstruct the hidden network structure from the infection status change of an information propagation process for evidence‐based spatial decision‐making. Unlike previous work, we not only consider the heterogeneity of the propagation agents, but also incorporate the heterogeneity of the text contents of information within the propagation process. In addition, the infection status is no longer restricted to the binary type (infected or not), and we allow the number of pieces of information texts to be counted which represents the degree of infection. The resulting model is a network‐based multivariate recurrent event model, in which the interactions between different types of text, between different agents, between agents and text types, and their mutual impacts on the whole propagation process can be comprehensively investigated. On that basis, a nonparametric mean‐field equation is derived to govern the propagation process, and a compressive sensing algorithm is provided to infer the hidden spatial propagation network from the infection status data. Finally, the proposed methodology is tested through synthetic data and a real data set of information diffusion on Twitter.

A TRANSITIVITY ANALYSIS ON STUDENTS’ WRITING RECOUNT TEXT

A TRANSITIVITY ANALYSIS ON STUDENTS’ WRITING RECOUNT TEXT *Septyana D Sianipar ** Siti Aisah Ginting ** Tiarnita M. S Siregar Abstract The objectives of this study were to find out the types of transitivity system used on students’ writing recount texts. Further, this study also aims to identify the types of transitivity systems realized in students’ writing recount texts. This study was conducted by using a descriptive qualitative design method. The data were 15 pieces of recount texts written by tenth grade students of SMAN 21 Medan being analyzed as the data. From this research, it was found that in all units of students’ writing recount texts, there were 249 clauses. In these clauses, all process types can be found with varieties of the usage, they were: material process, mental process, relational process, behavioral process, verbal process, and existential process. As for the participants, almost all can be found in students’ recount texts: actor, goal, range, senser, phenomenon, token, value, carrier, attribute, possessor, possessed, behaver, sayer, verbiage, existent, and beneficiary ones. Only two participants were not found in the texts: the range and the target ones. As for circumstances, there were seven circumstances found in the students’ recount texts: extent, location, manner, cause, accompaniment, and matter. The contingency, angle, and role aspects however, were not found. The realization of the transitivity system on students writing recount texts, were found with varieties of the usage in constructing language features of the students’ recount texts. The processes were realized by personal pronoun, direct and indirect objects. Finally, the circumstances were realized by prepositional and adverbial phrases. Keywords: Transitivity System, Writing, Students’ recount texts.