Tagging Scheme Research Articles

Neural multi-task learning for end-to-end Arabic aspect-based sentiment analysis

Most existing aspect-based sentiment analysis (ABSA) methods perform the tasks of aspect extraction and sentiment classification independently, assuming that the aspect terms are already determined when handling the aspect sentiment classification task. However, such settings are neither practical nor appropriate in real-life applications, as aspects must be extracted prior to sentiment classification. This study aims to overcome this shortcoming by jointly identifying aspect terms and the corresponding sentiments using a multi-task learning approach based on a unified tagging scheme. The proposed model uses the Bidirectional Encoder Representations from Transformers (BERT) model to produce the input representations, followed by a Bidirectional Gated Recurrent Unit (BiGRU) layer for further contextual and semantic coding. An attention layer is added on top of BiGRU to force the model to focus on the important parts of the sentence. Finally, a Conditional Random Fields (CRF) layer is used to handle inter-label dependencies. Experiments conducted on a reference Arabic hotel dataset show that the proposed model significantly outperforms the baseline and related work models.

Label-semantics enhanced multi-layer heterogeneous graph convolutional network for Aspect Sentiment Quadruplet Extraction

Aspect-based sentiment analysis (ABSA) is a pivotal area within natural language processing (NLP) that focuses on extracting fine-grained sentiment information from text. A particularly demanding task within ABSA is Aspect Sentiment Quadruplet Extraction (ASQE), which aims to identify quadruplets comprising aspect terms, their corresponding opinion terms, sentiment polarity, and categories. This level of analysis is crucial for downstream applications such as sentiment monitoring and user experience research, offering nuanced insights into textual data. However, current methodologies fall short of fully leveraging the rich linguistic features of sentences and the semantic information of labels embedded within sentences. To address this gap, this paper introduces a novel framework, the Label-Semantics Enhanced Multi-layer Heterogeneous Graph Convolutional Network (LSEMH-GCN), specifically designed for ASQE. Our model integrates sentence linguistic features and label semantics to construct a graph neural network tailored for this task. It employs a multi-layer graph convolutional network that synergizes various linguistic features, and utilizes Biaffine attention to enrich the label probability distribution for token pairs with semantic label information. Furthermore, our approach introduces a token pair vector concatenation strategy alongside an advanced asymmetric label tagging scheme to enhance quadruplet extraction. Comprehensive evaluations on benchmark datasets reveal that LSEMH-GCN significantly surpasses existing state-of-the-art models, establishing a new benchmark for ASQE. Our model achieves an average F1 score improvement of 15.52% and 12.30% on the Restaurant-ACOS and Laptop-ACOS datasets, respectively.

Double Cyclization Tandem Mass for Identification and Quantification of Phosphatidylcholines Using Isobaric Six-Plex Capillary nLC-MS/MS.

Multiplexing of phosphatidylcholine analysis is hindered by a lack of appropriate derivatization. Presented here is a tagging scheme that uses a quaternary amine tag and targets the hydroxy group of the phosphate, which switches the net charge from neutral to +2. Quantitative yields were achieved from >99% reaction completion derived by dimethoxymethyl morpholinium (DMTMM) activation. Fragmentation of phosphatidylcholines (PCs) and lysophosphatidylcholines (LPCs) releases two trimethylamines and the acyl chains through neutral loss and generates a unique double cyclization constant mass reporter. Selective incorporation of isotopes onto the tag produces a six-plex set of isobaric reagents. For equivalent six-plex-labeled samples, <14% RSD was achieved, followed by a dynamic range of 1:10 without signal compression. Quantification of PCs/LPCs in human hepatic cancer cells was conducted as six-plex using data-dependent analysis tandem MS. We report a six-plex qualitative and quantitative isobaric tagging strategy expanding the limits of analyzing PCs/LPCs.

Progressive Multigranularity Information Propagation for Coupled Aspect-Opinion Extraction.

Coupled aspect-opinion extraction aims to identify aspect-opinion pairs in the form of (aspect term, opinion term) or triplets in the form of (aspect term, opinion term, sentiment polarity) from user-generated texts. Compared to the traditional aspect-based sentiment prediction or extraction tasks, coupled aspect-opinion extraction needs to associate aspects with their corresponding opinions and organize opinion-related information into structured outputs. The existing works either divide this task into subproblems (i.e., term extraction and relation prediction) or utilize a unified tagging scheme. However, these methods only focus on atomic word-level interactions and ignore the intensive information propagation among different granularities (e.g., words and word pairs). To address this limitation, we propose a progressive multigranularity information propagation network that progressively explores three types of correlations with different granularities. Specifically, our model starts with the most basic word-level correlations by composing all possible word pairs. In the second stage, the pairwise relation information is used to update the word features. The last stage propagates information among word pairs to produce the relation scores. We treat the task as a unified relation prediction problem and construct an end-to-end framework that iteratively conducts the three-stage information propagation to refine the textual representations. Comprehensive experiments on different aspect-based sentiment analysis benchmarks clearly demonstrate the effectiveness of the proposed approach.

One-pot method for preparing DNA, RNA, and protein for multiomics analysis

Typical multiomics studies employ separate methods for DNA, RNA, and protein sample preparation, which is labor intensive, costly, and prone to sampling bias. We describe a method for preparing high-quality, sequencing-ready DNA and RNA, and either intact proteins or mass-spectrometry-ready peptides for whole proteome analysis from a single sample. This method utilizes a reversible protein tagging scheme to covalently link all proteins in a lysate to a bead-based matrix and nucleic acid precipitation and selective solubilization to yield separate pools of protein and nucleic acids. We demonstrate the utility of this method to compare the genomes, transcriptomes, and proteomes of four triple-negative breast cancer cell lines with different degrees of malignancy. These data show the involvement of both RNA and associated proteins, and protein-only dependent pathways that distinguish these cell lines. We also demonstrate the utility of this multiomics workflow for tissue analysis using mouse brain, liver, and lung tissue.

Extraction and attribution of public figures statements for journalism in Indonesia using deep learning

News articles are usually written by journalists based on statements taken from interviews with public figures. Attribution from such statements provides important information and it can be extracted from news articles to build a knowledge base by developing a sequential tagging scheme such as entity recognition. This research applies two deep learning architectures: recurrent neural networks-based and transformer-based, to establish public figures statement attribution and extraction models in the Indonesian Language. The experiments are conducted using five deep-learning model architectures with two different corpus sizes to investigate the impact of corpus size on each model's performance. The experiments show that the best model for the RNN-based architecture is PFSA-ID-BLWCA which achieves 81.34 % F1 score, and the best model for the transformer-based is PFSA-ID-TWCA which obtains 81.01 % F1 score. This research also discovers that the size of the corpus influences the model performances. Furthermore, the study lays a foundation to overcome the attribution extraction in another language, especially low-resource languages, with some necessary adjustments.

A Transfer Learning-Based Pairwise Information Extraction Framework Using BERT and Korean-Language Modification Relationships

Most named entity recognition approaches employing BERT-based transfer learning focus solely on extracting independent and simple tags, neglecting the sequence and dependency features inherent in the named-entity tags. Consequently, these basic BERT-based methods fall short in domains requiring the extraction of more intricate information, such as the detailed characteristics of products, services, and places from user reviews. In this paper, we introduce an end-to-end information extraction framework comprising three key components: (1) a tagging scheme that effectively represents detailed characteristics; (2) a BERT-based transfer learning model designed for extracting named-entity tags, utilizing both general linguistic features learned from a large corpus and the sequence and symmetric-dependency features of the named-entity tags; and (3) a pairwise information extraction algorithm that pairs features with their corresponding symmetric modifying words to extract detailed information.

Aspect sentiment quadruple extraction based on the sentence-guided grid tagging scheme

Aspect sentiment quadruple extraction based on the sentence-guided grid tagging scheme

STAGE: Span Tagging and Greedy Inference Scheme for Aspect Sentiment Triplet Extraction

Aspect Sentiment Triplet Extraction (ASTE) has become an emerging task in sentiment analysis research, aiming to extract triplets of the aspect term, its corresponding opinion term, and its associated sentiment polarity from a given sentence. Recently, many neural networks based models with different tagging schemes have been proposed, but almost all of them have their limitations: heavily relying on 1) prior assumption that each word is only associated with a single role (e.g., aspect term, or opinion term, etc. ) and 2) word-level interactions and treating each opinion/aspect as a set of independent words. Hence, they perform poorly on the complex ASTE task, such as a word associated with multiple roles or an aspect/opinion term with multiple words. Hence, we propose a novel approach, Span TAgging and Greedy infErence (STAGE), to extract sentiment triplets in span-level, where each span may consist of multiple words and play different roles simultaneously. To this end, this paper formulates the ASTE task as a multi-class span classification problem. Specifically, STAGE generates more accurate aspect sentiment triplet extractions via exploring span-level information and constraints, which consists of two components, namely, span tagging scheme and greedy inference strategy. The former tag all possible candidate spans based on a newly-defined tagging set. The latter retrieves the aspect/opinion term with the maximum length from the candidate sentiment snippet to output sentiment triplets. Furthermore, we propose a simple but effective model based on the STAGE, which outperforms the state-of-the-arts by a large margin on four widely-used datasets. Moreover, our STAGE can be easily generalized to other pair/triplet extraction tasks, which also demonstrates the superiority of the proposed scheme STAGE.

KinRob: An ontology based robot for solving kinematic problems

Intelligent answering technology, which enables computers to solve problems automatically, is often used to develop tutorial systems, and has a wide range of application prospects. However, due to the lack of linguistic analysis and understanding methods, there are few researches on intelligent algorithms for solving kinematics problems. Developing such an algorithm is challenging, because solving kinematics problems is a complex task that includes text understanding, problem analysis, and automatic solution. To understand all these complexities involved in kinematics problems requires background knowledge. And only when an automatic solver contains a powerful internal knowledge representation system can it perform these tasks. We, thus, develop KinRob, an tutorial system for solving kinematics problems by combining neural network and ontology. Firstly, we propose an ontology for KinRob, which defines the knowledge of kinematics, and can help the robot understand a kinematics problem. Secondly, to match the text in natural language with the ontology, we propose a novel tagging scheme based on the kinematic problem understanding model in named entity recognition (NER). Finally, extensive experiments are conducted, and the experimental results show that the performance of the proposed method on a dataset of kinematic problems from authoritative sources better than the baseline algorithms.

A Token Classification-Based Attention Model for Extracting Multiple Emotion–Cause Pairs in Conversations

People exchange emotions through conversations with others and provide different answers depending on the reasons for their emotions. During a conversation, it is important to find not only such emotions but also their cause. Emotion-cause pair extraction (ECPE) is a task used to determine emotions and their causes in a single pair within a text, and various studies have been conducted to accomplish ECPE tasks. However, existing studies have limitations in that some models conduct the task in two or more steps, whereas others extract only one emotion-cause pair for a given text. We propose a novel methodology for extracting multiple emotion-cause pairs simultaneously from a given conversation with a single model. Our proposed model is a token-classification-based emotion-cause pair extraction model, which applies the BIO (beginning-inside-outside) tagging scheme to efficiently extract multiple emotion-cause pairs in conversations. The proposed model showed the best performance on the RECCON benchmark dataset in comparative experiments with existing studies and was experimentally verified to efficiently extract multiple emotion-cause pairs in conversations.

Green patents and green codes: How different methodologies lead to different results

Green patents are valid instruments to measure eco-innovation which aims at reducing the negative impact on the environment and providing an efficient use of resources. There are three methodologies available to identify green patents based on the code classification: ENV-TECH (developed by OECD), IPC Green Inventory (WIPO) and Y02/Y04S Tagging scheme (EPO). Our results are: the systematic organisation of green codes for each methodology and the development of algorithms for their periodic update; the application of these methodologies to two large datasets; the comparison of the three methodologies. The relevant findings are that ENV-TECH and IPC Green Inventory should be used in combination to identify more green patents, with the inclusion of the "CPC" ENV-TECH green codes when applicable. The Tagging scheme identifies fewer green patents than the combination of the other two methodologies. The three methodologies overlap only partially (in 22.47% of cases) and their mutual integration is recommended.

A Deep-Learning Framework for Analysing Students' Review in Higher Education.

As part of continuous process improvements to teaching and learning, the management of tertiary institutions requests students to review modules towards the end of each semester. These reviews capture students' perceptions about various aspects of their learning experience. Considering the large volume of textual feedback, it is not feasible to manually analyze all the comments, hence the need for automated approaches. This study presents a framework for analyzing students' qualitative reviews. The framework consists of four distinct components: aspect-term extraction, aspect-category identification, sentiment polarity determination, and grades' prediction. We evaluated the framework with the dataset from the Lilongwe University of Agriculture and Natural Resources (LUANAR). A sample size of 1,111 reviews was used. A microaverage F1-score of 0.67 was achieved using Bi- LSTM-CRF and BIO tagging scheme for aspect-term extraction. Twelve aspect categories were then defined for the education domain and four variants of RNNs models (GRU, LSTM, Bi-LSTM, and Bi-GRU) were compared. A Bi-GRU model was developed for sentiment polarity determination and the model achieved a weighted F1-score of 0.96 for sentiment analysis. Finally, a Bi-LSTM-ANN model which combined textual and numerical features was implemented to predict students' grades based on the reviews. A weighted F1-score of 0.59 was obtained, and out of 29 students with "F" grade, 20 were correctly identified by the model.

HIM: An End-to-End Hierarchical Interaction Model for Aspect Sentiment Triplet Extraction

Aspect Sentiment Triplet Extraction (ASTE) is an emerging task of fine-grained sentiment analysis, which aims to extract aspect terms, associated opinion terms, and sentiment polarities in the form of triplets. Thus, ASTE involves two groups of subtasks: aspect/opinion term extraction and aspect-opinion-pair sentiment classification. Due to the high correlations of subtasks, three categories of joint methods have been proposed, including <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">end-to-end tagging-based methods</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">cascaded span-based methods</i> , and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sequence-to-sequence generation-based methods</i> . These methods basically learn either a shared feature space or a shared sentence encoder to capture interactions across all subtasks by parameter sharing. However, they fail to learn deep and mutual interactive features for ASTE. In this work, we present a novel tagging scheme to cast ASTE as a unified boundary-words relation classification problem. Subsequently, we propose an end-to-end Hierarchical Interaction Model (HIM), exploiting deep and mutual interactions across subtasks mainly with two interaction modules. The first-level interaction module primarily leverages multi-task learning models to capture implicit subtask interactions. Then, the second-level interaction module, namely Gated Interaction Network (GIN), adopts a novel gated control mechanism and a newly-designed Conditional BiLSTM (Cond-BiLSTM) network to capture explicit subtask interactions. Moreover, to refine the unreliable outputs of the first-level module, we develop a General word-Pair Relationship Learning (G-PRL) component. With the task-shared features as input, G-PRL further facilitates interactions between term extraction and pair classification. We conduct experiments on two benchmarks and achieve promising results. Extensive analyses demonstrate the effectiveness and flexibility of our work.

Evaluation of end-to-end aspect-based sentiment analysis methods employing novel benchmark dataset for aspect, and opinion review analysis

Aspect-based sentiment analysis (ABSA) deals with the determination of sentiments for opinion targets. While historically this research task has been addressed with pipeline approaches, more recent works use neural networks to jointly deal with the aspect term and opinion term extraction, as well as the polarity classification. Although learned together, most NN-based approaches and all pipeline approaches do not model correlations between the tasks. This is also based on the absence of adequate datasets which are annotated for all sub-tasks in a unified tagging scheme. We address this bottleneck and introduce the first purposely designed and annotated dataset for ABSA. The DAORA dataset covers 2,100 Tripadvisor reviews, and it is annotated on aspect terms, opinion terms, as well as aspect term polarity, using a unified tagging scheme. It was designed especially for end-to-end aspect-based sentiment analysis of real-world reviews and does not use any sentence repetition or removal. We evaluate the DAORA dataset in several experiments employing state-of-the-art models for ABSA. We set benchmarks and analyze the strengths as well as weaknesses of the data and approaches.

Dependency graph enhanced interactive attention network for aspect sentiment triplet extraction

Aspect sentiment triplet extraction is an extremely daunting task designed to identify the triplets from comments, where each triplet is composed of an aspect term, the related opinion term, and the sentiment between them. Existing research efforts majorly construct a novel tagging scheme to avoid the disadvantages of pipeline methods. However, the improvement is limited due to neglecting the implicit grammatical relationships among the three elements in a triplet. To cope with this limitation, we put forward an innovative Dependency Graph Enhanced Interactive Attention Network, which explicitly introduces the syntactic and semantic relationships between words. Specifically, an interactive attention mechanism is conceived to jointly consider both the contextual features learned from Bi-directional Long Short-Term Memory and the syntactic dependencies learned from the correspondent dependency graph in an iterative interaction manner. In addition, we notice that words with different Part-of-Speech categories have different contributions to the semantic expression of sentences. Accordingly, the information of different Part-of-Speech categories is recognized during the modeling process to properly capture the semantic relationships. Experiments on the benchmark datasets originally derived from SemEval Challenges illustrate that our presented approach has superiority over strong baselines.

Aspect term extraction via information-augmented neural network

Aspect term extraction (ATE) aims at identifying the aspect terms that are expressed in a sentence. Recently, Seq2Seq learning has been employed in ATE and significantly improved performance. However, it suffers from some weaknesses, such as lacking the ability to encode the more informative information and integrate information of surrounding words in the encoder. The static word embeddings employed in ATE fall short of modeling the dynamic meaning of words. To alleviate the problems mentioned above, this paper proposes the information-augmented neural network (IANN) which is a novel Seq2Seq learning framework. In IANN, a specialized neural network is developed as the key module of the encoder, named multiple convolution with recurrence network (MCRN), to encode the more informative information and integrate information of surrounding words in the encoder. The contextualized embedding layer is designed to capture the dynamic word sense. Besides, the novel AO ({Aspect, Outside}) tags are proposed as the less challenging tagging scheme. A lot of experiments have been performed on three widely used datasets. These experiments demonstrate that the proposed IANN acquires state-of-the-art results and validate that the proposed IANN is a powerful method for the ATE task.

Stepwise Multi-Task Learning Model for Holder Extraction in Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis is a text analysis technique that categorizes data by aspect and identifies the sentiment attributed to each one and a task for a fine-grained sentiment analysis. In order to accurately perform a fine-grained sentiment analysis, a sentiment word within a text, a target it modifies, and a holder who represents the sentiment word are required; however, they should be extracted in sequence because the sentiment word is an important clue for extracting the target, which is key evidence of the holder. Namely, the three types of information sequentially become an important clue. Therefore, in this paper, we propose a stepwise multi-task learning model for holder extraction with RoBERTa and Bi-LSTM. The tasks are sentiment word extraction, target extraction, and holder extraction. The proposed model was trained and evaluated under Laptop and Restaurant datasets in SemEval 2014 through 2016. We have observed that the performance of the proposed model was improved by using stepwised features that are the output of the previous task. Furthermore, the generalization effect has been observed by making the final output format of the model a BIO tagging scheme. This can avoid overfitting to a specific domain of the review text by outputting BIO tags instead of the words.

A Unified Knowledge Extraction Method Based on BERT and Handshaking Tagging Scheme

In the actual knowledge extraction system, different applications have different entity classes and relationship schema, so the generalization and migration ability of knowledge extraction are very important. By training a knowledge extraction model in the source domain and applying the model to an arbitrary target domain directly, open domain knowledge extraction technology becomes crucial to mitigate the generalization and migration ability issues. Traditional knowledge extraction models cannot be directly transferred to new domains and also cannot extract undefined relation types. In order to deal with the above issues, in this paper, we proposed an end-to-end Chinese open-domain knowledge extraction model, TPORE (Extract Open-domain Relations through Token Pair linking), which combined BERT with a handshaking tagging scheme. TPORE can alleviate the nested entities and nested relations issues. Additionally, a new loss function that conducts a pairwise comparison of target category score and non-target category score to automatically balance the weight was adopted, and the experiment results indicate that the loss function can bring speed and performance improvements. The extensive experiments demonstrate that the proposed method can significantly surpass strong baselines. Specifically, our approach can achieve new state-of-the-art Chinese open Relation Extraction (ORE) benchmarks (COER and SAOKE). In the COER dataset, F1 increased from 66.36% to 79.63%, and in the SpanSAOKE dataset, F1 increased from 46.0% to 54.91%. In the medical domain, our method can obtain close performance compared with the SOTA method in the CMeIE and CMeEE datasets.

Hybrid neural tagging model for open relation extraction

Open Relation Extraction (ORE) task remains a challenge to obtain a semantic representation by discovering arbitrary relations from the unstructured text. Conventional methods heavily depend on feature engineering or syntactic parsing, which are inefficient or error-cascading. Recently, leveraging supervised deep learning methods to address the ORE task is a promising way. However, there are two main challenges: (1) The lack of enough labeled corpus to support supervised training; (2) The exploration of specific neural architecture that adapts to the characteristics of open relation extracting. In this paper, we build a large-scale, high-quality training corpus in a fully automated way. And wedesign a tagging scheme to assist in transforming the ORE task into a sequence tagging processing. Furthermore, we propose a hybrid neural network model (HNN4ORT) for open relation tagging. The model employs the Ordered Neurons LSTM to encode potential syntactic information to capture the associations among the arguments and relations. It also emerges a novel Dual Aware Mechanism, including Local-aware Attention and Global-aware Convolution. The dual awarenesses complement each other. Takes the sentence-level semantics as a global perspective, and at the same time, the model implements salient local features to achieve sparse annotation. Experiment results on various testing sets show that our model achieves state-of-the-art performance compared toconventional methods or other neural models.