Dependency Syntax Research Articles

Извлечение отношений из NER-размеченных предложений для обучения онтологии

This paper is a continuation of the research on training and using a neural-network language model for ontology learning. The paper deals with the problem of analyzing domain natural language texts and extracting possible relations between concepts from them. A brief overview of existing approaches to solving this problem is presented. A procedure for extracting relations between concepts based on the analysis of the sentence syntax dependency tree is proposed. The results of evaluating the efficiency of extracting relations using this procedure are considered.

Transparent Aspect-Level Sentiment Analysis Based on Dependency Syntax Analysis and Its Application on COVID-19

Aspect-level sentiment analysis identifies fine-grained emotion for target words. There are three major issues in current models of aspect-level sentiment analysis. First, few models consider the natural language semantic characteristics of the texts. Second, many models consider the location characteristics of the target words, but ignore the relationships among the target words and among the overall sentences. Third, many models lack transparency in data collection, data processing, and results generating in sentiment analysis. In order to resolve these issues, we propose an aspect-level sentiment analysis model that combines a bidirectional Long Short-Term Memory (LSTM) network and a Graph Convolutional Network (GCN) based on Dependency syntax analysis (Bi-LSTM-DGCN). Our model integrates the dependency syntax analysis of the texts, and explicitly considers the natural language semantic characteristics of the texts. It further fuses the target words and overall sentences. Extensive experiments are conducted on four benchmark datasets, i.e., Restaurant14, Laptop, Restaurant16, and Twitter. The experimental results demonstrate that our model outperforms other models like Target-Dependent LSTM (TD-LSTM), Attention-based LSTM with Aspect Embedding (ATAE-LSTM), LSTM+SynATT+TarRep and Convolution over a Dependency Tree (CDT). Our model is further applied to aspect-level sentiment analysis on “government” and “lockdown” of 1,658,250 tweets about “#COVID-19” that we collected from March 1, 2020 to July 1, 2020. The experimental results show that Twitter users’ positive and negative sentiments fluctuated over time. Through the transparency analysis in data collection, data processing, and results generating, we discuss the reasons for the evolution of users’ emotions over time based on the tweets and on our models.

MS-Transformer: Introduce multiple structural priors into a unified transformer for encoding sentences

Transformers have been widely utilized in recent NLP studies. Unlike CNNs or RNNs, the vanilla Transformer is position-insensitive, and thus is incapable of capturing the structural priors between sequences of words. Existing studies commonly apply one single mask strategy on Transformers for incorporating structural priors while failing at modeling more abundant structural information of texts. In this paper, we aim at introducing multiple types of structural priors into Transformers, proposing the Multiple Structural Priors Guided Transformer (MS-Transformer) that transforms different structural priors into different attention heads by using a novel multi-mask based multi-head attention mechanism. In particular, we integrate two categories of structural priors, including the sequential order and the relative position of words. For the purpose of capturing the latent hierarchical structure of the texts, we extract these information not only from the word contexts but also from the dependency syntax trees. Experimental results on three tasks show that MS-Transformer achieves significant improvements against other strong baselines.

A Neural Machine Translation Model Based on Sequence to Dependency

A supervised self-attention network is proposed to introduce the dependency structure into the Transformer model. This scheme is mainly proposed based on the characteristics of the self-attention mechanism in Transformer, which converts the dependency syntax tree into two equivalent adjacency matrices, and then uses the adjacency matrix to supervise the self-attention network in the Transformer’s encoder. So that Transformer learns how to model the dependency structure of the source language. Although this scheme is simple, the effect is remarkable. In particular, there is no need to modify the network structure, as long as two additional loss functions are added in the process of training Transformer. In the decoding process, there is no need to use an external syntactic analyzer to perform syntactic analysis on the source language, but it can automatically construct the dependent syntactic structure of the source language. Experiments demonstrate that the quality of the constructed dependency syntactic structure is also good.

Morfoskładnia w Słowniku właściwych użyć języka

This article outlines the theoretical basis of describing morphosyntactic problems in The Dictionary of Proper Uses of Language. The main point was to 1) show the principles of the reference grammatical model and 2) to describe the main phases of the procedure of language facts evaluation. In the first case, we refer to the dependency syntax model, which reflects the relationships between the elements of syntactic structures through assigning them to a specific type of binding. In the second case, in contrast to the traditional nor-mative model, the importance of the frequency of the specific syntactic structures is much more underlined. Ultimately, the combination of prevalence and frequency of language elements allows for constructing an objective assessment procedure.

Independent Infinitive in Directive Speech Acts for Direct Communication in the Polish Language and Its Equivalents in the French Language

This article discusses the Polish independent infinitive, which constitutes a predicate in imperative utterances, and its French functional equivalents. The analysis was conducted at two levels. In the first part, the author describes the independent infinitive in the Polish language referring to the Polish formal structural syntax (Saloni, Świdziński 2012). This is to determine which place is occupied by this unit in a sentence, both in relation to other uses of the infinitive and in comparison to other units with the function of a predicate in statements of the same modality. The French structural equivalent has been determined on the basis of the same criterion of syntax dependency. However, even though both languages have corresponding structures, they do not use them in the same way. Only in Polish it is possible to form sentences with infinitive predicates in the spoken language, in face-to-face conversation. What are the factors that favour choosing this form? The author answers this question in her semantic and pragmatic analysis, conducted in the methodological framework of speech act theory (Searle 1979, Vanderveken 1988). She presents imperatives as a class of speech acts, which are extensively developed and specifies those, which can be executed by means of utterances with infinitive predicates. Additionally, factors of social and psychological character have been taken into consideration, as those which favour selecting the discussed form. What structure constitutes its functional equivalent in the French language? An analysis of a body composed of examples originating primarily from dialogues in contemporary literary works and their approved translations has allowed, on the one hand, to confirm the intuitive belief that grammar forms perform this function, in face-to-face oral communication the French language has only the command mode forms (l’impératif in French). On the other hand, we can launch a discussion about possibilities to translate them into a language which does not allow for an analogous use of the available infinitive structure.

Research on Grammar Checking System Using Computer Big Data and Convolutional Neural Network Constructing Classification Model

This article proposes an automatic grammatical correction method for typos and word order errors that may occur in Chinese writing by beginners. The thesis first constructs heuristic rules and expands the corpus by analyzing the characteristics of different grammatical error types in the data set. Secondly, when the paper uses classification methods to detect grammatical errors, it extracts sentence-level binary and ternary part-of-speech combinations, n-gram models based on part-of-speech statistics and other three types of features to construct single classification and ensemble classification models, and then use convolutional neural the network constructs classification models from different angles. Finally, when the paper adopts the method based on sequence labeling for grammatical error detection, it mainly uses dependency syntax tree features, and realizes grammatical error detection by constructing a conditional random field model. This method can automatically detect grammatical errors while also identifying the sentence the location of the error. On this basis, the paper implements a simple Chinese grammatical error automatic detection system, which can provide help for the optimization of questions and answers in the question-and-answer system

Image captioning based on dependency syntax

Image captioning based on dependency syntax

Research on Topic Mining of Medical Surgical Mask Reviews Sold on E-commerce Platform

Since the outbreak of the epidemic in 2020, the development potential of pharmaceutical e-commerce is huge. Mining and studying the review information of e-commerce platform to improve the dimensions of products is an important way for the healthy development of medical e-commerce. In this paper, the review information of medical surgical mask, a hot sales product of medical e-commerce, is extracted. TF-IDF algorithm, dependency syntax and other models are used to mine the topic information contained in the review, and then python is used to analyze the emotional polarity of the review, and finally the emotional assignments of the main review topics are obtained. The results show that consumers have higher satisfaction with the appearance of goods, but lower satisfaction with packaging. On the basis of the research results, corresponding reference suggestions are proposed for medical e-commerce.

Transition based neural network dependency parsing of Tibetan

In order to improve the performance of Tibetan natural language processing applications such as machine translation, sentiment analysis and other tasks, this article proposes a neural network-based method for syntactic analysis of Tibetan language dependence. Part of the corpus of Qinghai Normal University’s part-of-speech tag set is marked by the corresponding mapping relationship is transformed into the corpus annotated by the national standard part-of-speech tag set. At the same time, the CoNLL format Tibetan language dependency syntax tree library is constructed, and the method of shift-reduce plus neural network is adopted to systematically study and analyze the Tibetan language dependency syntax. Thereby improving the quality of Tibetan dependency syntactic analysis, and its accuracy rate reaches UAS:94.59%

Short Text Sentiment Analysis Based on Multi-Channel CNN With Multi-Head Attention Mechanism

In view of the limited text features of short texts, features of short texts should be mined from various angles, and multiple sentiment feature combinations should be used to learn the hidden sentiment information. A novel sentiment analysis model based on multi-channel convolutional neural network with multi-head attention mechanism (MCNN-MA) is proposed. This model combines word features with part of speech features, position features and dependency syntax features separately to form three new combined features, and inputs them into the multi-channel convolutional neural network, as well as integrates the multi-head attention mechanism to more fully learn the sentiment information in the text. Finally, experiments are carried out on two Chinese short text data sets. The experimental results show that the MCNN-MA model has a higher classification accuracy and a relatively low training time cost compared with other baseline models.

Investigating Typed Syntactic Dependencies for Targeted Sentiment Classification Using Graph Attention Neural Network

Targeted sentiment classification predicts the sentiment polarity on given target mentions in input texts. Dominant methods employ neural networks for encoding the input sentence and extracting relations between target mentions and their contexts. Recently, graph neural network has been investigated for integrating dependency syntax for the task, achieving the state-of-the-art results. However, existing methods do not consider dependency label information, which can be intuitively useful. To solve the problem, we investigate a novel relational graph attention network that integrates typed syntactic dependency information. Results on standard benchmarks show that our method can effectively leverage label information for improving targeted sentiment classification performances. Our final model significantly outperforms state-of-the-art syntax-based approaches.

An Arabic Grammar Auditor Based on Dependency Grammar

The Arabic language has many complex grammar rules that may seem complicated to the average user or learner. Automatic grammar checking systems can improve the quality of the text, reduce the costs of the proofreading process, and play a role in grammar teaching. This paper presents an initiative toward developing a novel and comprehensive Arabic auditor that can address vowelized texts. We called the “Arabic Grammar Detector” (AGD-أَجِــدْ). AGD was successfully implemented based on a dependency grammar and decision tree classifier model. Its purpose is to extract patterns of grammatical rules from a projective dependency graph in order to designate the appropriate syntax dependencies of a sentence. The current implementation covers almost all regular Arabic grammar rules for nonvowelized texts as well as partially or fully vowelized texts. AGD was evaluated using the Tashkeela corpus. It can detect more than 94% of grammatical errors and hint at their causes and possible corrections.

Dependency-based syntax-aware word representations

Dependency syntax has been demonstrated highly useful for a number of natural language processing (NLP) tasks. Typical approaches of utilizing dependency syntax include Tree-RNN and Tree-Linearization, both of which exploit explicit 1-best tree outputs from a well-trained parser as inputs. However, these approaches may suffer from error propagation due to the inevitable errors contained in the 1-best tree outputs. In this work, we propose a novel approach to integrate dependency syntax without using the discrete tree outputs. The key idea is to use the intermediate hidden representations of a well-trained encoder-decoder dependency parser, which are referred to as Dependency-based Syntax-Aware Word Representations (Dep-SAWRs). Then, we simply concatenate such Dep-SAWRs with the conventional context-insensitive word embeddings to compose input word representations, without requiring to modify the model architecture of the downstream tasks. We evaluate the proposed method on four kinds of typical NLP tasks, including sentence classification, sentence matching, sequence labeling and machine translation. Experimental results show that the proposed approach is highly promising. On the one hand, it can utilize dependency syntax effectively, bringing consistently better performance on the four tasks compared with baselines without using syntax. On the other hand, the proposed method can outperform the Tree-RNN and Tree-Linearization approaches in most settings, and meanwhile are highly efficient in syntax integration. In addition, the proposed method would be easily extendable to encoding other structural attributes of language.

Syntax grounded graph convolutional network for joint entity and event extraction

Event extraction aims to determine entity mentions, event triggers and argument roles in text, such as news articles. For this task, dependency syntax has been recognized as a valuable source of information. Previous work integrates dependency trees by using one-best results from a parser, where potential incorrect dependencies may impact the event extraction performance. We propose to use syntax features in an implicit approach, by adopting Soft Head Vectors (SHV) drawn from a well-trained parser as an adjacency matrix, densely connecting all words with the probabilistic head scores. SHV can be also regarded as dependency forests where multiple possible structures can be taken into account. A two-phase graph neural network is adopted to represent the forests, automatically differentiating the salient syntactic information from noisy parsers. Experiments on the two public datasets show that our model outperforms various baselines including graph-based one-best tree as well as recent transition-based decoding, giving the state-of-the-art results in the literature.

A NOVEL HYBRID MODEL BY USING CONVOLUTIONAL NEURAL NETWORK AND LONG SHORT-TERM MEMORY FOR TEXT SENTIMENT ANALYSIS

Machine learning has been widely applied in sentiment analysis. However, natural language has a structure-dependent relationship, relying on a single network for feature extraction, which limits the precision of text sentiment classification. To extract semantic features and perform classification effectively, the convolutional neural network and long short-term memory model (CNN–LSTM) was proposed in the study. The Word2vec method was employed to train the initial word vector. The CNN convolutional layer and the max-pooling layer were introduced to extract the local features of the text. The LSTM module was adopted to capture the long-term dependencies between word sequences. The features extracted with LSTM and CNN were fused, and efficiency was improved using dropout regularization technology. A case study was performed on the movie review dataset to analyze the classification effect utilizing the traditional and proposed methods. Results demonstrate that compared with the single CNN and LSTM models, the evaluation index F1 of this model improves by 1.93% and 0.97%, respectively. Compared with the randomly initialized feature vector, the precision of the model improves 2.74% by the word embedding.The proposed method can receive parallel input of the text information, which reduces the training time of the network model. The mechanism combining CNN and LSTM compensates for the shortcomings of relying only on word embedding for feature extraction, enabling the model to obtain in-depth sentiment feature and effectively-identified sentiment polarity without external knowledge such as dependency syntax analysis. The proposed method provides a specific reference for the sentiment classification of comment texts in social media. Keywords: Sentiment analysis, CNN, LSTM, Feature fusion

Relation Extraction Exploiting Full Dependency Forests

Dependency syntax has long been recognized as a crucial source of features for relation extraction. Previous work considers 1-best trees produced by a parser during preprocessing. However, error propagation from the out-of-domain parser may impact the relation extraction performance. We propose to leverage full dependency forests for this task, where a full dependency forest encodes all possible trees. Such representations of full dependency forests provide a differentiable connection between a parser and a relation extraction model, and thus we are also able to study adjusting the parser parameters based on end-task loss. Experiments on three datasets show that full dependency forests and parser adjustment give significant improvements over carefully designed baselines, showing state-of-the-art or competitive performances on biomedical or newswire benchmarks.

Relation Extraction with Convolutional Network over Learnable Syntax-Transport Graph

A large majority of approaches have been proposed to leverage the dependency tree in the relation classification task. Recent works have focused on pruning irrelevant information from the dependency tree. The state-of-the-art Attention Guided Graph Convolutional Networks (AGGCNs) transforms the dependency tree into a weighted-graph to distinguish the relevance of nodes and edges for relation classification. However, in their approach, the graph is fully connected, which destroys the structure information of the original dependency tree. How to effectively make use of relevant information while ignoring irrelevant information from the dependency trees remains a challenge in the relation classification task. In this work, we learn to transform the dependency tree into a weighted graph by considering the syntax dependencies of the connected nodes and persisting the structure of the original dependency tree. We refer to this graph as a syntax-transport graph. We further propose a learnable syntax-transport attention graph convolutional network (LST-AGCN) which operates on the syntax-transport graph directly to distill the final representation which is sufficient for classification. Experiments on Semeval-2010 Task 8 and Tacred show our approach outperforms previous methods.

Dependency Treebanks of Ancient Greek Prose

This dataset is a collection of dependency syntax trees of representative texts from ancient Greek prose authors (Aeschines, Antiphon, Appian, Athenaeus, Demosthenes, Dionysius of Halicarnassus, Herodotus, Josephus, Lysias, Plutarch, Polybius, Thucydides, and Xenophon), totaling to date 550,000+ tokens. It is hand-annotated by one person, using the Arethusa program on the Perseids website. Original texts were obtained from the Perseus Digital Library, and some (as indicated) were computer pre-parsed at the Pedalion Project. The database is stored in a stable form (2019-12-31) on Zenodo (DOI: 10.5281/zenodo.3596076 ) and in a continuously updated form on GitHub in .xml format ( https://vgorman1.github.io/ ). The repository can be used for pedagogical purposes and for research in linguistics analysis and corpus linguistics, stylistics, natural language processing, classification, and literary and historical analysis.

Towards a dependency treebank for Spanish Sign Language (LSE)

This paper presents the foundations, procedures, tests and first results of a dependency treebank of the Spanish Sign Language (LSE). Dependency syntax offers many advantages over other alternatives for the systematic and exhaustive syntactic analysis of a corpus. Nevertheless, the visual modality that is characteristic of sign languages poses unique challenges for their syntactic analysis, among which the most prominent is the simultaneity of expression: both hands, face and other non-manual components. Taking into account these and other particularities of sign languages, the paper explores the main difficulties faced when one tries to apply some usual categories and relations from the syntactic analysis of spoken and written languages to LSE.