Dependency Tree Research Articles

Leader-Based Community Detection Algorithm in Attributed Networks

The community structure plays an indispensable role in developing the deep structure of complex networks. In recent years, some researchers have realized the importance of leader nodes in the community detection process. However, most of the existing leader-based algorithms only use the topological information of networks or attribute information to supplement the topological information, resulting in a significant loss of information integrity. In this paper, we propose a leader-based method that combines topological and attribute information (TALB), uses attribute information among nodes in the network to construct an attribute similarity matrix, and then combines it with network topological information to establish dependency relationships among nodes. As a result, a dependency tree is formed, and the final result of the community division is obtained. Experiments on synthetic networks and real networks show that our proposed method is more effective and practical than the existing leader-based algorithms.

Syntax Vector Learning Using Correspondence for Natural Language Understanding

Natural language understanding (NLU) is a core technique for implementing natural user interfaces. In this study, we propose a neural network architecture to learn syntax vector representation by employing the correspondence between texts and syntactic structures. For representing the syntactic structures of sentences, we used three methods: dependency trees, phrase structure trees, and part of speech tagging. A pretrained transformer is used to propose text-to-vector and syntax-to-vector projection approaches. The texts and syntactic structures are projected onto a common vector space, and the distances between the two vectors are minimized according to the correspondence property to learn the syntax representation. We conducted massive experiments to verify the effectiveness of the proposed methodology using Korean corpora, i.e., Weather, Navi, and Rest, and English corpora, i.e., the ATIS, SNIPS, Simulated Dialogue-Movie, Simulated Dialogue-Restaurant, and NLU-Evaluation datasets. Through the experiments, we concluded that our model is quite effective in capturing a syntactic representation and the learned syntax vector representations are useful.

A Graph Convolutional Network With Multiple Dependency Representations for Relation Extraction

Dependency analysis can assist neural networks to capture semantic features within a sentence for entity relation extraction (RE). Both hard and soft strategies of encoding dependency tree structure have been developed to balance the beneficial extra information against the unfavorable interference in the task of RE. A wide application of graph convolutional network (GCN) in the field of natural language processing (NLP) has demonstrated its effectiveness in encoding the input sentence with the dependency tree structure, as well as its efficiency in parallel computation. This study proposes a novel GCN-based model using multiple representations to depict the dependency tree from various perspectives, and combines those dependency representations afterward to obtain a better sentence representation for relation classification. This model can maximally draw from the sentence the semantic features relevant to the relationship between entities. Results show that our model achieves state-of-the-art performance in terms of the F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> score (68.0) on the Text Analysis Conference relation extraction dataset (TACRED). In addition, we verify that the renormalization parameter in the GCN operation should be carefully chosen to help GCN-based models achieve its best performance.

Research on syntactic dependency tree and Ontology constraint in remote Supervising relation extraction

At present, research on remote-supervised relationship extraction fail to fully consider the position information among words in a sentence, and the extracted relationship may be inconsistent with the actual relationship. Therefore, this paper combines syntactic dependency tree and ontological constraint to carry out remote-supervised relationship extraction. The syntactic dependency tree is used to obtain the position weight of each word in the sentence and the domain ontology constraint is introduced to improve the accuracy of the extraction relationship. Experiments on Freebase+NYT data set show that this model can effecttively reduce the noise interference of wrong labels, and the model improves the accuracy by 2% compared with other reference models, thus better realizing relation extraction and laying a relevant foundation for the construction of high-precision knowledge map.

Effectiveness of Syntactic Dependency Information for Higher-Order Syntactic Attention Network

Recently, as a replacement of syntactic tree-based approaches, such as tree-trimming, Long Short-Term Memory (LSTM)-based methods have been commonly used to compress sentences because LSTM can generate fluent compressed sentences. However, the performance of these methods degrades significantly while compressing long sentences because they do not explicitly handle long-distance dependencies between the words. To solve this problem, we proposed a higher-order syntactic attention network (HiSAN) that can handle higher-order dependency features as an attention distribution on LSTM hidden states. Furthermore, to avoid the influence of incorrect parse results, we trained HiSAN by maximizing the probability of a correct output together with the attention distribution. Experiments on the Google sentence compression dataset show that our method improved the performance from baselines in terms of F1 as well as ROUGE-1, -2, and -L scores. In subjective evaluations, HiSAN outperformed baseline methods in both readability and informativeness. Besides, in this study, we additionally investigated the performance of HiSAN after training it without any syntactic dependency tree information. The results of our investigation show that HiSAN can compress sentences without relying on any syntactic dependency information while maintaining accurate compression rates, and also shows the effectiveness of syntactic dependency information in compressing long sentences with higher F1 scores.

Pattern-based bootstrapping framework for biomedical relation extraction

The progress made in the realm of ‘-omics’ technologies has led to a tremendous increase in the quantum of biomedical research published. Information extraction from this huge unstructured mass of data needs automation through text mining methods. Biomedical relation extraction is one such vital automation processes for extracting biomedical relations hidden in scientific literature. In the recent past, several supervised machine learning methods have been used to identify biomedical relations. However, given the variations in textual expression, huge corpus size and small task-specific training data, semi-supervised techniques appear to perform better. To this end, we propose a system that uses the semi-supervised bootstrapping algorithm to extract biomedical relations from text. The unlabelled corpus used contains sentences with biomedical entities represented as patterns with the dependency tree feature. Bootstrapping starts with a seed set and iteratively learns new patterns from the unlabelled corpus. We have designed a three-level masking technique to generate new patterns, and incorporated three types of scoring to help select appropriate patterns. The pattern-based bootstrapping method performs well with a minimum seed set. The system is able to extract 37,450 patterns from the unlabelled corpus that represents different biomedical relations. These patterns, in turn, are able to identify 460,886 relation pairs with 1327 single, and 1012 coupled, trigger words that convey the semantics of the biomedical relation. More than 64% of the identified relations have evidence in the CTD database.

Using Reproducible Data Visualizations to Augment Decision-Making During Suppression of Small Counts

IntroductionWhen reporting disease rates to the public, a health system must take precaution to protect released data from re-identification risks. While specific guidelines and methods vary across data systems and governances 1 , redaction of cells with small values is a key component in any approach for preparing data for public release. These preparations, when conducted manually, have proven to be arduous, time consuming, and prone to human error. Although finding a “small” value (e.g. “&lt; 5 ” ) is straightforward, detecting conditions in which suppressed values could be recalculated from related cells involves human judgement.&#x0D; Objectives and ApproachGuided by the real-world objective to reports the rates of chronic diseases in British Columbia, we aimed to design a reproducible workflow that would augment human decision-making and offer a nimble quality control tool, approachable by epidemiologists without technical background. Our workflow (1) splits data into disease-by-year data frames of a specific form, (2) applies a sequence of algorithms trained to recognize conditions that made recalculation of suppressed values possible and (3) prints a graph for each case of suggested automatic redaction to be confirmed by a human.&#x0D; ResultsThe augmented suppression system was successfully integrated into the maintenance of Chronic Disease Dashboard, an online reporting tool of the Observatory for Population and Public Health designed to address the gap in surveillance of chronic diseases in British Columbia. Anticipating the evolution of suppression logic, we isolated the logical tests responsible for redaction and provided several options to vary the degree of preserved information.&#x0D; Conclusion / ImplicationsInstead of employing a complex generalizable solution, we make a case for organizing the procedure for small cell redaction as a data visualization task, allowing for straightforward quality control of suppression decision and thus more approachable to a non-technical audience, as well as for employing such learning devices as workflow maps and function dependency trees for structuring applied projects and ensuring their reproducibility.

Distant Supervision Relation Extraction via adaptive dependency-path and additional knowledge graph supervision

Relation Extraction systems train an extractor by aligning relation instances in Knowledge Base with a large amount of labeled corpora. Since the labeled datasets are very expensive, Distant Supervision Relation Extraction (DSRE) utilizes rough corpus annotated with Knowledge Graph to reduce the cost of acquisition. Nevertheless, the data noise problem limits the performance of the DSRE. Dependency trees can be used to filter the wrong-labeled instances in the distant supervision bag. However, existing dependency tree relation extraction strategies are all based on manually-set paths between the subject and object entities, and suffer from the problem of pruning the trees too aggressively or too insufficiently. To circumvent the shortcomings, in this paper, we propose a novel DSRE framework A2DSRE, based on the Adaptive dependency-path and Additional KG supervision. To obtain the dependency paths related to entity relations adaptively, we introduce an advanced graph neural network—GeniePath into DSRE, which assigns higher weights to those direct neighbor words that contribute more to relation prediction through breadth exploration, and conducts depth exploration to determine the correlation between relations and high-order neighbors. In this way, the irrelevant nodes are pruned while the relevant nodes are kept, our method can obtain more appropriate paths associated with relations. At the same time, to further reduce the noises in the data, we incorporate additional supervision information from the knowledge graph by retracting the margin between the representation of the bag and the pre-training knowledge graph embedding. Extensive numerical experiments validate the effectiveness of our new method.

Semantic Enhanced Distantly Supervised Relation Extraction via Graph Attention Network

Distantly Supervised relation extraction methods can automatically extract the relation between entity pairs, which are essential for the construction of a knowledge graph. However, the automatically constructed datasets comprise amounts of low-quality sentences and noisy words, and the current Distantly Supervised methods ignore these noisy data, resulting in unacceptable accuracy. To mitigate this problem, we present a novel Distantly Supervised approach SEGRE (Semantic Enhanced Graph attention networks Relation Extraction) for improved relation extraction. Our model first uses word position and entity type information to provide abundant local features and background knowledge. Then it builds the dependency trees to remove noisy words that are irrelevant to relations and employs Graph Attention Networks (GATs) to encode syntactic information, which also captures the important semantic features of relational words in each instance. Furthermore, to make our model more robust against noisy words, the intra-bag attention module is used to weight the bag representation and mitigate noise in the bag. Through extensive experiments on Riedel New York Times (NYT) and Google IISc Distantly Supervised (GIDS) datasets, we demonstrate SEGRE’s effectiveness.

A Fine-grained Sentiment Analysis Method Based on Dependency Tree and Graph Attention Network

With the rapid development of the Internet, e-commerce and social media have continued to develop and grow. Merchants tend to better understand users’ attitudes and emotional tendencies through comments. As the number of user reviews increases, the previous sentiment analysis methods have highlighted the problems of high cost and high error rate. How to use more advanced methods to analyse comments has become an urgent problem to be solved. In order to solve the above problems, in this article, we propose a fine-grained sentiment analysis method based on dependency tree and graph neural network, which can help businesses and social network platforms to identify users’ sentiment tendencies and can be subsequently used in recommendation systems and public opinion analysis systems. The experimental results show that our scheme has achieved the best results.

Tree Rotations for Dependency Trees: Converting the Head-Directionality of Noun Phrases

To overcome the lack of NLP resources for the low-resource languages, we can utilize tools that are already available for other highresource languages and then modify the output to conform to the target language. In this study, we proposed an approach to convert an Indonesian constituency treebank to a dependency treebank by utilizing an English NLP tool (Stanford CoreNLP) to create the initial dependency treebank. Some annotations in this initial treebank did not conform to Indonesian grammar, especially noun phrases’ head-directionality. Noun phrases in English usually have head-final direction, while in Indonesian is the opposite, head-initial. We proposed a variant of tree rotations algorithm named headSwap for dependency trees. We used this algorithm to convert the head-directionality for noun phrases that were initially labeled as a compound. Moreover, we also proposed a set of rules to rename the dependency relation labels to conform to the recent guidelines. To evaluate our proposed method, we created a gold standard of 2,846 tokens that were annotated manually. Experiment results showed that our proposed method improved the Unlabeled Attachment Score (UAS) with a margin of 32.5% from 61.6 to 94.1% and the Labeled Attachment Score (LAS) with a margin of 41% from 44.1 to 85.1%. Finally, we created a new Indonesian dependency treebank that converted automatically using our proposed method that consists of 25,416 tokens. The dependency parser model built using this treebank has UAS of 75.90% and LAS of 70.38%.

Multi-head self-attention based gated graph convolutional networks for aspect-based sentiment classification

Aspect-based sentiment classification aims to predict the sentiment polarity of specific aspects appeared in a sentence. Nowadays, most current methods mainly focus on the semantic information by exploiting traditional attention mechanisms combined with recurrent neural networks to capture the interaction between the contexts and the targets. However, these models did not consider the importance of the relevant syntactical constraints. In this paper, we propose to employ a novel gated graph convolutional networks on the dependency tree to encode syntactical information, and we design a Syntax-aware Context Dynamic Weighted layer to guide our model to pay more attention to the local syntax-aware context. Moreover, Multi-head Attention is utilized for capturing both semantic information and interactive information between semantics and syntax. We conducted experiments on five datasets and the results demonstrate the effectiveness of the proposed model.

Improving AMR parsing by exploiting the dependency parsing as an auxiliary task

Abstract meaning representations (AMRs) represent sentence semantics as rooted labeled directed acyclic graphs. Though there is a strong correlation between the AMR graph of a sentence and its corresponding dependency tree, the recent neural network AMR parsers do neglect the exploitation of dependency structure information. In this paper, we explore a novel approach to exploiting dependency structures for AMR parsing. Unlike traditional pipeline models, we treat dependency parsing as an auxiliary task for AMR parsing under the multi-task learning framework by sharing neural network parameters and selectively extracting syntactic representation by the attention mechanism. Particularly, to balance the gradients and focus on the AMR parsing task, we present a new dynamical weighting scheme in the loss function. The experimental results on the LDC2015E86 and LDC2017T10 dataset show that our dependency-auxiliary AMR parser significantly outperforms the baseline and its pipeline counterpart, and demonstrate that the neural AMR parsers can be greatly boosted with the help of effective methods of integrating syntax.

Syntax grounded graph convolutional network for joint entity and event extraction

Event extraction aims to determine entity mentions, event triggers and argument roles in text, such as news articles. For this task, dependency syntax has been recognized as a valuable source of information. Previous work integrates dependency trees by using one-best results from a parser, where potential incorrect dependencies may impact the event extraction performance. We propose to use syntax features in an implicit approach, by adopting Soft Head Vectors (SHV) drawn from a well-trained parser as an adjacency matrix, densely connecting all words with the probabilistic head scores. SHV can be also regarded as dependency forests where multiple possible structures can be taken into account. A two-phase graph neural network is adopted to represent the forests, automatically differentiating the salient syntactic information from noisy parsers. Experiments on the two public datasets show that our model outperforms various baselines including graph-based one-best tree as well as recent transition-based decoding, giving the state-of-the-art results in the literature.

FAT-RE: A faster dependency-free model for relation extraction

Recent years have seen the dependency tree as effective information for relation extraction. Two problems still exist in previous methods: (1) dependency tree relies on external tools and needs to be carefully integrated with a trade-off between pruning noisy words and keeping semantic integrity; (2) dependency-based methods still have to encode sequential context as a supplement, which needs extra time. To tackle the two problems, we propose a faster dependency-free model in this paper: regarding the sentence as a fully-connected graph, we customize the vanilla transformer architecture to remove the irrelevant information via filtering mechanism and further aggregate the sentence information through the enhanced query. Our model yields comparable results on the SemEval2010 Task8 dataset and better results on the TACRED dataset, without requiring external information from the dependency tree but with improved time efficiency.

Trees, assemblies, chains, and windows

Abstract For describing grammatical organization, metaphors based on a variety of source domains – including trees, networks, chains, paths, and windows – all appear to have some validity. In Cognitive Grammar, they pertain to facets of assemblies, where semantic and phonological structures are connected by relations of symbolization, composition, and categorization. Assemblies have a temporal dimension; consisting in sequenced processing activity that runs concurrently on different time scales, they involve both seriality and hierarchy. In their hierarchical aspect, they are comparable to constituency trees, and in their connections, to dependency trees. Assembly elements, which can be characterized at any level of specificity, are connected in both syntagmatic and paradigmatic relations. A person’s linguistic ability comprises a vast assembly of conventional units, a portion of which are activated as part of the transient assembly constituting a particular expression. Lexicon and grammar effect the implementation of semantic functions – affective, interactive, descriptive, and discursive – which emerge with varying degrees of salience depending on their symbolization by segmental, prosodic, and other means. Assemblies thus make possible a unified approach to processing, structure, function, and use.

Cross-sentence N-ary relation classification using LSTMs on graph and sequence structures

Relation classification is an important semantic processing task in the field of Natural Language Processing (NLP). The past works mainly focused on binary relations in a single sentence. Recently, cross-sentence N-ary relation classification, which detects relations among n entities across multiple sentences, has been arousing people’s interests. The dependency tree based methods and some Graph Neural Network (GNN) based methods have been carried out to convey rich structural information. However, it is challenging for researchers to fully use the relevant information while ignore the irrelevant information from the dependency trees. In this paper, we propose a Graph Attention-based LSTM (GA LSTM) network to make full use of the relevant graph structure information. The dependency tree of multiple sentences is divided into many subtrees whose root node is a word in the sentence and the leaf nodes are regarded as the neighborhood. A graph attention mechanism is used to aggregate the local information in the neighborhood. Using this network, we identify the relevant information from the dependency tree. On the other hand, because the GNNs highly depend on the graph structure of the sentence and lack context sequence structural information, their effectiveness to the task is limited. To tackle this problem, we propose an N-gram Graph LSTM (NGG LSTM) network, which updates the hidden states by aggregating graph neighbor node information and the inherent sequence structural information of sentence. The experimental results show that our methods outperform most of the existing methods.

SK-GCN: Modeling Syntax and Knowledge via Graph Convolutional Network for aspect-level sentiment classification

Aspect-level sentiment classification is a fundamental subtask of fine-grained sentiment analysis. The syntactic information and commonsense knowledge are important and useful for aspect-level sentiment classification, while only a limited number of studies have explored to incorporate them via flexible graph convolutional neural networks (GCN) for this task. In this paper, we propose a new Syntax- and Knowledge-based Graph Convolutional Network (SK-GCN) model for aspect-level sentiment classification, which leverages the syntactic dependency tree and commonsense knowledge via GCN. In particular, to enhance the representation of the sentence toward the given aspect, we develop two strategies to model the syntactic dependency tree and commonsense knowledge graph, namely SK-GCN1 and SK-GCN2 respectively. SK-GCN1 models the dependency tree and knowledge graph via Syntax-based GCN (S-GCN) and Knowledge-based GCN (K-GCN) independently, and SK-GCN2 models them jointly. We also apply pre-trained BERT to this task and obtain new state-of-the-art results. Extensive experiments on five benchmark datasets demonstrate that our approach can effectively improve the performance of aspect-level sentiment classification compared with the state-of-the-art methods.

Ancient Divergence Driven by Geographic Isolation and Ecological Adaptation in Forest Dependent Sundaland Tree Squirrels

A surprising amount of hidden phylogenetic diversity exists in the small to medium size, drab colored squirrels of the genusSundasciurus. This genus is endemic to Sundaland and the Philippines, where it is widespread. An earlier revision of this genus found that the high elevation ‘populations’ of the widespread, lowland slender squirrel (S. tenuis) were different species. Previous phylogenies based on mitochondrial cytochromebsequences also suggested that the widespread, lowland Low’s squirrel (S. lowii) and the narrow endemic Fraternal squirrel (S. fraterculus) are not reciprocally monophyletic. Additionally, deep divergences have been identified between lineages within Low’s squirrel that date to the early Pliocene. Here we focus on evaluating the relationships and differences within and between populations of these two nominal species using whole mitochondrial genome sequences, nuclear intron sequences, and morphology. We reassess the taxonomy of this group, revalidate the species status of Robinson’s squirrel (Sundasciurus robinsoniBonhote, 1903) support the species level recognition of the Natuna squirrel (Sundasciurus natunensisThomas, 1895) and identify three other lineages that require further study. We estimate times of divergence and integrate geologic history to find that most of the divergences are pre-Pleistocene, and thus predate the Pleistocene flooding of Sundaland. Biogeographic, and ecological factors may have played a more important role than climatic factors in generating these patterns. While divergence in allopatry seems to be the main process driving speciation in lowland Sundaland squirrels (Sundasciurus), ecomorphological and behavioral adaptations in this clade suggest an important role of niche divergence.

Joint extraction of entities and relations using graph convolution over pruned dependency trees

We present a novel end-to-end deep neural network model based on graph convolutional networks for simultaneous joint extraction of entities and relations among them. Our model captures context and syntactic information from sentence by stacking a graph convolutional layer over bidirectional sequential LSTM layers. We sequentially concatenate the subject, object and sentence representations for obtaining the directionality of relations. Besides, in order to address long entity-distances problem, we further apply a path-centric pruning procedure to input trees in order to preserve useful information while maximally removing irrelevant words. Experiments are conducted on NYT dataset, and the proposed model achieves the state-of-the-art results on entity and relation extraction task. Our source code is available on Github: https://github.com/michael-hon/LSTM-GCN-ER.