Syntactic Dependency Research Articles

Interactive memory networks based on syntactic dependencies for aspect-level sentiment classification

Interactive memory networks based on syntactic dependencies for aspect-level sentiment classification

Improving semi-autoregressive machine translation with the guidance of syntactic dependency parsing structure

The advent of non-autoregressive machine translation (NAT) accelerates the decoding superior to autoregressive machine translation (AT) significantly, while bringing about a performance decrease. Semi-autoregressive neural machine translation (SAT), as a compromise, enjoys the merits of both autoregressive and non-autoregressive decoding. However, current SAT methods face the challenges of information-limited initialization and rigorous termination. This paper develops a layer-and-length-based syntactic labeling method and introduces a syntactic dependency parsing structure-guided two-stage semi-autoregressive translation (SDPSAT) structure, which addresses the above challenges with a syntax-based initialization and termination. Additionally, we also present a Mixed Training strategy to shrink exposure bias. Experiments on seven widely-used datasets reveal that our SDPSAT surpasses traditional SAT models with reduced word repetition and achieves competitive results with the AT baseline at a 2×∼3× speedup.

The Optimal Placement of the Head in the Noun Phrase. The Case of Demonstrative, Numeral, Adjective and Noun

ABSTRACT The word order of a sentence is shaped by multiple principles. The principle of syntactic dependency distance minimization is in conflict with the principle of surprisal minimization (or predictability maximization) in single head syntactic dependency structures: while the former predicts that the head should be placed at the centre of the linear arrangement, the latter predicts that the head should be placed at one of the ends (either first or last). A critical question is when surprisal minimization (or predictability maximization) should surpass syntactic dependency distance minimization. In the context of single head structures, it has been predicted that this is more likely to happen when two conditions are met, i.e. (a) fewer words are involved and (b) words are shorter. Here, we test the prediction on the noun phrase when it is composed of demonstrative, numeral, adjective, and noun. We find that, across preferred orders in languages, the noun tends to be placed at one of the ends, confirming the theoretical prediction. We also show evidence of anti-locality effects: syntactic dependency distances in preferred orders are longer than expected by chance.

Between Syntactic Carpentry and Dependency Grammar Similarities and Differences

Between Syntactic Carpentry and Dependency Grammar Similarities and Differences

Integrating AI with medical industry chain data: enhancing clinical nutrition research through semantic knowledge graphs.

In clinical nutrition research, the medical industry chain generates a wealth of multidimensional spatial data across various formats, including text, images, and semi-structured tables. This data's inherent heterogeneity and diversity present significant challenges for processing and mining, which are further compounded by the data's diverse features, which are difficult to extract. To address these challenges, we propose an innovative integration of artificial intelligence (AI) with the medical industry chain data, focusing on constructing semantic knowledge graphs and extracting core features. These knowledge graphs are pivotal for efficiently acquiring insights from the vast and granular big data within the medical industry chain. Our study introduces the Clinical Feature Extraction Knowledge Mapping ( ) model, designed to augment the attributes of medical industry chain knowledge graphs through an entity extraction method grounded in syntactic dependency rules. The model is applied to real and large-scale datasets within the medical industry chain, demonstrating robust performance in relation extraction, data complementation, and feature extraction. It achieves superior results to several competitive baseline methods, highlighting its effectiveness in handling medical industry chain data complexities. By representing compact semantic knowledge in a structured knowledge graph, our model identifies knowledge gaps and enhances the decision-making process in clinical nutrition research.

Enhanced UrduAspectNet: Leveraging Biaffine Attention for superior Aspect-Based Sentiment Analysis

Urdu, with its rich linguistic complexity, poses significant challenges for computational sentiment analysis. This study presents an enhanced version of UrduAspectNet, specifically designed for Aspect-Based Sentiment Analysis (ABSA) in Urdu. We introduce key innovations including the incorporation of Biaffine Attention into the model architecture, which synergizes XLM-R embeddings, a bidirectional LSTM (BiLSTM), and dual Graph Convolutional Networks (GCNs). Additionally, we utilize dependency parsing to create the adjacency matrix for the GCNs, capturing syntactic dependencies to enhance relational representation. The improved model, termed Enhanced UrduAspectNet, integrates POS and lemma embeddings, processed through BiLSTM and GCN layers, with Biaffine Attention enhancing the extraction of intricate aspect and sentiment relationships. We also introduce the use of BIO tags for aspect term identification, improving the granularity of aspect extraction. Experimental results demonstrate significant improvements in both aspect extraction and sentiment classification accuracy. This research advances Urdu sentiment analysis and sets a precedent for leveraging sophisticated NLP techniques in underrepresented languages.

IDSV-GCN: Integrating Dual Syntactic Views Graph Convolutional Network for aspect-based sentiment analysis

Aspect-based sentiment analysis (ABSA) aims to predict the sentiment polarity towards given aspects in a sentence. Recent research has highlighted that graph neural networks based on syntactic dependency trees effectively convey rich syntactic information and are suitable for ABSA tasks. Most existing work typically adopt either dependency type or minimum distance between nodes in the original syntactic tree to construct different views. However, existing work overlooks the high-dimensional local feature information in dependency relations, resulting in inadequate exploration of this view. Meanwhile, most work ignores that the minimum distance view and the dependency view can be complementary and integrated in syntactic analysis. Unfortunately, directly adding the two views introduces noise issues, leading to unsatisfactory results after their integration. To address these issues, this paper introduces an integrating dual syntactic views graph convolutional network (IDSV-GCN) model that seamlessly integrates dependency type view and minimum distance view, enabling comprehensive utilization of syntactic structure and information. To better address the computation of weights in the dependency type view, we innovatively propose a fusion attention mechanism. This approach effectively captures both global and local feature information. To more effectively integrate the two views and reduce the impact of noise, we employ a syntactic mask layer to filter out noise in the dependency type view and highlighting important dependency feature information. Then, we construct a minimum distance weight matrix based on the matrix obtained from the dependency type view. Finally, we merge the two matrices and feed them into the GCN to facilitate the model’s learning of syntactic information. Experimental results on three widely-used benchmark datasets demonstrate that our proposed IDSV-GCN model outperforms state-of-the-art methods.

Incremental Discourse-Update Constrains Number Agreement Attraction Effect.

While a large body of work in sentence comprehension has explored how different types of linguistic information are used to guide syntactic parsing, less is known about the effect of discourse structure. This study investigates this question, focusing on the main and subordinate discourse contrast manifested in the distinction between restrictive relative clauses (RRCs) and appositive relative clauses (ARCs) in American English. In three self-paced reading experiments, we examined whether both RRCs and ARCs interfere with the matrix clause content and give rise to the agreement attraction effect. While the standard attraction effect was consistently observed in the baseline RRC structures, the effect varied in the ARC structures. These results collectively suggest that discourse structure indeed constrains syntactic dependency resolution. Most importantly, we argue that what is at stake is not the static discourse structure properties at the global sentence level. Instead, attention should be given to the incremental update of the discourse structure in terms of which discourse questions are active at any given moment of a discourse. The current findings have implications for understanding the way discourse structure, specifically the active state of discourse questions, constrains memoryretrieval.

Integrating graph convolutional networks to enhance prompt learning for biomedical relation extraction

Background and Objective:Biomedical relation extraction aims to reveal the relation between entities in medical texts. Currently, the relation extraction models that have attracted much attention are mainly to fine-tune the pre-trained language models (PLMs) or add template prompt learning, which also limits the ability of the model to deal with grammatical dependencies. Graph convolutional networks (GCNs) can play an important role in processing syntactic dependencies in biomedical texts. Methods:In this work, we propose a biomedical relation extraction model that fuses GCNs enhanced prompt learning to handle limitations in syntactic dependencies and achieve good performance. Specifically, we propose a model that combines prompt learning with GCNs for relation extraction, by integrating the syntactic dependency information analyzed by GCNs into the prompt learning model, by predicting the correspondence with [MASK] tokens labels for relation extraction. Results:Our model achieved F1 scores of 85.57%, 80.15%, 95.10%, and 84.11% in the biomedical relation extraction datasets GAD, ChemProt, PGR, and DDI, respectively, all of which outperform some existing baseline models. Conclusions:In this paper, we propose enhancing prompt learning through GCNs, integrating syntactic information into biomedical relation extraction tasks. Experimental results show that our proposed method achieves excellent performance in the biomedical relation extraction task.

Distance-reconstructed dependency enhanced aspect-based sentiment analysis with sentiment strength

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task that requires the detection of sentiment polarity towards specific aspects. Dependency tree-based graph convolutional network models (GCN) have been widely applied in ABSA. These studies utilize syntactic analysis tools to derive syntactic dependency graphs, which are then inputted into GCN. It enables iterative learning of sentiment dependency information from contextual words to aspect terms. However, directly using syntactic dependency graphs can lead to two shortcomings. First, the long-distance sentiment information in long sentences will be weakened during GCN iterative computations. Second, the syntactic dependency graphs of short sentences lack sentiment strength information. To address these two shortcomings, we propose a Distance-Reconstructed Dependency Enhanced Sentiment analysis approach with Sentiment Strength based on GCN (DRD-SE-GCN) approach, in which the syntactic dependency graph of long sentences is reconstructed based on distance, thus narrowing the gap between sentiment words and target aspect words. It ensures that the sentiment dependency information carried by the long-distance sentiment words will not be weakened or lost during the GCN iteration. Additionally, the weights of the syntactic dependency graph for short sentences are reassigned so as to carry the strength information of sentiment words. Experimental results on four standard datasets show that our proposed model can effectively improve the accuracy of aspect-level sentiment analysis.

RoGraphER: Enhanced Extraction of Chinese Medical Entity Relationships Using RoFormer Pre-Trained Model and Weighted Graph Convolution

Unstructured Chinese medical texts are rich sources of entity and relational information. The extraction of entity relationships from medical texts is pivotal for the construction of medical knowledge graphs and aiding healthcare professionals in making swift and informed decisions. However, the extraction of entity relationships from these texts presents a formidable challenge, notably due to the issue of overlapping entity relationships. This study introduces a novel extraction model that leverages RoFormer’s rotational position encoding (RoPE) technique for an efficient implementation of relative position encoding. This approach not only optimizes positional information utilization but also captures syntactic dependency information by constructing a weighted adjacency matrix. During the feature fusion phase, the model employs an entity attention mechanism for a deeper integration of features, effectively addressing the challenge of overlapping entity relationships. Experimental outcomes demonstrate that our model achieves an F1 score of 83.42 on datasets featuring overlapping entity relations, significantly outperforming other baseline models.

TCKGCN: Graph convolutional network for aspect-based sentiment analysis with three-channel knowledge fusion

At present, most aspect-based sentiment analysis studies addresses the syntactic dependency problem of review statements by building syntactic dependency graphs. However, this approach is insensitive to syntactic structural information and susceptible to noise interference. Existing fusion methods are unable to effectively capture the complementary relationships between information, leading to large and less efficient model structures. To tackle these issues, we introduce a unique and original scheme that fuses semantic information, ConceptNet conceptual knowledge, and SenticNet affective knowledge, intending to improve the capability of the model to capture sentence features. Specifically, the process involves four steps: first, a semantic attention mechanism is constructed as the first channel to parse the semantic relationship within the context; second, the expressive meaning of the aspects is enhanced with concept mapping as the second channel, with concepts injected into aspectual words; third, the representation ability of sentence dependency graphs is strengthened through adding the affective knowledge of SenticNet as a third channel. Subsequently, the sensitivity of the model to affective information is increased, and finally, the optimization of aspect and context coordination is strengthened by integrating an interactive attention mechanism. The overall findings underscore the efficacy of the model, showing accuracy improvements of 1.05 % and 2.80 % on the Twitter and Lap14 datasets, respectively, compared to state-of-the-art models. Moreover the macro-F1 scores showed significant improvements of 1.99 % and 1.61 %, respectively, enabling more effective capturing of aspect-specific sentiment expressions.

Rhymes and Syntax: A Morpho-Syntactic Analysis of Czech Poetry

A linguistically informed distant reading presupposes an adequate performance of Natural Language Processing tools. This article describes our evaluation of the UDPipe parser on a manually annotated sample of nineteenth-century Czech poetry in the following steps: (1) creation of a documented data set for this domain (poetry, nineteenth century, Czech); (2) domain-specific annotation decisions; (3) error analysis. The sample consisted of 29 randomly selected poems which were first automatically tagged and parsed with the UDPipe parser and then manually checked word by word. The following features were checked: word segmentation (chunking), lemmatization, part of speech assignment, assignment of more fine-grained morphological details, the position in the syntactic dependency tree (selection of the syntactic parent), as well as the label of the syntactic relation between the word and its parent. The findings were analyzed. The most typical parser errors are associated with complex noun phrases that contain other noun(s) as modifier(s), especially when these occur in a poetry-specific word order, that is, preposed to the governing noun. On the other hand, neither archaic orthography nor neologisms posed substantial issues.

Location-enhanced syntactic knowledge for biomedical relation extraction

Biomedical relation extraction has long been considered a challenging task due to the specialization and complexity of biomedical texts. Syntactic knowledge has been widely employed in existing research to enhance relation extraction, providing guidance for the semantic understanding and text representation of models. However, the utilization of syntactic knowledge in most studies is not exhaustive, and there is often a lack of fine-grained noise reduction, leading to confusion in relation classification. In this paper, we propose an attention generator that comprehensively considers both syntactic dependency type information and syntactic position information to distinguish the importance of different dependency connections. Additionally, we integrate positional information, dependency type information, and word representations together to introduce location-enhanced syntactic knowledge for guiding our biomedical relation extraction. Experimental results on three widely used English benchmark datasets in the biomedical domain consistently outperform a range of baseline models, demonstrating that our approach not only makes full use of syntactic knowledge but also effectively reduces the impact of noisy words.

Hyperraising, evidentiality, and phase deactivation

AbstractThis paper investigates an interaction between locality requirements and syntactic dependencies through the lens of hyperraising constructions in Cantonese and Vietnamese. We offer a novel piece of evidence from subject displacement in support of the claim that phasehood can be deactivated by syntactic dependencies during the derivation. We show that (i) hyperraising (to subject) constructions are attested in both languages, and that (ii) only attitude verbs that encode an indirect evidential component allow hyperraising constructions. We propose a phase deactivation account for hyperraising, where the phasehood of a CP is deactivated by an Agree relation in terms of an evidential feature with the embedding verb. The findings of this paper suggest that locality requirements in natural languages are less rigid than previously thought, and that there is a non-trivial semantic dimension to hyperraising phenomena.

A syntactic evidence network model for fact verification

In natural language processing, fact verification is a very challenging task, which requires retrieving multiple evidence sentences from a reliable corpus to verify the authenticity of a claim. Although most of the current deep learning methods use the attention mechanism for fact verification, they have not considered imposing attentional constraints on important related words in the claim and evidence sentences, resulting in inaccurate attention for some irrelevant words. In this paper, we propose a syntactic evidence network (SENet) model which incorporates entity keywords, syntactic information and sentence attention for fact verification. The SENet model extracts entity keywords from claim and evidence sentences, and uses a pre-trained syntactic dependency parser to extract the corresponding syntactic sentence structures and incorporates the extracted syntactic information into the attention mechanism for language-driven word representation. In addition, the sentence attention mechanism is applied to obtain a richer semantic representation. We have conducted experiments on the FEVER and UKP Snopes datasets for performance evaluation. Our SENet model has achieved 78.69% in Label Accuracy and 75.63% in FEVER Score on the FEVER dataset. In addition, our SENet model also has achieved 65.0% in precision and 61.2% in macro F1 on the UKP Snopes dataset. The experimental results have shown that our proposed SENet model has outperformed the baseline models and achieved the state-of-the-art performance for fact verification.

DualGCN: Exploring Syntactic and Semantic Information for Aspect-Based Sentiment Analysis.

The task of aspect-based sentiment analysis aims to identify sentiment polarities of given aspects in a sentence. Recent advances have demonstrated the advantage of incorporating the syntactic dependency structure with graph convolutional networks (GCNs). However, their performance of these GCN-based methods largely depends on the dependency parsers, which would produce diverse parsing results for a sentence. In this article, we propose a dual GCN (DualGCN) that jointly considers the syntax structures and semantic correlations. Our DualGCN model mainly comprises four modules: 1) SynGCN: instead of explicitly encoding syntactic structure, the SynGCN module uses the dependency probability matrix as a graph structure to implicitly integrate the syntactic information; 2) SemGCN: we design the SemGCN module with multihead attention to enhance the performance of the syntactic structure with the semantic information; 3) Regularizers: we propose orthogonal and differential regularizers to precisely capture semantic correlations between words by constraining attention scores in the SemGCN module; and 4) Mutual BiAffine: we use the BiAffine module to bridge relevant information between the SynGCN and SemGCN modules. Extensive experiments are conducted compared with up-to-date pretrained language encoders on two groups of datasets, one including Restaurant14, Laptop14, and Twitter and the other including Restaurant15 and Restaurant16. The experimental results demonstrate that the parsing results of various dependency parsers affect their performance of the GCN-based models. Our DualGCN model achieves superior performance compared with the state-of-the-art approaches. The source code and preprocessed datasets are provided and publicly available on GitHub (see https://github.com/CCChenhao997/DualGCN-ABSA).

More Than Syntaxes: Investigating Semantics to Zero-shot Cross-lingual Relation Extraction and Event Argument Role Labelling

Syntactic dependency structures are commonly utilized as language-agnostic features to solve the word order difference issues in zero-shot cross-lingual relation and event extraction tasks. However, while sentences in multiple forms can be employed to express the same meaning, the syntactic structure may vary considerably in specific scenarios. To fix this problem, we find semantics are rarely considered, which could provide a more consistent semantic analysis of sentences and be served as another bridge between different languages. Therefore, in this article, we introduce Syntax and Semantic Driven Network (SSDN) to equip syntax and semantic knowledge across languages simultaneously. Specifically, predicate–argument structures from semantic role labelling are explicitly incorporated into word representations. Then, a semantic-aware relational graph convolutional network and a transformer-based encoder are utilized to model both semantic dependency and syntactic dependency structures, respectively. Finally, a fusion module is introduced to integrate output representations adaptively. We conduct experiments on the widely used Automatic Content Extraction 2005 English, Chinese, and Arabic datasets. The evaluation results demonstrate that the proposed method achieves the state-of-the-art performance. Further study also indicates SSDN could produce robust representations that facilitate the transfer operations across languages.

Dual-channel relative position guided attention networks for aspect-based sentiment analysis

Aspect-based sentiment analysis (ABSA) aims to match sentiment tendencies for different aspects of a sentence to understand the product experience of the user. It is a pressing challenge for existing ABSA methods to synthesize sentences’ semantic relevance and syntactic dependency for more comprehensive sentiment representations. In this paper, we propose a Dual-Channel Relative Position Guided Attention Network (Dual-RPGA). Dual-RPGA deeply learns semantic and syntactic representations of sentiment to provide reliable knowledge for dynamic fusion and prediction of sentiment. First, we design a syntactic graph attention network (Syn-GAT) to learn the syntactic relative position between aspect and context, which guides the sentiment syntactic representation. Then, we build a semantic attention network (Sem-Attention). It computes semantic attention and similarity coefficients for aspects and contexts to enhance sentiment semantic expressions. Finally, we design a fusion network (Bi-Fusion) that realizes dynamic feature interactions of sentiment semantics and syntactics to perform sentiment prediction. We conduct extensive experiments on two groups of datasets to validate the performance of Dual-RPGA on the ABSA task. The results show that Dual-RPGA outperforms the optimal baseline by 0.58%∼1.49% of the Acc score, which verifies that Dual-RPGA performs better on the ABSA task.

Syntax and Relation Enhanced Query Generation for Text to SQL Parsing

Text-to-SQL parsing is the task of converting natural language questions into executable SQL queries, a significant branch of semantic parsing, which has gained increasing attention in recent years. This technology lowers the barrier for people to access databases, enhancing the convenience and availability of data. However, the primary challenge for text-to-SQL parsing lies in domain adaptation, which concerns whether the model can be applied to new databases and effectively align natural language questions with the corresponding tables or columns within the database. To address these issues, research has introduced SRSQL (Syntax and Relation-Augmented Query Generation), which incorporates syntax information and predefined relationships into the model, effectively utilizing syntactic dependencies and pattern linking to improve performance. Using a Transformer-based decoder, SRSQL generates SQL queries in the form of Abstract Syntax Trees (AST), significantly boosting prediction accuracy. Experimental results show that SRSQL outperforms comparative models, particularly on challenging benchmarks like Spider and Spider-SYN.