Dependency Parsing Research Articles

Improving semi-autoregressive machine translation with the guidance of syntactic dependency parsing structure

The advent of non-autoregressive machine translation (NAT) accelerates the decoding superior to autoregressive machine translation (AT) significantly, while bringing about a performance decrease. Semi-autoregressive neural machine translation (SAT), as a compromise, enjoys the merits of both autoregressive and non-autoregressive decoding. However, current SAT methods face the challenges of information-limited initialization and rigorous termination. This paper develops a layer-and-length-based syntactic labeling method and introduces a syntactic dependency parsing structure-guided two-stage semi-autoregressive translation (SDPSAT) structure, which addresses the above challenges with a syntax-based initialization and termination. Additionally, we also present a Mixed Training strategy to shrink exposure bias. Experiments on seven widely-used datasets reveal that our SDPSAT surpasses traditional SAT models with reduced word repetition and achieves competitive results with the AT baseline at a 2×∼3× speedup.

Automatic dependency parsing of Estonian: what linguistic features to include?

Automatic dependency parsing of Estonian: what linguistic features to include?

Enhanced UrduAspectNet: Leveraging Biaffine Attention for superior Aspect-Based Sentiment Analysis

Urdu, with its rich linguistic complexity, poses significant challenges for computational sentiment analysis. This study presents an enhanced version of UrduAspectNet, specifically designed for Aspect-Based Sentiment Analysis (ABSA) in Urdu. We introduce key innovations including the incorporation of Biaffine Attention into the model architecture, which synergizes XLM-R embeddings, a bidirectional LSTM (BiLSTM), and dual Graph Convolutional Networks (GCNs). Additionally, we utilize dependency parsing to create the adjacency matrix for the GCNs, capturing syntactic dependencies to enhance relational representation. The improved model, termed Enhanced UrduAspectNet, integrates POS and lemma embeddings, processed through BiLSTM and GCN layers, with Biaffine Attention enhancing the extraction of intricate aspect and sentiment relationships. We also introduce the use of BIO tags for aspect term identification, improving the granularity of aspect extraction. Experimental results demonstrate significant improvements in both aspect extraction and sentiment classification accuracy. This research advances Urdu sentiment analysis and sets a precedent for leveraging sophisticated NLP techniques in underrepresented languages.

Cross-lingual dependency parsing for a language with a unique script

Abstract Syntactic parsing is one of the areas in Natural Language Processing. The development of large-scale multilingual language models has enabled cross-lingual parsing approaches, which allows us to develop parsers for languages that do not have treebanks available. However, these approaches rely on the assumption that languages share orthographic representations and lexical entries. In this article, we investigate methods for developing a dependency parser for Xibe, a low-resource language that is written in a unique script. We first investigate lexicalized monolingual dependency parsing experiments to examine the effectiveness of word, part-of-speech, and character embeddings as well as pre-trained language models. Results show that character embeddings can significantly improve performance, while pre-trained language models decrease performance since they do not recognize the Xibe script. We also train delexicalized monolingual models, which yield competitive results to the best lexicalized model. Since the monolingual models are trained on a very small training set, we also investigate lexicalized and delexicalized cross-lingual models. We use six closely related languages as source language, which cover a wide range of scripts. In this setting, the delexicalized models achieve higher performance than lexicalized models. A final experiment shows that we can increase performance of the cross-lingual model by combining source languages and selecting the most similar sentences to Xibe as training set. However, all cross-lingual parsing results are still considerably lower than the monolingual model. We attribute the low performance of cross-lingual methods to syntactic and annotation differences as well as to the impoverished input of Universal Dependency Part-of-Speech tags that the delexicalized model has access to.

Multi-source domain adaptation for dependency parsing via domain-aware feature generation

Multi-source domain adaptation for dependency parsing via domain-aware feature generation

NLP-based ergonomics MSD risk root cause analysis and risk controls recommendation

An ergonomics assessment of the physical risk factors in the workplace is instrumental in predicting and preventing musculoskeletal disorders (MSDs). Using Artificial Intelligence (AI) has become increasingly popular for ergonomics assessments because of the time savings and improved accuracy. However, most of the effort in this area starts and ends with producing risk scores, without providing guidance to reduce the risk. This paper proposes a holistic job improvement process that performs automatic root cause analysis and control recommendations for reducing MSD risk. We apply deep learning-based Natural Language Processing (NLP) techniques such as Part of Speech (PoS) tagging and dependency parsing on textual descriptions of the physical actions performed in the job (e.g. pushing) along with the object (e.g. cart) being acted upon. The action-object inferences provide the entry point to an expert-based Machine Learning (ML) system that automatically identifies the targeted work-related causes (e.g. cart movement forces are too high, due to caster size too small) of the identified MSD risk (e.g. excessive shoulder forces). The proposed framework utilises the root causes identified to recommend control strategies (e.g. provide larger diameter casters, minimum diameter 8" or 203 mm) most likely to mitigate risk, resulting in a more efficient and effective job improvement process.

LLM-Assisted Data Augmentation for Chinese Dialogue-Level Dependency Parsing

Abstract Dialogue-level dependency parsing, despite its growing academic interest, often encounters underperformance issues due to resource shortages. A potential solution to this challenge is data augmentation. In recent years, large language models (LLMs) have demonstrated strong capabilities in generation, which can facilitate data augmentation greatly. In this study, we focus on Chinese dialogue-level dependency parsing, presenting three simple and effective strategies with LLM to augment the original training instances, namely word-level, syntax-level, and discourse-level augmentations, respectively. These strategies enable LLMs to either preserve or modify dependency structures, thereby assuring accuracy while increasing the diversity of instances at different levels. We conduct experiments on the benchmark dataset released by Jiang et al. (2023) to validate our approach. Results show that our method can greatly boost the parsing performance in various settings, particularly in dependencies among elementary discourse units. Lastly, we provide in-depth analysis to show the key points of our data augmentation strategies.

TaskFinder: A Semantics-Based Methodology for Visualization Task Recommendation

Data visualization has entered the mainstream, and numerous visualization recommender systems have been proposed to assist visualization novices, as well as busy professionals, in selecting the most appropriate type of chart for their data. Given a dataset and a set of user-defined analytical tasks, these systems can make recommendations based on expert coded visualization design principles or empirical models. However, the need to identify the pertinent analytical tasks beforehand still exists and often requires domain expertise. In this work, we aim to automate this step with TaskFinder, a prototype system that leverages the information available in textual documents to understand domain-specific relations between attributes and tasks. TaskFinder employs word vectors as well as a custom dependency parser along with an expert-defined list of task keywords to extract and rank associations between tasks and attributes. It pairs these associations with a statistical analysis of the dataset to filter out tasks irrelevant given the data. TaskFinder ultimately produces a ranked list of attribute–task pairs. We show that the number of domain articles needed to converge to a recommendation consensus is bounded for our approach. We demonstrate our TaskFinder over multiple domains with varying article types and quantities.

Aspect-specific Parsimonious Segmentation via Attention-based Graph Convolutional Network for Aspect-Based Sentiment Analysis

Aspect-Based Sentiment Analysis (ABSA) involves determining sentiment polarity in sentences concerning specific aspects. Recent approaches in ABSA have incorporated dependency parsing to gain a deeper understanding of word associations within aspect-related terms. Nevertheless, these models typically concentrate solely on dependency connections while neglecting their specific characteristics or types. This can lead to sentiment inconsistencies, particularly when analyzing sentences with multiple aspects and diverse polarities. To address this challenge, we propose an innovative solution known as Aspect-specific Parsimonious Segmentation via Attention-based Graph Convolutional Network (APSA-GCN) for ABSA. Our model consists of two key modules that harness a variety of features, including syntax, semantics, and lexicon-based representations. The aspect-specific parsimonious segmentation framework segments the data and directs it to parsimonious dual filters, retaining only aspect-specific sentences called explicit data for lexicon-based representation. Subsequently, the Attention-based Graph Convolutional Network (GCN) is employed to extract syntactic and semantic features more explicitly, thereby enriching the representation of aspect-related features. Extensive experiments were conducted on various benchmark datasets, and the results confirm the model’s efficacy as our method achieves state-of-the-art performance. Moreover, our introduced APS framework can be effortlessly incorporated into other benchmark models to enhance the outcomes of ABSA.

An accurate transformer-based model for transition-based dependency parsing of free word order languages

Transformer models are the state-of-the-art in Natural Language Processing (NLP) and the core of the Large Language Models (LLMs). We propose a transformer-based model for transition-based dependency parsing of free word order languages. We have performed experiments on five treebanks from the Universal Dependencies (UD) dataset version 2.12. Our experiments show that a transformer model, trained with the dynamic word embeddings performs better than a multilayer perceptron trained on the state-of-the-art static word embeddings even if the dynamic word embeddings have a vocabulary size ten times smaller than the static word embeddings. The results show that the transformer trained on dynamic word embeddings achieves an unlabeled attachment score (UAS) of 84.17% for Urdu language which is ≈3.6% and ≈1.9% higher than the UAS scores of 80.56857% and 82.26859% achieved by the multilayer perceptron (MLP) using two static state-of-the-art word embeddings. The proposed approach is investigated for Arabic, Persian and Uyghur languages, in addition to Urdu, for UAS scores and the results suggest that the proposed solution outperform the MLP-based approaches.

Exploiting Contextual Target Attributes for Target Sentiment Classification

In the past few years, pre-trained language models (PTLMs) have brought significant improvements to target sentiment classification (TSC). Existing PTLM-based models can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that transfer the classification task to the text/word generation task. Despite the improvements achieved by these models, we argue that they have their respective limitations. For fine-tuning-based models, they cannot make the best use of the PTLMs’ strong language modeling ability because the pre-train task and downstream fine-tuning task are not consistent. For prompting-based models, although they can sufficiently leverage the language modeling ability, it is hard to explicitly model the target-context interactions, which are widely realized as a crucial point of this task. In this paper, we present a new perspective of leveraging PTLM for TSC: simultaneously leveraging the merits of both language modeling and explicit target-context interactions via contextual target attributes. Specifically, we design the domain- and target-constrained cloze test, which can leverage the PTLMs’ strong language modeling ability to generate the given target’s attributes pertaining to the review context. The attributes contain the background and property information of the target, which can help to enrich the semantics of the review context and the target. To exploit the attributes for tackling TSC, we first construct a heterogeneous information graph by treating the attributes as nodes and combining them with (1) the syntax graph automatically produced by the off-the-shelf dependency parser and (2) the semantics graph of the review context, which is derived from the self-attention mechanism. Then we propose a heterogeneous information gated graph convolutional network to model the interactions among the attribute information, the syntactic information, and the contextual information. The experimental results on three benchmark datasets demonstrate the superiority of our model, which achieves new state-of-the-art performance.

Customer Intent Mining from Service Inquiries with Newly Improved Deep Embedded Clustering

Discovering customer intents from text or speech data plays a vital role in text mining and automated dialogue response. It is challenging to process thousands of customer interactions daily. Deep Embedded Clustering (DEC) and Improved DEC (IDEC) with Kullback–Leibler loss handle a lot of data inefficiently due to the asymmetric nature of the loss. To address the challenge, an unsupervised learning approach to discover intents and automatically produce the labels from a collection of unlabeled utterances in the context of the banking domain is proposed. The proposed approach focuses on improving both architectures of DEC and IDEC by combining the Jensen–Shannon (JS) divergence to simultaneously learn feature representations and cluster assignments, and the Second-order Clipped Stochastic Optimization (Sophia). Then, a set of intent labels for each cluster is generated by using a dependency parser in the second stage. Experimental results showed that the proposed approach is capable of generating meaningful intent labels and short text clustering with high performance.

DualGCN: Exploring Syntactic and Semantic Information for Aspect-Based Sentiment Analysis.

The task of aspect-based sentiment analysis aims to identify sentiment polarities of given aspects in a sentence. Recent advances have demonstrated the advantage of incorporating the syntactic dependency structure with graph convolutional networks (GCNs). However, their performance of these GCN-based methods largely depends on the dependency parsers, which would produce diverse parsing results for a sentence. In this article, we propose a dual GCN (DualGCN) that jointly considers the syntax structures and semantic correlations. Our DualGCN model mainly comprises four modules: 1) SynGCN: instead of explicitly encoding syntactic structure, the SynGCN module uses the dependency probability matrix as a graph structure to implicitly integrate the syntactic information; 2) SemGCN: we design the SemGCN module with multihead attention to enhance the performance of the syntactic structure with the semantic information; 3) Regularizers: we propose orthogonal and differential regularizers to precisely capture semantic correlations between words by constraining attention scores in the SemGCN module; and 4) Mutual BiAffine: we use the BiAffine module to bridge relevant information between the SynGCN and SemGCN modules. Extensive experiments are conducted compared with up-to-date pretrained language encoders on two groups of datasets, one including Restaurant14, Laptop14, and Twitter and the other including Restaurant15 and Restaurant16. The experimental results demonstrate that the parsing results of various dependency parsers affect their performance of the GCN-based models. Our DualGCN model achieves superior performance compared with the state-of-the-art approaches. The source code and preprocessed datasets are provided and publicly available on GitHub (see https://github.com/CCChenhao997/DualGCN-ABSA).

A syntactic evidence network model for fact verification

In natural language processing, fact verification is a very challenging task, which requires retrieving multiple evidence sentences from a reliable corpus to verify the authenticity of a claim. Although most of the current deep learning methods use the attention mechanism for fact verification, they have not considered imposing attentional constraints on important related words in the claim and evidence sentences, resulting in inaccurate attention for some irrelevant words. In this paper, we propose a syntactic evidence network (SENet) model which incorporates entity keywords, syntactic information and sentence attention for fact verification. The SENet model extracts entity keywords from claim and evidence sentences, and uses a pre-trained syntactic dependency parser to extract the corresponding syntactic sentence structures and incorporates the extracted syntactic information into the attention mechanism for language-driven word representation. In addition, the sentence attention mechanism is applied to obtain a richer semantic representation. We have conducted experiments on the FEVER and UKP Snopes datasets for performance evaluation. Our SENet model has achieved 78.69% in Label Accuracy and 75.63% in FEVER Score on the FEVER dataset. In addition, our SENet model also has achieved 65.0% in precision and 61.2% in macro F1 on the UKP Snopes dataset. The experimental results have shown that our proposed SENet model has outperformed the baseline models and achieved the state-of-the-art performance for fact verification.

Triplétoile: Extraction of knowledge from microblogging text

Numerous methods and pipelines have recently emerged for the automatic extraction of knowledge graphs from documents such as scientific publications and patents. However, adapting these methods to incorporate alternative text sources like micro-blogging posts and news has proven challenging as they struggle to model open-domain entities and relations, typically found in these sources. In this paper, we propose an enhanced information extraction pipeline tailored to the extraction of a knowledge graph comprising open-domain entities from micro-blogging posts on social media platforms. Our pipeline leverages dependency parsing and classifies entity relations in an unsupervised manner through hierarchical clustering over word embeddings. We provide a use case on extracting semantic triples from a corpus of 100 thousand tweets about digital transformation and publicly release the generated knowledge graph. On the same dataset, we conduct two experimental evaluations, showing that the system produces triples with precision over 95% and outperforms similar pipelines of around 5% in terms of precision, while generating a comparatively higher number of triples.

Can Words Speak Louder Than Actions? Using Top Management Teams’ Language to Predict Myopic Marketing Spending

Myopic marketing spending—curtailing marketing and research and development expenses to boost earnings—damages firms’ long-term value. Despite this, top management teams are often myopic, and by the time investors or boards detect such short-termism, it is too late to react or intervene. This research introduces a novel prediction method by analyzing the language top management teams use in earnings calls, specifically focusing on marketing and earnings emphasis, to predict future instances of myopic marketing spending. Through linguistic dependency parsing of almost 11 million sentences extracted from nearly 25,000 quarterly earnings call transcripts of 1,197 firms between 2008 and 2019, the authors demonstrate that the proposed approach can predict myopic marketing spending at a quarterly frequency for up to one year in advance. They find that an increase of one standard deviation in earnings emphasis is associated with a 23.68% increase in the likelihood of future myopic marketing spending. Investments based on the proposed approach produce 1.61% additional annual abnormal returns compared with models that exclusively use known predictors of myopic marketing spending, while offering earlier foresight and more frequent opportunities for intervention. This reduces information asymmetry for investors and boards of directors.

Research on Aspect-Level Sentiment Analysis Based on Adversarial Training and Dependency Parsing

Aspect-level sentiment analysis is used to predict the sentiment polarity of a specific aspect in a sentence. However, most current research cannot fully utilize semantic information, and the models lack robustness. Therefore, this article proposes a model for aspect-level sentiment analysis based on a combination of adversarial training and dependency syntax analysis. First, BERT is used to transform word vectors and construct adjacency matrices with dependency syntactic relationships to better extract semantic dependency relationships and features between sentence components. A multi-head attention mechanism is used to fuse the features of the two parts, simultaneously perform adversarial training on the BERT embedding layer to enhance model robustness, and, finally, to predict emotional polarity. The model was tested on the SemEval 2014 Task 4 dataset. The experimental results showed that, compared with the baseline model, the model achieved significant performance improvement after incorporating adversarial training and dependency syntax relationships.

Enterprise violation risk deduction combining generative AI and event evolution graph

AbstractIn the current realms of scientific research and commercial applications, the risk inference of regulatory violations by publicly listed enterprises has attracted considerable attention. However, there are some problems in the existing research on the deduction and prediction of violation risk of listed enterprises, such as the lack of analysis of the causal logic association between violation events, the low interpretability and effectiveness of the deduction and the lack of training data. To solve these problems, we propose a framework for enterprise violation risk deduction based on generative AI and event evolution graphs. First, the generative AI technology was used to generate a new text summary of the lengthy and complex enterprise violation announcement to realize a concise overview of the violation matters. Second, by fine‐tuning the generative AI model, an event entity and causality extraction framework based on automated data augmentation are proposed, and the UIE (Unified Structure Generation for Universal Information Extraction) event entity extraction model is used to create the event entity extraction for listed enterprises ‘violations. Then, a causality extraction model CDDP‐GAT (Event Causality Extraction Based on Chinese Dictionary and Dependency Parsing of GAT) is proposed. This model aims to identify and analyse the causal links between corporate breaches, thereby deepening the understanding of the event logic. Then, the merger of similar events was realized, and the causal correlation weights between enterprise violation‐related events were evaluated. Finally, the listed enterprise's violation risk event evolution graph was constructed, and the enterprise violation risk deduction was carried out to form an expert system of financial violations. The deduction results show that the method can effectively reveal signs of enterprise violations and adverse consequences.

Sentiment Analysis Web App

This research explores how our approach to seeking information has evolved, particularly in understanding others' perspectives through platforms like online reviews and personal blogs. With the widespread use of technology, there's a growing interest in systems that can handle opinions. The paper focuses on techniques for creating opinion- centric information systems, particularly dealing with challenges in sentiment-aware applications. It categorizes information into facts and opinions, concentrating on expressions of positive or negative sentiments. Using typed dependency parsing, the study investigates the functional connections among words in sentiment analysis, allowing for a thorough examination of grammar and semantics in textual data. emphasizing the role of machine learning in ASP.NET Core for effective feedback and opinion handling.

Design and Implementation of an Intelligent Cross-Border E-Commerce English Learning Platform Based on a Knowledge Map

At present, most online education platforms still have problems such as single learning modes and loose knowledge structures. Using the knowledge map, the study employs a personalized adaptive intelligent adjustment strategy based on the structural expression of the CBEC English subject system. Firstly, the study uses the Scapy framework to crawl the subject knowledge data. Then use the LTP platform to process sentences containing multiple entities. Input the sentence into the dependency parser to analyze and extract the entity relationship. Finally, according to the relevance between knowledge points and topics in the knowledge map, the final learning path recommendation result is obtained. And cluster the similarity of curriculum content to build a complete curriculum system. Based on the above operations, a knowledge map-based smart learning platform for the CBEC English discipline has been designed and implemented to provide a smart, personalized learning environment for learners. According to the experimental analysis, the average satisfaction of learners with the learning platform designed by the study is 81.56%, which can meet the learning needs of learners and provide an excellent mobile learning environment for students.