Discourse Parsing Research Articles

ChatMDG: A discourse parsing graph fusion based approach for multi-party dialogue generation

Comprehending multi-party dialogue generation poses a challenge due to intricate speaker interactions, where multiple participants engage in a dynamic exchange of questions and responses, assuming diverse roles such as speaker, receiver, and observer, with these roles evolving across conversational turns. Most existing research on multi-party dialogue generation only considers semantic information contained in each sentence and does not take into account the dialogue flow information implicit in multi-role interaction, leading to difficulties in accurately understanding the dialogue state in multi-party dialogue. To fill these gaps, we introduce an information fusion based approach for Multi-party Dialogue Generation named ChatMDG, which integrates role interaction into a semantic-enriched graph with context-based embeddings to cooperatively capture both global and local information in multi-party dialogue. Specifically, we proposes a graph-based network to represent the complex role-interaction dialogue structure for discourse parsing and then designs the dialogue flow encoding method to fuse role-interaction information with semantic states effectively. Furthermore, ChatMDG presents interaction strategies to correspondingly generate reactive and proactive utterances based on the fused embeddings, which lead to more dialogue coherence and user engagement. Experimental results show that ChatMDG significantly improves the accuracy of the multi-party response generation task, especially in complex scenarios with multiple interactions.

Cross-genre argument mining: Can language models automatically fill in missing discourse markers?

Available corpora for Argument Mining differ along several axes, and one of the key differences is the presence (or absence) of discourse markers to signal argumentative content. Exploring effective ways to use discourse markers has received wide attention in various discourse parsing tasks, from which it is well-known that discourse markers are strong indicators of discourse relations. To improve the robustness of Argument Mining systems across different genres, we propose to automatically augment a given text with discourse markers such that all relations are explicitly signaled. Our analysis unveils that popular language models taken out-of-the-box fail on this task; however, when fine-tuned on a new heterogeneous dataset that we construct (including synthetic and real examples), they perform considerably better. We demonstrate the impact of our approach on an Argument Mining downstream task, evaluated on different corpora, showing that language models can be trained to automatically fill in discourse markers across different corpora, improving the performance of a downstream model in some, but not all, cases. Our proposed approach can further be employed as an assistant tool for better discourse understanding.

TOMDS (Topic-Oriented Multi-Document Summarization): Enabling Personalized Customization of Multi-Document Summaries

In a multi-document summarization task, if the user can decide on the summary topic, the generated summary can better align with the reader’s specific needs and preferences. This paper addresses the issue of overly general content generation by common multi-document summarization models and proposes a topic-oriented multi-document summarization (TOMDS) approach. The method is divided into two stages: extraction and abstraction. During the extractive stage, it primarily identifies and retrieves paragraphs relevant to the designated topic, subsequently sorting them based on their relevance to the topic and forming an initial subset of documents. In the abstractive stage, building upon the transformer architecture, the process includes two parts: encoding and decoding. In the encoding part, we integrated an external discourse parsing module that focuses on both micro-level within-paragraph semantic relationships and macro-level inter-paragraph connections, effectively combining these with the implicit relationships in the source document to produce more enriched semantic features. In the decoding part, we incorporated a topic-aware attention mechanism that dynamically zeroes in on information pertinent to the chosen topic, thus guiding the summary generation process more effectively. The proposed model was primarily evaluated using the standard text summary dataset WikiSum. The experimental results show that our model significantly enhanced the thematic relevance and flexibility of the summaries and improved the accuracy of grammatical and semantic comprehension in the generated summaries.

Understanding English Rhetorical Strategies Based on Neurosemantic Analysis

Abstract This paper designs and implements a model for analyzing rhetorical strategies of English texts to address practical needs. This paper combines the true word confusion set and the N meta-model to correct word errors. In addition, a discourse parser is constructed based on the theory of rhetorical structure, which analyzes the semantics of the English text from the intra-sentence and inter-sentence perspectives to extract the key information of the discourse. The information is inputted into a memory network that resembles a tree, and the memory units and gating mechanism are employed to obtain semantic structural features. Finally, a hierarchical network model based on the attention mechanism is designed, using a bidirectional long short-term memory network to extract the global dependency information of the text and adding the attention mechanism to make it adaptively select important features related to English rhetorical strategies. Combining learning of a tree-based long short-term memory network and a hierarchical network based on an attention mechanism is used to generate a model for semantic analysis of English text. This paper’s model has the highest accuracy, recall, and F1 value among the mentioned models in the 20newsgroups dataset, with 93.77%, 91.23%, and 90.15%, respectively. The performance on the three rhetorical strategies of explicit metaphor, personification, and analogy is excellent, with classification accuracy above 90%, which will accomplish the task of analyzing and understanding English modification strategies.

Hierarchical graph fusion network and a new argumentative dataset for multiparty dialogue discourse parsing

Discourse parsing in multi-party dialogue aims to extract the relationships between elementary discourse units (EDUs) such as arguments and utterances, and has numerous applications like chatbots or virtual assistants. Two significant challenges have been encountered in previous works: the obstacles in fusing various contexts in the modeling; and the lack of argumentative dialogue datasets in this field. To tackle context fusion challenges in the modeling, we introduce the Hierarchical Graph Fusion Network (HGFN). This method introduces sufficient contexts by hierarchically modeling the dialogue and minimizes context noise through a novel routing mechanism. It specifically: (1) Encodes multiple levels of contexts using hierarchical graph neural networks. During this stage, the router allows information exchange across different levels, expanding the model’s receptive field in dialogue. (2) Fuses matching signals from multiple levels of contexts with a fusion network. Here, the router restricts the information flows to the same context level, efficiently minimizing noise from irrelevant context. Furthermore, despite the significance of argumentative multi-party dialogue in real-world applications, this area remains largely unexplored due to dataset scarcity. To address this challenge, we develop two meticulously annotated datasets, MRDL and MRDR. Unlike the prevailing datasets that primarily focus on short and colloquial conversations, our datasets feature intricate argumentative dialogues and are publicly accessible at https://github.com/AI0Research/MRDL-and-MRDR. Our new datasets and the HGFN model could promote further advancements in this field. Extensive experiments are conducted and it is revealed that the HGFN model surpasses the state-of-the-art, particularly in complex, argumentative dialogues.

Conserving Semantic Unit Information and Simplifying Syntactic Constituents to Improve Implicit Discourse Relation Recognition.

Implicit discourse relation recognition (IDRR) has long been considered a challenging problem in shallow discourse parsing. The absence of connectives makes such relations implicit and requires much more effort to understand the semantics of the text. Thus, it is important to preserve the semantic completeness before any attempt to predict the discourse relation. However, word level embedding, widely used in existing works, may lead to a loss of semantics by splitting some phrases that should be treated as complete semantic units. In this article, we proposed three methods to segment a sentence into complete semantic units: a corpus-based method to serve as the baseline, a constituent parsing tree-based method, and a dependency parsing tree-based method to provide a more flexible and automatic way to divide the sentence. The segmented sentence will then be embedded at the level of semantic units so the embeddings could be fed into the IDRR networks and play the same role as word embeddings. We implemented our methods into one of the recent IDRR models to compare the performance with the original version using word level embeddings. Results show that proper embedding level better conserves the semantic information in the sentence and helps to enhance the performance of IDRR models.

Hierarchical Clause Annotation: Building a Clause-Level Corpus for Semantic Parsing with Complex Sentences

Most natural-language-processing (NLP) tasks suffer performance degradation when encountering long complex sentences, such as semantic parsing, syntactic parsing, machine translation, and text summarization. Previous works addressed the issue with the intuition of decomposing complex sentences and linking simple ones, such as rhetorical-structure-theory (RST)-style discourse parsing, split-and-rephrase (SPRP), text simplification (TS), simple sentence decomposition (SSD), etc. However, these works are not applicable for semantic parsing such as abstract meaning representation (AMR) parsing and semantic dependency parsing due to misalignments with semantic relations and unavailabilities to preserve the original semantics. Following the same intuition and avoiding the deficiencies of previous works, we propose a novel framework, hierarchical clause annotation (HCA), for capturing clausal structures of complex sentences, based on the linguistic research of clause hierarchy. With the HCA framework, we annotated a large HCA corpus to explore the potentialities of integrating HCA structural features into semantic parsing with complex sentences. Moreover, we decomposed HCA into two subtasks, i.e., clause segmentation and clause parsing, and provide neural baseline models for more-silver annotations. In evaluating the proposed models on our manually annotated HCA dataset, the performances of clause segmentation and parsing resulted in 91.3% F1-scores and 88.5% Parseval scores, respectively. Due to the same model architectures employed, the performance differences of the clause/discourse segmentation and parsing subtasks was reflected in our HCA corpus and compared discourse corpora, where our sentences contained more segment units and fewer interrelations than those in the compared corpora.

Toward a shallow discourse parser for Turkish

Abstract One of the most interesting aspects of natural language is how texts cohere, which involves the pragmatic or semantic relations that hold between clauses (addition, cause-effect, conditional, similarity), referred to as discourse relations. A focus on the identification and classification of discourse relations appears as an imperative challenge to be resolved to support tasks such as text summarization, dialogue systems, and machine translation that need information above the clause level. Despite the recent interest in discourse relations in well-known languages such as English, data and experiments are still needed for typologically different and less-resourced languages. We report the most comprehensive investigation of shallow discourse parsing in Turkish, focusing on two main sub-tasks: identification of discourse relation realization types and the sense classification of explicit and implicit relations. The work is based on the approach of fine-tuning a pre-trained language model (BERT) as an encoder and classifying the encoded data with neural network-based classifiers. We firstly identify the discourse relation realization type that holds in a given text, if there is any. Then, we move on to the sense classification of the identified explicit and implicit relations. In addition to in-domain experiments on a held-out test set from the Turkish Discourse Bank (TDB 1.2), we also report the out-domain performance of our models in order to evaluate its generalization abilities, using the Turkish part of the TED Multilingual Discourse Bank. Finally, we explore the effect of multilingual data aggregation on the classification of relation realization type through a cross-lingual experiment. The results suggest that our models perform relatively well despite the limited size of the TDB 1.2 and that there are language-specific aspects of detecting the types of discourse relation realization. We believe that the findings are important both in providing insights regarding the performance of the modern language models in a typologically different language and in the low-resource scenario, given that the TDB 1.2 is 1/20th of the Penn Discourse TreeBank in terms of the number of total relations.

Towards intelligent policy analysis: A discourse structure parsing technique for Chinese government document

Government document analysis can help organizations to understand key policy points and make right response. To realize intelligent analysis of a volume of policies, we propose a discourse parsing technique for in-depth understanding of Chinese government documents (CGDs). Based on Superstructure Schema and Rhetorical Structure Theory (RST), we characterize the stylistic features and macrostructure patterns of CGDs and develop a discourse analysis framework to specify its functional structure and semantic system. A tree schema is adopted to portray the hierarchy of textual fragments in full text and formalize their representations in machine-based parsing, then a multi-head partition self-attention model is constructed to evaluate the discourse analysis framework. Experiment results show that our parsing model embedding with inherent CGD discourse features performs better than baselines. The highest f1-score in span and label prediction tasks researches 0.9209. The parsing model can be migrated to other discourse parsing tasks by considering specific genre features of different documents.

A speaker-aware multiparty dialogue discourse parser with heterogeneous graph neural network

Discourse parsing for multiparty dialogue aims to detect the discourse structure and relations in a dialogue to obtain a discourse dependency graph. Existing models for this task have proven the effect of speaker information. However, no model explicitly learns representations of the speaker for parsing the discourse structure of dialogues. In this paper, to further exploit the effect of speaker information, we propose a novel model HG-MDP, which uses a heterogeneous graph neural network to encode dialogue graphs and we use an iterative update for the aggregation of speaker nodes and utterance nodes. Finally, we adopt updated utterance nodes to predict discourse dependency links and relations using the biaffine module. To validate our HG-MDP model, we perform experiments on the two existing benchmarks STAC and Molweni corpus. The results prove the effectiveness of the speaker modeling module on two datasets and we achieve the state-of-the-art on the Molweni dataset.

Predicting Above-Sentence Discourse Structure Using Distant Supervision from Topic Segmentation

RST-style discourse parsing plays a vital role in many NLP tasks, revealing the underlying semantic/pragmatic structure of potentially complex and diverse documents. Despite its importance, one of the most prevailing limitations in modern day discourse parsing is the lack of large-scale datasets. To overcome the data sparsity issue, distantly supervised approaches from tasks like sentiment analysis and summarization have been recently proposed. Here, we extend this line of research by exploiting distant supervision from topic segmentation, which can arguably provide a strong and oftentimes complementary signal for high-level discourse structures. Experiments on two human-annotated discourse treebanks confirm that our proposal generates accurate tree structures on sentence and paragraph level, consistently outperforming previous distantly supervised models on the sentence-to-document task and occasionally reaching even higher scores on the sentence-to-paragraph level.

STEM: Unsupervised STructural EMbedding for Stance Detection

Stance detection is an important task, supporting many downstream tasks such as discourse parsing and modeling the propagation of fake news, rumors, and science denial. In this paper, we propose a novel framework for stance detection. Our framework is unsupervised and domain-independent. Given a claim and a multi-participant discussion -- we construct the interaction network from which we derive topological embedding for each speaker. These speaker embedding enjoy the following property: speakers with the same stance tend to be represented by similar vectors, while antipodal vectors represent speakers with opposing stances. These embedding are then used to divide the speakers into stance-partitions. We evaluate our method on three different datasets from different platforms. Our method outperforms or is comparable with supervised models while providing confidence levels for its output. Furthermore, we demonstrate how the structural embedding relate to the valence expressed by the speakers. Finally, we discuss some limitations inherent to the framework.

ASER: Towards large-scale commonsense knowledge acquisition via higher-order selectional preference over eventualities

Commonsense knowledge acquisition and reasoning have long been a core artificial intelligence problem. However, in the past, there has been a lack of scalable methods to collect commonsense knowledge. In this paper, we propose to develop principles for collecting commonsense knowledge based on selectional preference, which is a common phenomenon in human languages that has been shown to be related to semantics. We generalize the definition of selectional preference from one-hop linguistic syntactic relations to higher-order relations over linguistic graphs. Unlike previous commonsense knowledge definitions (e.g., ConceptNet), the selectional preference (SP) knowledge only relies on statistical distributions over linguistic graphs, which can be efficiently and accurately acquired from the unlabeled corpora with modern tools, rather than human-defined relations. As a result, acquiring SP knowledge is a much more scalable way of acquiring commonsense knowledge. Following this principle, we develop a large-scale eventuality (a linguistic term covering activity, state, and event)-based knowledge graph ASER, where each eventuality is represented as a dependency graph, and the relation between them is a discourse relation defined in shallow discourse parsing. The higher-order selectional preference over collected linguistic graphs reflects various kinds of commonsense knowledge. For example, dogs are more likely to bark than cats as the eventuality “dog barks” appears 14,998 times in ASER while “cat barks” only appears 6 times. “Be hungry” is more likely to be the reason rather than result of “eat food” as the edge 〈“be hungry,” Cause, “eat food”〉 appears in ASER while 〈“eat food,” Cause, “be hungry”〉 does not. Moreover, motivated by the observation that humans understand events by abstracting the observed events to a higher level and can thus transfer their knowledge to new events, we propose a conceptualization module on top of the collected knowledge to significantly boost the coverage of ASER. In total, ASER contains 648 million edges between 438 million eventualities. After conceptualization with Probase, a selectional preference based concept-instance relational knowledge base, our concept graph contains 15 million conceptualized eventualities and 224 million edges between them. Detailed analysis is provided to demonstrate its quality. All the collected data, APIs, and tools that can help convert collected SP knowledge into the format of ConceptNet are available at https://github.com/HKUST-KnowComp/ASER.

Clausal Adjuncts of Romance Languages. A Corpus-driven Comparison

"Clausal Adjuncts of Romance Languages – a Corpus-Driven Comparison. Continuing the traditional comparative analysis of Romance languages, the present paper brings evidence for the fact that the syntactic configuration (the word order) of a Romance complex sentence (henceforth CS) with clausal adjuncts is the result of many competing forces. The analysis is based on an original media discourse corpus, collected from three Romance varieties (Romanian, Italian, and French), and the focus will be both on the word order of the temporal, causal, and concessive adjuncts (their occurrence in sentence-initial, in sentence-final position, sometimes even in sentence-middle position), and on the comparative analysis of the results. What I intend to prove is that the ordering of a CS in the three Romance languages is motivated by three types of factors: discourse pragmatics, semantics, and parsing. Keywords: clausal adjuncts, morphosyntactic analysis, word order, Romance languages "

Out-of-Domain Discourse Dependency Parsing via Bootstrapping: An Empirical Analysis on Its Effectiveness and Limitation

Abstract Discourse parsing has been studied for decades. However, it still remains challenging to utilize discourse parsing for real-world applications because the parsing accuracy degrades significantly on out-of-domain text. In this paper, we report and discuss the effectiveness and limitations of bootstrapping methods for adapting modern BERT-based discourse dependency parsers to out-of-domain text without relying on additional human supervision. Specifically, we investigate self-training, co-training, tri-training, and asymmetric tri-training of graph-based and transition-based discourse dependency parsing models, as well as confidence measures and sample selection criteria in two adaptation scenarios: monologue adaptation between scientific disciplines and dialogue genre adaptation. We also release COVID-19 Discourse Dependency Treebank (COVID19-DTB), a new manually annotated resource for discourse dependency parsing of biomedical paper abstracts. The experimental results show that bootstrapping is significantly and consistently effective for unsupervised domain adaptation of discourse dependency parsing, but the low coverage of accurately predicted pseudo labels is a bottleneck for further improvement. We show that active learning can mitigate this limitation.

An Analysis on Semantic Interpretation of Tamil Literary Texts

The interaction between a computer and a human or natural language is known as Natural Language Processing (NLP). The ultimate goal is to make the natural language text understandable, which in turn, requires its meaning to be captured. Text can be analyzed on several levels, such as lexical, syntax, semantics, discourse, and pragmatics. These NLP tasks deal with text at different levels, such as word, phrase, sentence, paragraph, and document. Discourse analysis is a type of text analysis that goes beyond the sentence level. The discourse analysis is currently performed on expository (essay) type of texts. There are currently no state-of-the-art NLP applications that handle Tamil literary texts at a discourse level. Tamil classical literature is rich with ethical, moral, and philosophical values that should be explored for the benefit of society. This paper proposes an automatic semantic interpretation framework for Tamil literary texts using discourse parsing by giving works on discourse parsing, text classification, discourse-based clustering and information retrieval, and Tamil language and Tamil literatures. This semantic interpretation can be developed as a smart mobile application using multimedia components. This paper also discusses how the Tamil literary text processing differs from the essay type of text.

A survey of discourse parsing

A survey of discourse parsing

Discourse Parsing for Contentious, Non-Convergent Online Discussions

Online discourse is often perceived as polarized and unproductive. While some conversational discourse parsing frameworks are available, they do not naturally lend themselves to the analysis of contentious and polarizing discussions. Inspired by the Bakhtinian theory of Dialogism, we propose a novel theoretical and computational framework, better suited for non-convergent discussions. We redefine the measure of a successful discussion, and develop a novel discourse annotation schema which reflects a hierarchy of discursive strategies. We consider an array of classification models -- from Logistic Regression to BERT. We also consider various feature types and representations, e.g., LIWC categories, standard embeddings, conversational sequences, and non-conversational discourse markers learnt separately. Given the 31 labels in the tagset, an average F-Score of 0.61 is achieved if we allow a different model for each tag, and 0.526 with a single model. The promising results achieved in annotating discussions according to the proposed schema paves the way for a number of downstream tasks and applications such as early detection of discussion trajectories, active moderation of open discussions, and teacher-assistive bots. Finally, we share the first labeled dataset of contentious non-convergent online discussions.

Hierarchical Macro Discourse Parsing Based on Topic Segmentation

Hierarchically constructing micro (i.e., intra-sentence or inter-sentence) discourse structure trees using explicit boundaries (e.g., sentence and paragraph boundaries) has been proved to be an effective strategy. However, it is difficult to apply this strategy to document-level macro (i.e., inter-paragraph) discourse parsing, the more challenging task, due to the lack of explicit boundaries at the higher level. To alleviate this issue, we introduce a topic segmentation mechanism to detect implicit topic boundaries and then help the document-level macro discourse parser to construct better discourse trees hierarchically. In particular, our parser first splits a document into several sections using the topic boundaries that the topic segmentation detects. Then it builds a smaller and more accurate discourse sub-tree in each section and sequentially forms a whole tree for a document. The experimental results on both Chinese MCDTB and English RST-DT show that our proposed method outperforms the state-of-the-art baselines significantly.

Unsupervised Learning of Discourse Structures using a Tree Autoencoder

Discourse information, as postulated by popular discourse theories, such as RST and PDTB, has been shown to improve an increasing number of downstream NLP tasks, showing positive effects and synergies of discourse with important real-world applications. While methods for incorporating discourse become more and more sophisticated, the growing need for robust and general discourse structures has not been sufficiently met by current discourse parsers, usually trained on small scale datasets in a strictly limited number of domains. This makes the prediction for arbitrary tasks noisy and unreliable. The overall resulting lack of high-quality, high-quantity discourse trees poses a severe limitation to further progress. In order the alleviate this shortcoming, we propose a new strategy to generate tree structures in a task-agnostic, unsupervised fashion by extending a latent tree induction framework with an auto-encoding objective. The proposed approach can be applied to any tree-structured objective, such as syntactic parsing, discourse parsing and others. However, due to the especially difficult annotation process to generate discourse trees, we initially develop a method to generate larger and more diverse discourse treebanks. In this paper we are inferring general tree structures of natural text in multiple domains, showing promising results on a diverse set of tasks.