Learning explicit and implicit Arabic discourse relations

Many discourse relations are explicitly marked with discourse connectives, and these examples could potentially serve as a plentiful source of training data for recognizing implicit discourse relations. However, there are important linguistic differences between explicit and implicit discourse relations, which limit the accuracy of such an approach. We account for these differences by applying techniques from domain adaptation, treating implicitly and explicitly-marked discourse relations as separate domains. The distribution of surface features varies across these two domains, so we apply a marginalized denoising autoencoder to induce a dense, domain-general representation. The label distribution is also domain-specific, so we apply a resampling technique that is similar to instance weighting. In combination with a set of automatically-labeled data, these improvements eliminate more than 80% of the transfer loss incurred by training an implicit discourse relation classifier on explicitly-marked discourse relations.

Discourse relations between two text segments play an important role in many Natural Language Processing (NLP) tasks. The connectives strongly indicate the sense of discourse relations, while in fact, there are no connectives in a large proportion of discourse relations, that is, implicit discourse relations. Compared with explicit relations, implicit relations are much harder to detect and have drawn significant attention. Until now, there have been many studies focusing on English implicit discourse relations, and few studies address implicit relation recognition in Chinese even though the implicit discourse relations in Chinese are more common than those in English. In our work, both the English and Chinese languages are our focus. The key to implicit relation prediction is to properly model the semantics of the two discourse arguments, as well as the contextual interaction between them. To achieve this goal, we propose a neural network based framework that consists of two hierarchies. The first one is the model hierarchy, in which we propose a max-margin learning method to explore the implicit discourse relation from multiple views. The second one is the feature hierarchy, in which we learn multilevel distributed representations from words, arguments, and syntactic structures to sentences. We have conducted experiments on the standard benchmarks of English and Chinese, and the results show that compared with several methods our proposed method can achieve the best performance in most cases.

BackgroundIdentification of discourse relations, such as causal and contrastive relations, between situations mentioned in text is an important task for biomedical text-mining. A biomedical text corpus annotated with discourse relations would be very useful for developing and evaluating methods for biomedical discourse processing. However, little effort has been made to develop such an annotated resource.ResultsWe have developed the Biomedical Discourse Relation Bank (BioDRB), in which we have annotated explicit and implicit discourse relations in 24 open-access full-text biomedical articles from the GENIA corpus. Guidelines for the annotation were adapted from the Penn Discourse TreeBank (PDTB), which has discourse relations annotated over open-domain news articles. We introduced new conventions and modifications to the sense classification. We report reliable inter-annotator agreement of over 80% for all sub-tasks. Experiments for identifying the sense of explicit discourse connectives show the connective itself as a highly reliable indicator for coarse sense classification (accuracy 90.9% and F1 score 0.89). These results are comparable to results obtained with the same classifier on the PDTB data. With more refined sense classification, there is degradation in performance (accuracy 69.2% and F1 score 0.28), mainly due to sparsity in the data. The size of the corpus was found to be sufficient for identifying the sense of explicit connectives, with classifier performance stabilizing at about 1900 training instances. Finally, the classifier performs poorly when trained on PDTB and tested on BioDRB (accuracy 54.5% and F1 score 0.57).ConclusionOur work shows that discourse relations can be reliably annotated in biomedical text. Coarse sense disambiguation of explicit connectives can be done with high reliability by using just the connective as a feature, but more refined sense classification requires either richer features or more annotated data. The poor performance of a classifier trained in the open domain and tested in the biomedical domain suggests significant differences in the semantic usage of connectives across these domains, and provides robust evidence for a biomedical sublanguage for discourse and the need to develop a specialized biomedical discourse annotated corpus. The results of our cross-domain experiments are consistent with related work on identifying connectives in BioDRB.

In this paper we propose a first empirical mapping between the RST-DT and the PDTB 3.0.We provide an original algorithm which allows the mapping of 6,510 (80.0%) explicit and implicit discourse relations between the overlapping articles of the RST-DT and PDTB 3.0 discourse annotated corpora.Results of the mapping show that while it is easier to align segments of implicit discourse relations, the mapping obtained between the aligned explicit discourse relations is more unambiguous.

In this work, we describe how EdgeMaps provide a new method for integrating the visualization of explicit and implicit data relations. Explicit relations are specific connections between entities already present in a given data set, while implicit relations are derived from multidimensional data based on similarity measures. Many data sets include both types of relations, which are often difficult to represent together in information visualizations. Node-link diagrams typically focus on explicit data connections while not incorporating implicit similarities between entities. Multidimensional scaling considers similarities between items; however, explicit links between nodes are not displayed. In contrast, EdgeMaps visualize both explicit and implicit relations by combining graph drawing and spatiatization techniques. We have applied this technique to three case studies [philosophers, painters, and musicians] and explored how integrated visualizations of explicit and implicit relations reveal novel patterns and relationships.

The paper presents the Chinese Discourse TreeBank, a corpus annotated with Penn Discourse TreeBank style discourse relations that take the form of a predicate taking two arguments. We first characterize the syntactic and statistical distributions of Chinese discourse connectives as well as the role of Chinese punctuation marks in discourse annotation, and then describe how we design our annotation strategy procedure based on this characterization. The Chinese-specific features of our annotation strategy include annotating explicit and implicit discourse relations in one single pass, defining the argument labels on semantic, rather than syntactic, grounds, as well as annotating the semantic type of implicit discourse relations directly. We also introduce a flat, 11-valued semantic type classification scheme for discourse relations. We finally demonstrate the feasibility of our approach with evaluation results.

Discourse relations represent how textual units logically connect with each other. Analyzing the discourse structure for texts could aid the understanding of the meaning behind paragraphs. There are many potential applications such as natural language interface and large-scale content-analysis. Although there are popular English discourse corpora for researchers, large-scale Chinese discourse corpora have not been available until recently. In addition, Chinese discourse analysis has many unique issues including the variety of discourse connectives, the common occurrences of parallel connectives, and the complex sentence structures. Discourse connectives are important clues for identifying discourse relations in Chinese texts. However, the ambiguity involved makes it a challenge to extract true connectives. In this thesis, we investigate four tasks regarding explicit discourse relations that are signaled by discourse connectives. Firstly, we deal with the extraction of explicit discourse connectives. Secondly, we investigate resolving linking ambiguities among connective components. Thirdly, we disambiguate the discourse relation type for each connective. Finally, we extract the arguments for each discourse connective. Several features are proposed to train Logistic Regression classifiers to disambiguate between discourse and non-discourse usages and the relation types for connectives. Additionally, we rank each connective candidate and develop a greedy algorithm to resolve linking ambiguities. Finally, the argument identification is formulated as a sequence labeling problem, and Conditional Random Fields are utilized to determine the argument boundaries. Besides explicit discourse relations, further investigation must be done to recognize implicit relations. Built upon these components, an end-to-end discourse parser for Chinese may be constructed in future studies.

Inferring implicatures and discourse relations from frame information

The PDTB-3 contains many more Implicit discourse relations than the previous PDTB-2. This is in part because implicit relations have now been annotated within sentences as well as between them. In addition, some now co-occur with explicit discourse relations, instead of standing on their own. Here we show that while this can complicate the problem of identifying the location of implicit discourse relations, it can in turn simplify the problem of identifying their senses. We present data to support this claim, as well as methods that can serve as a non-trivial baseline for future state-of-the-art recognizers for implicit discourse relations.

In implicit discourse relation classification, we want to predict the relation between adjacent sentences in the absence of any overt discourse connectives. This is challenging even for humans, leading to shortage of annotated data, a fact that makes the task even more difficult for supervised machine learning approaches. In the current study, we perform implicit discourse relation classification without relying on any labeled implicit relation. We sidestep the lack of data through explicitation of implicit relations to reduce the task to two sub-problems: language modeling and explicit discourse relation classification, a much easier problem. Our experimental results show that this method can even marginally outperform the state-of-the-art, in spite of being much simpler than alternative models of comparable performance. Moreover, we show that the achieved performance is robust across domains as suggested by the zero-shot experiments on a completely different domain. This indicates that recent advances in language modeling have made language models sufficiently good at capturing inter-sentence relations without the help of explicit discourse markers.

Though an important task in natural language processing, discourse relation recognition has not until recently received as much attention as it deserves, maybe due to its complexity. In Chinese, discourse relations are mostly implicit, making the task even harder. There are more explicit discourse relations in English than Chinese. In this paper, we propose a new approach to Chinese discourse relation recognition, which utilizes English-Chinese alignment corpus to discover implicit Chinese discourse relations. Results show our method achieves 60% accuracy in argument detection and 40% accuracy in discourse relation recognition.

Following the conventions developed in general domain, most of the current work on clinical temporal relation identification aims to identify a comprehensive set of temporal relations from source documents. This includes both explicit relations that is described in the documents and implicit relations that are identifiable only through inference. Although such an approach may provide a complete view of temporal information provided in a document, some temporal relations may not be practically essential, depending on the clinical application at hand. In addition, the performances of current systems that identify both explicit and implicit relations are still low and how to enhance the performances to be enough for practical use is not clear yet. In this paper, we propose focus on a subset of temporal relations, in order to provide insights into how to develop practically useful temporal information extraction methods for clinical text. We focus on “direct” temporal relations, which are intra-sentential temporal relations between a time expression and an event mention with limited syntactic distance. A corpus of 120 discharge summaries is constructed, leveraging an existing corpus, the 2012 i2b2 corpus. We show that the direct temporal relations constitute a major category of temporal relations. In addition, we show that the performance of the state-of-the art temporal relation extraction system, which is developed for both implicit and explicit relations, on direct temporal relations is still low. This indicates the need for development of methods tailored to direct temporal relations.

The present work deals with signals for marking discourse relations, which play an outstanding role for the understanding of a text. The aim of the study is to use German and Taiwanese newspaper editorials to capture signals that mark discourse relations. An attempt is made to classify signals according to their linking functions. This should contribute to the fact that implicit and explicit discourse relations can be differentiated from one another and compared with each other. In the literature search, a contrastive study on discourse relations around newspaper editorials is not yet available. It is therefore interesting to look at how signals that mark discourse relation are used in the newspaper editorial, and whether language-specific differences can be derived from them, and what conclusions can be drawn from the results of the analysis on foreign language didactics.

Discourse relation classification is an important component for automatic discourse parsing and natural language understanding. The performance bottleneck of a discourse parser comes from implicit discourse relations, whose discourse connectives are not overtly present. Explicit discourse connectives can potentially be exploited to collect more training data to collect more data and boost the performance. However, using them indiscriminately has been shown to hurt the performance because not all discourse connectives can be dropped arbitrarily. Based on this insight, we investigate the interaction between discourse connectives and the discourse relations and propose the criteria for selecting the discourse connectives that can be dropped independently of the context without changing the interpretation of the discourse. Extra training data collected only by the freely omissible connectives improve the performance of the system without additional features.

The knowledge concept recommendation in Massive Open Online Courses (MOOCs) is a significant issue that has garnered widespread attention. Existing methods primarily rely on the explicit relations between users and knowledge concepts on the MOOC platforms for recommendation. However, there are numerous implicit relations (e.g., shared interests or same knowledge levels between users) generated within the users' learning activities on the MOOC platforms. Existing methods fail to consider these implicit relations, and these relations themselves are difficult to learn and represent, causing poor performance in knowledge concept recommendation and an inability to meet users' personalized needs. To address this issue, we propose a novel framework based on contrastive learning, which can represent and balance the explicit and implicit relations for knowledge concept recommendation in MOOCs (CL-KCRec). Specifically, we first construct a MOOCs heterogeneous information network (HIN) by modeling the data from the MOOC platforms. Then, we utilize a relation-updated graph convolutional network and stacked multi-channel graph neural network to represent the explicit and implicit relations in the HIN, respectively. Considering that the quantity of explicit relations is relatively fewer compared to implicit relations in MOOCs, we propose a contrastive learning with prototypical graph to enhance the representations of both relations to capture their fruitful inherent relational knowledge, which can guide the propagation of students' preferences within the HIN. Based on these enhanced representations, to ensure the balanced contribution of both towards the final recommendation, we propose a dual-head attention mechanism for balanced fusion. Experimental results demonstrate that CL-KCRec outperforms several state-of-the-art baselines on real-world datasets in terms of HR, NDCG and MRR.