Ellipsis Resolution Research Articles

CGM: Copy Mechanism GPT with Mask for Ellipsis and Anaphora Resolution in Dialogue

GPT (Generative Pre-trained Transformer) is a generative language model that demonstrates outstanding performance in the field of text generation. Generally, the attention mechanism of the transformer model behaves similarly to a copy distribution. However, due to the absence of a dedicated encoder, it is challenging to ensure that the input is retained for generation. We propose a model that emphasizes the copy mechanism in GPT. We generate masks for the input words to initialize the distribution and explicitly encourage copying through training. To demonstrate the effectiveness of our approach, we conducted experiments to restore ellipsis and anaphora in dialogue. In a single domain, we achieved 0.4319 (BLEU), 0.6408 (Rouge-L), 0.9040 (simCSE), and 0.9070 (BERTScore), while in multi-domain settings we obtained 0.4611 (BLEU), 0.6379 (Rouge-L), 0.8902 (simCSE), and 0.8999 (BERTScore). Additionally, we evaluated the operation of the copy mechanism on out-of-domain data, yielding excellent results. We anticipate that applying the copy mechanism to GPT will be useful for utilizing language models in constrained situations.

Expectations modulate retrieval interference during ellipsis resolution

Memory operations during language comprehension are subject to interference: retrieval is harder when items are linguistically similar to each other. We test how such interference effects might be modulated by linguistic expectations. Theories differ in how these factors might interact; we consider three possibilities: (i) predictability determines the need for retrieval, (ii) predictability affects cue-preference during retrieval, or (iii) word predictability moderates the effect of noise in memory during retrieval. We first demonstrate that expectations for a target word modulate retrieval interference in Mandarin noun-phrase ellipsis in an electroencephalography (EEG) experiment. This result obtains in globally ungrammatical sentences – termed “facilitatory interference.” Such a pattern is inconsistent with theories that focus only on the need for retrieval. To tease apart cue-preferences from noisy-memory representations, we operationalize the latter using a Transformer neural network language model. Confronting the model with our stimuli reveals an interference effect, consistent with prior work, but that effect does not interact with predictability in contrast to human EEG results. Together, these data are most consistent with the hypothesis that the predictability of target items affects cue-preferences during retrieval.

Efficient Learning Approach for Pronominal Anaphora and Ellipsis Identification and Resolution in Arabic Texts

The anaphors and ellipses resolution is a challenging task for most of NLP applications. However, few works have addressed this problem in Arabic. In fact the richness of the Arabic language and the scarcity of labeled data make the task more difficult. This paper offers a generic approach that deals with the identification and the resolution of both pronominal anaphors and ellipses in Arabic texts. To identify referential anaphors and ellipses positions, we propose a self-training SVM method based on a set of pattern-based and linguistic-based criteria. For the resolution step, we propose a novel hybrid method combining a reinforcement learning method with Word Embedding models. The reinforcement learning method uses an adapted version of Q-learning algorithm to finds the optimal combination of features. It exploits a set of morphological and syntactical features. The Word Embedding based method uses word representation models to check the semantic validity of the candidates. The evaluation of the identification approach gives a precision of 99.23% for pronominal anaphors and 94.33% for ellipses. The resolution approach reaches a precision of 80% for pronominal anaphors and 97.5% for ellipses. Our proposed approaches can be easily adapted to English and French to provide multilingual approaches.

Ellipsis with a Coordinated Antecedent: An Alternative to V‐stranding VP‐ellipsis

AbstractGribanova (2013b) observes that when the antecedent clause for an object gap in Russian contains a disjunction of ditransitive VPs, the gap is interpreted as if it contained the disjunction, thus favoring a V‐stranding VP‐ellipsis (VSVPE) analysis over a simple Argument Ellipsis (AE) analysis. We show that VSVPE fails to predict the attested interpretation of the gap when the antecedent VP contains a conjunction. The data are fully explained by combining AE with a pragmatic (scalar) inference triggered in situations of indeterminate ellipsis resolution, leading to multiple “cycles” in the dynamic semantics.

What Do You Mean ‘Why?’: Resolving Sluices in Conversations

In conversation, we often ask one-word questions such as ‘Why?’ or ‘Who?’. Such questions are typically easy for humans to answer, but can be hard for computers, because their resolution requires retrieving both the right semantic frames and the right arguments from context. This paper introduces the novel ellipsis resolution task of resolving such one-word questions, referred to as sluices in linguistics. We present a crowd-sourced dataset containing annotations of sluices from over 4,000 dialogues collected from conversational QA datasets, as well as a series of strong baseline architectures.

Verb Phrase Ellipsis and Reflexive Anaphora Resolution in Second Language Acquisition: A Study of Spanish Learners of English

This paper analyses the choice of sloppy and strict interpretations of reflexive anaphora in verb phrase ellipsis from the perspective of Relevance Theory (RT) (Sperber and Wilson 1986, 1995, 2002, 2008; Wilson and Sperber 2002, 2004). Forty-four Spanish learners of English and 29 native speakers of English were administered two judgement tasks designed to gauge the effect of the Communicative Principle of Relevance on their interpretation of reflexive anaphora in bare, referential and non-referential contexts. Results showed that, in accordance with this principle, the sloppy interpretation is favoured in bare and non-referential contexts, while strict readings prevail in referential contexts, these preferences being less marked for L2 learners than for native speakers. Moreover, the sloppy interpretation is chosen more frequently when native speakers are given a non-referential context, whereas it decreases when L2 learners are provided with the very same context, indicating that the syntax-pragmatics interface makes up an information processing load and acts as a distractor for L2 English learners.

Agreement Mismatch in Partitive Relatives

Agreement Mismatch in Partitive Relatives

Resolution of Ellipsis in Stacked VPs

Resolution of Ellipsis in Stacked VPs

A Neural Network Approach to Verb Phrase Ellipsis Resolution

Verb Phrase Ellipsis (VPE) is a linguistic phenomenon, where some verb phrases as syntactic constituents are omitted and typically referred by an auxiliary verb. It is ubiquitous in both formal and informal text, such as news articles and dialogues. Previous work on VPE resolution mainly focused on manually constructing features extracted from auxiliary verbs, syntactic trees, etc. However, the optimization of feature representation, the effectiveness of continuous features and the automatic composition of features are not well addressed. In this paper, we explore the advantages of neural models on VPE resolution in both pipeline and end-to-end processes, comparing the differences between statistical and neural models. Two neural models, namely multi-layer perception and the Transformer, are employed for the subtasks of VPE detection and resolution. Experimental results show that the neural models outperform the state-of-the-art baselines in both subtasks and the end-to-end results.

Structural priming in production through ‘silence’: An investigation of verb phrase ellipsis and null complement anaphora

There are two common competing conceptions of how ellipsis can be resolved: in the first, ellipsis is resolved by constructing unpronounced syntactic representations at the ellipsis site; in the second, ellipsis can be resolved by consulting the semantic/discourse information present in the antecedent, without the mediation of any syntax at all. In four syntactic priming experiments, we examine whether resolving English VP ellipsis and Null Complement Anaphora involves accessing the syntactic representations, or only the semantic representations, of the antecedent clause. Our findings suggest both VPE and Null Complement Anaphora can trigger structural priming effects, but the conditions under which they trigger priming are different. These results have implications for both theories of structural priming mechanism and theories of ellipsis resolution.

Ellipsis resolution in Persian complex predicates

Abstract The study of Ellipsis has been of special interest to scholars of linguistics. One reason for this interest is the fact that the meaning of a sentence with ellipsis cannot purely be based on its antecedent. Elliptical constructions in the literature are categorised according to their syntactic structures. This article intends to discuss ellipsis in Persian Complex predicate constructions in which parts of the verbal construction along with some of its arguments are elided. We will argue that ellipsis resolution in such constructions is licensed due to the logical form of such verbal constructions and the relationship that holds amongst parallel elements in the source and the target sentence. Finally, we will propose that distinguishing between primary and secondary occurrences of anaphoric expressions in the logical form can account for the correct and grammatical reading of intended anaphoric expressions in elliptical sites.

Using Tectogrammatical Annotation for Studying Actors and Actions in Sallust’s Bellum Catilinae

Abstract In the context of the Index Thomisticus Treebank project, we have enhanced the full text of Bellum Catilinae by Sallust with semantic annotation. The annotation style resembles the one used for the so called “tectogrammatical” layer of the Prague Dependency Treebank. By exploiting the results of semantic role labeling, ellipsis resolution and coreference analysis, this paper presents a network-based study of the main Actors and Actions (and their relations) in Bellum Catilinae.

Voice Mismatch Meets Neurolinguistics

This paper aims to investigate Korean advanced L2 English learners’ strategies for ellipsis resolution during sentence processing. Ellipsis resolution is known to involve several stages of information processing from the initial step of detecting an ellipsis-licensing element by the parser to the final stage of integrating the ellipsis site with the information retrieved from the antecedent of the ellipsis site. In examining these steps, we have manipulated three factors: (i) TP vs. VP-ellipsis; (ii) two types of discourse coherence relations (resemblance(-contrast) vs. cause-effect relations); (iii) voice match vs. mismatch. We found through the ERP recordings that voice mismatch in TP ellipsis elicited N400, followed by P600, irrespective of discourse coherence relations. In contrast, voice mismatch in VP-ellipsis registered N400 only in resemblance(-contrast) relation, but not in cause-effect relation. These findings lead us to conclude that Korean advanced L2 learners of English seem to undergo the full sequence of processing stages required for ellipsis resolution.

An ERP Study of Ellipsis Resolution in Korean

Park, Myung-Kwan and Chung, Wonil. 2018. An ERP study of ellipsis resolution in Korean. Language and Information, 22.1, 29-56. The purpose of this paper is to investigate the prediction/integration of syntactic structure during sentence processing, employing the construction that allows a TP ellipsis or Sluicing-like interpretation in Korean. To this aim we recorded event-related potentials (ERPs) from Korean native speakers while they read ellipsis constructions where the syntactic context allows or disallows remnants/ survivors (mis)matching with their explicit/implicit correlates in terms of Case/case (postposition) particle, also reflecting voice (mis)match between the antecedent and the ellipsis clauses. The Korean speakers were able to predict clausal syntactic structure on the basis of the antecedent clause and integrate it into the ellipsis site, thereby quickly detecting voice match/mismatch, as suggested by an ERP difference between matched and mismatched remnants/survivors neighboring the ellipsis site.

Ellipsis and Coreference Resolution in a Computerized Virtual Patient Dialogue System.

This paper describes the design of an ellipsis and coreference resolution module integrated in a computerized virtual patient dialogue system. Real medical diagnosis dialogues have been collected and analyzed. Several groups of diagnosis-related concepts were defined and used to construct rules, patterns, and features to detect and resolve ellipsis and coreference. The best F-scores of ellipsis detection and resolution were 89.15% and 83.40%, respectively. The best F-scores of phrasal coreference detection and resolution were 93.83% and 83.40%, respectively. The accuracy of pronominal anaphora resolution was 92% for the 3rd-person singular pronouns referring to specific entities, and 97.31% for other pronouns.

Antecedent contained deletions in native and non-native sentence processing

Abstract We report the results from an eye-movement monitoring study investigating native (L1) and non-native (L2) speakers’ real-time processing of antecedent-contained deletion (ACD), a type of verb phrase ellipsis in which the ellipsis gap forms part of its own antecedent. The resulting interpretation problem is traditionally thought to be solved by quantifier raising, a covert scope-shifting operation that serves to remove the gap from within its antecedent. Our L2 group comprised advanced, native German-speaking L2 learners of English. The analysis of the eye-movement data showed that both L1 and L2 English speakers tried to recover the missing verb phrase after encountering the gap. Only the native speakers showed evidence of ellipsis resolution being affected by quantification, however. No effects of quantification following gap detection were found in the L2 group, by contrast, indicating that recovery of the elided material was accomplished independently from the object’s quantificational status in this group.

Detection and Resolution of Verb Phrase Ellipsis

Verb phrase (VP) ellipsis is the omission of a verb phrase whose meaning can be reconstructed from the linguistic or real-world context. It is licensed in English by auxiliary verbs, often modal auxiliaries: She can go to Hawaii but he can’t [e]. This paper describes a system called ViPER (VP Ellipsis Resolver) that detects and resolves VP ellipsis, relying on linguistic principles such as syntactic parallelism, modality correlations, and the delineation of core vs. peripheral sentence constituents. The key insight guiding the work is that not all cases of ellipsis are equally difficult: some can be detected and resolved with high confidence even before we are able to build systems with human-level semantic and pragmatic understanding of text.

Multiple Focus Strategies inPro‐Drop Languages: Evidence from Ellipsis in Spanish

AbstractIn this paper I use the case of Spanish to argue that language regularities may give rise to multiple strategies for marking focus. In addition to the well‐known observation that focus can be marked by word order and intonation, I present experimental results regarding ellipsis that show that overt full DPs in subject position in Spanish are marked as focused. This strategy is linked to a language regularity, namely the availability of silent (pro) subjects. In Spanish, there is a preference forprosubjects. Overt full DPs in subject position are marked, and the presence of overt full DPs is used to indicate focus on the subject (thepro‐drop hypothesis). I will provide a novel syntactic analysis of the ellipsis structures, discuss discourse licensing conditions and present two experiments that investigate preferences in ellipsis resolution and argue in favor of thepro‐drop hypothesis. Experiment 1 compares structural preferences for ellipsis resolution across bare argument ellipsis and replacives, investigating the role of syntax and the information‐structure status of the subject. Experiment 2 compares the resolution of ellipsis with antecedents with overt DP subjects versusprosubjects. The paper also establishes links with the processing of ellipsis in other languages.

On the Negativity of Negative Fragment Answers and Ellipsis Resolution

Based on the assumption that the amwu-(N)-to expressions in Korean are Negative Concord Items, this paper presents evidence for the inherent negativity of the amwu-(N)-to expressions. Further, along the line of the recent studies which argue that ellipsis ‘is’ subject to some syntactic identity condition over and above any semantic identity condition, this paper explores the syntactic resolution of Negative fragment ellipsis: the amwu-(N)-to expressions can be used without negation only in ellipsis exactly because ellipsis repairs a violation of Neg-feature checking requirement. In consequence, the puzzle of Island-Insensitivity of fragments in Korean can be handled.

Eliding the derivation: A minimalist formalization of ellipsis

In this paper I use the formal framework of minimalist grammars to implement a version of the traditional approach to ellipsis as ‘deletion under syntactic (derivational) identity’, which, in conjunction with canonical analyses of voice phenomena, immediately allows for voice mismatches in verb phrase ellipsis, but not in sluicing. This approach to ellipsis is naturally implemented in a parser by means of threading a state encoding a set of possible antecedent derivation contexts through the derivation tree. Similarities between ellipsis and pronominal resolution are easily stated in these terms. In the context of this implementation, two approaches to ellipsis in the transformational community are naturally seen as equivalent descriptions at different levels: the LF-copying approach to ellipsis resolution is best seen as a description of the parser, whereas the phonological deletion approach a description of the underlying relation between form and meaning.