Unification Grammar Research Articles

Obituary: Martin Kay

Obituary: Martin Kay

ARTEMIS: parsing non-peripheral complex sentences in ASD-STE100

The simulation of natural language understanding is one of the main objectives of natural language processing (NLP). Within the different applications designed for this purpose, the ARTEMIS prototype follows the paradigm of unification grammars, and unlike other trending computational resources, it is theoretically grounded in Role and Reference Grammar (RRG). The syntax-to-semantics linking algorithm proposed in this functional grammar lies at the basis of a parsing process that starts with a natural language sentence, extracts its morphosyntactic features and provides a representation of these in terms of the so-called layered structure of the clause (LSC). The Grammar Development Environment (GDE) in ARTEMIS is a major component where feature- based production rules (syntactic, lexical and constructional) are stored and ready to allow the generation of the enhanced layered structure of the clause of natural language expressions. Syntactic rules that account for phrasal constituents and simple sentences have already been described, but it is now turn to focus on the study of non-peripheral complex sentences. In an attempt to validate these syntactic rules and to avoid some of the common problems that may arise in parsing applications, our research will concentrate on the analysis of RRG’s juncture-nexus combinations as found in a controlled natural language (CNL) such as ASD-STE100 (Aerospace and Defence Simplified Technical English).

Computational Psycholinguistics

Computational Psycholinguistics

Parsing adverbial complex sentences in ASD-STE100 within ARTEMIS

One of the main objectives of Natural Language Processing is the simulation of natural language understanding. Within the applications designed for this purpose today, ARTEMIS follows the paradigm of unification grammars (Sag, I., Wasow, T. & Bender, E. 2003), and unlike other trending computational resources, it is theoretically grounded in linguistic models like RRG (Van Valin & LaPolla 1997 and Van Valin 2005), whose linking algorithm lies at the basis of the parsing process of our interlingua-based system.
 A fundamental component of ARTEMIS is the Grammar Development Environment (GDE), where feature-based production rules (syntactic, constructional and lexical) are stored and ready to allow the generation of the enhanced layered structure of the clause of natural language expressions (Periñán-Pascual 2013: 222). Syntactic rules for phrasal constituents and simple sentences have already been described (Cortés-Rodríguez 2016; Cortés-Rodríguez & Mairal-Usón 2016; Díaz-Galán & Fumero-Pérez 2015; Fumero-Pérez & Díaz-Galán 2017; Martín-Díaz 2017 and Martín-Díaz 2018), but it now turns to focus on complex sentences, and, to be more precise, on adverbial subordination.
 Bearing in mind the validation of these syntactic rules and the common problems that may arise in such parsing applications, our research will concentrate on the analysis of these structures as found in a Controlled Natural Language: ASD-STE100.

ARTEMIS: State of the Art and Future Horizons

The syntactic parser ARTEMIS (Automatically Representing Text Meaning via an Interlingua-based System) is a prototype intended to process natural language within the environment of the Functional Grammar Knowledge Base (FunGramKB) (Perinan-Pascual & Arcas-Tunez, 2010). Different from other parsing devices, ARTEMIS is grounded on two functional linguistic models: Role and Reference Grammar (RRG) (Van Valin & La Polla, 1997; Van Valin, 2005) and the Lexical Constructional Model (LCM) (Ruiz de Mendoza & Mairal Uson, 2008). However, certain adjustments have to be made to both models in order to meet the requirements derived from the fact that ARTEMIS follows the paradigm of Unification Grammars, in such a way that to comply with this paradigm, the GDE (Grammar Development Environment) within ARTEMIS needs to integrate two types of constructs: a catalogue of Attribute-Value Matrixes (AVMs) to describe grammatical units, and a set of production rules (grammatical, lexical and constructional) to allow it to produce a feature based grammar. It is the aim of this paper to give an overview of the investigation carried out so far within ARTEMIS in relation to these two aspects. We will do so by revisiting the literature in relation to the adjustments made to the linguistic models, especially to the RRG, and by reviewing the efforts made to describe the units and design the rules necessary for the parsing of simple sentences in English. Our paper will conclude by pointing at prospective research needed for the completion of this project.

On the Universal Generation Problem for Unification Grammars

The universal generation problem for unification grammars is the problem of determining whether a given grammar derives any terminal string with a given feature structure. It is known that the problem is decidable for LFG and PATR grammars if only acyclic feature structures are taken into consideration. In this brief note, we show that the problem is undecidable for cyclic structures. This holds even for grammars that are off-line parsable.

Implicit structured sequence learning: an fMRI study of the structural mere-exposure effect.

In this event-related fMRI study we investigated the effect of 5 days of implicit acquisition on preference classification by means of an artificial grammar learning (AGL) paradigm based on the structural mere-exposure effect and preference classification using a simple right-linear unification grammar. This allowed us to investigate implicit AGL in a proper learning design by including baseline measurements prior to grammar exposure. After 5 days of implicit acquisition, the fMRI results showed activations in a network of brain regions including the inferior frontal (centered on BA 44/45) and the medial prefrontal regions (centered on BA 8/32). Importantly, and central to this study, the inclusion of a naive preference fMRI baseline measurement allowed us to conclude that these fMRI findings were the intrinsic outcomes of the learning process itself and not a reflection of a preexisting functionality recruited during classification, independent of acquisition. Support for the implicit nature of the knowledge utilized during preference classification on day 5 come from the fact that the basal ganglia, associated with implicit procedural learning, were activated during classification, while the medial temporal lobe system, associated with explicit declarative memory, was consistently deactivated. Thus, preference classification in combination with structural mere-exposure can be used to investigate structural sequence processing (syntax) in unsupervised AGL paradigms with proper learning designs.

Polynomially parsable unification grammars

Unification grammars (UG) are a grammatical formalism that underlies several contemporary linguistic theories, including Lexical-functional Grammar and Head-driven Phrase-structure Grammar. UG is an especially attractive formalism because of its expressivity, which facilitates the expression of complex linguistic structures and relations. Formally, UG is Turing-complete, generating the entire class of recursively enumerable languages. This expressivity, however, comes at a price: the universal recognition problem is undecidable for arbitrary unification grammars. We define a constrained version of UG that guarantees efficient processing, while allowing the expression of complex linguistic structures. We do so by proving that the constrained formalism is equivalent to Range Concatenation Grammar, a formalism that generates exactly the class of languages recognizable in deterministic polynomial time. We thus obtain a grammatical formalism that is on one hand highly expressive, and on the other efficient to compute with.

Unification-Based Parsing Applications for Intelligent Foreign Language Tutoring Systems

This paper is a description of a computer based intelligent tutoring system for French as a second language, and of its ancillary grammar editor, which is built on an object oriented, unification-based natural language processor. First I review some of the considerations that motivate using intelligent tutoring systems in second language education. Second, I discuss the details of object oriented programming and unification grammar. Third, I give some examples of how this software interacts with students and of what sort of grammatical phenomena it is capable of handling. I will show how the parser straightforwardly handles some thorny problems of complex grammatical structures (e.g. conjunctions; reflexive binding; phrase embedding; dislocated, missing, or superfluous parts of speech) which have had an inhibitory effect on CALL parsing applications in the past.

Towards Modular Development of Typed Unification Grammars

Development of large-scale grammars for natural languages is a complicated endeavor: Grammars are developed collaboratively by teams of linguists, computational linguists, and computer scientists, in a process very similar to the development of large-scale software. Grammars are written in grammatical formalisms that resemble very-high-level programming languages, and are thus very similar to computer programs. Yet grammar engineering is still in its infancy: Few grammar development environments support sophisticated modularized grammar development, in the form of distribution of the grammar development effort, combination of sub-grammars, separate compilation and automatic linkage, information encapsulation, and so forth. This work provides preliminary foundations for modular construction of (typed) unification grammars for natural languages. Much of the information in such formalisms is encoded by the type signature, and we subsequently address the problem through the distribution of the signature among the different modules. We define signature modules and provide operators of module combination. Modules may specify only partial information about the components of the signature and may communicate through parameters, similarly to function calls in programming languages. Our definitions are inspired by methods and techniques of programming language theory and software engineering and are motivated by the actual needs of grammar developers, obtained through a careful examination of existing grammars. We show that our definitions meet these needs by conforming to a detailed set of desiderata. We demonstrate the utility of our definitions by providing a modular design of the HPSG grammar of Pollard and Sag.

What artificial grammar learning reveals about the neurobiology of syntax

In this paper we examine the neurobiological correlates of syntax, the processing of structured sequences, by comparing FMRI results on artificial and natural language syntax. We discuss these and similar findings in the context of formal language and computability theory. We used a simple right-linear unification grammar in an implicit artificial grammar learning paradigm in 32 healthy Dutch university students (natural language FMRI data were already acquired for these participants). We predicted that artificial syntax processing would engage the left inferior frontal region (BA 44/45) and that this activation would overlap with syntax-related variability observed in the natural language experiment. The main findings of this study show that the left inferior frontal region centered on BA 44/45 is active during artificial syntax processing of well-formed (grammatical) sequence independent of local subsequence familiarity. The same region is engaged to a greater extent when a syntactic violation is present and structural unification becomes difficult or impossible. The effects related to artificial syntax in the left inferior frontal region (BA 44/45) were essentially identical when we masked these with activity related to natural syntax in the same subjects. Finally, the medial temporal lobe was deactivated during this operation, consistent with the view that implicit processing does not rely on declarative memory mechanisms that engage the medial temporal lobe. In the context of recent FMRI findings, we raise the question whether Broca’s region (or subregions) is specifically related to syntactic movement operations or the processing of hierarchically nested non-adjacent dependencies in the discussion section. We conclude that this is not the case. Instead, we argue that the left inferior frontal region is a generic on-line sequence processor that unifies information from various sources in an incremental and recursive manner, independent of whether there are any processing requirements related to syntactic movement or hierarchically nested structures. In addition, we argue that the Chomsky hierarchy is not directly relevant for neurobiological systems.

Toward the Formal Verification of a Unification System $^{\ast}$

Unification grammars are widely used for encoding human knowledge. For unification systems, one major difficulty is the debugging of rules. In this paper, the authors suggest a novel method based on model checking to theoretically verify a complex grammar system for a unification-based parser. We propose the modeling method of the grammar and, more importantly, the abstraction method to compress the state space. We apply partial Kripke structures to model the rules. We prove that the state space can be reduced by several orders of magnitude while still keeping the behaviors of a noncompressed one. Practical verification issues are discussed, including the restrictions on specifications, the properties to check, etc. The proposed method will contribute to the effective debugging and application of unification grammars.

Associative Grammar Combination Operators for Tree-Based Grammars

Polarized unification grammar (PUG) is a linguistic formalism which uses polarities to better control the way grammar fragments interact. The grammar combination operation of PUG was conjectured to be associative. We show that PUG grammar combination is not associative, and even attaching polarities to objects does not make it order-independent. Moreover, we prove that no non-trivial polarity system exists for which grammar combination is associative. We then redefine the grammar combination operator, moving to the powerset domain, in a way that guarantees associativity. The method we propose is general and is applicable to a variety of tree-based grammar formalisms.

Toward the Formal Verification of a Unification System

Unification grammars are widely used for encoding human knowledge. For unification systems, one major difficulty is the debugging of rules. In this paper, the authors suggest a novel method based on model checking to theoretically verify a complex grammar system for a unification-based parser. We propose the modeling method of the grammar and, more importantly, the abstraction method to compress the state space. We apply partial Kripke structures to model the rules. We prove that the state space can be reduced by several orders of magnitude while still keeping the behaviors of a noncompressed one. Practical verification issues are discussed, including the restrictions on specifications, the properties to check, etc. The proposed method will contribute to the effective debugging and application of unification grammars.

Highly Constrained Unification Grammars

Unification grammars are widely accepted as an expressive means for describing the structure of natural languages. In general, the recognition problem is undecidable for unification grammars. Even with restricted variants of the formalism, off-line parsable grammars, the problem is computationally hard. We present two natural constraints on unification grammars which limit their expressivity and allow for efficient processing. We first show that non-reentrant unification grammars generate exactly the class of context-free languages. We then relax the constraint and show that one-reentrant unification grammars generate exactly the class of mildly context-sensitive languages. We thus relate the commonly used and linguistically motivated formalism of unification grammars to more restricted, computationally tractable classes of languages.

From UBGs to CFGs A practical corpus-driven approach

AbstractWe present a simple and intuitive unsound corpus-driven approximation method for turning unification-based grammars, such as HPSG, CLE, or PATR-II into context-free grammars (CFGs). Our research is motivated by the idea that we can exploit (large-scale), hand-written unification grammars not only for the purpose of describing natural language and obtaining a syntactic structure (and perhaps a semantic form), but also to address several other very practical topics. Firstly, to speed up deep parsing by having a cheap recognition pre-flter (the approximated CFG). Secondly, to obtain an indirect stochastic parsing model for the unification grammar through a trained PCFG, obtained from the approximated CFG. This gives us an efficient disambiguation model for the unification-based grammar. Thirdly, to generate domain-specific subgrammars for application areas such as information extraction or question answering. And finally, to compile context-free language models which assist the acoustic model of a speech recognizer. The approximation method is unsound in that it does not generate a CFG whose language is a true superset of the language accepted by the original unification-based grammar. It is a corpus-driven method in that it relies on a corpus of parsed sentences and generates broader CFGs when given more input samples. Our open approach can be fine-tuned in different directions, allowing us to monotonically come close to the original parse trees by shifting more information into the context-free symbols. The approach has been fully implemented inJAVA.

Unification Grammars and Off-Line Parsability

Unification grammars are known to be Turing-equivalent; given a grammar G and a word w, it is undecidable whether w ∈ L(G). In order to ensure decidability, several constraints on grammars, commonly known as off-line parsability (OLP), were suggested, such that the recognition problem is decidable for grammars which satisfy OLP. An open question is whether it is decidable if a given grammar satisfies OLP. In this paper we investigate various definitions of OLP and discuss their interrelations, proving that some of the OLP variants are indeed undecidable. We then present a novel, decidable OLP constraint which is more liberal than the existing decidable ones.

Tree Unification Grammar: Problems and Proposals for Topology, TAG, and German

This work presents a lexicalized grammar formalism which can be seen as a variant of multi-component tree adjoining grammar (TAG). This formalism is well-suited for describing the syntax of German because it relates a syntactic dependency graph with a hierarchy of topological domains. The topological phrase structure encodes the placement of verbal and nominal elements in the (ordered) field structure, just as in the classical topological analysis of the German sentence. This module constitutes an intermediate step between the semantic and the prosodic representations of the sentence and allows deriving cases of scrambling that are problematic for classical TAGs and for some phrase structure based formalisms.

An approach to the formal specification of lingware

This paper has two purposes. First, it suggests a formal approach for specifying and verifying lingware. This approach is based on a unified notation of the main existing formalisms for describing linguistic knowledge (i.e. Formal Grammars, Unification Grammars, HPSG, etc.) on the one hand, and the integration of data and processing on the other. Accordingly, a lingware specification includes all related aspects in a unified framework. This facilitates the development of a lingware system, since one has to follow a single development process instead of two separate ones. Secondly, it presents an environment for the formal specification of lingware, based on the suggested approach, which is neither restricted to a particular kind of application nor to a particular class of linguistic formalisms. This environment provides interfaces enabling the specification of both linguistic knowledge and functional aspects of a lingware system. Linguistic knowledge is specified with the usual grammatical formalisms, whereas functional aspects are specified with a suitable formal notation. Both descriptions will be integrated into the same framework to obtain a complete requirement specification that can be refined towards an executable program.

On the Semantics of Unification Grammars

An essential prerequisite for modular grammar design is a clear, mathematically well-founded definition for the semantics of grammar formalisms, facilitating reasoning about grammars and their computational properties. This paper shows that existing definitions for the semantics of unification grammars, both operational and denotational, are not compositional with respect to a simple and natural grammar combination operator. Adapting results from the semantics of logic programming languages, we suggest a denotational semantics that we show to be both compositional and fully-abstract. This semantics induces an equivalence relation by which two grammars are equivalent if and only if they can be interchanged in any context. This provides a clear, mathematically sound way for defining grammar modularity.