Top-down Parser Research Articles

Diverging neural dynamics for syntactic structure building in naturalistic speaking and listening

The neural correlates of sentence production are typically studied using task paradigms that differ considerably from the experience of speaking outside of an experimental setting. In this fMRI study, we aimed to gain a better understanding of syntactic processing in spontaneous production versus naturalistic comprehension in three regions of interest (BA44, BA45, and left posterior middle temporal gyrus). A group of participants (n = 16) was asked to speak about the events of an episode of a TV series in the scanner. Another group of participants (n = 36) listened to the spoken recall of a participant from the first group. To model syntactic processing, we extracted word-by-word metrics of phrase-structure building with a top-down and a bottom-up parser that make different hypotheses about the timing of structure building. While the top-down parser anticipates syntactic structure, sometimes before it is obvious to the listener, the bottom-up parser builds syntactic structure in an integratory way after all of the evidence has been presented. In comprehension, neural activity was found to be better modeled by the bottom-up parser, while in production, it was better modeled by the top-down parser. We additionally modeled structure building in production with two strategies that were developed here to make different predictions about the incrementality of structure building during speaking. We found evidence for highly incremental and anticipatory structure building in production, which was confirmed by a converging analysis of the pausing patterns in speech. Overall, this study shows the feasibility of studying the neural dynamics of spontaneous language production.

Relative clauses as a benchmark for Minimalist parsing

Minimalist grammars have been used recently in a series of papers to explain well-known contrasts in human sentence processing in terms of subtle structural differences. These proposals combine a top-down parser with complexity metrics that relate parsing difficulty to memory usage. So far, though, there has been no large-scale exploration of the space of viable metrics. Building on this earlier work, we compare the ability of 1600 metrics to derive several processing effects observed with relative clauses, many of which have been proven difficult to unify. We show that among those 1600 candidates, a few metrics (and only a few) can provide a unified account of all these contrasts. This is a welcome result for two reasons: First, it provides a novel account of extensively studied psycholinguistic data. Second, it significantly limits the number of viable metrics that may be applied to other phenomena, thus reducing theoretical indeterminacy.

Left recursion in Parsing Expression Grammars

Parsing Expression Grammars (PEGs) are a formalism that can describe all deterministic context-free languages through a set of rules that specify a top-down parser for some language. PEGs are easy to use, and there are efficient implementations of PEG libraries in several programming languages.A frequently missed feature of PEGs is left recursion, which is commonly used in Context-Free Grammars (CFGs) to encode left-associative operations. We present a simple conservative extension to the semantics of PEGs that gives useful meaning to direct and indirect left-recursive rules, and show that our extensions make it easy to express left-recursive idioms from CFGs in PEGs, with similar results. We prove the conservativeness of these extensions, and also prove that they work with any left-recursive PEG.PEGs can also be compiled to programs in a low-level parsing machine. We present an extension to the semantics of the operations of this parsing machine that let it interpret left-recursive PEGs, and prove that this extension is correct with regard to our semantics for left-recursive PEGs.

Two Models of Minimalist, Incremental Syntactic Analysis

Minimalist grammars (MGs) and multiple context-free grammars (MCFGs) are weakly equivalent in the sense that they define the same languages, a large mildly context-sensitive class that properly includes context-free languages. But in addition, for each MG, there is an MCFG which is strongly equivalent in the sense that it defines the same language with isomorphic derivations. However, the structure-building rules of MGs but not MCFGs are defined in a way that generalizes across categories. Consequently, MGs can be exponentially more succinct than their MCFG equivalents, and this difference shows in parsing models too. An incremental, top-down beam parser for MGs is defined here, sound and complete for all MGs, and hence also capable of parsing all MCFG languages. But since the parser represents its grammar transparently, the relative succinctness of MGs is again evident. Although the determinants of MG structure are narrowly and discretely defined, probabilistic influences from a much broader domain can influence even the earliest analytic steps, allowing frequency and context effects to come early and from almost anywhere, as expected in incremental models.

From EBNF to PEG

Parsing Expression Grammar PEG encodes a recursive-descent parser with limited backtracking. The parser has many useful properties, and with the use of memoization, it works in a linear time. In its appearance, PEG is almost identical to a grammar in Extended Backus-Naur Form EBNF, but usually defines a different language. However, in some cases only minor typographical changes are sufficient to convert an EBNF grammar into its PEG parser. As recently shown by Medeiros, this is, in particular, true for LL1 grammars. But this is also true for many non-LL1 grammars, which is interesting because the backtracking of PEG is often a convenient way to circumvent just the LL1 restriction. We formulate a number of conditions for EBNF grammar to become its own PEG parser, and arrive at a condition that we call LL1p, meaning that a top-down parser can choose its next action by looking at the input within the reach of one parsing procedure rather than by looking at the next letter. An extension to LLkp for k > 1 seems possible.

A systematic approach to the implementation of attribute grammars with conventional compiler construction tools

This article describes structure-preserving coding patterns to code arbitrary non-circular attribute grammars as syntax-directed translation schemes for bottom-up and top-down parser generation tools. In these translation schemes, semantic actions are written in terms of a small repertory of primitive attribution operations. By providing alternative implementations for these attribution operations, it is possible to plug in different semantic evaluation strategies in a seamlessly way (e.g., a demand-driven strategy, or a data-driven one). The pattern makes possible the direct implementation of attribute grammar-based specifications with widely-used translation schemedriven tools for the development of both bottom-up (e.g. YACC, BISON, CUP) and top-down (e.g., JavaCC, ANTLR) language translators. As a consequence, initial translation schemes can be successively refined to yield final efficient implementations. Since these implementations still preserve the ability to be extended with new features described at the attribute grammar level, the advantages from the point of view of development and maintenance become apparent.

The information conveyed by words in sentences.

A method is presented for calculating the amount of information conveyed to a hearer by a speaker emitting a sentence generated by a probabilistic grammar known to both parties. The method applies the work of Grenander (1967) to the intermediate states of a top-down parser. This allows the uncertainty about structural ambiguity to be calculated at each point in a sentence. Subtracting these values at successive points gives the information conveyed by a word in a sentence. Word-by-word information conveyed is calculated for several small probabilistic grammars, and it is suggested that the number of bits conveyed per word is a determinant of reading times and other measures of cognitive load.

Top-Down Parsing of Conjunctive Languages

This paper generalizes the notion of a strong LL(k) context-free grammar for the case of conjunctive grammars and develops a top-down parsing algorithm for the resulting language family. A top-down parser of a conjunctive language attempts to construct a derivation of the input string, at each step using a finite lookahead to determine which grammar rule to apply. Fragments of formulae that form the derivation are stored in a tree-structured pushdown. Two ways to implement top-down parsers are suggested: a table-driven parser with a tree-structured pushdown and a recursive descent parser. Both techniques naturally extend their context-free counterparts.

Probabilistic Top-Down Parsing and Language Modeling

This paper describes the functioning of a broad-coverage probabilistic top-down parser, and its application to the problem of language modeling for speech recognition. The paper first introduces key notions in language modeling and probabilistic parsing, and briefly reviews some previous approaches to using syntactic structure for language modeling. A lexicalized probabilistic top-down parser is then presented, which performs very well, in terms of both the accuracy of returned parses and the efficiency with which they are found, relative to the best broad-coverage statistical parsers. A new language model that utilizes probabilistic top-down parsing is then outlined, and empirical results show that it improves upon previous work in test corpus perplexity. Interpolation with a trigram model yields an exceptional improvement relative to the improvement observed by other models, demonstrating the degree to which the information captured by our parsing model is orthogonal to that captured by a trigram model. A small recognition experiment also demonstrates the utility of the model.

Recursive parser optimization by rewriting context-free grammars

The analysis of programming languages is a fundamental activity in the process of comprehending, maintaining, documenting and reengineering old software. However, the analysis may sometimes be inefficient because of the parsing methodology adopted. Prolog parsers, for instance, have two main drawbacks: the time spent on extracting information as a result of the backtracking mechanism, and the large memory occupation involved. This paper presents an algorithm that optimizes a top-down parser by imposing a set of rules on the grammar to be analyzed. A methodology is then defined for rewritting a context-free grammar ( LL( n) n > = 1), and a tool that can achieve this is illustrated. This tool, called FACTOTUM, has been implemented in LPA Prolog, and works on a PC (IBM-MSDOS) through the Windows medium. FACTOTUM is composed of nearly 250 productions, and is an important part of another reverse engineering tool called RE_Tool (Giannone and Maresca, 1995) which can analyze codes in a multilanguage environment. RE_Tool has been implemented in the software engineering laboratories of DIS at the University of Naples Federico II, where it is currently under experimentation.

Chart parsers as inference systems for fixed-mode logic programs

Logic programs resemble context-free grammars. Moreover, Prolog’s proof procedure can be viewed as a generalization of a simple top-down parser with backtracking. This simple parser has disadvantages that motivated the design of more sophisticated parsing methods. As similar disadvantages occur in Prolog’s proof procedure, it may be desirable to develop other proof procedures for logic programs than the one used by Prolog. The resemblance between definite clauses and productions suggests looking at parsing to develop such procedures. We obtain proof procedures for fixed-mode logic programs, based on “chart” parsers.

Using memorization to achieve polynomial complexity of purely functional executable specifications of non-deterministic top-down parsers

Norvig [1991] has shown that a process with properties that are similar to those of Earley's algorithm can be generated by a simple top-down backtracking parser when augmented by memoization. Unfortunately, Norvig's approach cannot be implemented directly in pure functional programming languages such as Miranda, Haskell, LML, etc, owing to the fact that such languages do not provide any form of mutable object. This restriction is a necessary condition for referential transparency, a property of programs that simplifies reasoning about them and which is widely regarded as one of the main advantages of the pure functional programming style. In this note, we show how Norvig's approach can be adapted and used to achieve polynomial complexity of non-deterministic top-down backtracking parsers that are implemented as executable specifications in pure functional programming languages. The technique that we present uses a slight variation of an approach to implementing memoization in pure functional languages that is described in Field and Harrison [1988].

Techniques for automatic memoization with applications to context-free parsing

It is shown that a process similar to Earley's algorithm can be generated by a simple top-down backtracking parser, when augmented by automatic memoization. The memoized parser has the same complex...

A top-down language analyzer

Parsing is the process which matches a given input against prestored possibilities (a grammar) to discover the structure of acceptable candidates. It is relevant to both natural and artificial languages--both English and command or query languages. Although parsing is necessary to computer processing of all languages, and the parser described in this paper (PARSYN) is a general-purpose top-down parser, the focus of this paper is its applicability as a computationally efficient processor of natural language. In natural language, a grammar is taken to cover a subset of the language; parsing proceeds both top-down looking for constituents and bottom-up replacing the words by syntactic categories for processing. Since procedural formulation is equivalent to an ATN, complete computational coverage of all natural language parsers can be obtained with template processing and an ATN. PARSYN is a computational accelerator for parsing which provides this coverage. The algorithm of PARSYN is a Recursive Transition Network (RTN) with interrupt capability; the combination can serve as an ATN or template processor. Yet, PARSYN is simple enough to be attached to a microprocessor based English-language processing system ( — BE—, “microBE”, for micro-processor-Based English). It is flexible enough to provide pattern-matching, template processing and parsing all with the same formulation, bringing these facilities to the micro-system level efficiently and economically.

Nonlinear Regression With Linear Constraints: An Extension of the Magnified Diagonal Method

A syntax-directed picture analysis system based on a formal picture description scheme is described. The system accepts a description of a set of pictures in terms of a grammar generating strings in a picture description language; the grammar is explicitly used to direct the analysis or parse, and to control the calls on pattern classification routines for primitive picture components. Pictures are represented by directed graphs with labeled edges, where the edges denote elementary picture components and the graph connectivity mirrors the picture component connectivity; blank and don't care “patterns” allow the description of simple relations between visible patterns. The bulk of the paper is concerned with the picture parsing algorithm which is an n -dimensional analog of a classical top-down string parser, and an application of an implemented system to the analysis of spark chamber film. The potential benefits of this approach, as demonstrated by the application, include ease of implementation and modification of picture processing systems, and simplification of the pattern recognition problem by automatically taking advantage of contextual information.

Parsing of Graph-Representable Pictures

A syntax-directed picture analysis system based on a formal picture description scheme is described. The system accepts a description of a set of pictures in terms of a grammar generating strings in a picture description language; the grammar is explicitly used to direct the analysis or parse, and to control the calls on pattern classification routines for primitive picture components. Pictures are represented by directed graphs with labeled edges, where the edges denote elementary picture components and the graph connectivity mirrors the picture component connectivity; blank and don't care “patterns” allow the description of simple relations between visible patterns. The bulk of the paper is concerned with the picture parsing algorithm which is an n -dimensional analog of a classical top-down string parser, and an application of an implemented system to the analysis of spark chamber film. The potential benefits of this approach, as demonstrated by the application, include ease of implementation and modification of picture processing systems, and simplification of the pattern recognition problem by automatically taking advantage of contextual information.