Recursive Descent Research Articles

Recursive descent parsing for Boolean grammars

The recursive descent parsing method for the context-free grammars is extended for their generalization, Boolean grammars, which include explicit set-theoretic operations in the formalism of rules and which are formally defined by language equations. The algorithm is applicable to a subset of Boolean grammars. The complexity of a direct implementation varies between linear and exponential, while memoization keeps it down to linear.

Better extensibility through modular syntax

We explore how to make the benefits of modularity available for syntactic specifications and present Rats! , a parser generator for Java that supports easily extensible syntax. Our parser generator builds on recent research on parsing expression grammars (PEGs), which, by being closed under composition, prioritizing choices, supporting unlimited lookahead, and integrating lexing and parsing, offer an attractive alternative to context-free grammars. PEGs are implemented by so-called packrat parsers, which are recursive descent parsers that memoize all intermediate results (hence their name). Memoization ensures linear-time performance in the presence of unlimited lookahead, but also results in an essentially lazy, functional parsing technique. In this paper, we explore how to leverage PEGs and packrat parsers as the foundation for extensible syntax. In particular, we show how make packrat parsing more widely applicable by implementing this lazy, functional technique in a strict, imperative language, while also generating better performing parsers through aggressive optimizations. Next, we develop a module system for organizing, modifying, and composing large-scale syntactic specifications. Finally, we describe a new technique for managing (global) parsing state in functional parsers. Our experimental evaluation demonstrates that the resulting parser generator succeeds at providing extensible syntax. In particular, Rats! enables other grammar writers to realize real-world language extensions in little time and code, and it generates parsers that consistently out-perform parsers created by two GLR parser generators.

English to Bangla Translator: The BANGANUBAD

This paper describes a new approach to English to Bangla translation. The English to Bangla Translator was developed. This system (BANGANUBAD) takes a paragraph of English sentences as input sentences and produces equivalent Bangla sentences. The BANGANUBAD system comprises a preprocessor, morphological recursive parser, semantic parser using English word ontology for context disambiguation, an electronic lexicon associated with grammatical information and a discourse processor. It also employs a lexical disambiguation analyzer. This system does not rely on a stochastic approach. Rather, it is based on a special kind of amalgamated architecture of transformer and rule-based NLE architectures along with various linguistic knowledge components.

Fuzzy context-free languages—Part 2: Recognition and parsing algorithms

In a companion paper [P.R.J. Asveld, Fuzzy context-free languages—Part 1: Generalized fuzzy context-free grammars, Theoret. Comput. Sci., (2005).] we used fuzzy context-free grammars in order to model grammatical errors resulting in erroneous inputs for robust recognizing and parsing algorithms for fuzzy context-free languages. In particular, this approach enables us to distinguish between small errors (“tiny mistakes”) and big errors (“capital blunders”). In this paper, we present some algorithms to recognize fuzzy context-free languages: particularly, a modification of Cocke–Younger–Kasami's algorithm and some recursive descent algorithms. Then we extend these recognition algorithms to corresponding parsing algorithms for fuzzy context-free languages. These parsing algorithms happen to be robust in some very elementary sense.

Parcels: A fast and feature-rich binary deployment technology

While development of a software system is important, it is also very important to have suitable mechanisms for actually deploying code. Current state-of-the-art deployment approaches force the developer to structure the code in such a way that deployment is possible, thereby severely inhibiting reuse and comprehensibility of the system. This paper presents parcels, an atomic deployment mechanism for objects and source code that supports shape changing of classes, method addition, method replacement, and partial loading. The key to making this deployment mechanism feasible and fast is a pickling algorithm that allows the unpickling to be done iteratively instead of with a recursive descent parser. Parcels were developed for VisualWorks Smalltalk, and have been the default deployment mechanism the past years for thousands of customers.

Design patterns for parsing

We provide a systematic transformation of an LL(1) grammar to an object model that consists of an object structure representing the non-terminal symbols and their corresponding grammar production rules, a union of classes representing the terminal symbols (tokens). We present a variant form of the visitor pattern and apply it to the above union of token classes to model a predictive recursive descent parser on the given grammar. Parsing a non-terminal is represented by a visitor to the tokens. For non-terminals that have more than one production rule, the corresponding visitors are chained together according to the chain of responsibility pattern in order to be processed correctly by a valid token. The abstract factory pattern, where each concrete factory corresponds to a non-terminal symbol, is used to manufacture appropriate parsing visitors.Our object-oriented formulation for predictive recursive descent parsing eliminates the traditional construction of the predictive parsing table and yields a parser that is declarative and has minimal conditionals. It not only serves to teach standard techniques in parsing but also as a non-trivial exercise of object modeling for objects-first introductory courses.

Selection of potential markers for epithelial ovarian cancer with gene expression arrays and recursive descent partition analysis.

Advanced-stage epithelial ovarian cancer has a poor prognosis with long-term survival in less than 30% of patients. When the disease is detected in stage I, more than 90% of patients can be cured by conventional therapy. Screening for early-stage disease with individual serum tumor markers, such as CA125, is limited by the fact that no single marker is up-regulated and shed in adequate amounts by all ovarian cancers. Consequently, use of multiple markers in combination might detect a larger fraction of early-stage ovarian cancers. To identify potential candidates for novel markers, we have used Affymetrix human genome arrays (U95 series) to analyze differences in gene expression of 41,441 known genes and expressed sequence tags between five pools of normal ovarian surface epithelial cells (OSE) and 42 epithelial ovarian cancers of different stages, grades, and histotypes. Recursive descent partition analysis (RDPA) was performed with 102 probe sets representing 86 genes that were up-regulated at least 3-fold in epithelial ovarian cancers when compared with normal OSE. In addition, a panel of 11 genes known to encode potential tumor markers [mucin 1, transmembrane (MUC1), mucin 16 (CA125), mesothelin, WAP four-disulfide core domain 2 (HE4), kallikrein 6, kallikrein 10, matrix metalloproteinase 2, prostasin, osteopontin, tetranectin, and inhibin] were similarly analyzed. The 3-fold up-regulated genes were examined and four genes [Notch homologue 3 (NOTCH3), E2F transcription factor 3 (E2F3), GTPase activating protein (RACGAP1), and hematological and neurological expressed 1 (HN1)] distinguished all tumor samples from normal OSE. The 3-fold up-regulated genes were analyzed using RDPA, and the combination of elevated claudin 3 (CLDN3) and elevated vascular endothelial growth factor (VEGF) distinguished the cancers from normal OSE. The 11 known markers were analyzed using RDPA, and a combination of HE4, CA125, and MUC1 expression could distinguish tumor from normal specimens. Expression at the mRNA level in the candidate markers was examined via semiquantitative reverse transcription-PCR and was found to correlate well with the array data. Immunohistochemistry was performed to identify expression of the genes at the protein level in 158 ovarian cancers of different histotypes. A combination of CLDN3, CA125, and MUC1 stained 157 (99.4%) of 158 cancers, and all of the tumors were detected with a combination of CLDN3, CA125, MUC1, and VEGF. Our data are consistent with the possibility that a limited number of markers in combination might identify >99% of epithelial ovarian cancers despite the heterogeneity of the disease.

Robust garden path parsing

This paper presents modifications to a standard probabilistic context-free grammar that enable a predictive parser to avoid garden pathing without resorting to any ad-hoc heuristic repair. The resulting parser is shown to apply efficiently to both newspaper text and telephone conversations with complete coverage and excellent accuracy. The distribution over trees is peaked enough to allow the parser to find parses efficiently, even with the much larger search space resulting from overgeneration. Empirical results are provided for both Wall St. Journal and Switchboard test corpora.

A BNF-based automatic test program generator for compatible microprocessor verification

A novel Backus-Naur-form- (BNF-) based method to automatically generate test programs from simple to complex ones for advanced microprocessors is presented in this paper. We use X86 architecture to illustrate our design method. Our method is equally applicable to other processor architectures by redefining BNF production rules. Design issues for an automatic program generator (APG) are first outlined. We have resolved the design issues and implemented the APG by a top-down recursive descent parsing method which was originated from compiler design. Our APG can produce not only random test programs but also a sequence of instructions for a specific module to be tested by specifying a user menu-driven file. In addition, test programs generated by our APG have the features of no infinite loop, not entering illegal states, controllable data dependency, flexible program size, and data cache testable. Our method has been shown to be efficient and feasible for the development of an APG compared with other approaches. We have also developed a coverage tool to integrate with the APG. Experimental evaluation of the generated test programs indicates that our APG, with the guidance of the coverage tool, only needs to generate a small number of test programs to sustain high coverage.

Object-oriented recursive descent parsing in C#

An object-oriented recursive descent parsing technique that is suitable for use in advanced programming classes where no language theory is required is presented. The C# code for a calculator is shown as an example.

Volumetric feature recognition for machining components with freeform surfaces

This paper describes a procedure for the extraction of features of a part containing a combination of 2.5D features and freeform surfaces. This work invokes a previous algorithm that was designed to recognize machining features from 2.5D parts destined to be machined on a 3-axis milling machine. The essence of that algorithm was a volume decomposition based on a recursive descent into the part, yielding a feature graph that captured both the geometry and the spatial relationships of the features. This work augments the previous algorithm with the ability to handle a limited class of components having freeform surfaces. Freeform features are defined similar to the 2.5D features as comprising a planar contour, but substituting a bottom freeform surface for the depth. Covering faces, defined as projection of the freeform surface on the faces of the bounding box of the surface, are used as equivalent planar faces for performing the recursive descent. Inter-feature open edges are used to signal the relationship between the freeform feature and other neighboring features. Examples of molds and components that were machined using the proposed algorithms are also presented.

Compiler generation tools for C#

Recursive descent parsing has the advantage of more convenient semantic processing than in the case of bottom-up parsing. However, it is limited to LL(1) grammars, which has often prevented its use for popular programming languages, such as Java or C#. The authors extended their compiler generator Coco/R, which generates recursive descent parsers, so that the user can specify rules for LL(1) conflict resolution. The simple technique works by evaluating user-defined Boolean expressions to allow the parser to make its parsing decisions where a one-symbol lookahead does not suffice. Thus multi-symbol lookahead or even semantic information can now guide the parsing process as well. Using their extended compiler generator, the authors implemented a compiler front end for C# that can be used as a framework for implementing a variety of tools, two of which are described in more detail: a white-box testing tool that measures path coverage, and a performance hint tool that evaluates source code according to user-defined rules about performance enhancements or coding style.

Packrat parsing:

Packrat parsing is a novel technique for implementing parsers in a lazy functional programming language. A packrat parser provides the power and flexibility of top-down parsing with backtracking and unlimited lookahead, but nevertheless guarantees linear parse time. Any language defined by an LL( k ) or LR( k ) grammar can be recognized by a packrat parser, in addition to many languages that conventional linear-time algorithms do not support. This additional power simplifies the handling of common syntactic idioms such as the widespread but troublesome longest-match rule, enables the use of sophisticated disambiguation strategies such as syntactic and semantic predicates, provides better grammar composition properties, and allows lexical analysis to be integrated seamlessly into parsing. Yet despite its power, packrat parsing shares the same simplicity and elegance as recursive descent parsing; in fact converting a backtracking recursive descent parser into a linear-time packrat parser often involves only a fairly straightforward structural change. This paper describes packrat parsing informally with emphasis on its use in practical applications, and explores its advantages and disadvantages with respect to the more conventional alternatives.

Top-Down Parsing of Conjunctive Languages

This paper generalizes the notion of a strong LL(k) context-free grammar for the case of conjunctive grammars and develops a top-down parsing algorithm for the resulting language family. A top-down parser of a conjunctive language attempts to construct a derivation of the input string, at each step using a finite lookahead to determine which grammar rule to apply. Fragments of formulae that form the derivation are stored in a tree-structured pushdown. Two ways to implement top-down parsers are suggested: a table-driven parser with a tree-structured pushdown and a recursive descent parser. Both techniques naturally extend their context-free counterparts.

There and Back Again

We illustrate a variety of programming problems that seemingly require two separate list traversals, but that can be efficiently solved in one recursive descent, without any other auxiliary storage but what can be expected from a control stack. The idea is to perform the second traversal when returning from the first.<br /> <br />This programming technique yields new solutions to traditional problems. For example, given a list of length 2n or 2n+1, where n is unknown, we can detect whether this list is a palindrome in n+1 recursive calls and no heap allocation.

An object-oriented LL(1) parser generator

This paper describes oops , an object-oriented parser generator implemented in Java [1]. Oops takes a grammar written in EBNF, checks that it is indeed LL(1), i.e., suitable for recursive descent parsing, and produces a parser as a set of serialized objects. A scanner must be provided and classes satisfying certain interfaces can be implemented which the parser uses to build parse trees.The paper discusses the ideas behind oops ---which are not specific to an implementation in Java---and shows the advantages of an object-oriented approach to grammar verification and parsing.

Towards a 'natural' narratology By Monika Fludernik (review)

BOOK NOTICES 199 Several ofthe papers extend natural language parsing into new territory. For example, Mats Wirén and Ralph Ronnquist ('Fully incremental parsing', 1 1-34) present a technique for parsing a text as it is typed and responding immediately to changes made anywhere in it. This is important for online grammar checking, language translation, and the like. Incremental parsing of programming languages has been around for a long time, but incremental parsing of natural language is, as far as I know, a new idea. Rudi Lutz ('Flowgraph parsing', 359-83) and Kent Wittenburg ('Predictive parsing for unordered relational languages', 385-407) extend parsing techniques to two-dimensional diagrams and flowcharts . After all, a diagram, like a sentence, is made of recognizable elements connected in a limited set of ways; some relation between natural language parsing and computer vision has been evident for some time, and the parsing of diagrams is perhaps the point of greatest contact. At least four papers deal in some way with dependency grammar or syntactic headship. Makoto Nagao ('Varieties of heuristics in sentence parsing', 79-90) advocates dependency analysis as the best approach to machine translation of Japanese. He is not alone; to judge from the papers presented at COLING-98 (Montréal, 1998, four years after the conferences reported in this volume), interest in dependency parsing is on the rise. Klaas Sikkel and Rieks op den Akker ('Predictive head-corner parsing', 169-82) build syntactic tree structures from the head outward. Papers by Stephan Raaijmakers (115-44) and Harry Bunt and Ko van der Sloot (91-1 14) use HPSG (headdriven phrase-structure grammar); the latter also uses Bunt's own formalism, DPSG (discontinuous phrase-structure grammar), i.e. constituency grammar with crossing branches. What is missing, of course, is transformational grammar, which is generally considered unsuitable for parsing. Even Eric Brill's interesting paper, 'Learning to parse with transformations' (221-40), turns out to deal with transforming a phrase-structure parser to improve its output rather than using transformations in the Chomskyan sense. [Michael A. Covington, University of Georgia.] Towards a 'natural' narratology. By Monika Fludernik. London: Routledge , 1996. Pp. xvi, 454. Encyclopedic in its coverage of examples and richly synthetic in its combination of theoretical paradigms for studying narratives (oral as well as written ), this book marks a significant contribution to several research domains. Analyzing the structures and functions of narrative from both a (synchronic) cognitive and a historical perspective, Fludernik addresses issues that will be of interest to discourse analysts, cognitive scientists, historians of the English language and of its literature, narratologists, theorists and historians of prose, and students of the historical and conceptual interconnections between orality and literacy. Chronicling the emergence of prose narrative in English, F develops a historical argument about how the experimental techniques found in the elaborate narratives of contemporary literature can ultimately be traced back to techniques used in 'simpler' conversational storytelling. Whatever their location along the scale of narrative complexity, stories rely crucially on what F descnbes as four cognitive parameters in her first chapter, 'Towards a "natural" narratology' (12-52). These four parameters include the real-world script of telling; the real-world schema of perception or viewing; the parameter by which one gains access to experiences that can then be told about or narrativized, i.e. experiencing; and finally the parameter of action or acting. Ch. 2, 'Natural narrative and other oral modes' (53-91), shows how these parameters guide the use and interpretation of stories in conversational settings. Yet—and this is one of F's chief claims in the book—the same basic parameters allow story recipients to 'narrativize ' (interpret as narratively structured) instances of discourse that can be quite difficult to comprehend in narrative terms. In the author's words, when faced with, say, challenging literary narratives of recent years, readers 'attempt to re-cognize what they find in the text in terms of the natural telling or experiencing or viewing parameters, or they try to recuperate the inconsistences in terms of actions and event structures at the most minimal level' (34). Hence the transition from oral to written narrative should be conceived as a continuum, and researchers should...

An object oriented approach to constructing recursive descent parsers

We discuss a technique to construct a recursive descent parser for a context free language using concepts found in object oriented design and implementation. A motivation for the technique is given. The technique is then introduced with snippets of a Smalltalk implementation. Some advantages and disadvantages of the technique are examined. Finally some areas of possible future work are discussed.

Run-time Extensible Deterministic Top-Down Parsing

When reading a text or listening to speech, words are processed on-line by humans in the order they come. Humans mainly use this kind of parsing even when they process deterministic text (programs). Intuitively there are some mental actions just after the morphological analysis of any newly recognized word. This mental action helps to understand the given word (or to position the word within the frame of the – still not complete – sentence). Within parsing of formal languages the concept closest to this idea is top-down parsing that is usually used only together with different classes of LL grammars. The advantage of top-down parsing of programming languages is the possibility to implement it by a recursive descent parser – i.e. by a system of procedures that may recursively call each other. Such a system may be `tuned' by handmade changes. The usage of LL grammars is not always possible, because the grammars of programming languages may have left recursive symbols. Programming language grammars are intuitively `close' to LL grammars. A good model for such grammars are the kind grammars studied in this paper. Kind grammars preserve all the important features of LL grammars that are advantageous for parsing.

Rdp—an iterator-based recursive descent parser generator with tree promotion operators

rdp is a parser generator which accepts Iterator Backus Naur Form productions decorated with attributes and ANSI-C actions and produces recursive descent parsers. It has special support for the generation of tree-based intermediate forms, built-in symbol table handling for the implementation of context-sensitive components of the language syntax and a support library that includes a generalised graph handling module that can output graphs in a form suitable for use with well known visualisation tools.