LR parsing for strings with placeholders

A new parsing method called LLLR parsing is defined and a method for producing LLLR parsers is described. An LLLR parser uses an LL parser as its backbone and parses as much of its input string using LL parsing as possible. To resolve LL conflicts it triggers small embedded LR parsers. An embedded LR parser starts parsing the remaining input and once the LL conflict is resolved, the LR parser produces the left parse of the substring it has just parsed and passes the control back to the backbone LL parser. The LLLR(k) parser can be constructed for any LR(k) grammar. It produces the left parse of the input string without any backtracking and, if used for a syntax-directed translation, it evaluates semantic actions using the top-down strategy just like the canonical LL(k) parser. An LLLR(k) parser is appropriate for grammars where the LL(k) conflicting nonterminals either appear relatively close to the bottom of the derivation trees or produce short substrings. In such cases an LLLR parser can perform a significantly better error recovery than an LR parser since the most part of the input string is parsed with the backbone LL parser. LLLR parsing is similar to LL(^*) parsing except that it (a) uses LR(k) parsers instead of finite automata to resolve the LL(k) conflicts and (b) does not perform any backtracking.

On different LL and LR parsers used in LLLR parsing

The methods of improving LR(k) parsers proposed by DeRemer and Korenjak are shown to be based on a single concept -- that of modifying the contextual information on which parsing decisions are made. This concept is then used to derive a straightforward algorithm for eliminating unit productions from an LR parser.

Component-based LR parsing

One of the most attractive techniques in optimizing LR parsers is to eliminate reductions by semantically insignificant productions of the form A → X (single productions), where X is a nonterminal or a terminal; such a modification can lead to substantial savings in both space and time. Therefore, much effort has been devoted to the development of methods for eliminating reductions by single productions from LR parsers (Aho and Ullman [1973a,1973b], Anderson, Eve and Horning[1973], Pager[1973a,1974], Demers[1974,1975], Backhouse [1976], Joliat[1976], Koskimies[1976], LaLonde [1976], Soisalon-Soininen[1976]).

Extending lookahead for LR parsers

Extracting error productions from a neural network-based LR parser

We discuss existing approaches to train LR parsers, which have been used for statistical resolution of structural ambiguity. These approaches are nonoptimal, in the sense that a collection of probability distributions cannot be obtained. In particular, some probability distributions expressible in terms of a context-free grammar cannot be expressed in terms of the LR parser constructed from that grammar, under the restrictions of the existing approaches to training of LR parsers. We present an alternative way of training that is provably optimal, and that allows all probability distributions expressible in the context-free grammar to be carried over to the LR parser. We also demonstrate empirically that this kind of training can be effectively applied on a large treebank.

MLR, an extended LR parser, is introduced, and its application to natural language parsing is discussed. An LR parser is a shift-reduce parser which is deterministically guided by a parsing table. A parsing table can be obtained automatically from a context-free phrase structure grammar. LR parsers cannot manage ambiguous grammars such as natural language grammars, because their parsing tables would have multiply-defined entries, which precludes deterministic parsing. MLR, however, can handle multiply-defined entries, using a dynamic programming method. When an input sentence is ambiguous, the MLR parser produces all possible parse trees without parsing any part of the input sentence more than once in the same way, despite the fact that the parser does not maintain a chart as in chart parsing. Our method also provides an elegant solution to the problem of multi-part-of-speech words such as "that". The MLR parser and its parsing table generator have been implemented at Carnegie-Mellon University.

This paper presents a text-based syntax completion method using an LR parser. We propose formal definitions of candidate text to be completed based on the sentential forms, and we design algorithms for computing candidates through reductions in the LR parsing. This is in contrast to the existing methods that have not clearly stated what candidates they intend to produce. This is also different from a transformation approach using an LR parser, which transforms the grammar of the programming language, a burdensome task at this moment. The advantage of our method is that LR parsers can be adopted without modification, and a syntax completion system can be built using them, without incurring efforts. We implemented the algorithms as an Emacs server to demonstrate the feasibility of their application.

This chapter is devoted to the practical issues involved in the construction and use of deterministic LR(1) parsers. We shall show how the practical versions of LR (1) parsers, most notably the LALR (1) parsers, can be constructed efficiently, and we shall present methods for encoding LR(1) parsers as efficient RAM programs. Two versions of RAM program implementation are considered: in the first implementation the parsing program is table-driven, that is, the rules of the parser are encoded in a two-dimensional array which is simulated by a program body; in the other implementation the tabular information is further transformed into a set of program statements.

Despite the popularity of LALR(1) parser generators such as Yacc/Bison and LL parser generators such as ANTLR, robust and effective LR(1) parser generators are rare due to expensive performance and implementation difficulty. This work employed relevant algorithms, including the Knuth canonical algorithm, Pager's practical general method, lane-tracing algorithm, unit production elimination algorithm and its extension, and the edge-pushing algorithm, implemented an efficient, practical and Yacc/Bison-compatible open-source parser generator Hyacc, which supports full LR(0)/LALR(1)/LR(1) and partial LR(k). Based on the implementation, an empirical study was conducted comparing different LR(1) parser generation algorithms and LALR(1) algorithms. The result shows that LR(1) parser generation based upon improved algorithms and carefully selected data structures can be sufficiently efficient to be of practical use with modern computing facilities.

A method for resolving LL(k) conflicts using small LR(k) parsers (called embedded left LR(k) parsers) is described. An embedded left LR(k) parser is capable of (a) producing the prefix of the left parse of the input string and (b) stopping not on the end-of-file marker but on any string from the set of lookahead strings fixed at the parser generation time. The conditions regarding the termination of the embedded left LR(k) parser if used within LL(k) (and similar) parsers are defined and examined in-depth. It is proved that an LL(k) parser augmented with a set of embedded left LR(k) parsers can parse any deterministic context-free grammar in the same asymptotic time as LR(k) parser. As the embedded left LR(k) parser produces the prefix of the left parse, the LL(k) parser augmented with embedded left LR(k) parsers still produces the left parse and the compiler writer does not need to bother with different parsing strategies during the compiler implementation.

Characteristic parsing: A framework for producing compact deterministic parsers, II

A wide variety of algorithms have been suggested for the repair of syntactic errors in a computer program. Since there is usually more than one possible repair for any syntax error, many algorithms employ a cost function to guide the the repair, and some [1,3,4,6], guarantee that the repair chosen will be least-cost, according to some definition. (The others, although guided by costs, do not guarantee least-cost in all cases.) Fischer et al. [4,6,7] define a “locally least-cost” repair using insertions and deletions, and provide algorithms for LL and LR parsers. A locally least-cost repair is a least-cost sequence of deletions and insertions such that one more symbol in the original string will be accepted by the parser. Backhouse [2,3] uses a similar definition. In both cases, the repair algorithms operate by examining a single symbol in the input at any time.