Arbitrary Context-free Grammars Research Articles

The Parallelism Tradeoff: Limitations of Log-Precision Transformers

AbstractDespite their omnipresence in modern NLP, characterizing the computational power of transformer neural nets remains an interesting open question. We prove that transformers whose arithmetic precision is logarithmic in the number of input tokens (and whose feedforward nets are computable using space linear in their input) can be simulated by constant-depth logspace-uniform threshold circuits. This provides insight on the power of transformers using known results in complexity theory. For example, if L≠P (i.e., not all poly-time problems can be solved using logarithmic space), then transformers cannot even accurately solve linear equalities or check membership in an arbitrary context-free grammar with empty productions. Our result intuitively emerges from the transformer architecture’s high parallelizability. We thus speculatively introduce the idea of a fundamental parallelism tradeoff: any model architecture as parallelizable as the transformer will obey limitations similar to it. Since parallelism is key to training models at massive scale, this suggests a potential inherent weakness of the scaling paradigm.

On the complexity and performance of parsing with derivatives

Current algorithms for context-free parsing inflict a trade-off between ease of understanding, ease of implementation, theoretical complexity, and practical performance. No algorithm achieves all of these properties simultaneously. Might et al. introduced parsing with derivatives, which handles arbitrary context-free grammars while being both easy to understand and simple to implement. Despite much initial enthusiasm and a multitude of independent implementations, its worst-case complexity has never been proven to be better than exponential. In fact, high-level arguments claiming it is fundamentally exponential have been advanced and even accepted as part of the folklore. Performance ended up being sluggish in practice, and this sluggishness was taken as informal evidence of exponentiality. In this paper, we reexamine the performance of parsing with derivatives. We have discovered that it is not exponential but, in fact, cubic. Moreover, simple (though perhaps not obvious) modifications to the implementation by Might et al. lead to an implementation that is not only easy to understand but also highly performant in practice.

Automating grammar comparison

We consider from a practical perspective the problem of checking equivalence of context-free grammars. We present techniques for proving equivalence, as well as techniques for finding counter-examples that establish non-equivalence. Among the key building blocks of our approach is a novel algorithm for efficiently enumerating and sampling words and parse trees from arbitrary context-free grammars; the algorithm supports polynomial time random access to words belonging to the grammar. Furthermore, we propose an algorithm for proving equivalence of context-free grammars that is complete for LL grammars, yet can be invoked on any context-free grammar, including ambiguous grammars. Our techniques successfully find discrepancies between different syntax specifications of several real-world languages, and are capable of detecting fine-grained incremental modifications performed on grammars. Our evaluation shows that our tool improves significantly on the existing available state of the art tools. In addition, we used these algorithms to develop an online tutoring system for grammars that we then used in an undergraduate course on computer language processing. On questions involving grammar constructions, our system was able to automatically evaluate the correctness of 95% of the solutions submitted by students: it disproved 74% of cases and proved 21% of them.

확장된 PLR(1) 문법에 대한 단일 틈 파싱

Gap parsing is an algorithm for parsing incomplete input strings which include some gaps. Gap parsing is different from conventional parsing, and as known results, one-gap parsing algorithms for arbitrary context-free grammar and LL(1) grammar have and time complexity, respectively. This paper presents a one-gap parsing algorithm for extended PLR(1) grammars. Extended PLR(1) grammars are the class of grammars smaller than LR(1) but much larger than LL(1). The one-gap parsing algorithm of the grammar class is shown to have the time complexity of , which is equal to the complexity of one-gap parsing algorithms for LL(1) grammars.

An interactive parser generator for context-free grammars

This paper describes a parser generator that accepts arbitrary context-free grammars. It generates a parser using the Earley algorithm [1]. It enables the user to develop, edit, and test a grammar in an interactive graphical environment. This GUI visualizes both the operation of the Earley algorithm as well as the generated parse trees. The generated parsers are saved as fully-functional Java source files, ready to be incorporated into an application. These Java programs can be reloaded into the GUI for further editing of the grammar. Employing this parser generator in a sophomore-level software development course enables students to become proficient in writing a parser with two days of lecture and one assignment.

Self-embedded context-free grammars with regular counterparts

In general, it is undecidable if an arbitrary context-free grammar has a regular solution. Past work has focused on special cases, such as one-letter grammars, non self-embedded grammars and the fi...

Cooperation in context-free grammars

A new dynamical measure of the descriptional complexity for context-free grammars and languages, namely the degree of cooperation, is introduced and studied. This measure is connected with respect to both families of languages considered, namely the regular and context-free languages. We prove that the degree of cooperation is computable for regular and unambigous context-free grammars and it is not computable for arbitrary context-free grammars. The computability status of this measure for languages remains to be investigated.

Pattern Ambiguities for Pure Context—Free Grammars

We consider here some special forms of ambiguity for pure context-free grammars, namely pattern avoiding ambiguity, pattern preserving ambiguity, pattern ambiguity, and grammar avoiding ambiguity. The first two properties are undecidable for arbitrary pure context-free grammars, and in a particular but general enough case we can effectively decide whether or not a pure context-free grammar is pattern preserving ambiguous.

A parallel parsing algorithm for arbitrary context-free grammars

A parallel parsing algorithm for arbitrary context-free grammars

(0,1)-TOTALITY IS UNDECIDABLE FOR ARBITRARY CONTEXT-FREE GRAMMARS

This note shows that it is undecidable whether or not an arbitrary context-free grammar is (0,l)-total, i.e., whether or not {0, 1} + ⊆ g(Szl(G)) holds for an arbitrary context-free grammar G, the left Szilard language Szl(G) of G, and a homomorphism g mapping the labels of G's productions into {0, 1} ∪ {λ}. These considerations are motivated by cryptosystems recently proposed by Andrasiu et al.

Iterated GSMs and Co-CFL

We study the infinite words generated by an iterated gsm. The motivation was two fold. The first one was given by the apparent similarity between the iterated gsm and the iterated morphism. However contrary to the appearences almost all questions become undecidable. The second one was to disprove the following conjecture of Berstel: The language of coprefixes of an infinite word w is context free iff w is generated by an iterated gsm. We use for that the infinite word: w = abca2 ba 2 bca 4 ba 4 ba 2 ba 4 bc ... (a 2n b)2n c .... We prove also that the degree of the ambiguity problem for coprefixes of iterated gsm is � 1-complete in the Kleene hierarchy. This result fills the gap between the degree of this problem for iterated morphisms which is � 0 and for arbitrary context-free grammars which is � 2-complete.

Incremental generation of parsers

An LR-based parser generator for arbitrary context-free grammars is described, which generates parsers by need and processes grammar modifications by updating already existing parsers. We motivate the need for these techniques in the context of interactive language definition environments, present all required algorithms, and give measurements comparing their performance with that of conventional techniques.

Incremental generation of parsers

An LR-based parser generator for arbitrary context-free grammars that generates parsers by need and handles modifications to its input grammar by updating the parser it has generated so far is described. The need for these techniques is discussed in the context of interactive language definition environments. All required algorithms are presented. Measurements are given comparing their performance with that of conventional techniques. >

Control sets on grammars using depth-first derivations

Control sets on grammars are extended to depth-first derivations. It is proved that a context-free language is generated by the depth-first derivations of an arbitrary context-free grammar controlled by an arbitrary regular set. This result is sharpened to obtain a new characterization of the family of derivation-bounded languages: a languageL is derivation bounded if and only if it is generated by the depth-first derivations of a context-free grammarG controlled by a regular subsetR of the Szilard language ofG. The left-derivation-bounded languages are characterized analogously using leftmost derivations. It is proved that a grammarG is nonterminal bounded if and only if the Szilard language defined using only the depth-first derivations ofG is regular. Finally, it is proved that if a family of languagesC is a trio, a semi-AFL, an AFL, or an AFL closed under λ-free substitution, then the family of languages generated using arbitrary context-free grammars controlled by members ofC is full, is closed under reversal, and has the closure properties assumed ofC.

On the complexity of LR(k) testing

The problem of determining whether an arbitrary context-free grammar is a member of some easily parsed subclass of grammars such as the LR(k) grammars is considered. The time complexity of this problem is analyzed both when k is considered to be a fixed integer and when k is considered to be a parameter of the test. In the first case, it is shown that for every k there exists an O(n k+2 ) algorithm for testing the LR(k) property, where n is the size of the grammar in question. On the other hand, if both k and the subject grammar are problem parameters, then the complexity of the problem depends very strongly on the representation chosen for k. More specifically, it is shown that this problem is NP-complete when k is expressed in unary. When k is expressed in binary the problem is complete for nondeterministic exponential time. These results carry over to many other parameterized classes of grammars, such as the LL(k), strong LL(k), SLR(k), LC(k), and strong LC(k) grammars.

An algorithm for the construction of bounded-context parsers

An algorithm is described which accepts an arbitrary context-free grammar and constructs a bounded-context parser for it whenever such a parser exists. In the first part of the paper the definition of a context-free grammar and the working of a bounded-context parser are recalled. The notion of reduction class for a context-free grammar is then introduced and its connection with the structure of a bounded-context parser is indicated. Next, pushdown automata which generate the different reduction classes of a context-free grammar are defined. Finally, the algorithm is described; it essentially carries out an exhaustive study of all possible runs of the pushdown automata generating the reduction classes. In the second part, the utility of the algorithm is discussed in the light of the experience gained from its use in compiler design. The algorithm is claimed to be particularly useful in the simultaneous design of a language and a compiler for it.

Structural equivalence of context-free grammars

Two context-free grammars are defined as being structurally-equivalent if they generate the same sentences and assign similar parse trees (differing only in the labelling of the nodes) to each. It is argued that this type of equivalence is more significant than weak equivalence, which requires only that the same sentences be generated. While the latter type of equivalence is in general undecidable, it is shown here that there exists a finite algorithm for determining if two arbitrary context-free grammars are structurally equivalent. A related result is a procedure for converting an arbitrary context-free grammar into a structurally equivalent “simple” grammar (S-grammar) where this is possible, or else indicating that no such grammar exists. The question of structural ambiguity is also studied and a procedure is given for determining if an arbitrary context-free grammar can generate the same string in 2 different ways with similar parse trees.