Extension Of Context-free Grammars Research Articles

Enumerating grammar-based extractions

We propose a new grammar-based language for defining information extractors from documents (text) that is built upon the well-studied framework of document spanners for extracting structured data from text. While previously studied formalisms for document spanners are mainly based on regular expressions, we use an extension of context-free grammars, called extraction grammars, to define the new class of context-free spanners. Extraction grammars are simply context-free grammars extended with variables that capture interval positions of the document, namely spans. While regular expressions are efficient for tokenizing and tagging, context-free grammars are also efficient for capturing structural properties. Indeed, we show that context-free spanners are strictly more expressive than their regular counterparts. We reason about the expressive power of our new class and present a pushdown-automata model that captures it. We show that extraction grammars can be evaluated with polynomial data complexity. Nevertheless, as the degree of the polynomial depends on the query, we present an enumeration algorithm for unambiguous extraction grammars that, after quintic preprocessing, outputs the results sequentially, without repetitions, with a constant delay between every two consecutive ones.

Multiple context-free path querying by matrix multiplication

Many graph analysis problems can be formulated as formal language-constrained path querying problems where the formal languages are used as constraints for navigational path queries. Recently, the context-free language (CFL) reachability formulation has become very popular and can be used in many areas, for example, querying graph databases, Resource Description Framework (RDF) analysis. However, the generative capacity of context-free grammars (CFGs) is too weak to generate some complex queries, for example, from natural languages, and the various extensions of CFGs have been proposed. Multiple context-free grammar (MCFG) is one of such extensions of CFGs. Despite the fact that, to the best of our knowledge, there is no algorithm for MCFL-reachability, this problem is known to be decidable. This paper is devoted to developing the first such algorithm for the MCFL-reachability problem. The essence of the proposed algorithm is to use a set of Boolean matrices and operations on them to find paths in a graph that satisfy the given constraints. The main operation here is Boolean matrix multiplication. As a result, the algorithm returns a set of matrices containing all information needed to solve the MCFL-reachability problem. The presented algorithm is implemented in Python using GraphBLAS API. An analysis of real RDF data and synthetic graphs for some MCFLs is performed. The study showed that using a sparse format for matrix storage and parallel computing for graphs with tens of thousands of edges the analysis time can be 10–20 minutes. The result of the analysis provides tens of millions of reachable vertex pairs. The proposed algorithm can be applied in problems of static code analysis, bioinformatics, network analysis, as well as in graph databases when a path query cannot be expressed using context-free grammars. The provided algorithm is linear algebra-based, hence, it allows one to use high-performance libraries and utilize modern parallel hardware.

The hardest language for grammars with context operators

In 1973, Greibach (“The hardest context-free language”, SIAM J. Comp., 1973) constructed a context-free language L0 with the property that every context-free language can be reduced to L0 by a homomorphism, thus representing it as an inverse homomorphic image h−1(L0). In this paper, a similar characterization is established for a family of grammars equipped with operators for referring to the left context of any substring, recently defined by Barash and Okhotin (“An extension of context-free grammars with one-sided context specifications”, Inform. Comput., 2014). An essential step of the argument is a new normal form for grammars with context operators, in which every nonterminal symbol defines only strings of odd length in left contexts of even length: the even-odd normal form. The characterization is completed by showing that the language family defined by grammars with context operators is closed under inverse homomorphisms; actually, it is closed under injective nondeterministic finite transductions.

Weighted Multiple Context-Free Grammars

Multiple context-free grammar (MCFG) is an extension of context-free grammar (CFG), which generates tuples of words. The expressive power of MCFG is between CFG and context-sensitive grammar while MCFG inherits good properties of CFG. In this paper, we introduce weighted multiple context-free grammar (WMCFG) as a quantitative extension of MCFG. Then we investigate properties of WMCFG such as polynomial-time computability of basic problems, its closure property and expressive power.

A Pumping Lemma for Permitting Semi-Conditional Languages

Permitting semi-conditional grammars (pSCGs) are extensions of context-free grammars where each rule is associated with a word [Formula: see text] and such a rule can be applied to a sentential form [Formula: see text] only if [Formula: see text] is a subword of [Formula: see text]. We consider permitting generalized SCGs (pgSCGs) where each rule [Formula: see text] is associated with a set of words [Formula: see text] and [Formula: see text] is applicable only if every word in [Formula: see text] occurs in [Formula: see text]. We investigate the generative power of pgSCGs with no erasing rules and prove a pumping lemma for their languages. Using this lemma we show that pgSCGs are strictly weaker than context-sensitive grammars. This solves a long-lasting open problem concerning the generative power of pSCGs. Moreover, we give a comparison of the generating power of pgSCGs and that of forbidding random context grammars with no erasing rules.

Weighted iterated linear control

We combine three extensions of context-free grammars: (a) associating its nonterminals with storage configurations, (b) equipping its rules with weights, and (c) controlling its derivations. For a commutative semiring K, we introduce the class of weighted languages generated by K-weighted linear context-free grammars with storage S and with derivations controlled by (S, K)-recognizable weighted languages. The control on the derivations can be iterated in a natural way. We characterize the n-th iteration of the control in terms of the n-th iteration of the one-turn pushdown operator on the storage S of the control weighted language. Moreover, for each proper semiring we prove that iterating the control yields an infinite, strict hierarchy of classes of weighted languages.

Generalized LR Parsing Algorithm for Grammars with One-Sided Contexts

The Generalized LR parsing algorithm for context-free grammars is notable for having a decent worst-case running time (cubic in the length of the input string, if implemented efficiently), as well as much better performance on “good” grammars. This paper extends the Generalized LR algorithm to the case of “grammars with left contexts” (M. Barash, A. Okhotin, “An extension of context-free grammars with one-sided context specifications”, Inform. Comput., 2014), which augment the context-free grammars with special operators for referring to the left context of the current substring, along with a conjunction operator (as in conjunctive grammars) for combining syntactical conditions. All usual components of the LR algorithm, such as the parsing table, shift and reduce actions, etc., are extended to handle the context operators. The resulting algorithm is applicable to any grammar with left contexts and has the same worst-case cubic-time performance as in the case of context-free grammars.

Improved normal form for grammars with one-sided contexts

Formal grammars equipped with operators for specifying the form of the context of a substring were recently studied by Barash and Okhotin (“An extension of context-free grammars with one-sided context specifications”, Inform. and Comput., 2014) further extending the author's earlier work on propositional connectives in grammars (A. Okhotin, “Conjunctive grammars”, J. Autom. Lang. Comb., 2001). These grammars allow two types of context specifications: for a substring v of a string uvx, a proper left context operator ◁D states that u is of the form described by D, and an extended left context operator ⊴E states that uv is described by E. This paper establishes a normal form for these grammars, in which extended left contexts are never used, whereas left contexts may be applied only to individual symbols, so that all rules are of the form A→B1C1&…&BmCm or A→a&◁D. This eliminates circular dependencies in the grammar and leads to a new, clearer version of the known cubic-time parsing algorithm. With some further improvements, the algorithm is accelerated from time O(n3) to time O(n3log⁡n).

Infinite Hierarchy of Permutation Languages

Permutation grammars are an extension of context-free grammars with rules having the same symbols on both sides but possibly in a different order. An example of a permutation rule of length 3 is ABC → CBA. If these non-context-free rules are of length at most n, then we say that permutation grammar is of order n and all such grammars generate a family of permutation languages Permn. In 2010 Nagy showed that there exists a language such that it cannot be generated by a grammar of order 2, but rules of length 3 are enough. In other words, a strict inclusion $Perm_2 \subsetneq Perm_3$ was obtained. We extend this result proving that $Perm_{4n \minus 2} \subsetneq Perm_{4n \minus 1}$ for n ≥ 1.

Unambiguous Boolean grammars

Boolean grammars are an extension of context-free grammars, in which conjunction and negation may be explicitly used in the rules. In this paper, the notion of ambiguity in Boolean grammars is defined. It is shown that the known transformation of a Boolean grammar to the binary normal form preserves unambiguity, and that every unambiguous Boolean language can be parsed in time O( n 2). Linear conjunctive languages are shown to be unambiguous, while the existence of languages inherently ambiguous with respect to Boolean grammars is left open.

Partial Word Order Syntax of Urdu/Sindhi and Linear Specification Language

Like most of the South-Asian languages Urdu and Sindhi are partial word order languages. Conventional syntax representation models like Context Free Grammars are not capable enough to cope with partial word order syntax. Linear Specification Language (LSL) is an extension of Context-Free Grammars (CFGs) which allows arbitrary partial order (free word order) on the right hand side of grammar rule. Partial word order in LSL is handled by using different types of linear precedence (LP) constraints. LSL by using LP constraints is capable enough to represent the syntax of partial word order sentence. Issues related to represent Urdu/Sindhi language sentences with their constituent parts in LSL are discussed. LSL versions for different types of Urdu and Sindhi sentences are presented.

A Cubic Time Extension of Context-Free Grammars

Context-free grammars and cubic parse time are so related in people's minds that they often think that parsing any extension of context-free grammars must need some extra time. Of course, this is not necessarily true and this paper presents a generalization of context-free grammars which nonetheless still has a cubic parse time complexity. This extension, which defines a subclass of context-sensitive languages, has both a theoretical and a practical interest. The class of languages defined by these grammars is closed under both intersection and complement (in fact this class contains both the intersection and the complement of context-free languages). Moreover, these languages belong to an extension of mildly context-sensitive languages in which the constant growth property is relaxed and which can thus potentially be used in natural language processing.

Learning semantic functions of attribute grammars

Attribute grammars can be considered as an extension of context-free grammars, where the attributes are associated with grammar symbols, and the semantic rules define the values of the attributes. ...

The equivalence of four extensions of context-free grammars

There is currently considerable interest among computational linguists in grammatical formalisms with highly restricted generative power. This paper concerns the relationship between the class of string languages generated by several such formalisms, namely, combinatory categorial grammars, head grammars, linear indexed grammars, and tree adjoining grammars. Each of these formalisms is known to generate a larger class of languages than context-free grammars. The four formalisms under consideration were developed independently and appear superficially to be quite different from one another. The result presented in this paper is that all four of the formalisms under consideration generate exactly the same class of string languages.

Formal Languages: Origins and Directions

Origins of the theory of formal languages and automata are surveyed starting from 1936 with the work of Turing and Post. Special attention is given to the machine translation projects of the 1950s and early 1960s and associated work in mathematical linguistics. The development of the Chomsky hierarchy of grammars, machines, and languages from 1956 to 1964 is traced. It is observed that the same important ideas emerged independently for the automatic analysis and translation of both natural and artificial languages. Since 1964, formal language theory is part of theoretical computer science. A few of the directions since 1964 are considered: restrictions and extensions of context-free grammars and pushdown store automata, unifying frameworks, and complexity questions.

The parsing problem for affix grammars

Affix grammars are an extension of context-free grammars which retain most of their advantages and eliminate most of their limitations with respect to the definition of programming languages and the specification of their translators. The extension allows definition of context-sensitive syntax features, and also allows semantics to be linked to syntax. In this paper, the parsing problem for affix grammars is explored and shown to be closely related to the parsing problem for context-free grammars. This enables a standard context-free parser constructor to be generalised to a constructor for affix grammars, essentially by addition of a preprocessor. The resulting constructors are compared with previously implemented or proposed constructors.

The parsing problem for affix grammars

Affix grammars are an extension of context-free grammars which retain most of their advantages and eliminate most of their limitations with respect to the definition of programming languages and th...