Context-free Research Articles

A Generalized Pumping Lemma for Weighted Recognizable Languages

Pumping lemmata are the main tool to prove that a certain language does not belong to a class of languages like the recognizable languages or the context-free languages. Essentially two pumping lemmata exist for the recognizable weighted languages: the classical one for the Boolean semiring (i.e., the unweighted case) and a similar one for fields. A joint generalization of these two pumping lemmata is provided that applies to all (right) Artinian semirings. These are (potentially non-commutative) semirings over which all finitely generated (right) semimodules have a finite bound on the length of chains of strictly increasing (right) subsemimodules. Since Artinian rings are exactly those that satisfy the Descending Chain Condition, the Artinian semirings include all skew fields and naturally also all finite semirings. The new pumping lemma thus covers most previously known pumping lemmata for recognizable weighted languages and a non-exhaustive list of semirings to which the new pumping lemma can be applied is provided as well.

Kernels of Context-Free Languages

While the closure of a language family [Formula: see text] under certain language operations is the least family of languages which contains all members of [Formula: see text] and is closed under all of the operations, a kernel of [Formula: see text] is a maximal family of languages which is a sub-family of [Formula: see text] and is closed under all of the operations. Here we investigate properties of kernels of general language families and operations defined thereon as well as kernels of (deterministic) (linear) context-free languages with a focus on Boolean operations. While the closures of language families are unique, this uniqueness is not obvious for kernels. We consider properties of language families and of operations that yield unique and non-unique, i.e. a set, of kernels. For the latter case, the question whether the union of all kernels coincides with the language family, or whether there are languages that do not belong to any kernel is addressed. Additionally, languages that are mandatory for each (Boolean) kernel and languages that are optional for (Boolean) kernels are studied. That is, we consider the intersection of all Boolean kernels as well as their union. The expressive capacities of these families are addressed leading to a hierarchical structure. Further closure properties are considered. Furthermore, we study descriptional complexity aspects of these families, where languages are represented by context-free grammars with proofs attached. It turns out that the size trade-offs between all families in question and deterministic context-free languages are non-recursive. That is, one can choose an arbitrarily large recursive function [Formula: see text], but the gain in economy of description eventually exceeds [Formula: see text] when changing from the latter system to the former.

Detection and treatment of string events in the limit

A string event is a pattern that occurs in a stream of characters. The need to detect and handle string events in infinite texts emerges in many scenarios, including online treatment of logs, web crawling, and syntax highlighting. This paper describes a technique to specify and treat string events. Users determine patterns of interest via a markup language. From such examples, tokens are generalized via a semi-lattice of regular expressions. Such tokens are combined into a context-free language that recognizes patterns in the text stream. These techniques are implemented in a text processing system called Lushu, which runs on the Java Virtual Machine (JVM). Lushu intercepts strings emitted by the JVM. Once patterns are detected, it invokes a user-specified action handler. As a proof of concept, this paper shows that Lushu outperforms state-of-the-art parsers and parser generators, such as Comby, BeautifulSoup4 and ZheFuscator, in terms of memory consumption and running time.

Explaining results of path queries on graphs: Single-path results for context-free path queries

Many graph query languages use, at their core, path queries that yield node pairs (m,n) that are connected by a path of interest. For the end-user, such node pairs only give limited insight as to why this result is obtained, as the pair does not directly identify the underlying path of interest.In this paper, we propose the single-path semantics to address this limitation of path queries. Under single-path semantics, path queries evaluate to a single path connecting nodes m and n and that satisfies the conditions of the query. To put our proposal in practice, we provide an efficient algorithm for evaluating context-free path queries using the single-path semantics. Additionally, we perform a short evaluation of our techniques that shows that the single-path semantics is practically feasible, even when query results grow large.In addition, we explore the formal relationship between the single-path semantics we propose the problem of finding the shortest string in the intersection of a regular language (representing a graph) and a context-free language (representing a path query). As our formal results show, there is a distinction between the complexity of the single-path semantics for queries that use a single edge label and queries that use multiple edge labels: for queries that use a single edge label, the length of the shortest path is linearly upper bounded by the number of nodes in the graph; whereas for queries that use multiple edge labels, the length of the shortest path has a worst-case quadratic lower bound.

Boosting the Performance of Alias-Aware IFDS Analysis with CFL-Based Environment Transformers

The IFDS algorithm is pivotal in solving field-sensitive data-flow problems. However, its conventional use of access paths for field sensitivity leads to the generation of a large number of data-flow facts. This causes scalability challenges in larger programs, limiting its practical application in extensive codebases. In response, we propose a new field-sensitive technique that reinterprets the generation of access paths as a Context-Free Language (CFL) for field-sensitivity and formulates it as an IDE problem. This approach significantly reduces the number of data-flow facts generated and handled during the analysis, which is a major factor in performance degradation. To demonstrate the effectiveness of this approach, we developed a taint analysis tool, IDEDroid, in the IFDS/IDE framework. IDEDroid outperforms FlowDroid, an established IFDS-based taint analysis tool, in the analysis of 24 major Android apps while improving its precision (guaranteed theoretically). The speed improvement ranges from 2.1× to 2,368.4×, averaging at 222.0×, with precision gains reaching up to 20.0% (in terms of false positives reduced). This performance indicates that IDEDroid is substantially more effective in detecting information-flow leaks, making it a potentially superior tool for mobile app vetting in the market.

Monotone Procedure Summarization via Vector Addition Systems and Inductive Potentials

This paper presents a technique for summarizing recursive procedures operating on integer variables. The motivation of our work is to create more predictable program analyzers, and in particular to formally guarantee compositionality and monotonicity of procedure summarization. To summarize a procedure, we compute its best abstraction as a vector addition system with resets (VASR) and exactly summarize the executions of this VASR over the context-free language of syntactic paths through the procedure. We improve upon this technique by refining the language of syntactic paths using (automatically synthesized) linear potential functions that bound the number of recursive calls within valid executions of the input program. We implemented our summarization technique in an automated program verification tool; our experimental evaluation demonstrates that our technique computes more precise summaries than existing abstract interpreters and that our tool’s verification capabilities are comparable with state-of-the-art software model checkers.

Decision Problems on Copying and Shuffling

We study decision problems of the form: given a regular or linear context-free language L, is there a word of a given fixed form in L, where given fixed forms are based on word operations copy, marked copy, shuffle and their combinations.

Remarks on context-free grammars with subregular control languages

A context-free grammar with control language is a pair (G,R) where G is a context-free grammar and R is a regular set over the set of productions of G. Its language consists of all terminal words where the sequence of applied productions belongs to R.We study context-free grammars with control languages belonging to subsets of the set of regular languages. We prove that we can obtain only context-free languages if we use regular commutative and strictly locally 1-testable languages. By strictly locally k-testable, k≥2, ordered, union-free, ordered, and regular circular control languages, we have no loss in the generative power, i.e., we generate the same family which is obtained by arbitrary regular control sets.

Context-Free Language Reachability via Skewed Tabulation

Context-Free Language Reachability via Skewed Tabulation

Design and Implementation of English to Yorùbá Verb Phrase Machine Translation System

The Yorùbá ethnic group has diverse language speakers across the world, and translating the language to other widely spoken languages must be emphasized. This study aims to develop an English to Yorùbá machine translation system which can translate the English verb phrase text to its Yorùbá equivalent. Words from both languages (Source Language and Target Language) were collected for the verb phrase group in the home domain. Lexical translation was done by assigning values of the matching word in the dictionary. The syntax of the two languages was realized using Context Free Grammar, the rewrite rules were validated with finite state automata. Human evaluation method was used and expert opinion scored. The evaluation shows the system performed better than that of sampled Google translation with over 70% of the responses matching that of the system’s output.

Classification of n-th Order Limit Language in Formal Language Classes

The study of splicing systems and their language has grown rapidly since Paun developed a splicing system known as a regular splicing scheme that produces a regular language. Since then, the researchers have been eager to classify the splicing language into certain classes in the Chomsky hierarchy, such as context-free language, context-sensitive language and recursive enumerable language. Previously, the study on the n-th order limit language was conducted from the biological perspective to the limit language produced. Still, no research has been done from the generation of language point of view. This research presents a generalization on the type of classes of the formal language, the n-th order limit language. The cases to obtain the n-th order limit language are revisited and used to obtain the types of language classes according to the Chomsky hierarchy produced by the n-th order limit language.

Scenario-Based Proofs for Concurrent Objects

Concurrent objects form the foundation of many applications that exploit multicore architectures and their importance has lead to informal correctness arguments, as well as formal proof systems. Correctness arguments (as found in the distributed computing literature) give intuitive descriptions of a few canonical executions or "scenarios" often each with only a few threads, yet it remains unknown as to whether these intuitive arguments have a formal grounding and extend to arbitrary interleavings over unboundedly many threads. We present a novel proof technique for concurrent objects, based around identifying a small set of scenarios (representative, canonical interleavings), formalized as the commutativity quotient of a concurrent object. We next give an expression language for defining abstractions of the quotient in the form of regular or context-free languages that enable simple proofs of linearizability. These quotient expressions organize unbounded interleavings into a form more amenable to reasoning and make explicit the relationship between implementation-level contention/interference and ADT-level transitions. We evaluate our work on numerous non-trivial concurrent objects from the literature (including the Michael-Scott queue, Elimination stack, SLS reservation queue, RDCSS and Herlihy-Wing queue). We show that quotients capture the diverse features/complexities of these algorithms, can be used even when linearization points are not straight-forward, correspond to original authors' correctness arguments, and provide some new scenario-based arguments. Finally, we show that discovery of some object's quotients reduces to two-thread reasoning and give an implementation that can derive candidate quotients expressions from source code.

Iterative-Epoch Online Cycle Elimination for Context-Free Language Reachability

Context-free language reachability (CFL-reachability) is a fundamental framework for implementing various static analyses. CFL-reachability utilizes context-free grammar (CFG) to extend the expressiveness of ordinary graph reachability from an unlabeled graph to an edge-labeled graph. Solving CFL-reachability requires a (sub)cubic time complexity with respect to the graph size, which limits its scalability in practice. Thus, an approach that can effectively reduce the graph size while maintaining the reachability result is highly desirable. Most of the existing graph simplification techniques for CFL-reachability work during the preprocessing stage, i.e., before the dynamic CFL-reachability solving process. However, in real-world CFL-reachability analyses, there is a large number of reducible nodes and edges that can only be discovered during dynamic solving, leaving significant room for on-the-fly improvements. This paper aims to reduce the graph size of CFL-reachability dynamically via online cycle elimination. We propose a simple yet effective approach to detect collapsible cycles in the graph based on the input context-free grammar. Our key insight is that symbols with particular forms of production rules in the grammar are the essence of transitivity of reachability relations in the graph. Specifically, in the graph, a reachability relation to a node v_i can be "transited" to another node v_j if there is a transitive relation from v_i to v_j, and cycles formed by transitive relations are collapsible. In this paper, we present an approach to identify the transitive symbols in a context-free grammar and propose an iterative-epoch framework for online cycle elimination. From the perspective of non-parallelized CFL-reachability solving, our iterative-epoch framework is well compatible with both the standard (unordered) solver and the recent ordered solver, and can significantly improve their performance. Our experiment on context-sensitive value-flow analysis for C/C++ and field-sensitive alias analysis for Java demonstrates promising performance improvement by our iterative-epoch cycle elimination technique. By collapsing cycles online, our technique accelerates the standard solver by 17.17× and 13.94× for value-flow analysis and alias analysis, respectively, with memory reductions of 48.8% and 45.0%. Besides, our technique can also accelerate the ordered solver by 14.32× and 8.36× for value-flow analysis and alias analysis, respectively, with memory reductions of 55.2% and 57.8%.

From words to pictures: Row-column combinations and Chomsky-Schützenberger theorem

The row-column combination RCC maps two (word) languages over the same alphabet onto the set of rectangular arrays, i.e., pictures, such that each row/column is a word of the first/second language. The resulting array is thus a crossword of the component words. Depending on the family of the components, different picture (2D) language families are obtained: e.g., the well-known tiling-system recognizable languages are the alphabetic projection of the crossword of local (regular) languages. We investigate the effect of the RCC operation especially when the components are context-free, also with application of an alphabetic projection. The resulting 2D families are compared with others defined in the past. The classical characterization of context-free languages, known as Chomsky-Schützenberger theorem, is extended to the crosswords in this way: the projection of a context-free crossword is equivalent to the projection of the intersection of a 2D Dyck language and the crossword of strictly locally testable language. The definition of 2D Dyck language relies on a new more flexible so-called Cartesian RCC operation on Dyck languages. The proof involves the version of the Chomsky-Schützenberger theorem that is non-erasing and uses a grammar-independent alphabet.

Two dimensional fuzzy context-free languages and tiling patterns

Fuzzy context-free languages are powerful compared to fuzzy regular languages as they are generated by fuzzy context-free grammars and fuzzy pushdown automata, which follow an enhanced computational mechanism. A two dimensional language (picture language) is a collection of two dimensional words, which are a rectangular array of symbols made up of finite alphabets. Two dimensional automata can recognize two dimensional languages that could not be recognized by one dimensional automata. In this paper, we introduce two dimensional fuzzy context-free languages generated by the two dimensional fuzzy context-free grammars and accepted by the two dimensional fuzzy pushdown automata in order to deal with the vagueness that arises in two dimensional context-free languages. We can construct a two dimensional fuzzy context free grammar from the given two dimensional fuzzy pushdown automata and vice versa. In addition, we prove that two dimensional fuzzy context-free languages are closed under union, column concatenation, column star, homomorphism, inverse homomorphism, reflection about right-most vertical, reflection about base, conjugation and half-turn and also show that two dimensional fuzzy context-free languages are not closed under matrix homomorphism, quarter-turn and transpose. Further, we have given the applications and the uses of closure properties in the formation of tiling patterns.

On the power of pushing or stationary moves for input-driven pushdown automata

Input-driven pushdown automata (IDPDAs) are pushdown automata where the next action on the pushdown store (push, pop, nothing) is solely governed by the input symbol. Nowadays such devices are usually defined such that every push operation pushes exactly one additional symbol on the pushdown store and, in addition, stationary moves are not allowed so that the devices work in real time. Here, we relax this strong definition and consider IDPDAs that may push more than one symbol in one step (push-IDPDA) or may perform stationary moves (stat-IDPDA). We study the computational power of the extended variants both in the deterministic and nondeterministic case, we investigate several decidability questions for the new automata classes, and we obtain interesting representations by inverse homomorphisms. Namely, every (1) deterministic, (2) real-time deterministic, and (3) nondeterministic context-free language can be characterized as the inverse homomorphic image of a language accepted by a (1) stat-IDPDA, (2) push-IDPDA, and (3) nondeterministic push-IDPDA.

Text analysis framework for identifying mutations among non-small cell lung cancer patients from laboratory data

BackgroundLaboratory data can provide great value to support research aimed at reducing the incidence, prolonging survival and enhancing outcomes of cancer. Data is characterized by the information it carries and the format it holds. Data captured in Alberta’s biomarker laboratory repository is free text, cluttered and rouge. Such data format limits its utility and prohibits broader adoption and research development. Text analysis for information extraction of unstructured data can change this and lead to more complete analyses. Previous work on extracting relevant information from free text, unstructured data employed Natural Language Processing (NLP), Machine Learning (ML), rule-based Information Extraction (IE) methods, or a hybrid combination between them.MethodsIn our study, text analysis was performed on Alberta Precision Laboratories data which consisted of 95,854 entries from the Southern Alberta Dataset (SAD) and 6944 entries from the Northern Alberta Dataset (NAD). The data covers all of Alberta and is completely population-based. Our proposed framework is built around rule-based IE methods. It incorporates topics such as Syntax and Lexical analyses to achieve deterministic extraction of data from biomarker laboratory data (i.e., Epidermal Growth Factor Receptor (EGFR) test results). Lexical analysis compromises of data cleaning and pre-processing, Rich Text Format text conversion into readable plain text format, and normalization and tokenization of text. The framework then passes the text into the Syntax analysis stage which includes the rule-based method of extracting relevant data. Rule-based patterns of the test result are identified, and a Context Free Grammar then generates the rules of information extraction. Finally, the results are linked with the Alberta Cancer Registry to support real-world cancer research studies.ResultsOf the original 5512 entries in the SAD dataset and 5017 entries in the NAD dataset which were filtered for EGFR, the framework yielded 5129 and 3388 extracted EGFR test results from the SAD and NAD datasets, respectively. An accuracy of 97.5% was achieved on a random sample of 362 tests.ConclusionsWe presented a text analysis framework to extract specific information from unstructured clinical data. Our proposed framework has shown that it can successfully extract relevant information from EGFR test results.

A Bit of Nondeterminism Makes Pushdown Automata Expressive and Succinct

We study the expressiveness and succinctness of history-deterministic pushdown automata (HD-PDA) over finite words, that is, pushdown automata whose nondeterminism can be resolved based on the run constructed so far, but independently of the remainder of the input word. These are also known as good-for-games pushdown automata. We prove that HD-PDA recognise more languages than deterministic PDA (DPDA) but not all context-free languages (CFL). This class is orthogonal to unambiguous CFL. We further show that HD-PDA can be exponentially more succinct than DPDA, while PDA can be double-exponentially more succinct than HD-PDA. We also study HDness in visibly pushdown automata (VPA), which enjoy better closure properties than PDA, and for which we show that deciding HDness is ExpTime-complete. HD-VPA can be exponentially more succinct than deterministic VPA, while VPA can be exponentially more succinct than HD-VPA. Both of these lower bounds are tight. We then compare HD-PDA with PDA for which composition with games is well-behaved, i.e. good-for-games automata. We show that these two notions coincide, but only if we consider potentially infinitely branching games. Finally, we study the complexity of resolving nondeterminism in HD-PDA. Every HDPDA has a positional resolver, a function that resolves nondeterminism and that is only dependant on the current configuration. Pushdown transducers are sufficient to implement the resolvers of HD-VPA, but not those of HD-PDA. HD-PDA with finite-state resolvers are determinisable.

Aperiodicity, Star-freeness, and First-order Logic Definability of Operator Precedence Languages

A classic result in formal language theory is the equivalence among non-counting, or aperiodic, regular languages, and languages defined through star-free regular expressions, or first-order logic. Past attempts to extend this result beyond the realm of regular languages have met with difficulties: for instance it is known that star-free tree languages may violate the non-counting property and there are aperiodic tree languages that cannot be defined through first-order logic. We extend such classic equivalence results to a significant family of deterministic context-free languages, the operator-precedence languages (OPL), which strictly includes the widely investigated visibly pushdown, alias input-driven, family and other structured context-free languages. The OP model originated in the '60s for defining programming languages and is still used by high performance compilers; its rich algebraic properties have been investigated initially in connection with grammar learning and recently completed with further closure properties and with monadic second order logic definition. We introduce an extension of regular expressions, the OP-expressions (OPE) which define the OPLs and, under the star-free hypothesis, define first-order definable and non-counting OPLs. Then, we prove, through a fairly articulated grammar transformation, that aperiodic OPLs are first-order definable. Thus, the classic equivalence of star-freeness, aperiodicity, and first-order definability is established for the large and powerful class of OPLs. We argue that the same approach can be exploited to obtain analogous results for visibly pushdown languages too.

Notes on subsemigroups of Nm0(+)

The paper summarizes and extends the knowledge of various subsemigroups of N m 0 (+) (= N0(+)m). It creates a theoretical basis for further study in this area and applications in other areas, such as the investigation of context-free languages. The last chapter introduces the notion of pure subsemigroups and presents one construction of a pure subsemigroup to a chosen semisubgroup of N m 0 (+).