Grammar Inference Research Articles

Building Greibach Normal Form Grammars Using Genetic Algorithms

Grammatical inference of context-free grammars using positive and negative language examples is among the most challenging task in modern artificial and natural language technology. Recently, several implementations combining various techniques, usually including the Backus–Naur form, have been proposed. In this paper, we explore a new implementation of grammatical inference using evolution methods focused on the Greibach normal form and exploiting its properties, and also propose new solutions both in the evolutionary processes and in the corresponding fitness estimation.

Building Analyses from Syntactic Inference in Local Languages: An HPSG Grammar Inference System

We present a grammar inference system that leverages linguistic knowledge recorded in the form of annotations in interlinear glossed text (IGT) and in a meta-grammar engineering system (the LinGO Grammar Matrix customization system) to automatically produce machine-readable HPSG grammars. Building on prior work to handle the inference of lexical classes, stems, affixes and position classes, and preliminary work on inferring case systems and word order, we introduce an integrated grammar inference system that covers a wide range of fundamental linguistic phenomena. System development was guided by 27 geneologically and geographically diverse languages, and we test the system's cross-linguistic generalizability on an additional 5 held-out languages, using datasets provided by field linguists. Our system out-performs three baseline systems in increasing coverage while limiting ambiguity and producing richer semantic representations, while also producing richer representations than previous work in grammar inference.

Learning grammar with a divide-and-concur neural network.

We implement a divide-and-concur iterative projection approach to context-free grammar inference. Unlike most state-of-the-art models of natural language processing, our method requires a relatively small number of discrete parameters, making the inferred grammar directly interpretable-one can read off from a solution how to construct grammatically valid sentences. Another advantage of our approach is the ability to infer meaningful grammatical rules from just a few sentences, compared to the hundreds of gigabytes of training data many other models employ. We demonstrate several ways of applying our approach: classifying words and inferring a grammar from scratch, taking an existing grammar and refining its categories and rules, and taking an existing grammar and expanding its lexicon as it encounters new words in new data.

Inferring Absolutely Non-Circular Attribute Grammars with a Memetic Algorithm

When valid syntactical structures are additionally constrained with context-sensitive information the Grammar Inference needs to be extended to the Semantic Inference. In this paper, it is shown that a complete compiler/interpreter for small Domain-Specific Languages (DSLs) can be generated automatically solely from given programs and their associated meanings using Semantic Inference. In this work a wider class of Attribute Grammars has been learned, while only S-attributed and L-attributed Grammars have previously been inferred successfully. Inferring Absolutely Non-Circular Attribute Grammars (ANC-AG) with complex dependencies among attributes has been achieved by integrating a Memetic Algorithm (MA) into the LISA.SI tool. The results show that the proposed Memetic Algorithm is at least four times faster on the selected benchmark than the previous method.

From Grammar Inference to Semantic Inference—An Evolutionary Approach

This paper describes a research work on Semantic Inference, which can be regarded as an extension of Grammar Inference. The main task of Grammar Inference is to induce a grammatical structure from a set of positive samples (programs), which can sometimes also be accompanied by a set of negative samples. Successfully applying Grammar Inference can result only in identifying the correct syntax of a language. With the Semantic Inference a further step is realised, namely, towards inducing language semantics. When syntax and semantics can be inferred, a complete compiler/interpreter can be generated solely from samples. In this work Evolutionary Computation was employed to explore and exploit the enormous search space that appears in Semantic Inference. For the purpose of this research work the tool LISA.SI has been developed on the top of the compiler/interpreter generator tool LISA. The first results are encouraging, since we were able to infer the semantics only from samples and their associated meanings for several simple languages, including the Robot language.

An interdisciplinary comparison of sequence modeling methods for next-element prediction

Data of sequential nature arise in many application domains in the form of, e.g., textual data, DNA sequences, and software execution traces. Different research disciplines have developed methods to learn sequence models from such datasets: (i) In the machine learning field methods such as (hidden) Markov models and recurrent neural networks have been developed and successfully applied to a wide range of tasks, (ii) in process mining process discovery methods aim to generate human-interpretable descriptive models, and (iii) in the grammar inference field the focus is on finding descriptive models in the form of formal grammars. Despite their different focuses, these fields share a common goal: learning a model that accurately captures the sequential behavior in the underlying data. Those sequence models are generative, i.e., they are able to predict what elements are likely to occur after a given incomplete sequence. So far, these fields have developed mainly in isolation from each other and no comparison exists. This paper presents an interdisciplinary experimental evaluation that compares sequence modeling methods on the task of next-element prediction on four real-life sequence datasets. The results indicate that machine learning methods, which generally do not aim at model interpretability, tend to outperform methods from the process mining and grammar inference fields in terms of accuracy.

A grammar inference approach for language self-adaptation and evolution in digital ecosystems

Socialization is the essential building process of any society in natural ecosystems. Effective socialization processes have been investigated for both “biotic” (human) and “abiotic” (virtual) entities, also within digital ecosystems in the perspective of common and self-adaptive languages. In this paper, we propose an approach for socialization, language self-adaptation, and evolution that enables an effective communicative interaction among digital entities acting in a digital ecosystem. The proposed method relies on an adaptable and extensible grammatical formalism, named Digital Ecosystem Grammar (DEG). This grammar enables digital entities to interpret the messages sent by other entities by using interaction, learning and evolution actions. Moreover, a grammar learning algorithm is applied to provide the self-adaptation mechanisms that allow the digital environment to adapt the interaction language according to new messages. The approach was suitable to support the characteristics of self-adaptation, context-awareness, evolvability, and semanticity of a digital ecosystem language.

Mobile Recruitment System for Nigerian Civil Service Commission via Cloud Computing

Traditional recruitment procedures are replaced in order to overcome most of its attendant challenges such as time-consuming and tiresome nature of recruiting a larger number of applicants into Civil Service from different parts of Nigeria considering multifaceted nature of the nation. This research work obliterate favouritism, nepotism and other corrupt means that were the usual practice in shortlisting prospective candidate for job, electronic recruitment system are enhanced with the availability of mobile platform that improve accessibility with emerging computing paradigm over the internet. Finite State Transducer (FST) in Machine Learning is used to learn from the pool of candidates for employment, GIATI(Grammar Inference and Alignment for Transducers Inference) are efficiently applied that is prospective candidates are offered job on a deductive rule-based Machine Learning. It is implemented with Android Studio and WAMP server. Its performance was tested using Ogun State as a pilot in the Federation and User Satisfaction were evaluated.

SEQUIN: a grammar inference framework for analyzing malicious system behavior

Targeted attacks on IT systems are a rising threat to the confidentiality of sensitive data and the availability of critical systems. The emergence of Advanced Persistent Threats (APTs) made it paramount to fully understand the particulars of such attacks in order to improve or devise effective defense mechanisms. Grammar inference paired with visual analytics (VA) techniques offers a powerful foundation for the automated extraction of behavioral patterns from sequential event traces. To facilitate the interpretation and analysis of APTs, we present SEQUIN, a grammar inference system based on the Sequitur compression algorithm that constructs a context-free grammar (CFG) from string-based input data. In addition to recursive rule extraction, we expanded the procedure through automated assessment routines capable of dealing with multiple input sources and types. This automated assessment enables the accurate identification of interesting frequent or anomalous patterns in sequential corpora of arbitrary quantity and origin. On the formal side, we extended the CFG with attributes that help describe the extracted (malicious) actions. Discovery-focused pattern visualization of the output is provided by our dedicated KAMAS VA prototype.

GrammarViz 3.0

The problems of recurrent and anomalous pattern discovery in time series, e.g., motifs and discords, respectively, have received a lot of attention from researchers in the past decade. However, since the pattern search space is usually intractable, most existing detection algorithms require that the patterns have discriminative characteristics and have its length known in advance and provided as input, which is an unreasonable requirement for many real-world problems. In addition, patterns of similar structure, but of different lengths may co-exist in a time series. Addressing these issues, we have developed algorithms for variable-length time series pattern discovery that are based on symbolic discretization and grammar inference—two techniques whose combination enables the structured reduction of the search space and discovery of the candidate patterns in linear time. In this work, we present GrammarViz 3.0—a software package that provides implementations of proposed algorithms and graphical user interface for interactive variable-length time series pattern discovery. The current version of the software provides an alternative grammar inference algorithm that improves the time series motif discovery workflow, and introduces an experimental procedure for automated discretization parameter selection that builds upon the minimum cardinality maximum cover principle and aids the time series recurrent and anomalous pattern discovery.

Synthesizing program input grammars

We present an algorithm for synthesizing a context-free grammar encoding the language of valid program inputs from a set of input examples and blackbox access to the program. Our algorithm addresses shortcomings of existing grammar inference algorithms, which both severely overgeneralize and are prohibitively slow. Our implementation, GLADE, leverages the grammar synthesized by our algorithm to fuzz test programs with structured inputs. We show that GLADE substantially increases the incremental coverage on valid inputs compared to two baseline fuzzers.

Faithful (meta-)encodings of programmable strategies into term rewriting systems

We consider the problem of inferring a grammar describing the output of a functional program given a grammar describing its input. Solutions to this problem are helpful for detecting bugs or proving safety properties of functional programs, and several rewriting tools exist for solving this problem. However, known grammar inference techniques are not able to take evaluation strategies of the program into account. This yields very imprecise results when the evaluation strategy matters. In this work, we adapt the Tree Automata Completion algorithm to approximate accurately the set of terms reachable by rewriting under the innermost strategy. We formally prove that the proposed technique is sound and precise w.r.t. innermost rewriting. We show that those results can be extended to the leftmost and rightmost innermost case. The algorithms for the general innermost case have been implemented in the Timbuk reachability tool. Experiments show that it noticeably improves the accuracy of static analysis for functional programs using the call-by-value evaluation strategy.

Content-based unsupervised segmentation of recurrent TV programs using grammatical inference

TV program segmentation raised as a major topic in the last decade for the task of high quality indexing of multimedia content. Earlier studies of TV program segmentation are either highly supervised (e.g., event detection) or too specific to a certain type of program (e.g., cluster-based methods), which is not practically usable for indexing tasks because of the lack of generality of programs types. In this paper, we address the problem of unsupervised TV program segmentation by leveraging grammatical inference, i.e., discovering a common structural model shared by a collection of episodes of a recurrent TV program by finding an optimal alignment of structural elements across episodes. Structural elements referring to a video segment with a particular syntactic meaning with respect to the video structure. The use of symbolic representation of structural elements makes grammatical inference feasible to be applied on TV program modeling, and makes TV program segmentation possible to rely on only minimal domain knowledge. The proposed approach is operated in two phases. The first phase aims at obtaining a symbolic representation of each episode, where the elements relevant to the structure are discovered based on recurrence mining. The second phase is that of grammatical inference from the symbolic representation of episodes. We investigate two inference techniques, one based on multiple sequence alignment and one relying on uniform resampling, to infer structural grammars for TV programs. A model of the structure is derived from the structural grammars and used to predict the structure of new episodes. Comparative evaluation on two grammar inference approaches demonstrates that the models obtained can reflect the structure of the program and predict the structure of unseen episodes, which is the main application of the proposed approach in industry, i.e., to assist librarians for segmentation tasks.

Source-Oriented Generalizations as Grammar Inference in Russian Vowel Deletion

Speakers learn detailed generalizations about the morphophonology of their language and extend them to nonce words. We propose a theory of this morphophonological knowledge that partitions the lexicon into uniform and productive sublexicons. Each sublexicon has its own phonotactic grammar, which the speaker uses as an inference mechanism to determine the relative productivity of each sublexicon. We report the results of an experiment on the generalization of mid vowel deletion (‘‘yer’’ deletion) in Russian, showing that speakers encode source-oriented generalizations about the shapes of words that can undergo vowel deletion, as well as product-oriented generalizations about words that result from vowel deletion. An implementation of our model learns the patterns of deletion and captures both source-oriented and product-oriented generalizations.

Software tools for understanding grammatical inference algorithms: Part I: Tools for regular grammars and finite-state automata

Software demonstrators are effective tools to show and understand scientific and engineering concepts in function, and they also allow rapid experiments. In the field of grammatical inference, there is a lack of 'ready-to-use' grammar synthesis tools, with simple interfaces showing intermediate stages of the grammar inference process, and the presented work addresses this issue by giving tools for experimentation with regular grammar and finite-state automata, to help students and researchers understand their properties, underlying concepts and applications.

Characterizing facial expressions by grammars of action unit sequences – A first investigation using ABL

We investigate the application of grammar inference to the analysis of facial expressions to discover underlying sequential regularities characteristic for a specific mental state. The input consists of sequences of action units (AUs), which represent basic facial signals. The typical classification task for facial expression analysis is to assign a set of AUs its corresponding mental state, e.g., an emotion. To our knowledge, there is no research investigating whether there is diagnostic information in the sequence in which the AUs occur in a given time interval. Our study is based on data of facial expressions of pain obtained in a psychological experiment with 347 pain episodes of 86 subjects represented as sequences of AUs. We applied the Alignment-Based Learning (ABL) approach to infer the underlying grammar for the set of all AUs which occurred in the sequences and for a reduced alphabet of the relevant AUs only. We used 10-fold cross-validation to estimate performance and we extended ABL with a frequency-based heuristics to reduce the number of grammar rules by eliminating such rules which do not contribute significantly to performance. The resulting grammar for the reduced AU alphabet provides a first approximation for a “grammar of pain”.

Study on mechanism for anti-hypertension efficacy of Eucommiae Cortex through assistant analysis systems for acting mechanisms of traditional Chinese medicine

Elucidate the mechanism of Eucommiae Cortex in treatment of hypertension, to provide the basis for further research and development of Eucommiae Cortex. Our study using the entity grammar systems inference rules to analyse the interactions of chemical constituents of Eucommiae Cortex and disease target proteins at the molecular level, and got a biological network of Eucommiae Cortex anti-hypertension which inciude 602 nodes and 2 354 edges. We got 3 treatment of hypertension pathways of Eucommiae Cortex by analyzing biological network, that is, by inhibition of vascular remodeling to improve the deterioration of hypertension, reduce activity of polymorphism genetic genes related to essential hypertension, inhibition of carbonic anhydrase 1 to maintain the osmotic pressure, Eucommiae Cortex play the role of anti-hypertension.

A grammar inference approach for predicting kinase specific phosphorylation sites.

Kinase mediated phosphorylation site detection is the key mechanism of post translational mechanism that plays an important role in regulating various cellular processes and phenotypes. Many diseases, like cancer are related with the signaling defects which are associated with protein phosphorylation. Characterizing the protein kinases and their substrates enhances our ability to understand the mechanism of protein phosphorylation and extends our knowledge of signaling network; thereby helping us to treat such diseases. Experimental methods for predicting phosphorylation sites are labour intensive and expensive. Also, manifold increase of protein sequences in the databanks over the years necessitates the improvement of high speed and accurate computational methods for predicting phosphorylation sites in protein sequences. Till date, a number of computational methods have been proposed by various researchers in predicting phosphorylation sites, but there remains much scope of improvement. In this communication, we present a simple and novel method based on Grammatical Inference (GI) approach to automate the prediction of kinase specific phosphorylation sites. In this regard, we have used a popular GI algorithm Alergia to infer Deterministic Stochastic Finite State Automata (DSFA) which equally represents the regular grammar corresponding to the phosphorylation sites. Extensive experiments on several datasets generated by us reveal that, our inferred grammar successfully predicts phosphorylation sites in a kinase specific manner. It performs significantly better when compared with the other existing phosphorylation site prediction methods. We have also compared our inferred DSFA with two other GI inference algorithms. The DSFA generated by our method performs superior which indicates that our method is robust and has a potential for predicting the phosphorylation sites in a kinase specific manner.

Segmentation of Document Using Discriminative Contextfree Grammar Inference and Alignment Similarities

Segmentation of Document Using Discriminative Contextfree Grammar Inference and Alignment Similarities

Inference of Probabilistic Grammars in Different Rules Systems of Natural Languages

Abstract Grammar induction is an important and current area within computational linguistics. The results in induction of string transformation rules can be applied not only in linguistics but in many other pattern recognition tasks. The task of grammar induction belongs in general to NP -hard problems. The paper analyses the main rule types in natural languages and in string transformation systems. The paper focuses on the inference of inflectional rule systems which differs in many aspects from the traditional production rule system of Chomsky grammars. The work compares the main candidate methods on the learning of objective-case in the Hungarian language.