Grammar Inference Research Articles

Performance Evaluation of Methods for handling Premature Convergence in GA - Case of Grammar Induction

Genetic algorithms are meta-heuristic algorithms based on the biological evolution. These algorithms are found to be useful for finding near to optimum results for the NP-category of problems. GA suffers with the disadvantage of premature convergence. The paper focuses on the implementation of various techniques of handling premature convergence and the statistical evaluation of the obtained results to identify the optimal method to the problem of grammar induction. General Terms Evolutionary Computations, Grammar Inference, Learning.

Improving Grammar Inference by a Memetic Algorithm

A memetic algorithm, a novel approach for solving NP-hard problems, has been applied in this paper for grammatical inference in the field of domain-specific languages (DSLs). DSLs are often designed by domain experts who have no knowledge about the syntax and semantics of programming languages. However, they are able to write sample programs to accomplish their goals and illustrate the features of their language. Grammatical inference is a technique to infer a context-free grammar from a set of positive (and negative) samples. This paper shows that grammatical inference may assist domain experts and software language engineers in developing DSLs by automatically producing a grammar, which describes a set of sample DSL programs. A memetic-algorithm-based tool is developed, which greatly improves results and robustness of the inference process.

Hybrid system for handling premature convergence in GA – Case of grammar induction

Grammar induction (also known as grammar inference or language learning) is the process of learning grammar from training data. This paper presents hybrid genetic algorithm (GA) Tool for handling premature convergence for the case of grammar induction. The tool is developed in Java and the results are tested for the set of 20 languages. It is an effort towards introduction of a hybrid GA system where the collection of more than one type of reproduction methods is used for the grammar induction. Paper also includes the discussion on the various approaches used by author for handling the premature convergence problem in using GA.

Learning CFG using Improved TBL algorithm

Grammar Induction is the process of learning context free grammar from finite set of data of the positive and negative sample of language. The Problem of learning context free grammars from example has two aspects viz. determining the grammatical structure (topology) of the unknown grammar and identifying non-terminals. To solve these problems through a new hypothesis representation method called tabular representation is used , This method uses a table like data structure similar to the parse table used in the Cocke-Younger-Kasami parsing algorithm (CYK Algorithm) for context free grammar of Chomsky Normal Form. In this paper we introduce some improvement to the tabular representation algorithm (TBL) called Improved TBL algorithm (ITBL) dedicated to inference of grammar in Chomsky normal form. This ITBL algorithm uses a Genetic Algorithm (GA) to solve partitioning problem and focuses on the modified fitness function. It gives faster and optimal solutions for 5-8 string length.

On the inference of non-confluent NLC graph grammars

Grammar inference deals with determining (preferably simple) models/grammars consistent with a set of observations. There is a large body of research on grammar inference within the theory of formal languages. However, there is surprisingly little known on grammar inference for graph grammars. In this article, we take a further step in this direction and work within the framework of node label controlled (NLC) graph grammars. Specifically, given a graph G and a set 𝒮 of disjoint and isomorphic subgraphs of G, we characterize whether or not there is a graph grammar consisting of one production such that G may be derived from G0, the graph obtained from G by ‘contraction’ of each subgraph in 𝒮 to a node labelled by N. This generalizes a previous result that assumes boundary NLC graph grammars, and leads one to consider the more involved ‘non-confluent’ graph grammar rules.

A memetic grammar inference algorithm for language learning

An unsupervised incremental algorithm for grammar inference and its application to domain-specific language development are described. Grammatical inference is the process of learning a grammar from the set of positive and optionally negative sentences. Learning general context-free grammars is still considered a hard problem in machine learning and is not completely solved yet. The main contribution of the paper is a newly developed memetic algorithm, which is a population-based evolutionary algorithm enhanced with local search and a generalization process. The learning process is incremental since a new grammar is obtained from the current grammar and false negative samples, which are not parsed by the current grammar. Despite being incremental, the learning process is not sensitive to the order of samples. All important parts of this algorithm are explained and discussed. Finally, a case study of a domain specific language for rendering graphical objects is used to show the applicability of this approach.

A Learning Algorithm for Multimodal Grammar Inference

The high costs of development and maintenance of multimodal grammars in integrating and understanding input in multimodal interfaces lead to the investigation of novel algorithmic solutions in automating grammar generation and in updating processes. Many algorithms for context-free grammar inference have been developed in the natural language processing literature. An extension of these algorithms toward the inference of multimodal grammars is necessary for multimodal input processing. In this paper, we propose a novel grammar inference mechanism that allows us to learn a multimodal grammar from its positive samples of multimodal sentences. The algorithm first generates the multimodal grammar that is able to parse the positive samples of sentences and, afterward, makes use of two learning operators and the minimum description length metrics in improving the grammar description and in avoiding the over-generalization problem. The experimental results highlight the acceptable performances of the algorithm proposed in this paper since it has a very high probability of parsing valid sentences.

Computational inference of grammars for larger-than-gene structures from annotated gene sequences

Larger than gene structures (LGS) are DNA segments that include at least one gene and often other segments such as inverted repeats and gene promoters. Mobile genetic elements (MGE) such as integrons are LGS that play an important role in horizontal gene transfer, primarily in Gram-negative organisms. Known LGS have a profound effect on organism virulence, antibiotic resistance and other properties of the organism due to the number of genes involved. Expert-compiled grammars have been shown to be an effective computational representation of LGS, well suited to automating annotation, and supporting de novo gene discovery. However, development of LGS grammars by experts is labour intensive and restricted to known LGS. This study uses computational grammar inference methods to automate LGS discovery. We compare the ability of six algorithms to infer LGS grammars from DNA sequences annotated with genes and other short sequences. We compared the predictive power of learned grammars against an expert-developed grammar for gene cassette arrays found in Class 1, 2 and 3 integrons, which are modular LGS containing up to 9 of about 240 cassette types. Using a Bayesian generalization algorithm our inferred grammar was able to predict > 95% of MGE structures in a corpus of 1760 sequences obtained from Genbank (F-score 75%). Even with 100% noise added to the training and test sets, we obtained an F-score of 68%, indicating that the method is robust and has the potential to predict de novo LGS structures when the underlying gene features are known. http://www2.chi.unsw.edu.au/attacca.

A survey of grammatical inference methods for natural language learning

The high complexity of natural language and the huge amount of human and temporal resources necessary for producing the grammars lead several researchers in the area of Natural Language Processing to investigate various solutions for automating grammar generation and updating processes. Many algorithms for Context-Free Grammar inference have been developed in the literature. This paper provides a survey of the methodologies for inferring context-free grammars from examples, developed by researchers in the last decade. After introducing some preliminary definitions and notations concerning learning and inductive inference, some of the most relevant existing grammatical inference methods for Natural Language are described and classified according to the kind of presentation (if text or informant) and the type of information (if supervised, unsupervised, or semi-supervised). Moreover, the state of the art of the strategies for evaluation and comparison of different grammar inference methods is presented. The goal of the paper is to provide a reader with introduction to major concepts and current approaches in Natural Language Learning research.

Generating Multimodal Grammars for Multimodal Dialogue Processing

This paper presents a new multimodal grammar generation system (MGGS) that allows defining a multimodal grammar in a very easy and intuitive way, overcoming the difficulties arising from the textual description of grammar production rules. The novelty of the proposed approach relies in adopting a “by example” paradigm to define a multimodal grammar. This paradigm consists in providing concrete examples of multimodal sentences, and enabling a grammar inference algorithm to automatically generate the grammar rules to parse those sentences. The main contribution of this approach is that it is general enough to be applicable for whatever modalities and in whichever domains. Moreover, the use of the grammar inference algorithm for automating the grammar writing and updating processes, reduces costs of the grammar development and maintenance. In the first part of this paper, we present the MGGS, describing in detail the methodology we have implemented in this system. More specifically, the multimodal attribute grammar and the grammar inference algorithm are illustrated. The second part describes the experiment aimed at observing participants while interacting with the system in order to provide some real data about the usability of the system. Results of this experiment showed that the proposed system facilitates the grammar definition and updating, and it is more suitable also for nonexpert people as it does not require the learning of the grammar notation.

Structural analysis of regulatory DNA sequences using grammar inference and Support Vector Machine

Regulatory DNA sequences such as promoters or splicing sites control gene expression and are important for successful gene prediction. Such sequences can be recognized by certain patterns or motifs that are conserved within a species. These patterns have many exceptions which makes the structural analysis of regulatory sequences a complex problem. Grammar rules can be used for describing the structure of regulatory sequences; however, the manual derivation of such rules is not trivial. In this paper, stochastic L-grammar rules are derived automatically from positive examples and counterexamples of regulatory sequences using genetic programming techniques. The fitness of grammar rules is evaluated using a Support Vector Machine (SVM) classifier. SVM is trained on known sequences to obtain a discriminating function which serves for evaluating a candidate grammar ruleset by determining the percentage of generated sequences that are classified correctly. The combination of SVM and grammar rule inference can mitigate the lack of structural insight in machine learning approaches such as SVM.

Learning context-free grammar using improved tabular representation

This paper describes an improved version of TBL algorithm [Y. Sakakibara, Learning context-free grammars using tabular representations, Pattern Recognition 38(2005) 1372–1383; Y. Sakakibara, M. Kondo, GA-based learning of context-free grammars using tabular representations, in: Proceedings of 16th International Conference in Machine Learning (ICML-99), Morgan-Kaufmann, Los Altos, CA, 1999] for inference of context-free grammars in Chomsky Normal Form. The TBL algorithm is a novel approach to overcome the hardness of learning context-free grammars from examples without structural information available. The algorithm represents the grammars by parsing tables and thanks to this tabular representation the problem of grammar learning is reduced to the problem of partitioning the set of nonterminals. Genetic algorithm is used to solve NP-hard partitioning problem. In the improved version modified fitness function and new delete specialized operator is applied. Computer simulations have been performed to determine improved a tabular representation efficiency. The set of experiments has been divided into 2 groups: in the first one learning the unknown context-free grammar proceeds without any extra information about grammatical structure, in the second one learning is supported by a partial knowledge of the structure. In each of the performed experiments the influence of partition block size in an initial population and the size of population at grammar induction has been tested. The new version of TBL algorithm has been experimentally proved to be not so much vulnerable to block size and population size, and is able to find the solutions faster than standard one.

A texture segmentation prototype for industrial inspection applications based on fuzzy grammar

PurposeThe purpose of this paper is to propose a set of techniques, in the domain of texture analysis, dedicated to the classification of industrial textures. One of the main purposes was to deal with a high diversity of textures, including structural and highly random patterns.Design/methodology/approachThe global system includes a texture segmentation phase and a classification phase. The approach for image texture segmentation is based on features extracted from wavelets transform, fuzzy spectrum and interaction maps. The classification architecture uses a fuzzy grammar inference system.FindingsThe classifier uses the aggregation of features from the several segmentation techniques, resulting in high flexibility concerning the diversity of industrial textures. The resulted system allows on‐line learning of new textures. This approach avoids the need for a global re‐learning of the all textures each time a new texture is presented to the system.Practical implicationsThese achievements demonstrate the practical value of the system, as it can be applied to different industrial sectors for quality control operations.Originality/valueThe global approach was integrated in a cork vision system, leading to an industrial prototype that has already been tested. Similarly, it was tested in a textile machine, for a specific fabric inspection, and gave results that corroborate the diversity of possible applications. The segmentation procedure reveals good performance that is indicated by high classification rates, revealing good perspectives for full industrialization.

Cork Parquet Quality Control Vision System Based on Texture Segmentation and Fuzzy Grammar

This paper presents a quality control vision system developed for the inspection of cork parquets that is already applied in the Portuguese cork industry. It is devoted specifically to the most critical quality issues: visibility of the lowest layer (BASE) on the noble layer (UPPER) and the homogeneity of this noble layer. Since these aspects are related with the texture of the raw material, the system was based on texture segmentation techniques. Features used were extracted from detail images of the wavelet transform. The classifier consists of a fuzzy grammar inference system. The segmentation procedure revealed a good performance indicated by high classification rates. Behavior in the industrial environment has been demonstrating high performance, revealing good perspectives for full spread industrialization.

A unified approach for determining the underlying causes of non-stationary disturbances

In this paper, we propose a framework for the automatic detection and classification of power distribution feeder disturbances based on their underlying causes. The segmentation algorithm based on either a Kalman Filter (KF) or a Wavelet Filter divides the quasi-stationary Root-Mean-Square (RMS) of the captured signal into pre-disturbance, disturbance and post-disturbance regions. The pre- and post-disturbance segments are essentially stationary while the non-stationary nature is extracted as the disturbance segment. Each region is then represented as a sequence of predefined wave patterns or primitives. A syntactically correct combination of these primitives will define the morphology of the mother RMS signal. The grammar, the production rules and the model for each class is built from a set of positive examples (I +) by using a stochastic Error-Correcting Grammar Inference (ECGI) engine. When used in combination with a k-nearest neighbour algorithm (kNN) classifier, this framework can recognise any event and learn new patterns.

AN UNSUPERVISED INCREMENTAL LEARNING ALGORITHM FOR DOMAIN-SPECIFIC LANGUAGE DEVELOPMENT

While grammar inference (or grammar induction) has found extensive application in the areas of robotics, computational biology, and speech recognition, its application to problems in programming language and software engineering domains has been limited. We have found a new application area for grammar inference which intends to make domain-specific language development easier for domain experts not well versed in programming language design, and finds a second application in construction of renovation tools for legacy software systems. As a continuation of our previous efforts to infer context-free grammars (CFGs) for domain-specific languages which previously involved a genetic-programming based CFG inference system, we discuss extensions to the inference capabilities of GenInc, an incremental learning algorithm for inferring CFGs. We show that these extensions enable GenInc to infer more comprehensive grammars, discuss the results of applying GenInc to various domain-specific languages and evaluate the results using a comprehensive suite of grammar metrics.

Introduction to the special issue on program comprehension through dynamic analysis (PCODA)

Introduction to the special issue on program comprehension through dynamic analysis (PCODA)

Improving dynamic software analysis by applying grammar inference principles

AbstractGrammar inference is a family of machine learning techniques that aim to infer grammars from a sample of sentences in some (unknown) language. Dynamic analysis is a family of techniques in the domain of software engineering that attempts to infer rules that govern the behaviour of software systems from a sample of executions. Despite their disparate domains, both fields have broadly similar aims; they try to infer rules that govern the behaviour of some unknown system from a sample of observations. Deriving general rules about program behaviour from dynamic analysis is difficult because it is virtually impossible to identify and supply a complete sample of necessary program executions. The problems that arise with incomplete input samples have been extensively investigated in the grammar inference community. This has resulted in a number of advances that have produced increasingly sophisticated solutions that are more successful at accurately inferring grammars from (potentially) sparse information about the underlying system. This paper investigates the similarities and shows how many of these advances can be applied with similar effect to dynamic analysis problems by a series of small experiments on random state machines. Copyright © 2008 John Wiley & Sons, Ltd.

INFERENCE OF EDGE REPLACEMENT GRAPH GRAMMARS

We describe an algorithm and experiments for inference of edge replacement graph grammars. This method generates candidate recursive graph grammar productions based on isomorphic subgraphs which overlap by two nodes. If there is no edge between the two overlapping nodes, the method generates a recursive graph grammar production with a virtual edge. We guide the search for the graph grammar based on the size of the grammar and the portion of the graph described by the grammar. We show experiments where we generate graphs from known graph grammars, use our method to infer the grammar from the generated graphs, and then measure the error between the original and inferred grammars. Experiments show that the method performs well on several types of grammars, and specifically that error decreases with increased numbers of unique labels in the graph.

Inferring Graph Grammars by Detecting Overlap in Frequent Subgraphs

Inferring Graph Grammars by Detecting Overlap in Frequent SubgraphsIn this paper we study the inference of node and edge replacement graph grammars. We search for frequent subgraphs and then check for an overlap among the instances of the subgraphs in the input graph. If the subgraphs overlap by one node, we propose a node replacement graph grammar production. If the subgraphs overlap by two nodes or two nodes and an edge, we propose an edge replacement graph grammar production. We can also infer a hierarchy of productions by compressing portions of a graph described by a production and then inferring new productions on the compressed graph. We validate the approach in experiments where we generate graphs from known grammars and measure how well the approach infers the original grammar from the generated graph. We show graph grammars found in biological molecules, biological networks, and analyze learning curves of the algorithm.