Grammatical Inference Research Articles

Robust models to infer flexible nondeterministic finite automata

Grammatical inference involves learning a formal grammar as a finite state machine or a set of rewrite rules. This paper focuses on inferring nondeterministic finite automata (NFA) from a given sample of words: the NFA must accept some words, and reject the others. To address this task we construct several over-constrained inference models capable of finding NFA of size k+1, which are directly convertible to NFA of size k. Additionally, we propose an NFA evaluation method based on random walks along with two methods used to enhance the acceptance rates of the inferred NFA. The effectiveness of our approach is demonstrated through the results of comprehensive experiments conducted on several benchmark sets. This paper is an extended version of the ICCS 2023 paper entitled “Inference of over-constrained NFA of size k+1 to efficiently and systematically derive NFA of size k for grammar learning” (Jastrzab et al., 2023 [1]).

The grammatical inference of the Quranic recitations between Interpretation Al-Zamakhshari and Al-Razi- a study of the names-

The grammatical inference of the Quranic recitations between Interpretation Al-Zamakhshari and Al-Razi- a study of the names-

D0L-System Inference from a Single Sequence with a Genetic Algorithm

In this paper, we proposed a new method for image-based grammatical inference of deterministic, context-free L-systems (D0L systems) from a single sequence. This approach is characterized by first parsing an input image into a sequence of symbols and then, using a genetic algorithm, attempting to infer a grammar that can generate this sequence. This technique has been tested using our test suite and compared to similar algorithms, showing promising results, including solving the problem for systems with more rules than in existing approaches. The tests show that it performs better than similar heuristic methods and can handle the same cases as arithmetic algorithms.

A Generative Urban Space Design Method Based on Shape Grammar and Urban Induction Patterns

This paper focuses on how to solve the demand of how to quickly explore more urban space design and layout in terms of the conservation and reuse of historical blocks under computer-aided design technology. Referring to urban inductive patterns, the author establishes an urban space design grammar of historical districts based on shape grammar and pattern language. It forms a complete generative urban space design method involving morphological analysis, rule-making, and scheme generation. Furthermore, taking the typical Russian-style historical block along the Chinese Eastern Railway—the historical district in front of Anda Station as an application example—this paper completed the generative urban space design process of block redevelopment and environmental reconstruction, which, using computer-aided technology, was under the premise of protecting the original historic block fabric. The final two group results of the experimental plot and the other plots of the historical district show that this method can quickly and accurately generate lots of urban space design schemes that meet the designers’ pre-thinking, and these schemes can be modified by real-time calculation and interactive operation.

Indonesian news classification application with named entity recognition approach

Nowadays, many netizens search for news via search engines with countless amounts of information, so it is increasingly difficult to determine when the number of news articles that appear changes very quickly and dynamically. Thus, it is necessary to process the extraction of news information to display the core information of the news. Problems arise, especially in Indonesian, which has a structure of various noun phrase entities with shallow parsing or grammatical induction. Named Entity Recognition (NER) has the opportunity to overcome this because it can extract news entities in depth, starting from proper nouns in text documents containing information search, machine translation, answering questions, and automatic summarization. This study aims to apply NER in Indonesian language news classification. This study uses Design-Based Research whose process includes (1) pre-implementation, (2) design, (3) implementation and revision, and finally, (4) reflection and evaluation. This application was developed on the platform python, streamlit, BeautifulSoup, gnews, and spacy library. The results of application accuracy testing have an F1-score value of 89.69% for all entities consisting of place, figure, day, date, and organization.

Paradoxes of MaxEnt markedness

Over the past two decades, theoretical linguistics has taken a probabilistic turn. Maximum entropy (ME) has been endorsed as a model of probabilistic phonology because of its classical guarantees for grammatical inference. Yet, little is known about the basic organizing principles of ME phonology beyond circumstantial evidence of ME’s ability to fit specific patterns of empirical frequencies. The study of ME typologies is difficult because they consist of infinitely many grammars that cannot be exhaustively listed and directly inspected. Uniform Probability Inequalities (Anttila and Magri 2018) are a new tool that solves the problem: they characterize cases where one phonological mapping has a probability smaller than another mapping and this probability inequality holds uniformly for every grammar in the typology. In other words, uniform probability inequalities are universals of probabilistic grammars. We present a new generalization about ME uniform probability inequalities and argue that they are phonologically paradoxical and prune the set of ME universals down to almost nothing. This suggests that ME is not a suitable model of phonology.

Sentential inference bridging between lexical/grammatical knowledge and text comprehension among native Chinese speakers learning Japanese.

The current study explored the role of sentential inference in connecting lexical/grammatical knowledge and overall text comprehension in foreign language learning. Using structural equation modeling (SEM), causal relationships were examined between four latent variables: lexical knowledge, grammatical knowledge, sentential inference, and text comprehension. The study analyzed 281 Chinese university students learning Japanese as a second language and compared two causal models: (1) the partially-mediated model, which suggests that lexical knowledge, grammatical knowledge, and sentential inference concurrently influence text comprehension, and (2) the wholly-mediated model, which posits that both lexical and grammatical knowledge impact sentential inference, which then further affects text comprehension. The SEM comparison analysis supported the wholly-mediated model, showing sequential causal relationships from lexical knowledge to sentential inference and then to text comprehension, without significant contribution from grammatical knowledge. The results indicate that sentential inference serves as a crucial bridge between lexical knowledge and text comprehension.

A Reinforcement Learning Based Grammatical Inference Algorithm Using Block-Based Delta Inverse Strategy

A resurgent interest for grammatical inference aka automaton learning has emerged in several intriguing areas of computer sciences such as machine learning, software engineering, robotics and internet of things. An automaton learning algorithm commonly uses queries to learn the regular grammar of a Deterministic Finite Automaton (DFA). These queries are posed to a Minimum Adequate Teacher (MAT) by the learner (Learning Algorithm). The membership and equivalence queries which the learning algorithm may pose, are often capable of having their answers provided by the MAT. The three main categories of learning algorithms are incremental, sequential, and complete learning algorithms. In the presence of a MAT, the time complexity of existing DFA learning algorithms is polynomial. Therefore, in some applications these algorithms may fail to learn the system. In this study, we have reduced the time complexity of DFA learning from polynomial to logarithmic form. For this, we propose an efficient complete DFA learning algorithm; the Block based DFA Learning through Inverse Query (BDLIQ) using block based delta inverse strategy, which is based on the idea of inverse queries that John Hopcroft introduced for state minimization of a DFA. The BDLIQ algorithm possess <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O(\vert \Sigma \vert N.log N)$ </tex-math></inline-formula> complexity when a MAT is available. The MAT is also made capable of responding to inverse queries. We provide theoretical and empirical analysis of the proposed algorithm. Results show that our suggested approach for complete learning; BDLIQ algorithm, is more efficient than the ID algorithm in terms of time complexity.

الرازي والقراءات القرآنية دراسة في المنهج والمورد

This research seeks to clarify the method of Imam Al-Fakhr Al-Razi and his resources in grammatical inference with Quranic readings through his interpretation of the (Mafaatihul-Gheib) with presentation of Quranic evidence, the title of the search came to be tagged with (Al-Razi and the Qur’anic Readings: A Study in the Method and the Resource) Mainly in the process of theorizing and analysis, because of its importance in the Qur’anic readings; this interpretation is an encyclopedia that includes various types of issues of different sciences, The research also aims to clarify his attribution to the readings and his approach to it, and to discover his method of inferring the Qur’anic readings and its influence on its predecessors, Because in most of his inferences from the Qur’anic readings, he supported his idea with the opinions of other scientists. The research is consisting of an introduction, two demands, and a conclusion which contains the most important results. The first demand was entitled: Al-Razi's method in his grammatical inference with the Qur’anic readings. The conclusion dealt with the most important findings of the research, we can mention among them: The research has proven that the imam cared about the readings and employed them in his interpretation, especially his inference from the readings related to grammatical issues. Al-Razi was clearly influenced by his predecessors, but this did not affect him to be a transmitter only, but rather to have an outstanding personality that transmits, criticizes, and gives weight. For he did not care to which school the scholar he took belongs to, visual, Kufic or Baghdadi. He also did not pay attention to the creedal sect to which he belongs, whether it was an Ash'ari or a Mu'tazilite or otherwise.

Building Greibach Normal Form Grammars Using Genetic Algorithms

Grammatical inference of context-free grammars using positive and negative language examples is among the most challenging task in modern artificial and natural language technology. Recently, several implementations combining various techniques, usually including the Backus–Naur form, have been proposed. In this paper, we explore a new implementation of grammatical inference using evolution methods focused on the Greibach normal form and exploiting its properties, and also propose new solutions both in the evolutionary processes and in the corresponding fitness estimation.

МОДЕЛІ РОЗУМІННЯ ІНШОМОВНОГО ТЕКСТУ

The article aims: 1) to analyze the theoretical development of models for understanding a foreign language text from a psychological point of view; 2) to emphasize their contribution to the general theory of understanding; 3) to structure, single out models of comprehension of a foreign language text. It also considers psycholinguistic and cognitive processes that affect the reader’s understanding of a foreign language text, analyzes resonant, constructivist and dynamic approaches to the processes of decoding lexical and grammatical information and inference of the meaning of the text. The main idea of the resonant approach is that in the process of reading, the reader automatically activates and removes from long-term memory all the information that corresponds to the semantic and phonological material of the text. The constructivist approach assumes that text comprehension is based on cognitively processed meanings of lexical units, which are always constructed in working memory depending on the content of the context and verbal/nonverbal experience of the reader. Currently, a dynamic approach to modelling the foreign text comprehension dominates, in which the resonant process and construction are considered to interact on a stage basis: the initial broad activation of lexical and grammatical concepts is accompanied by cognitive processing and extraction of relevant meanings from different knowledge structures in the long run. Thus, all these processes actively interact with the individual cognitive base of the reader, which does not lead to the creation of a literal, but to a more complex mental representation of a foreign text. The created representation depends not only on the previous knowledge of the reader, but also enriches his frames, which contain information about the typical and the possible in relevant situations.

Active Grammatical Inference for Non-Markovian Planning

Planning in finite stochastic environments is canonically posed as a Markov decision process where the transition and reward structures are explicitly known. Reinforcement learning (RL) lifts the explicitness assumption by working with sampling models instead. Further, with the advent of reward machines, we can relax the Markovian assumption on the reward. Angluin's active grammatical inference algorithm L* has found novel application in explicating reward machines for non-Markovian RL. We propose maintaining the assumption of explicit transition dynamics, but with an implicit non-Markovian reward signal, which must be inferred from experiments. We call this setting non-Markovian planning, as opposed to non-Markovian RL. The proposed approach leverages L* to explicate an automaton structure for the underlying planning objective. We exploit the environment model to learn an automaton faster and integrate it with value iteration to accelerate the planning. We compare against recent non-Markovian RL solutions which leverage grammatical inference, and establish complexity results that illustrate the difference in runtime between grammatical inference in planning and RL settings.

Online perceptual learning and natural language acquisition for autonomous robots

In this work, the problem of bootstrapping knowledge in language and vision for autonomous robots is addressed through novel techniques in grammar induction and word grounding to the perceptual world. In particular, we demonstrate a system, called OLAV, which is able, for the first time, to (1) learn to form discrete concepts from sensory data; (2) ground language (n-grams) to these concepts; (3) induce a grammar for the language being used to describe the perceptual world; and moreover to do all this incrementally, without storing all previous data. The learning is achieved in a loosely-supervised manner from raw linguistic and visual data. Moreover, the learnt model is transparent, rather than a black-box model and is thus open to human inspection. The visual data is collected using three different robotic platforms deployed in real-world and simulated environments and equipped with different sensing modalities, while the linguistic data is collected using online crowdsourcing tools and volunteers. The analysis performed on these robots demonstrates the effectiveness of the framework in learning visual concepts, language groundings and grammatical structure in these three online settings.

Simple Robust Grammar Induction with Combinatory Categorial Grammars

We present a simple EM-based grammar induction algorithm for Combinatory Categorial Grammar (CCG) that achieves state-of-the-art performance by relying on a minimal number of very general linguistic principles. Unlike previous work on unsupervised parsing with CCGs, our approach has no prior language-specific knowledge, and discovers all categories automatically. Additionally, unlike other approaches, our grammar remains robust when parsing longer sentences, performing as well as or better than other systems. We believe this is a natural result of using an expressive grammar formalism with an extended domain of locality.

Distillation of weighted automata from recurrent neural networks using a spectral approach

This paper is an attempt to bridge the gap between deep learning and grammatical inference. Indeed, it provides an algorithm to extract a (stochastic) formal language from any recurrent neural network trained for language modelling. In detail, the algorithm uses the already trained network as an oracle—and thus does not require the access to the inner representation of the black-box—and applies a spectral approach to infer a weighted automaton. As weighted automata compute linear functions, they are computationally more efficient than neural networks and thus the nature of the approach is the one of knowledge distillation. We detail experiments on 62 data sets (both synthetic and from real-world applications) that allow an in-depth study of the abilities of the proposed algorithm. The results show the WA we extract are good approximations of the RNN, validating the approach. Moreover, we show how the process provides interesting insights toward the behavior of RNN learned on data, enlarging the scope of this work to the one of explainability of deep learning models.

Montague Grammar Induction

We propose a computational model for inducing full-fledged combinatory categorial grammars from behavioral data. This model contrasts with prior computational models of selection in representing syntactic and semantic types as structured (rather than atomic) objects, enabling direct interpretation of the modeling results relative to standard formal frameworks. We investigate the grammar our model induces when fit to a lexicon-scale acceptability judgment dataset – Mega Acceptability – focusing in particular on the types our model assigns to clausal complements and the predicates that select them.

Split-Based Algorithm for Weighted Context-Free Grammar Induction

The split-based method in a weighted context-free grammar (WCFG) induction was formalised and verified on a comprehensive set of context-free languages. WCFG is learned using a novel grammatical inference method. The proposed method learns WCFG from both positive and negative samples, whereas the weights of rules are estimated using a novel Inside–Outside Contrastive Estimation algorithm. The results showed that our approach outperforms in terms of F1 scores of other state-of-the-art methods.

Statistical exploratory analysis of mask-fill reproduction operators of Genetic Algorithms

To be successful, a search algorithm needs to balance exploration and exploitation. In Genetic Algorithms (GAs) this is achieved through proportionate selection of individuals and reproduction operators. GAs can suffer premature convergence, when the diversity of the population decreases over time and search gets trapped in a local optimum, returning a very poor quality solution. Mask-fill reproduction operators utilizes bit-masking oriented data structure (BMODS) that maintains a good ratio between exploration and exploitation. Mask-fill reproduction operators have been utilized in various applications, but the exploration and exploitation ability of mask-fill reproduction operators have not been compared against other reproduction operators. This paper describes a rigorous and practical statistical methodology for the exploratory analysis of the mask-fill reproduction operators. First, the issues of robust experimental design and setting the control parameters for implementing a GA is addressed. Second, the impact of various reproduction operator combinations are analysed. In this study, three crossover operators and five mutation operators are considered which creates fifteen crossover–mutation operator combinations. Third, the methodology is demonstrated by considering grammatical inference (GI) problem as domain of enquiry. A hybrid genetic algorithm integrated with Sequitur algorithm (GAWS) is proposed for GI. Numerical results are presented to describe the effect of crossover–mutation operator combinations. The performance of the proposed GAWS is compared against the state-of-the-art algorithms. Statistical test are conducted to determine the performance significance of both crossover–mutation operator combinations and the proposed GAWS algorithm.

Parsing Expression Grammars and Their Induction Algorithm

Grammatical inference (GI), i.e., the task of finding a rule that lies behind given words, can be used in the analyses of amyloidogenic sequence fragments, which are essential in studies of neurodegenerative diseases. In this paper, we developed a new method that generates non-circular parsing expression grammars (PEGs) and compares it with other GI algorithms on the sequences from a real dataset. The main contribution of this paper is a genetic programming-based algorithm for the induction of parsing expression grammars from a finite sample. The induction method has been tested on a real bioinformatics dataset and its classification performance has been compared to the achievements of existing grammatical inference methods. The evaluation of the generated PEG on an amyloidogenic dataset revealed its accuracy when predicting amyloid segments. We show that the new grammatical inference algorithm achieves the best ACC (Accuracy), AUC (Area under ROC curve), and MCC (Mathew’s correlation coefficient) scores in comparison to five other automata or grammar learning methods.

The Return of Lexical Dependencies: Neural Lexicalized PCFGs

In this paper we demonstrate that context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies. This contrasts to the most popular current methods for grammar induction, which focus on discovering either constituents or dependencies. Previous approaches to marry these two disparate syntactic formalisms (e.g., lexicalized PCFGs) have been plagued by sparsity, making them unsuitable for unsupervised grammar induction. However, in this work, we present novel neural models of lexicalized PCFGs that allow us to overcome sparsity problems and effectively induce both constituents and dependencies within a single model. Experiments demonstrate that this unified framework results in stronger results on both representations than achieved when modeling either formalism alone.1