Stochastic Context-free Grammars Research Articles

Estimation of Distribution Algorithm for Grammar-Guided Genetic Programming.

Genetic variation operators in grammar-guided genetic programming are fundamental to guide the evolutionary process in search and optimization problems. However, they show some limitations, mainly derived from an unbalanced exploration and local-search trade-off. This article presents an estimation of distribution algorithm for grammar-guided genetic programming to overcome this difficulty and thus increase the performance of the evolutionary algorithm. Our proposal employs an extended dynamic stochastic context-free grammar to encode and calculate the estimation of the distribution of the search space from some promising individuals in the population. Unlike traditional estimation of distribution algorithms, the proposed approach improves exploratory behavior by smoothing the estimated distribution model. Therefore, this algorithm is referred to as SEDA, smoothed estimation of distribution algorithm. Experiments have been conducted to compare overall performance using a typical genetic programming crossover operator, an incremental estimation of distribution algorithm, and the proposed approach after tuning their hyperparameters. These experiments involve challenging problems to test the local search and exploration features of the three evolutionary systems. The results show that grammar-guided genetic programming with SEDA achieves the most accurate solutions with an intermediate convergence speed.

RNA Secondary Structure Thermodynamics.

Several different ways to predict RNA secondary structures have been suggested in the literature. Statistical methods, such as those that utilize stochastic context-free grammars (SCFGs), or approaches based on machine learning aim to predict the best representative structure for the underlying ensemble of possible conformations. Their parameters have therefore been trained on larger subsets of well-curated, known secondary structures. Physics-based methods, on the other hand, usually refrain from using optimized parameters. They model secondary structures from loops as individual building blocks which have been assigned a physical property instead: the free energy of the respective loop. Such free energies are either derived from experiments or from mathematical modeling. This rigorous use of physical properties then allows for the application of statistical mechanics to describe the entire state space of RNA secondary structures in terms of equilibrium probabilities. On that basis, and by using efficient algorithms, many more descriptors of the conformational state space of RNA molecules can be derived to investigate and explain the many functions of RNA molecules. Moreover, compared to other methods, physics-based models allow for a much easier extension with other properties that can be measured experimentally. For instance, small molecules or proteins can bind to an RNA and their binding affinity can be assessed experimentally. Under certain conditions, existing RNA secondary structure prediction tools can be used to model this RNA-ligand binding and to eventually shed light on its impact on structure formation and function.

Confidence-based interactable neural-symbolic visual question answering

Visual question answering (VQA) task demands proficiency in processing multi-modal information, and the ability to reason effectively using the information. One promising method for this task is neural-symbolic (NS) learning, which leverages the strengths of both neural network (NN) learning and symbolic reasoning to achieve efficient VQA. However, current NS approaches do not account for the uncertain nature of NN learning and can only provide a single answer to a question without any indication of its confidence, thereby limiting their ability to handle incorrect reasoning. To address this limitation, we propose a confidence-based neural-symbolic (CBNS) approach, which evaluates the confidence of the NN inferences based on uncertainty quantification and makes confidence-based reasoning. The proposed approach comprises three main components: (1) a probabilistic question parser that generates multiple program candidates, each with a corresponding confidence evaluation; (2) a probabilistic scene perception module that provides object-based scene representation and confidence evaluations for each attribute of objects in an image; and (3) a confidence-based program executor that provides answers with confidence evaluations throughout the inference process by leveraging the confidence evaluations of the scene representation and programs. Additionally, we present a data augmentation method to improve the training efficiency of NS learning. The proposed approach allows user interactions and feedback on the weak links based on confidence evaluations. Experiments on CLEVR and GQA datasets demonstrate that the proposed approach was effective in identifying the correctness of predictions and led to a promising performance improvement with a significantly reduced computation cost.

Coordinated Motion Planning for Heterogeneous Autonomous Vehicles Based on Driving Behavior Primitives

Heterogeneous autonomous vehicle (HAV) coordinated motion planning must guide each vehicle out of the conflict zone based on the differences in vehicle platform characteristics. Decomposing complex driving tasks into primitives is an effective way to improve algorithm efficiency. Hence, the purpose of this paper is to complete the coordinated motion planning tasks through offline driving behavior primitive (DBP) library generation, online extension and selection of DBPs. The proposed algorithm applies dynamic movement primitives and singular value decomposition to learn driving behavior patterns from driving data, integrates them into a model-based optimization generation method as constraints, and builds a DBP library by fusing driving data and vehicle model. Based on the generated DBP library and primitive association probabilities learned from labeled driving segments via stochastic context-free grammar, the planning method completes the independent DBP extension of each vehicle in the conflict zone, generates an interaction DBP tree, and uses the mixed-integer linear programming algorithm to optimally select the primitives to be executed. This study demonstrates that the generated DBP library not only expands the types of primitives, but also distinguishes the characteristics of HAVs. We also present how to utilize the DBP libraries to obtain coordinated motion planning results with spatiotemporal information in the form of DBP extension and selection. The results obtained by real vehicle platforms and simulation show that the proposed method can accomplish coordinated motion planning tasks without relying on specific scene elements and highlight the unique motion characteristics of HAVs.

An Arabic Probabilistic Parser Based on a Property Grammar

The specificities of Arabic parsing, such as agglutination, vocalization, and the relatively order-free words in Arabic sentences, remain major issues to consider. To promote its robustness, such parseing should define different types of constraints. Property Grammar (PG) formalism verifies the satisfiability of the constraints directly on the units of the structure, thanks to its properties (or relations). In this context, we propose to build a probabilistic parser with syntactic properties, using a PG, and we measure the production rules in terms of different implicit information and in particular the syntactic properties. We experimented with our parser on the treebank ATB, using the parsing algorithm CYK, and we obtained encouraging results. Our method is also automatic for implementation of most property types. Its generalization for other languages or corpus domains (using treebanks) could be a good perspective. Its combination with pre-trained models of BERT may also make our parser faster.

Using language-specific input methods and pronunciation rules to improve the guesses of passwords

Password-based authentication has the advantage of easy implementation and no requirement for additional hardware; therefore, it will remain one of the primary identification methods in the near future. However, users tend to choose passwords that are easy to remember, so these passwords may be vulnerable to guessing attacks. Based on the idea that some users are influenced by their native language when choosing passwords, we propose an improved password-guessing method that adds grammars regarding Asian-language input procedures and pronunciation rules to the most updated Probabilistic Context-Free Grammar (PCFG) v4.1. The experimental results show that the improved PCFG v4.1 can increase the success rate of password cracking compared to PCFG v4.1. The improvements range from 2% to 14% for different password datasets. Additionally, we compare the proposed method to many other guessing methods; ours can achieve an excellent performance. Moreover, the characteristics of passwords that are not cracked are analyzed, and we suggest some criteria for more robust passwords.

An Eclectic Approach to Network Service Failure Detection Based on Multicriteria Analysis with an Example of Mixing Probabilistic Context Free Grammar Models

A method of failure detection in telecommunication networks is presented. This is a meta-method that correlates alarms raised by failure-detection modules based on various philosophies. The correlation takes into account two main characteristics of each module and the whole metamethod: the percentage of false alarms and the percentage of omitted failures. The trade-off between them is tackled with aspiration-based multicriteria analysis. The alarms are cor- related using linear classification by support vector machines. An example of the profitability of correlating alarms in such way is shown. This is an example of probabilistic context free grammars (PCFGs), used to model the proper runtime paths of network services (and thus usable for detecting an improper behavior of the services). It is shown that the linearly mixing PCFGs can add context handling to the PCFG model, thus augmenting the capabilities of the model.

A syntactic method for robot imitation learning of complex sequence task

PurposeAiming at the problems of weak generalization of robot imitation learning methods and higher accuracy requirements of low-level detectors, this study aims to propose an imitation learning method based on structural grammar.Design/methodology/approachThe paper proposes a hybrid training model based on artificial immune algorithm and the Baum–Welch algorithm to extract the action information of the demonstration activity to form the {action-object} sequence and extract the symbol description of the scene to form the symbol primitives sequence. Then, probabilistic context-free grammar is used to characterize and manipulate these sequences to form a grammar space. Minimum description length criteria are used to evaluate the quality of the grammar in the grammar space, and the improved beam search algorithm is used to find the optimal grammar.FindingsIt is found that the obtained general structure can parse the symbol primitive sequence containing noise and obtain the correct sequence, thereby guiding the robot to perform more complex and higher-order demonstration tasks.Practical implicationsUsing this strategy, the robot completes the fourth-order Hanoi tower task has been verified.Originality/valueAn imitation learning method for robots based on structural grammar is first proposed. The experimental results show that the method has strong generalization ability and good anti-interference performance.

Discriminative estimation of probabilistic context-free grammars for mathematical expression recognition and retrieval

We present a discriminative learning algorithm for the probabilistic estimation of two-dimensional probabilistic context-free grammars (2D-PCFG) for mathematical expressions recognition and retrieval. This algorithm is based on a generalization of the H-criterion as the objective function and the growth transformations as the optimization method. For the development of the discriminative estimation algorithm, the N-best interpretations provided by the 2D-PCFG have been considered. Experimental results are reported on two available datasets: Im2Latex and IBEM. The first experiment compares the proposed discriminative estimation method with the classic Viterbi-based estimation method. The second one studies the performance of the estimated models depending on the length of the mathematical expressions and the number of admissible errors in the metric used.

ParsingPhrase: Parsing-based automated quality phrase mining

Phrases represent independent semantics in natural language but usually have indeterminate lengths and different combinations. So, extracting meaningful phrases from unstructured texts will substantially reduce semantic ambiguity and lay the foundation for downstream natural language tasks. Most existing research obtains candidate phrases by N-grams, which includes meaningless word sequences and degrades algorithm performance. In this paper, we propose a novel phrase-mining algorithm, called ParsingPhrase, which effectively extracts combination phrases from text and improves phrase quality by syntactic features. It consists of three stages. Firstly, all sentences in texts are represented as parsing trees by PCFG (Probabilistic Context-Free Grammar). We propose PBMP (Parsing-Based Phrase mining) to obtain candidate phrases from those parsing trees. Then, we introduce a new phrase evaluation indicator, called Significance, that relies on the role of phrases to measure their importance. We integrate the Significance with conventional evaluation indexes for a more reasonable phrase evaluation. Finally, we optimize the phrase quality again by exploiting the optimal phrase composition features of sentences. To the best of our knowledge, it is the first work to employ parsing for combination phrase mining and evaluation, meanwhile offering a solution for syntactic disambiguation. Experiments on three real corpora demonstrate that the ParsingPhrase exceeds state-of-the-art baselines, is 7% higher in the candidate phrase conversion rate, and is 6% better in terms of Precision.

Dimensionally-consistent equation discovery through probabilistic attribute grammars

Equation discovery, also known as symbolic regression, is a machine learning task of inducing closed-form equations from data and background knowledge. The latter takes various forms. Domain-specific knowledge can constrain the space of candidate equations to those that make sense in the scientific or engineering domain of use. Cross-domain knowledge, on the other hand, imposes general rules for model acceptability, such as parsimony, understandability, or consistency of the equations with the dimensional units of the variables. In this paper, we propose using attribute grammars to ensure the induced equations' dimensional consistency. Attribute grammars are flexible enough to combine cross-domain knowledge on dimensional consistency with domain-specific knowledge expressed as a probabilistic context-free grammar. At the same time, we show that attribute grammars can be efficiently transformed into probabilistic context-free grammars for equation discovery with existing algorithms. Finally, we provide empirical evidence that attribute grammars ensuring dimensional consistency of equations can significantly improve the performance of equation discovery on the standard set of a hundred Feynman benchmarks.

Learning of Structurally Unambiguous Probabilistic Grammars

The problem of identifying a probabilistic context free grammar has two aspects: the first is determining the grammar's topology (the rules of the grammar) and the second is estimating probabilistic weights for each rule. Given the hardness results for learning context-free grammars in general, and probabilistic grammars in particular, most of the literature has concentrated on the second problem. In this work we address the first problem. We restrict attention to structurally unambiguous weighted context-free grammars (SUWCFG) and provide a query learning algorithm for \structurally unambiguous probabilistic context-free grammars (SUPCFG). We show that SUWCFG can be represented using \emph{co-linear multiplicity tree automata} (CMTA), and provide a polynomial learning algorithm that learns CMTAs. We show that the learned CMTA can be converted into a probabilistic grammar, thus providing a complete algorithm for learning a structurally unambiguous probabilistic context free grammar (both the grammar topology and the probabilistic weights) using structured membership queries and structured equivalence queries. A summarized version of this work was published at AAAI 21.

LiDetector : License Incompatibility Detection for Open Source Software

Open-source software (OSS) licenses dictate the conditions, which should be followed to reuse, distribute, and modify software. Apart from widely-used licenses such as the MIT License, developers are also allowed to customize their own licenses (called custom license), whose descriptions are more flexible. The presence of such various licenses imposes challenges to understand licenses and their compatibility. To avoid financial and legal risks, it is essential to ensure license compatibility when integrating third-party packages or reusing code accompanied with licenses. In this work, we propose LiDetector , an effective tool that extracts and interprets OSS licenses (including both official licenses and custom licenses), and detects license incompatibility among these licenses. Specifically, LiDetector introduces a learning-based method to automatically identify meaningful license terms from an arbitrary license, and employs Probabilistic Context-Free Grammar (PCFG) to infer rights and obligations for incompatibility detection. Experiments demonstrate that LiDetector outperforms existing methods with 93.28% precision for term identification, and 91.09% accuracy for right and obligation inference, and can effectively detect incompatibility with 10.06% FP rate and 2.56% FN rate. Furthermore, with LiDetector , our large-scale empirical study on 1,846 projects reveals that 72.91% of the projects are suffering from license incompatibility, including popular ones such as the MIT License and the Apache License. We highlighted lessons learned from perspectives of different stakeholders and made all related data and the replication package publicly available to facilitate follow-up research.

Exploring a diverse world of effector domains and amyloid signaling motifs in fungal NLR proteins.

NLR proteins are intracellular receptors constituting a conserved component of the innate immune system of cellular organisms. In fungi, NLRs are characterized by high diversity of architectures and presence of amyloid signaling. Here, we explore the diverse world of effector and signaling domains of fungal NLRs using state-of-the-art bioinformatic methods including MMseqs2 for fast clustering, probabilistic context-free grammars for sequence analysis, and AlphaFold2 deep neural networks for structure prediction. In addition to substantially improving the overall annotation, especially in basidiomycetes, the study identifies novel domains and reveals the structural similarity of MLKL-related HeLo- and Goodbye-like domains forming the most abundant superfamily of fungal NLR effectors. Moreover, compared to previous studies, we found several times more amyloid motif instances, including novel families, and validated aggregating and prion-forming properties of the most abundant of them in vitro and in vivo. Also, through an extensive in silico search, the NLR-associated amyloid signaling was identified in basidiomycetes. The emerging picture highlights similarities and differences in the NLR architectures and amyloid signaling in ascomycetes, basidiomycetes and other branches of life.

#Segments: A Dominant Factor of Password Security to Resist against Data-driven Guessing

Understanding which factors dominate password security is vital for users to create their secure passwords. Prior works generally consider the password length and the number of character classes as the dominant factors. However, creating secure passwords based on the above two factors becomes much more challenging than before due to the emergence of powerful data-driven guessing methods, e.g., the Probabilistic Context-free Grammars (PCFG) and its variations, Markov-based methods, and neural-network-based methods. In this paper, inspired by the segments used in PCFG, where a segment is a continuous string whose characters have a strong correlation, we conduct a comprehensive empirical analysis and find that the number of segments (# Segments for short) is a dominant factor of password security to resist against data-driven guessing. That is, the increase of # Segments generally leads to a significant improvement of password security. The observation helps us explore an optimised identification method for segments, referred to as re-segment, which reduces # Segments as much as possible to obtain accurate # Segments by leveraging five popular patterns (i.e., keyboard, abbreviation, leet, mixture, and component), to evaluate password security more accurately from an adversary’s viewpoint. Then we propose an efficient data-driven guessing method, referred to as ReSeg-PCFG, by leveraging re-segment based on the latest version of PCFG. Our study shows that ReSeg-PCFG outperforms the state-of-the-art data-driven guessing methods in almost all scenarios; e.g., it outperforms the latest version of PCFG by up to 79.34% at 1014 guesses, a commonly used threshold of off-line attacks.

Understanding digit‐only financial account passwords: ID card, structure, and security

Password-based authentication is widely used in website access. Current studies on password security have focused considerable attention on letter-only passwords or passwords that include both letters and digits. However, little research has been conducted on commercial and digit-only passwords. In this paper, we comprehensively evaluate digit-only passwords by analyzing approximately 130,000 passwords from the 12306 data set. We observe that approximately 20% of six-digit passwords belonging to Chinese users include the user's identity document (ID) card information. Most of the passwords are found to be related to the user's birth dates. Additionally, we demonstrate that male users are more likely to set passwords with digits extracted from their ID card numbers than female users. It is the first time that the relation between real-user passwords and ID card series numbers was considered. It focuses on actual attack methodologies and real-user passwords, which renders this study an explicit study on digit-only password security. We propose a password-encryption authentication model based on FF3-1 to implement secure storage of passwords. We experiment with ciphertext and plaintext passwords using three–three and four–two structures combined with probabilistic context-free grammars. The cracking success rates of ciphertext passwords are found to be 0.086% and 1.06%, and they can recover 59.83% and 33.2% of the passwords in the test set. Conversely, using plaintext passwords results in the recovery of 98% and 85% of passwords in the test set. Our results demonstrate that the use of encryption for password storage can significantly reduce the cracking success rate of attackers.

ALP: Data Augmentation Using Lexicalized PCFGs for Few-Shot Text Classification

Data augmentation has been an important ingredient for boosting performances of learned models. Prior data augmentation methods for few-shot text classification have led to great performance boosts. However, they have not been designed to capture the intricate compositional structure of natural language. As a result, they fail to generate samples with plausible and diverse sentence structures. Motivated by this, we present the data Augmentation using Lexicalized Probabilistic context-free grammars (ALP) that generates augmented samples with diverse syntactic structures with plausible grammar. The lexicalized PCFG parse trees consider both the constituents and dependencies to produce a syntactic frame that maximizes a variety of word choices in a syntactically preservable manner without specific domain experts. Experiments on few-shot text classification tasks demonstrate that ALP enhances many state-of-the-art classification methods. As a second contribution, we delve into the train-val splitting methodologies when a data augmentation method comes into play. We argue empirically that the traditional splitting of training and validation sets is sub-optimal compared to our novel augmentation-based splitting strategies that further expand the training split with the same number of labeled data. Taken together, our contributions on the data augmentation strategies yield a strong training recipe for few-shot text classification tasks.

LPG-PCFG: An Improved Probabilistic Context- Free Grammar to Hit Low-Probability Passwords.

With the development of the Internet, information security has attracted more attention. Identity authentication based on password authentication is the first line of defense; however, the password-generation model is widely used in offline password attacks and password strength evaluation. In real attack scenarios, high-probability passwords are easy to enumerate; extremely low-probability passwords usually lack semantic structure and, so, are tough to crack by applying statistical laws in machine learning models, but these passwords with lower probability have a large search space and certain semantic information. Improving the low-probability password hit rate in this interval is of great significance for improving the efficiency of offline attacks. However, obtaining a low-probability password is difficult under the current password-generation model. To solve this problem, we propose a low-probability generator–probabilistic context-free grammar (LPG–PCFG) based on PCFG. LPG–PCFG directionally increases the probability of low-probability passwords in the models’ distribution, which is designed to obtain a degeneration distribution that is friendly for generating low-probability passwords. By using the control variable method to fine-tune the degeneration of LPG–PCFG, we obtained the optimal combination of degeneration parameters. Compared with the non-degeneration PCFG model, LPG–PCFG generates a larger number of hits. When generating and times, the number of hits to low-probability passwords increases by and , respectively.

Absence of phase transition in random language model

The random language model, proposed as a simple model of human languages, is defined by the averaged model of a probabilistic context-free grammar. This grammar expresses the process of sentence generation as a tree graph with nodes having symbols as variables. Previous studies proposed that a phase transition, which can be considered to represent the emergence of order in language, occurs in the random language model. We discuss theoretically that the analysis of the ``order parameter'' introduced in previous studies can be reduced to solving the maximum eigenvector of the transition probability matrix determined by a grammar. This helps analyze the distribution of a quantity determining the behavior of the order parameter and reveals that no phase transition occurs. Our results suggest the need to study a more complex model such as a probabilistic context-sensitive grammar, in order for phase transitions to occur.

Language Design as Information Renormalization

Here we consider some well-known facts in syntax from a physics perspective, allowing us to establish equivalences between both fields with many consequences. Mainly, we observe that the operation MERGE, put forward by N. Chomsky in 1995, can be interpreted as a physical information coarse-graining. Thus, MERGE in linguistics entails information renormalization in physics, according to different time scales. We make this point mathematically formal in terms of language models. In this setting, MERGE amounts to a probability tensor implementing a coarse-graining, akin to a probabilistic context-free grammar. The probability vectors of meaningful sentences are given by stochastic tensor networks (TN) built from diagonal tensors and which are mostly loop-free, such as Tree Tensor Networks and Matrix Product States, thus being computationally very efficient to manipulate. We show that this implies the polynomially-decaying (long-range) correlations experimentally observed in language, and also provides arguments in favour of certain types of neural networks for language processing. Moreover, we show how to obtain such language models from quantum states that can be efficiently prepared on a quantum computer, and use this to find bounds on the perplexity of the probability distribution of words in a sentence. Implications of our results are discussed across several ambits.