Grammar-based Genetic Programming Research Articles

Leveraging explainable AI for enhanced decision making in humanitarian logistics: An Adversarial CoevoluTION (ACTION) framework

This study examines the potential of AI-enabled wargames to enhance strategic decision-making in humanitarian assistance and disaster relief (HADR). We introduce an Adversarial CoevoluTION (ACTION) framework, which showcases AI’s capacity to evolve adaptable policies capable of responding to dynamic changes and adversarial actions in HADR wargame scenarios. The framework presented employs a grammar-based genetic programming algorithm to evolve intelligent and interpretable player policies. We apply the ACTION framework to a HADR wargame case study, commonly used by the Australian Defence Science and Technology Group for research purposes. The case study centres on a hypothetical disaster relief scenario in the fictional Joadia Islands, struck by a tsunami, necessitating the evacuation of dispersed civilians. Experimental results illustrate that the ACTION framework can evolve policies that adapt to environmental uncertainties and respond effectively to adversarial actions. This study offers evidence of the potential and practical application of AI-enabled technology in real-life humanitarian situations. Our findings suggest practical guidelines for humanitarian practitioners to enhance the efficiency and effectiveness of logistics planning for humanitarian aid, ultimately leading to improved outcomes in HADR scenarios.

Dynamic Grammar Pruning for Program Size Reduction in Symbolic Regression

Grammar is a key input in grammar-based genetic programming. Grammar design not only influences performance, but also program size. However, grammar design and the choice of productions often require expert input as no automatic approach exists. This research work discusses our approach to automatically reduce a bloated grammar. By utilizing a simple Production Ranking mechanism, we identify productions which are less useful and dynamically prune those to channel evolutionary search towards better (smaller) solutions. Our objective in this work was program size reduction without compromising generalization performance. We tested our approach on 13 standard symbolic regression datasets with Grammatical Evolution. Using a grammar embodying a well-defined function set as a baseline, we compare effective genome length and test performance with our approach. Dynamic grammar pruning achieved significantly better genome lengths for all datasets, while significantly improving generalization performance on three datasets, although it worsened in five datasets. When we utilized linear scaling during the production ranking stages (the first 20 generations) the results dramatically improved. Not only were the programs smaller in all datasets, but generalization scores were also significantly better than the baseline in 6 out of 13 datasets, and comparable in the rest. When the baseline was also linearly scaled as well, the program size was still smaller with the Production Ranking approach, while generalization scores dropped in only three datasets without any significant compromise in the rest.

Automated multi-objective system identification using grammar-based genetic programming

In order to use existing identification tools effectively, a user must make critical choices a priori that ultimately determine the quality of estimated models. Furthermore, while estimated models are typically optimized for a single identification criterion, engineering applications typically impose multiple performance specifications that may contradict each other. In this contribution, we develop a system identification methodology that automatically selects parametric model structures from a wide range of dynamic system models based on measured data. The problem of inferring model structures and estimating model parameters within these structures is encapsulated in a bi-level optimization problem. The optimization problem is formulated for multiple user-specified identification objectives. Finally, the range of dynamical systems considered for the optimization problem is specified using Tree Adjoining Grammar. A solution approach based on genetic programming is developed, and its asymptotic properties and computational complexity is analysed. The empirical performance of the proposed identification techniques is studied using a simulation example.

Performance Prediction of the Full-Scale KNR Biological Nutrient Removal Process Using Multivariate Dynamic Process Model with the Grammar-Based Genetic Programming (GBGP) System

Biological wastewater treatment systems are characterized by large temporal variability of inflow, variable concentrations of components in the incoming wastewater to the plant, and highly variable biological reactions within the process. This paper proposes an automatic process model induction system using an evolutionary computional intelligence, called grammar-based genetic programming (GBGP), that is specially designed to automatically discover multivariate dynamic process models that best fit observed process data. This automatic process model induction system combines an evolutionary self-organizing system of genetic programming paradigm with various mathematical functions for a multivariate nonlinear model evolution using a grammar system via the mechanism of genetics and natural selection. In the GBGP, a four-step modelling procedure was performed: (1) initialization: generatr an initial population of P models using contex-free grammar. generation K=0; (2) optimize each model (variables and constants) in the population; (3) execute and evaluate the fitness of each model in the population; (4) genetic loop: repeat until termination criterion is met (maximum generation Kmax). The results demonstrate that the multivariate dynamic process modelling technique presented using a grammar-based genetic programming (GBGP) provides a valuable tool for predicting the out puts with high levels of accuracy and identifying key operating variables for the full-scale KNR biological nutrient removal process. These modells are derived automatically in the form of understandable mathematical formulas that enable engineers to extract important knowledge hidden in the data and develop better operation and control strategies.

Mining Road Traffic Rules with Signal Temporal Logic and Grammar-Based Genetic Programming

Traffic systems, where human and autonomous drivers interact, are a very relevant instance of complex systems and produce behaviors that can be regarded as trajectories over time. Their monitoring can be achieved by means of carefully stated properties describing the expected behavior. Such properties can be expressed using Signal Temporal Logic (STL), a specification language for expressing temporal properties in a formal and human-readable way. However, manually authoring these properties is a hard task, since it requires mastering the language and knowing the system to be monitored. Moreover, in practical cases, the expected behavior is not known, but it has instead to be inferred from a set of trajectories obtained by observing the system. Often, those trajectories come devoid of human-assigned labels that can be used as an indication of compliance with expected behavior. As an alternative to manual authoring, automatic mining of STL specifications from unlabeled trajectories would enable the monitoring of autonomous agents without sacrificing human-readability. In this work, we propose a grammar-based evolutionary computation approach for mining the structure and the parameters of an STL specification from a set of unlabeled trajectories. We experimentally assess our approach on a real-world road traffic dataset consisting of thousands of vehicle trajectories. We show that our approach is effective at mining STL specifications that model the system at hand and are interpretable for humans. To the best of our knowledge, this is the first such study on a set of unlabeled real-world road traffic data. Being able to mine interpretable specifications from this kind of data may improve traffic safety, because mined specifications may be helpful for monitoring traffic and planning safety promotion strategies.

EvoStencils: a grammar-based genetic programming approach for constructing efficient geometric multigrid methods

For many systems of linear equations that arise from the discretization of partial differential equations, the construction of an efficient multigrid solver is challenging. Here we present EvoStencils, a novel approach for optimizing geometric multigrid methods with grammar-guided genetic programming, a stochastic program optimization technique inspired by the principle of natural evolution. A multigrid solver is represented as a tree of mathematical expressions that we generate based on a formal grammar. The quality of each solver is evaluated in terms of convergence and compute performance by automatically generating an optimized implementation using code generation that is then executed on the target platform to measure all relevant performance metrics. Based on this, a multi-objective optimization is performed using a non-dominated sorting-based selection. To evaluate a large number of solvers in parallel, they are distributed to multiple compute nodes. We demonstrate the effectiveness of our implementation by constructing geometric multigrid solvers that are able to outperform hand-crafted methods for Poisson’s equation and a linear elastic boundary value problem with up to 16 million unknowns on multi-core processors with Ivy Bridge and Broadwell microarchitecture.

Probabilistic Contextual and Structural Dependencies Learning in Grammar-Based Genetic Programming.

Genetic Programming is a method to automatically create computer programs based on the principles of evolution. The problem of deceptiveness caused by complex dependencies among components of programs is challenging. It is important because it can misguide Genetic Programming to create suboptimal programs. Besides, a minor modification in the programs may lead to a notable change in the program behaviours and affect the final outputs. This article presents Grammar-Based Genetic Programming with Bayesian Classifiers (GBGPBC) in which the probabilistic dependencies among components of programs are captured using a set of Bayesian network classifiers. Our system was evaluated using a set of benchmark problems (the deceptive maximum problems, the royal tree problems, and the bipolar asymmetric royal tree problems). It was shown to be often more robust and more efficient in searching the best programs than other related Genetic Programming approaches in terms of the total number of fitness evaluation. We studied what factors affect the performance of GBGPBC and discovered that robust variants of GBGPBC were consistently weakly correlated with some complexity measures. Furthermore, our approach has been applied to learn a ranking program on a set of customers in direct marketing. Our suggested solutions help companies to earn significantly more when compared with other solutions produced by several well-known machine learning algorithms, such as neural networks, logistic regression, and Bayesian networks.

Using Multi-objective Grammar-based Genetic Programming to Integrate Multiple Social Theories in Agent-based Modeling.

Different theoretical mechanisms have been proposed for explaining complex social phenomena. For example, explanations for observed trends in population alcohol use have been postulated based on norm theory, role theory, and others. Many mechanism-based models of phenomena attempt to translate a single theory into a simulation model. However, single theories often only represent a partial explanation for the phenomenon. The potential of integrating theories together, computationally, represents a promising way of improving the explanatory capability of generative social science. This paper presents a framework for such integrative model discovery, based on multi-objective grammar-based genetic programming (MOGGP). The framework is demonstrated using two separate theory-driven models of alcohol use dynamics based on norm theory and role theory. The proposed integration considers how the sequence of decisions to consume the next drink in a drinking occasion may be influenced by factors from the different theories. A new grammar is constructed based on this integration. Results of the MOGGP model discovery process find new hybrid models that outperform the existing single-theory models and the baseline hybrid model. Future work should consider and further refine the role of domain experts in defining the meaningfulness of models identified by MOGGP.

Multi-view Genetic Programming Learning to Obtain Interpretable Rule-Based Classifiers for Semi-supervised Contexts. Lessons Learnt

Multi-view learning analyzes the information from several perspectives and has largely been applied on semi-supervised contexts. It has not been extensively analyzed for inducing interpretable rule-based classifiers. We present a multi-view and grammar-based genetic programming model for inducing rules for semi-supervised contexts. It evolves several populations and views, and promotes both accuracy and agreement among the views. This work details how and why common practices may not produce the expected results when inducing rule-based classifiers under this methodology.

The impact of genetic programming in education

Since its inception genetic programming, and later variations such as grammar-based genetic programming and grammatical evolution, have contributed to various domains such as classification, image processing, search-based software engineering, amongst others. This paper examines the role that genetic programming has played in education. The paper firstly provides an overview of the impact that genetic programming has had in teaching and learning. The use of genetic programming in intelligent tutoring systems, predicting student performance and designing learning environments is examined. A critical analysis of genetic programming in education is provided. The paper then examines future directions of research and challenges in the application of genetic programming in education.

Automated Self-Optimization in Heterogeneous Wireless Communications Networks

Traditional single-tiered wireless communications networks cannot scale to satisfy exponentially rising demand. Operators are increasing capacity by densifying their existing macro cell deployments with co-channel small cells. However, cross-tier interference and load balancing issues present new optimization challenges in channel sharing heterogeneous networks (HetNets). One-size-fits-all heuristics for allocating resources are highly suboptimal, but designing ad hoc controllers requires significant human expertise and manual fine-tuning. In this paper, a unified, flexible, and fully automated approach for end-to-end optimization in multi-layer HetNets is presented. A hill climbing algorithm is developed for reconfiguring cells in real time in order to track dynamic traffic patterns. Schedulers for allocating spectrum to user equipment are automatically synthesized using grammar-based genetic programming. The proposed methods for configuring the HetNet and scheduling in the time–frequency domain can address ad hoc objective functions. Thus, the operator can flexibly tune the tradeoff between peak rates and fairness. Far cell edge downlink rates are increased by up to 250% compared with non-adaptive baselines. Alternatively, peak rates are increased by up to 340%. The experiments illustrate the utility and future potential of natural computing techniques in software-defined wireless communications networks.

Multiobjective grammar-based genetic programming applied to the study of asthma and allergy epidemiology

BackgroundAsthma and allergies prevalence increased in recent decades, being a serious global health problem. They are complex diseases with strong contextual influence, so that the use of advanced machine learning tools such as genetic programming could be important for the understanding the causal mechanisms explaining those conditions. Here, we applied a multiobjective grammar-based genetic programming (MGGP) to a dataset composed by 1047 subjects. The dataset contains information on the environmental, psychosocial, socioeconomics, nutritional and infectious factors collected from participating children. The objective of this work is to generate models that explain the occurrence of asthma, and two markers of allergy: presence of IgE antibody against common allergens, and skin prick test positivity for common allergens (SPT).ResultsThe average of the accuracies of the models for asthma higher in MGGP than C4.5. IgE were higher in MGGP than in both, logistic regression and C4.5. MGGP had levels of accuracy similar to RF, but unlike RF, MGGP was able to generate models that were easy to interpret.ConclusionsMGGP has shown that infections, psychosocial, nutritional, hygiene, and socioeconomic factors may be related in such an intricate way, that could be hardly detected using traditional regression based epidemiological techniques. The algorithm MGGP was implemented in c ++ and is available on repository: http://bitbucket.org/ciml-ufjf/ciml-lib.

Mining Context-Aware Association Rules Using Grammar-Based Genetic Programming.

Real-world data usually comprise features whose interpretation depends on some contextual information. Such contextual-sensitive features and patterns are of high interest to be discovered and analyzed in order to obtain the right meaning. This paper formulates the problem of mining context-aware association rules, which refers to the search for associations between itemsets such that the strength of their implication depends on a contextual feature. For the discovery of this type of associations, a model that restricts the search space and includes syntax constraints by means of a grammar-based genetic programming methodology is proposed. Grammars can be considered as a useful way of introducing subjective knowledge to the pattern mining process as they are highly related to the background knowledge of the user. The performance and usefulness of the proposed approach is examined by considering synthetically generated datasets. A posteriori analysis on different domains is also carried out to demonstrate the utility of this kind of associations. For example, in educational domains, it is essential to identify and understand contextual and context-sensitive factors that affect overall and individual student behavior and performance. The results of the experiments suggest that the approach is feasible and it automatically identifies interesting context-aware associations from real-world datasets.

Tactics Exploration Framework based on Genetic Programming

Engagement-level simulation is a quantitative way to evaluate the effectiveness of weapon systems before construction and acquisition, minimizing the risk of investment. Though contractors have built simulation systems with high fidelity models of weapon systems and battlefields, developing competent tactics to give full play to new weapon systems in simulation experiments is labor intensive, as most classical tactics tend to be out of date. In this work, we proposed a tactics exploration framework (TEF) that applied grammar-based genetic programming (GP) to generating and evolving tactics in the engagement-level simulation. Tactics are represented with modular behavior trees (BTs) for compatibility with the genetic operators. Experiments to explore submarine tactics have been conducted to observe and study the exploration process. The experimental results show that the TEF based on GP is efficient to explore tactics in the formalism of BTs.

Grammar-based generation of variable-selection heuristics for constraint satisfaction problems

We propose a grammar-based genetic programming framework that generates variable-selection heuristics for solving constraint satisfaction problems. This approach can be considered as a generation hyper-heuristic. A grammar to express heuristics is extracted from successful human-designed variable-selection heuristics. The search is performed on the derivation sequences of this grammar using a strongly typed genetic programming framework. The approach brings two innovations to grammar-based hyper-heuristics in this domain: the incorporation of if-then-else rules to the function set, and the implementation of overloaded functions capable of handling different input dimensionality. Moreover, the heuristic search space is explored using not only evolutionary search, but also two alternative simpler strategies, namely, iterated local search and parallel hill climbing. We tested our approach on synthetic and real-world instances. The newly generated heuristics have an improved performance when compared against human-designed heuristics. Our results suggest that the constrained search space imposed by the proposed grammar is the main factor in the generation of good heuristics. However, to generate more general heuristics, the composition of the training set and the search methodology played an important role. We found that increasing the variability of the training set improved the generality of the evolved heuristics, and the evolutionary search strategy produced slightly better results.

Grammar-based generation of stochastic local search heuristics through automatic algorithm configuration tools

Several grammar-based genetic programming algorithms have been proposed in the literature to automatically generate heuristics for hard optimization problems. These approaches specify the algorithmic building blocks and the way in which they can be combined in a grammar; the best heuristic for the problem being tackled is found by an evolutionary algorithm that searches in the algorithm design space defined by the grammar.In this work, we propose a novel representation of the grammar by a sequence of categorical, integer, and real-valued parameters. We then use a tool for automatic algorithm configuration to search for the best algorithm for the problem at hand. Our experimental evaluation on the one-dimensional bin packing problem and the permutation flowshop problem with weighted tardiness objective shows that the proposed approach produces better algorithms than grammatical evolution, a well-established variant of grammar-based genetic programming. The reasons behind such improvement lie both in the representation proposed and in the method used to search the algorithm design space.

Knowledge Discovering in Corporate Securities Fraud by Using Grammar Based Genetic Programming

Securities fraud is a common worldwide problem, resulting in serious negative consequences to securities market each year. Securities Regulatory Commission from various countries has also attached great importance to the detection and prevention of securities fraud activities. Securities fraud is also increasing due to the rapid expansion of securities market in China. In accomplishing the task of securities fraud detection, China Securities Regulatory Commission (CSRC) could be facilitated in their work by using a number of data mining techniques. In this paper, we investigate the usefulness of Logistic regression model, Neural Networks (NNs), Sequential minimal optimization (SMO), Radial Basis Function (RBF) networks, Bayesian networks and Grammar Based Genet- ic Programming (GBGP) in the classification of the real, large and latest China Corporate Securities Fraud (CCSF) database. The six data mining techniques are compared in terms of their performances. As a result, we found GBGP outperforms others. This paper describes the GBGP in detail in solving the CCSF problem. In addition, the Synthetic Minority Over-sampling Technique (SMOTE) is applied to generate synthetic minority class examples for the imbalanced CCSF dataset.

Aplicación de Algoritmos Evolutivos como Técnica de Minería de Datos para la Mejora de Cursos Hipermedia Adaptativos basados en Web

This paper shows how to use evolutionary algorithms to data mining methods for discovering prediction rules in databases. These rules will be used to improve web-based adpative hypermedia courses. The idea is to discover important rules among the usage data picked up during the students’ execuctions. This information may be very useful to the author of the course, who can decide what modification will be the most apropiate to improve the performance of students. In order to do the discovering of rules we have used grammar-based genetic programming (GBGP) with multiobjective optimization technics. In this work we also present a graphic tools for discovering rules to facilitate the usage of the proposed methodology to improve the courses.

Automatic reactor model synthesis with genetic programming

Successful modeling of wastewater treatment plant (WWTP) processes requires an accurate description of the plant hydraulics. Common methods such as tracer experiments are difficult and costly and thus have limited applicability in practice; engineers are often forced to rely on their experience only. An implementation of grammar-based genetic programming with an encoding to represent hydraulic reactor models as program trees should fill this gap: The encoding enables the algorithm to construct arbitrary reactor models compatible with common software used for WWTP modeling by linking building blocks, such as continuous stirred-tank reactors. Discharge measurements and influent and effluent concentrations are the only required inputs. As shown in a synthetic example, the technique can be used to identify a set of reactor models that perform equally well. Instead of being guided by experience, the most suitable model can now be chosen by the engineer from the set. In a second example, temperature measurements at the influent and effluent of a primary clarifier are used to generate a reactor model. A virtual tracer experiment performed on the reactor model has good agreement with a tracer experiment performed on-site.

Grammar-based Genetic Programming: a survey

Grammar formalisms are one of the key representation structures in Computer Science. So it is not surprising that they have also become important as a method for formalizing constraints in Genetic Programming (GP). Practical grammar-based GP systems first appeared in the mid 1990s, and have subsequently become an important strand in GP research and applications. We trace their subsequent rise, surveying the various grammar-based formalisms that have been used in GP and discussing the contributions they have made to the progress of GP. We illustrate these contributions with a range of applications of grammar-based GP, showing how grammar formalisms contributed to the solutions of these problems. We briefly discuss the likely future development of grammar-based GP systems, and conclude with a brief summary of the field.