Human-competitive Results Research Articles

Guest Editorial Special Issue on Evolutionary Computer Vision

Evolutionary Computer Vision (ECV) is at the intersection of two major research fields of artificial intelligence: 1) computer vision (CV) and 2) evolutionary computation (EC). This special issue brings an overview of state-of-the-art contributions to the latest research and development in the discipline. CV includes methods for acquiring, processing, analyzing, and understanding images. The aim is to design computational models of human and animal perception. ECV is an interdisciplinary research area where analytical methods combined with powerful stochastic optimization and metaheuristic approaches produced human-competitive results. From an engineering standpoint, ECV aims to design software and hardware solutions useful for solving challenging CV problems. From a scientific viewpoint, the goal is to enhance our current understanding of visual processing in nature and replicate this within a seeing machine. ECV is a well-established research discipline as evolutionary algorithms are more efficient than classical optimization approaches for the discontinuous, nondifferentiable, multimodal, and noisy search, optimization, and learning problems arising in many CV tasks. EC has also demonstrated its ability as a robust approach to cope with the fundamental steps of image processing, image analysis, and image understanding included in the CV pipeline (e.g., restoration, segmentation, registration, classification, reconstruction, or tracking).

2021 Humies winners awarded at GECCO!

The GECCO Conference, held virtually this year in Lille, France, July 10-14, was the home for the 18th Annual Human-Competitive Results Awards. The Humies competition annually awards $10,000 in cash prizes for computational results that are deemed to be competitive with results produced by human beings, but are generated automatically by computer. This year, there were 20 entries submitted, and the judges selected six of those to be finalists, based on the papers and entry forms they submitted. Because GECCO was virtual, the six Humies finalists did not present live talks, but instead submitted 10-minute videos that were shown during the Humies session and viewed by the judges in making their selections of the prize winners. All 20 entries and the six finalists' videos are all available to the general public at the website for the Humies, www.human-competitive.org, so anyone can watch them there.

Multi-Objective Software Effort Estimation: A Replication Study

Replication studies increase our confidence in previous results when the findings are similar each time, and help mature our knowledge by addressing both internal and external validity aspects. However, these studies are still rare in certain software engineering fields. In this paper, we replicate and extend a previous study, which denotes the current state-of-the-art for multi-objective software effort estimation, namely CoGEE. We investigate the original research questions with an independent implementation and the inclusion of a more robust baseline (LP4EE), carried out by the first author, who was not involved in the original study. Through this replication, we strengthen both the internal and external validity of the original study. We also answer two new research questions investigating the effectiveness of CoGEE by using four additional evolutionary algorithms (i.e., IBEA, MOCell, NSGA-III, SPEA2) and a well-known Java framework for evolutionary computation, namely JMetal (rather than the previously used R software), which allows us to strengthen the external validity of the original study. The results of our replication confirm that: (1) CoGEE outperforms both baseline and state-of-the-art benchmarks statistically significantly ( <inline-formula><tex-math notation="LaTeX">$p &lt;0.001$</tex-math></inline-formula> ); (2) CoGEE’s multi-objective nature makes it able to reach such a good performance; (3) CoGEE’s estimation errors lie within claimed industrial human-expert-based thresholds. Moreover, our new results show that the effectiveness of CoGEE is generally not limited to nor dependent on the choice of the multi-objective algorithm. Using CoGEE with either NSGA-II, NSGA-III, or MOCell produces human competitive results in less than a minute. The Java version of CoGEE has decreased the running time by over 99.8 percent with respect to its R counterpart. We have made publicly available the Java code of CoGEE to ease its adoption, as well as, the data used in this study in order to allow for future replication and extension of our work.

Human-competitive results awards - "Humies" 2019 - announces winners at GECCO

The Sixteenth Annual Humies Competition, which awards $10,000 in cash prizes for computational results deemed to be competitive with results produced by human beings, but are generated automatically by computer, was held as part of the GECCO-2019 program, the Genetic and Evolutionary Computation Conference, held July 13--17 in Prague, Czech Republic. The awards, sponsored by John Koza (who is widely acknowledged as the "Father of Genetic Programming") annually solicit newly published papers that describe work fulfilling one or more of eight criteria, including such features as winning a regulated competition against humans or other programs, producing results that are publishable in their own right, not because they were created by a computer program, patentability, and several others, described fully on the competition's website, www.human-competitive.org.

Hybrid Genetic Programming for the Development of Metamaterials Designs With Improved Characteristics

The expansion of a hybrid genetic program's (HGP) functionality to allow for the development of broadband two-dimensional metamaterials designs with advanced characteristics is presented. Artificial magnetic conductor (AMC) ground planes and a tunable terahertz absorber generated by the HGP are compared with designs available in the literature. The HGP is shown to produce human-competitive results. The AMC ground planes synthesized by the HGP were found to produce improved results including wider bandwidths and larger reflection coefficient magnitudes than that of the human designs. For one of the designed AMCs, the HGP's bandwidth is 73.3% larger and the minimum reflection magnitude is 1.0% larger than the reference AMC. Similarly, the absorber synthesized by the HGP has larger bandwidths than that of a recently published absorber optimized by random hill climbing. Three bias voltages were tested with the tunable absorber. The bandwidths of the HGP absorber are 23.1%, 37.6%, and 400% larger than the reference absorber, for biases of 4, 2, and 0.5 V/nm, respectively. Four example designs are discussed together with comparative results to illustrate the advantages of the developed HGP-enabled design method.

Human Competitiveness of Genetic Programming in Spectrum-Based Fault Localisation

We report on the application of Genetic Programming to Software Fault Localisation, a problem in the area of Search-Based Software Engineering (SBSE). We give both empirical and theoretical evidence for the human competitiveness of the evolved fault localisation formulæ under the single fault scenario, compared to those generated by human ingenuity and reported in many papers, published over more than a decade. Though there have been previous human competitive results claimed for SBSE problems, this is the first time that evolved solutions have been formally proved to be human competitive. We further prove that no future human investigation could outperform the evolved solutions. We complement these proofs with an empirical analysis of both human and evolved solutions, which indicates that the evolved solutions are not only theoretically human competitive, but also convey similar practical benefits to human-evolved counterparts.

A genetic programming hyper-heuristic for the multidimensional knapsack problem

Purpose– Hyper-heuristics are a class of high-level search techniques which operate on a search space of heuristics rather than directly on a search space of solutions. The purpose of this paper is to investigate the suitability of using genetic programming as a hyper-heuristic methodology to generate constructive heuristics to solve the multidimensional 0-1 knapsack problemDesign/methodology/approach– Early hyper-heuristics focused on selecting and applying a low-level heuristic at each stage of a search. Recent trends in hyper-heuristic research have led to a number of approaches being developed to automatically generate new heuristics from a set of heuristic components. A population of heuristics to rank knapsack items are trained on a subset of test problems and then applied to unseen instances.Findings– The results over a set of standard benchmarks show that genetic programming can be used to generate constructive heuristics which yield human-competitive results.Originality/value– In this work the authors show that genetic programming is suitable as a method to generate reusable constructive heuristics for the multidimensional 0-1 knapsack problem. This is classified as a hyper-heuristic approach as it operates on a search space of heuristics rather than a search space of solutions. To our knowledge, this is the first time in the literature a GP hyper-heuristic has been used to solve the multidimensional 0-1 knapsack problem. The results suggest that using GP to evolve ranking mechanisms merits further future research effort.

Evolutionary game design

The "Humies" awards are an annual competition held in conjunction with the Genetic and Evolutionary Computation Conference (GECCO), in which cash prizes totalling $10,000 are awarded to the most human-competitive results produced by any form of evolutionary computation published in the previous year. This article describes the gold medal-winning entry from the 2012 "Humies" competition, based on the LUDI system for playing, evaluating and creating new board games. LUDI was able to demonstrate human-competitive results in evolving novel board games that have gone on to be commercially published, one of which, Yavalath, has been ranked in the top 2.5% of abstract board games ever invented. Further evidence of human-competitiveness was demonstrated in the evolved games implicitly capturing several principles of good game design, outperforming human designers in at least one case, and going on to inspire a new sub-genre of games.

Foreword to the invited impact paper on automatic software repair

The authors of this paper, Claire Le Goues, Stephanie Forrest and Westley Weimer, have been at the forefront of developments in automated software patching (also known as automated software repair). Their approach uses genetic programming, a search-based algorithm, in order to search for patches which fix software systems. The invited impact paper presented in this issue of the journal will serve as a useful introduction to the area for those new to automated software repair and a roadmap for future work for those actively engaged in the topic. Impact papers are invited submissions to the journal, and they are reviewed speedily. An invited paper covers work that has already demonstrated an impact on the development of research, and so the review process starts from the presumption that the paper will ultimately be accepted. However, papers are reviewed by several expert reviewers (as normal), and authors are requested to respond to these reviewers’ comments (as has happened in the case of this paper). The authors’ work on automated repair is well known and highly cited and has undoubtedly already had an impact on the direction of research work, both for work on search-based software engineering (SBSE) and also on work on automated software repair. It has received a distinguished paper award at the International Conference on Software Engineering (ICSE) and also the gold medal for human competitive results at the genetic and evolutionary computation conference (GECCO). It was therefore felt timely and appropriate that the authors be invited to submit an impact paper to the Journal. This invited impact paper provides a survey of existing results in automated software repair, together with directions for future research in this area of SBSE. The results that have been achieved hitherto are, indeed, impressive, and we can look forward to a very bright future for this exciting and important area of research. I am confident that the reader will gain a great deal from this paper, whether or not they start out from a position of interest in either SBSE or automated software repair and so I commend the paper to the reader.

Automating the Packing Heuristic Design Process with Genetic Programming

The literature shows that one-, two-, and three-dimensional bin packing and knapsack packing are difficult problems in operational research. Many techniques, including exact, heuristic, and metaheuristic approaches, have been investigated to solve these problems and it is often not clear which method to use when presented with a new instance. This paper presents an approach which is motivated by the goal of building computer systems which can design heuristic methods. The overall aim is to explore the possibilities for automating the heuristic design process. We present a genetic programming system to automatically generate a good quality heuristic for each instance. It is not necessary to change the methodology depending on the problem type (one-, two-, or three-dimensional knapsack and bin packing problems), and it therefore has a level of generality unmatched by other systems in the literature. We carry out an extensive suite of experiments and compare with the best human designed heuristics in the literature. Note that our heuristic design methodology uses the same parameters for all the experiments. The contribution of this paper is to present a more general packing methodology than those currently available, and to show that, by using this methodology, it is possible for a computer system to design heuristics which are competitive with the human designed heuristics from the literature. This represents the first packing algorithm in the literature able to claim human competitive results in such a wide variety of packing domains.

Trustworthy Genetic Programming-Based Synthesis of Analog Circuit Topologies Using Hierarchical Domain-Specific Building Blocks

This paper presents MOJITO, a system that performs structural synthesis of analog circuits, returning designs that are trustworthy by construction. The search space is defined by a set of expert-specified, trusted, hierarchically-organized analog building blocks, which are organized as a parameterized context-free grammar. The search algorithm is a multiobjective evolutionary algorithm that uses an age-layered population structure to balance exploration versus exploitation. It is validated with experiments to search across >;100 000 different one-stage and two-stage opamp topologies, returning human-competitive results. The runtime is orders of magnitude faster than open-ended systems, and unlike the other evolutionary algorithm approaches, the resulting circuits are trustworthy by construction. The approach generalizes to other problem domains which have accumulated structural domain knowledge, such as robotic structures, car assemblies, and modeling biological systems.

Human-competitive results produced by genetic programming

Genetic programming has now been used to produce at least 76 instances of results that are competitive with human-produced results. These human-competitive results come from a wide variety of fields, including quantum computing circuits, analog electrical circuits, antennas, mechanical systems, controllers, game playing, finite algebras, photonic systems, image recognition, optical lens systems, mathematical algorithms, cellular automata rules, bioinformatics, sorting networks, robotics, assembly code generation, software repair, scheduling, communication protocols, symbolic regression, reverse engineering, and empirical model discovery. This paper observes that, despite considerable variation in the techniques employed by the various researchers and research groups that produced these human-competitive results, many of the results share several common features. Many of the results were achieved by using a developmental process and by using native representations regularly used by engineers in the fields involved. The best individual in the initial generation of the run of genetic programming often contains only a small number of operative parts. Most of the results that duplicated the functionality of previously issued patents were novel solutions, not infringing solutions. In addition, the production of human-competitive results, as well as the increased intricacy of the results, are broadly correlated to increased availability of computing power tracked by Moore’s law. The paper ends by predicting that the increased availability of computing power (through both parallel computing and Moore’s law) should result in the production, in the future, of an increasing flow of human-competitive results, as well as more intricate and impressive results.

Automatedab initiosynthesis of complete designs of four patented optical lens systems by means of genetic programming

AbstractThis paper describes how genetic programming has been used as an invention machine to automatically synthesize complete designs for four optical lens systems that duplicated the functionality of previously patented lens systems. The automatic synthesis of the complete design is doneab initio, that is, without starting from a preexisting good design and without prespecifying the number of lenses, the topological arrangement of the lenses, or the numerical or nonnumerical parameters associated with any lens. One of the genetically evolved lens systems infringed a previously issued patent, whereas the others were noninfringing novel designs that duplicated (or improved upon) the performance specifications contained in the patents. One of the patents was issued in the 21st century. The designs were created in a substantially similar and routine way, suggesting that the approach described in the paper can be readily applied to other similar problems in the field of optical design. The genetically evolved designs are instances of human-competitive results produced by genetic programming in the field of optical design.

Invention and creativity in automated design by means of genetic programming

Some designs are sufficiently creative that they are considered to be inventions. The invention process is typically characterized by a singular moment when the prevailing thinking concerning a long-standing problem is, in a “flash of genius,” overthrown and replaced by a new approach that could not have been logically deduced from what was previously known. This paper discusses such logical discontinuities using an example based on the history of one of the most important inventions of the 20th century in electrical engineering, namely, the invention of negative feedback by AT&amp;T's Harold S. Black. This 1927 invention overthrew the then prevailing idiom of positive feedback championed by Westinghouse's Edwin Howard Armstrong. The paper then shows how this historically important discovery can be readily replicated by an automated design and invention technique patterned after the evolutionary process in nature, namely, genetic programming. Genetic programming employs Darwinian natural selection along with analogs of recombination (crossover), mutation, gene duplication, gene deletion, and mechanisms of developmental biology to breed an ever improving population of structures. Genetic programming rediscovers negative feedback by conducting an evolutionary search for a structure that satisfies Black's stated high-level goal (i.e., reduction of distortion in amplifiers). Like evolution in nature, genetic programming conducts its search probabilistically without resort to logic using a process that is replete with logical discontinuities. The paper then shows that genetic programming can routinely produce many additional inventive and creative results. In this regard, the paper discusses the automated rediscovery of numerous 20th-century patented inventions involving analog electrical circuits and controllers, the Sallen–Key filter, and six 21st-century patented inventions. In addition, two patentable new inventions (controllers) have been created in the same automated way by means of genetic programming. The paper discusses the promising future of automated invention by means of genetic programming in light of the fact that, to date, increased computer power has yielded progressively more substantial results, including numerous human-competitive results, in synchrony with Moore's law. The paper argues that evolutionary search by means of genetic programming is a promising approach for achieving creative, human-competitive, automated design because illogic and creativity are inherent in the evolutionary process.

Routine automated synthesis of five patented analog circuits using genetic programming

This article reports on a project in which we browsed patents issued after January 1, 2000 to commercial enterprises or university research institutions for analog electrical circuits. We then employed genetic programming to automatically design (synthesize) entities that duplicated the functionality of five post-2000 issued patents. The automated method works from a high-level statement of the circuit’s intended function. The article addresses the question of what is actually delivered by the operation of the artificial problem-solving method in relation to the amount of intelligence that is supplied by the humans employing the method (something we refer to as the yield of an automated method). The article also addresses the question of the routineness of the artificial problem-solving method – that is, the amount of effort required to make the transition from problem to problem within a particular domain. The conclusion is that the artificial method routinely delivers high-yield, human-competitive (i.e., previously patented) results.

What's AI done for me lately? Genetic programming's human-competitive results

The automated problem-solving technique of genetic programming has generated at least 36 human-competitive results. In six cases, it automatically duplicated the functionality of inventions patented after January 2000.

Automatic Creation of Human-Competitive Programs and Controllers by Means of Genetic Programming

Genetic programming is an automatic method for creating a computer program or other complex structure to solve a problem. This paper first reviews various instances where genetic programming has previously produced human-competitive results. It then presents new human-competitive results involving the automatic synthesis of the design of both the parameter values (i.e., tuning) and the topology of controllers for two illustrative problems. Both genetically evolved controllers are better than controllers designed and published by experts in the field of control using the criteria established by the experts. One of these two controllers infringes on a previously issued patent. Other evolved controllers duplicate the functionality of other previously patented controllers. The results in this paper, in conjunction with previous results, reinforce the prediction that genetic programming is on the threshold of routinely producing human-competitive results and that genetic programming can potentially be used as an “invention machine” to produce patentable new inventions.