Novelty Search Research Articles

Deep reinforcement learning assisted novelty search in Voronoi regions for constrained multi-objective optimization

Solving constrained multi-objective optimization problems (CMOPs) requires optimizing multiple conflicting objectives while satisfying various constraints. Existing constrained multi-objective evolutionary algorithms (CMOEAs) cross infeasible regions by ignoring constraints. However, these methods might neglect promising search directions, leading to insufficient exploration of the search space. To address this issue, this paper proposes a deep reinforcement learning assisted constrained multi-objective quality-diversity algorithm. The proposed algorithm designs a diversity maintenance mechanism to promote evenly coverage of the final solution set on the constrained Pareto front. Specifically, first, a novelty-oriented archive is created using a centroid Voronoi tessellation, which divides the search space into a desired number of Voronoi regions. Each region acts as a repository of non-dominated solutions with different phenotypic characteristics to provide diversity information and supplementary evolutionary trails. Secondly, to improve resource utilization, a deep Q-network is adopted to learn a policy to select suitable Voronoi regions for offspring generation based on their novelty scores. The exploration of these regions aims to find a set of diverse, high-performing solutions to accelerate convergence and escape local optima. Compared with eight state-of-the-art CMOEAs, experimental studies on four benchmark suites and nine real-world applications demonstrate that the proposed algorithm exhibits superior or at least competitive performance, especially on problems with discrete and narrow feasible regions.

Preliminary Analysis of Simple Novelty Search.

Novelty search is a powerful tool for finding diverse sets of objects in complicated spaces. Recent experiments on simplified versions of novelty search introduce the idea that novelty search happens at the level of the archive space, rather than individual points. The sparseness measure and archive update criterion create a process that is driven by a two measures: (1) spread out to cover the space while trying to remain as efficiently packed as possible, and (2) metrics inspired by k nearest neighbor theory. In this paper, we generalize previous simplifications of novelty search to include traditional population (μ,λ) dynamics for generating new search points, where the population and the archive are updated separately. We provide some theoretical guidance regarding balancing mutation and sparseness criteria and introduce the concept of saturation as a way of talking about fully covered spaces. We show empirically that claims that novelty search is inherently objectiveless are incorrect. We leverage the understanding of novelty search as an optimizer of archive coverage, suggest several ways to improve the search, and demonstrate one simple improvement-generating some new points directly from the archive rather than the parent population.

Synthesising Diverse and Discriminatory Sets of Instances using Novelty Search in Combinatorial Domains.

Gathering sufficient instance data to either train algorithm-selection models or understand algorithm footprints within an instance space can be challenging. We propose an approach to generating synthetic instances that are tailored to perform well with respect to a target algorithm belonging to a predefined portfolio but are also diverse with respect to their features. Our approach uses a novelty search algorithm with a linearly weighted fitness function that balances novelty and performance to generate a large set of diverse and discriminatory instances in a single run of the algorithm. We consider two definitions of novelty: (1) with respect to discriminatory performance within a portfolio of solvers; (2) with respect to the features of the evolved instances. We evaluate the proposed method with respect to its ability to generate diverse and discriminatory instances in two domains (knapsack and bin-packing), comparing to another well-known quality diversity method, Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) and an evolutionary algorithm that only evolves for discriminatory behaviour. The results demonstrate that the novelty search method outperforms its competitors in terms of coverage of the space and its ability to generate instances that are diverse regarding the relative size of the "performance gap" between the target solver and the remaining solvers in the portfolio. Moreover, for the Knapsack domain, we also show that we are able to generate novel instances in regions of an instance space not covered by existing benchmarks using a portfolio of state-of-the-art solvers. Finally, we demonstrate that the method is robust to different portfolios of solvers (stochastic approaches, deterministic heuristics and state-of-the-art methods), thereby providing further evidence of its generality.

Using Genetic Algorithms to Automate Scenario Generation and Enhance the Training Value of Serious Games for Adaptive Instruction

This paper discusses an end-to-end process to enhance the variability of serious game scenario conditions to support adaptive instructional strategies that select changes in sce-nario difficulty to match learning objectives to trainee ca-pabilities. This process can also be used to model or recog-nize human responses to serious game stimuli to support trainee feedback during after-action reviews (AARs). Adaptive instruction is a learning experience where inter-ventions (content selection and feedback) are tailored for each individual trainee or group of trainees. Adaptive train-ing has been shown to be as much as one to two standard deviations more effective than traditional classroom meth-ods. Our process starts with a few parent scenarios. The population of parent scenarios is automatically expanded using a Novelty search (genetic algorithm) methodology to optimize feature variability in the game-based scenarios. Specifically, this paper focuses on the process of creating new training scenarios of varying difficulty and conditions based on changes in scenario environmental factors (i.e., light, precipitation, wind, and cloud cover). The resulting new scenarios may then be run within a serious game with embedded intelligent agents to represent realistic entity behaviors under the scenario-specified conditions. The ability to morph conditions and scenario difficulty during an adaptive training event is a significant benefit of this process.

A frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning

The construction of microgrid has fully promoted the large-scale access of distributed power and the rapid development of electric vehicles (EVs). And microgrid system faces operational issues such as strong random disturbances from distributed power sources and loads, as well as unexpected events during operation. These issues can lead to unstable microgrid frequency, excessive discharge of EVs, and increased control costs. To solve the issues, a frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning (EDRL) is proposed in this paper. First, consider the effect of the vehicle-to-grid (V2G) process on the minimum time required to fully charge an EV, and the impact of the output distribution of micro turbines (MTs) on the regulation cost, and based on the coupling relationship between generator terminal voltage regulation and system frequency control, a multimicrogrid comprehensive control model is constructed. Second, in order to deal with engineering tasks with deceptive rewards and sparse rewards such as integrated frequency control, evolutionary algorithms and deep reinforcement learning algorithms are combined to effectively assist the training process to jump out of local optimal solutions and approach optimal control strategies. Meanwhile, the algorithm is improved based on the novelty search and intelligent partition strategy, so that the strategy has better convergence characteristics and can reduce the cost of information transmission and computational complexity under the premise of ensuring the control effect. Furthermore, the state space, action space and reward function of the controller are defined. Finally, the simulation results show that the proposed controller has the ability of coordinated control, can effectively reduce the adjustment cost of MT unit and the unnecessary discharging of EVs while ensuring the frequency regulation requirements of each submicrogrid, which is far superior to PID control, fuzzy control, and traditional deep reinforcement learning control.

Openness to Experience and Player Satisfaction in a Simulation Game

Background Openness to experience (OTE) is one of the Big Five traits that describe the personality of the individual. Player satisfaction (Satisfaction) is composed of the factors of excitement, challenge, learning experience, team victory and self-discovery in a simulation game. OTE and Satisfaction appear to have a symbiotic relationship that feeds on their characteristics and outcomes. This study is undertaken to understand the relationship between OTE and Satisfaction in a simulation game. Objectives of the Study The objectives are to develop a scale of OTE in the context of a simulation game, to identify the factors of OTE and to study the interactive effects of OTE and Satisfaction and their factors. Methods An instrument of 40 statements was administered to 190 post-graduate management students at the end of a brand-related simulation game. It had 12 statements that represented OTE and 28 statements of the Satisfaction scale. The data was purified and processed for factor analysis; the variables and their factors were subjected to correlation and regression. Results Two OTE factors, each of eigenvalue greater than one, were extracted and named search for novelty and passion to know. Discussion The conclusions of this study may be generalisable only to sample profiles that are most similar to the study sample, but not to other contexts due to the ambiguous effect of personality, contexts and cultures on the study variables. The strong positive correlations between OTE, Satisfaction and their factors show their bases in a common platform, i.e. the experience of the simulation game. The search for novelty predicts Satisfaction more powerfully than OTE predicts Satisfaction. Self-discovery, learning experience and excitement affect OTE positively and more powerfully than challenge, team victory or Satisfaction. Further studies of passion, intrinsic motivation and self-efficacy may enhance our understanding of their impact on Satisfaction.

Laikmeta komentārs teātra valodā. Latviešu dramaturģija Rīgas Strādnieku teātra repertuārā: modernisma aspekti

The repertoire of the Riga Workers’ Theatre (1926–1934) reflected the trends of the time and leftist progressive ideas. City life as a nodal point of modern reality became inspiration and material for plays, poetry performances, dramatised feuilletons and interludes commenting on current events. The aesthetic programme of the theatre envisaged the creation of a peculiar sovereign artistic style, where the new dramaturgy played an important role. Topical and recently created works by contemporary authors were chosen for the repertoire with an emphasis on the search for novelty. In the first years, while the director was Roberts Dukurs, the dramaturge was Jānis Grots, then Arvīds Grigulis; after this function was taken over by the director Kārlis Dziļleja. The article aims to introduce the main principles of the Riga Workers’ Theatre repertoire policy, focusing on an overview of the contributions of Latvian authors. Rainis, Andrejs Upīts, and Pāvils Gruzna played a dominant role in the ideological decisions of the repertoire. A circle of progressive authors gathered around the theatre, creating a contemporary drama laboratory. Young authors Aleksandrs Čaks, Valdis Grēviņš, Fricis Rokpelnis, Jūlijs Vanags, and others experienced their first works on stage in this theatre. The research object is dramaturgical works and testimonies describing the productions – programme leaflets, reviews, reminiscences, etc. The article discusses innovative trends in drama and the expansion of the concept of dramaturgy to the dynamics of stage set, musical and choreographic action. As a result, the consistently executed repertory policy heralded a new type of drama programme by achieving a synthesis of contemporary content and new visual, musical and choreographic forms. The article concludes that creating innovative drama forms and genre experiments on the Riga Workers’ Theatre stage significantly contributed to the development of Latvian dramaturgy.

Using temperament and character dimensions (TCI) to analyze the personality profiles of adults and older adults with cancer managed in outpatient settings.

This study aimed to investigate profiles of personality evaluated by temperament and character dimensions (TCI) in 638 adult and older adult patients (CP) who had recently been diagnosed with breast, colon, lung, and other kinds of cancer (female and male subjects were assessed). Tests: Temperament and Character Inventory (TCI). Statistical analysis: cluster K-means analysis for personality traits. Two different personality profiles emerged: "Low self-determination and pessimism" (Profile 1) and "Self-determination and self-caring (medium)" (Profile 2). The following significant differences were observed in the TCI dimensions between the two profiles: Temperament-Novelty-Seeking (NS) (p < 0.001); Harm-Avoidance (HA) (p < 0.001); Reward-Dependence (RD) (p < 0.001); Persistence (PS) (p < 0.001); Character-Self-Directness (SD) (p < 0.001); Cooperativeness (C) (p > 0.001); Self-Transcendence (ST) (p < 0.001). No differences in the two profiles were found between adult and elderly patients. Profile 1 - "Low self-determination and pessimism": Patients with this profile present low resistance to frustration, poor search for novelty and solutions (NS), anxiety and pessimism (medium HA), high social attachment and dependence on the approval of others (medium-high RD), and low self-determination (PS) as temperament dimensions; and medium-low self-direction, low autonomy and ability to adapt (SD-medium-low), medium cooperativeness (C), and low self-transcendence (ST) as character dimensions. Profile 2 - "Self-determination and self-caring (medium)": Patients with this profile have resistance to frustration, ability to search for novelty and solutions (medium-NS), low anxiety and pessimism (HA), low social attachment and dependence on approval (medium-low-RD), and determination (medium-high PS) as dimensions of temperament; and autonomy and capacity for adaptation and self-direction (SD), capacity for cooperation (high-CO), and self-transcendence (medium-high-ST) as character dimensions. Personality screening allows a better understanding of the difficulties of the individual patient and the planning of targeted psychotherapeutic interventions that promote quality of life and good adaptation to the disease course.

A Q-learning Novelty Search Strategy for Evaluating Robustness of Deep Reinforcement Learning in Open-world Environments

A Q-learning Novelty Search Strategy for Evaluating Robustness of Deep Reinforcement Learning in Open-world Environments

Open-Ended Evolution for Minecraft Building Generation

This paper proposes a procedural content generator which evolves Minecraft buildings according to an open-ended and intrinsic definition of novelty. To realize this goal we evaluate individuals' novelty in the latent space using a 3D autoencoder, and alternate between phases of exploration and transformation. During exploration the system evolves multiple populations of CPPNs through CPPN-NEAT and constrained novelty search in the latent space (defined by the current autoencoder). We apply a set of repair and constraint functions to ensure candidates adhere to basic structural rules and constraints during evolution. During transformation, we reshape the boundaries of the latent space to identify new interesting areas of the solution space by retraining the autoencoder with novel content. In this study we evaluate five different approaches for training the autoencoder during transformation and its impact on populations' quality and diversity during evolution. Our results show that by retraining the autoencoder we can achieve better open-ended complexity compared to a static model, which is further improved when retraining using larger datasets of individuals with diverse complexities.

GENDER-SPECIFIC LIFE SCENARIOS IN INDIVIDUALS WITH DIFFERENT LEVELS OF TOLERANCE TO UNCERTAINTY

Purpose. Currently, the scenario line of research in personality psychology seems to be particularly promising, since this line of research allows us to identify the specific features of the transformation of the conscious construction of human life paths under changing conditions of uncertainty. The aim of the study is to determine the gender specificity of life scenarios in individuals with different levels of tolerance to uncertainty.&#x0D; Materials and Methods. The sampling consisted of 200 people with different levels of tolerance towards uncertainty. The methods of the study included the semantic differential Lifestyle by V.P. Serkin and the New Questionnaire of Tolerance Towards Uncertainty by T.V. Kornilova. The data obtained were processed by means of factor analysis with the help of the program IBM SPSS Statistics 23.&#x0D; Results. The results showed that the life scenarios of people with different levels of tolerance to uncertainty are gender-specific: men with a high tolerance to uncertainty show predominant characteristics of the scenario associated with an active life position and the search for novelty, whereas women show a scenario aimed at activity in the social sphere (the desire to actively participate in the fate of others, to help and support). Men with a low tolerance towards uncertainty show such scenario patterns as striving for security, stability and predictability of events, while women – in addition to striving for a convenient, comfortable life – show tactical patterns: emotional restraint and calculation.&#x0D; Field of results application. The results of the study can be used in socio-psychological training and counselling to identify and correct an individual’s life scenario.

Genetic Improvement @ ICSE 2023

Following the formal presentations, which included keynotes by Prof. Myra B. Cohen of Iowa State University and Dr. Sebastian Baltes of SAP as well as six papers (which are recorded in the pro- ceedings) there was a wide ranging discussion at the twelfth inter- national Genetic Improvement workshop, GI-2023 @ ICSE held on Saturday 20th May 2023 in Melbourne and online via Zoom. Topics included GI to improve testing, and remove unpleasant surprises in cloud computing costs, incorporating novelty search, large language models (LLM ANN) and GI benchmarks.

Discovering Rules for Rule-Based Machine Learning with the Help of Novelty Search

Automated prediction systems based on machine learning (ML) are employed in practical applications with increasing frequency and stakeholders demand explanations of their decisions. ML algorithms that learn accurate sets of rules, such as learning classifier systems (LCSs), produce transparent and human-readable models by design. However, whether such models can be effectively used, both for predictions and analyses, strongly relies on the optimal placement and selection of rules (in ML this task is known as model selection). In this article, we broaden a previous analysis on a variety of techniques to efficiently place good rules within the search space based on their local prediction errors as well as their generality. This investigation is done within a specific pre-existing LCS, named SupRB, where the placement of rules and the selection of good subsets of rules are strictly separated—in contrast to other LCSs where these tasks sometimes blend. We compare two baselines, random search and (1, lambda )-evolution strategy (ES), with six novelty search variants: three novelty-/fitness weighing variants and for each of those two differing approaches on the usage of the archiving mechanism. We find that random search is not sufficient and sensible criteria, i.e., error and generality, are indeed needed. However, we cannot confirm that the more complicated-to-explain novelty search variants would provide better results than (1, lambda )-ES which allows a good balance between low error and low complexity in the resulting models.

Premature convergence in morphology and control co-evolution: a study

This article addresses the co-evolution of morphology and control in evolutionary robotics, focusing on the challenge of premature convergence and limited morphological diversity. We conduct a comparative analysis of state-of-the-art algorithms, focusing on QD (Quality-Diversity) algorithms, based on a well-defined methodology for benchmarking evolutionary algorithms. We introduce carefully chosen indicators to evaluate their performance in three core aspects: task performance, phenotype diversity, and genotype diversity. Our findings highlight MNSLC (Multi-BC NSLC), with the introduction of aligned novelty to NSLC (Novelty Search with Local Competition), as the most effective algorithm for diversity preservation (genotype and phenotype diversity), while maintaining a competitive level of exploitability (task performance). MAP-Elites, although exhibiting a well-balanced trade-off between exploitation and exploration, fall short in protecting morphological diversity. NSLC, while showing similar performance to MNSLC in terms of exploration, is the least performant in terms of exploitation, contrasting with QN (Fitness-Novelty MOEA), which exhibits much superior exploitation, but inferior exploration, highlighting the effects of local competition in skewing the balance toward exploration. Our study provides valuable insights into the advantages, disadvantages, and trade-offs of different algorithms in co-evolving morphology and control.

Пошук новизни у методі нейроеволюції для позиціонування роботизованої кінцівки

The article considers the use of the neuroevolution algorithm for neural network policies search when creating a controller for a robotic arm, in particular for the subtask of positioning the end effector. Neuro-evolution is a family of machine learning methods that use evolutionary algorithms by imitating the process of natural selection. This approach has been found to be particularly effective for the positioning task, where the final position can be achieved in many optimal ways and therefore requires reinforcement learning. It is noted that the final result of neuroevolution is an optimal network topology, which makes the model more resource-efficient and easier to analyze. The paper considers the process of neural network policy search for controlling a two-dimensional robot with two links. According to the results of the experiments, an increase in the efficiency of the best solution found using novelty search for the NEAT algorithm is noted compared to the NEAT algorithm without novelty search. It was established that the proposed approach allows to obtain an effective neural network policy, which has a minimal configuration, which will allow to increase the speed of the controller, that is critical for the operation of a real system. Thus, the use of novelty search as a method of optimizing the neuroevolutionary process to solve the positioning problem allows to increase the efficiency of the learning process and obtain the optimal network topology.

Automatic modular design of robot swarms based on repertoires of behaviors generated via novelty search

Several methods have already been proposed to automatically design control software for robot swarms by assembling predefined modules. Yet, so far, the modules on which these methods operate have always been defined manually in a process that is time consuming, requires domain knowledge, and must be performed by an expert. Motivated by the goal of automatizing the definition of these modules, we propose an approach in which repertoires of modules, in the form of neural networks, are automatically generated via a quality-diversity evolutionary algorithm. To illustrate the proposal, we introduce Nata, a novel approach belonging to the AutoMoDe family. Nata automatically generates probabilistic finite-state machines in which states are selected from a repertoire of neural networks, and transition conditions are selected from a set of rules based on the sensory capabilities of the robotic platform considered. Both the repertoire of neural networks and the set of transition rules are automatically generated a priori, once and for all, in a mission-agnostic way. We study Nata on three missions, both in simulation and with real robots. Nata is the first modular automatic design method that assembles modules that were themselves generated automatically.

The Evolution of Money Culture and the Transformation of Narrative Culture in Pre-modern Chinese Fiction

Abstract The money culture of a given time and place can have a profound influence on literature, a medium that allows for the artistic expression of aesthetic taste. This article takes money culture as its analytical lens to explore the cultural transformation of literary concepts that were prevalent in Chinese novels. It will delve into the transition from agricultural production and lifestyles, which were focused on productive and consumptive survival, to industrialized production and lifestyles, which were centered on monetary survival. Additionally, it will trace the evolution from farming literature, which is rooted in Confucian morality, to commercial literature, which is characterized by themes of wealth, lust, and the search for novelty.

Desperately searching for something

There is a growing interest in novelty search : that is, in sampling a parameter space to search for radical or unexpected behaviour(s), occurring as a consequence of parameter choice, being input to some downstream complex system, process, or service that will not yield to analysis, without imposing any specific pre-ordained objective function, or fitness function to be optimised. We mean “parameter” in the widest sense, including system learnables, non-autonomous forcing, sequencing and all inputs.Depending upon the nature of the underlying parameter space of interest one may adopt a rather wide range of search algorithms. We do consider that this search activity has meta-objectives, though: one is of achieving diversity (efficiently reaching out across the space in some way); and one is of achieving some minimum density (not leaving out large unexplored holes). These are in tension. In general, the computational costs of both of these qualities become restrictive as the dimension of the parameter spaces increase; and consequently their balance is harder to maintain. We may also wish for a substantial random element of search to provide some luck in discovery and to avoid any naive preset sampling patterns.We consider archive-based methods within a range of spaces: finite discrete spaces, where the problem is straightforward (provided we are patient with the random element); Euclidean spaces, of increasing dimension, that become very lonely places; and infinite dimensional spaces. Our aim is to discuss a raft of distinctive search concepts, that respond to identified challenges, and rely on a rather diverse range of mathematical ideas. This arms practitioners with a range of highly practical methods.However applications requiring novelty search arise, one should avoid rushing to code-up a standard evolving search algorithm and instead give some thought to the nature and requirements of the search: there is a range of effective options available. We give some considered advice.

A particle swarm optimization algorithm with novelty search for combustion systems with ultra-low emissions and minimum fuel consumption

The particle swarm optimization algorithm is primarily inspired by the natural behaviour of swarms and achieves important results in different applications. However, it is not exempt from stagnation in local optima and has a tendency to prematurely converge to them. Novelty Search is a concept that appeared recently in different fields of computational intelligence. It aims at exploring non-visited areas of the search space through solutions that bring novelty to already discovered solutions. The novelty of this work can be divided into two steps: on one side, this article proposes a variant of the particle swarm optimization algorithm which uses Novelty Search concepts to improve the algorithm’s performance. Our proposal is first checked and compared using the CEC 2005 benchmark suite and then, we apply it to solve a real-world optimization problem: the design of a combustion system targeting the reduction of pollutant emissions and fuel consumption. The combustion chamber design phase usually is a complex and time-consuming process even with advanced supercomputers, since it depends on several input variables which are highly non-linear and with crossed interaction. Then, the second contribution of this work is to develop a methodology that couples a computational fluid dynamics (CFD) simulation tool with the new optimization algorithm for minimizing the specific fuel consumption of a compression-ignited engine, while constraining the NOx and soot emissions. A 3D-CFD model of the combustion system was built to predict and analyse the performance of the combustion system and hence, select the parameters with a higher impact on the system. The method reduces the computational time and includes tools for the automatic preparation of the input parameters and geometry of the system. The input parameters correspond to geometrical variables that control the bowl shape, the number of holes in the injector, the injection pressure, the swirl number and the exhaust gas recirculation rate. Results show how the simulation tool and the new PSO with Novelty Search algorithm allow us to obtain a new combustion system that minimizes the fuel consumption by 3%, simultaneously reducing NOx and soot emissions.

Value orientations of Italian adolescents: empirical evidence

ABSTRACT Little is known about the value orientations of adolescents in Italy. This paper presents a sociological analysis of Italian adolescents’ values. The study draws on a survey of 1,994 students from 44 Italian schools and applies Schwartz’s theory of basic human values. The analysis revealed the centrality of independence and the search for novelty (openness to change) among the adolescents in the sample and the low priority given to conservation. Gender appears to be crucial in understanding adolescents’ values. Significant differences emerged between male and female adolescents: female adolescents attributed more importance to hedonism, self-direction and stimulation than their male peers and less importance to self-achievement and power, in line with previous studies. Overall, the study found that Italian adolescents hold fragmented or even conflicting values, and hedonism and tradition emerged as their two most important values. Based on these findings, the paper discusses implications and provides suggestions for future research.