Evolutionary Search Method Research Articles

Multi-objective evolutionary neural architecture search for network intrusion detection

Network Intrusion Detection (NID) becomes significantly important for protecting the security of information systems, as the frequency and complexity of network attacks are increasing with the rapid development of the Internet. Recent research studies have proposed various neural network models for NID, but they need to manually design the network architectures based on expert knowledge, which is very time-consuming. To solve this problem, this paper proposes a Multi-objective Evolutionary Neural Architecture Search (MENAS) method, which can automatically design neural network models for NID. First, a comprehensive search space is designed and then a weight-sharing mechanism is used to construct a supernet for NID, allowing each subnet to inherit weights from the supernet for direct performance evaluation. Subsequently, the subnets are encoded as chromosomes for multi-objective evolutionary search, which simultaneously optimizes two objectives: enhancing the model’s detection performance and reducing its complexity. To improve the search capability, a path-based crossover method is designed, which can iteratively refine the subnets’ architectures by simultaneously optimizing their accuracy and complexity for NID. At last, our MENAS method has been validated through extensive experiments on three well-known NID datasets: NSL-KDD, UNSW-NB15, and CICIDS2017. The experiments show that our MENAS method obtains an average 1.45% improvement on accuracy and an average 68.70% reduction on floating-point operations through multi-objective optimization process on six scenarios, which outperforms some state-of-the-art NID methods.

Enhancing Service Quality of On-Demand Transportation Systems Using a Hybrid Approach with Customized Heuristics

As customers’ expectations continue to rise, advanced on-demand transport services face the challenge of meeting new requirements. This study addresses a specific transportation issue belonging to dial-a-ride problems, including constraints aimed at fulfilling customer needs. In order to provide more efficient on-demand transportation solutions, we propose a new hybrid evolutionary computation method. This method combines customized heuristics including two exchanged mutation operators, a crossover, and a tabu search. These optimization techniques have been empirically proven to support advanced designs and reduce operational costs, while significantly enhancing service quality. A comparative analysis with an evolutionary local search method from the literature has demonstrated the effectiveness of our approach across small-to-large-scale problems. The main results show that service providers can optimize their scheduling operations, reduce travel costs, and ensure a high level of service quality from the customer’s perspective.

Multi-objective Evolutionary Neural Architecture Search for Recurrent Neural Networks

Artificial neural network (NN) architecture design is a nontrivial and time-consuming task that often requires a high level of human expertise. Neural architecture search (NAS) serves to automate the design of NN architectures and has proven to be successful in automatically finding NN architectures that outperform those manually designed by human experts. NN architecture performance can be quantified based on multiple objectives, which include model accuracy and some NN architecture complexity objectives, among others. The majority of modern NAS methods that consider multiple objectives for NN architecture performance evaluation are concerned with automated feed forward NN architecture design, which leaves multi-objective automated recurrent neural network (RNN) architecture design unexplored. RNNs are important for modeling sequential datasets, and prominent within the natural language processing domain. It is often the case in real world implementations of machine learning and NNs that a reasonable trade-off is accepted for marginally reduced model accuracy in favour of lower computational resources demanded by the model. This paper proposes a multi-objective evolutionary algorithm-based RNN architecture search method. The proposed method relies on approximate network morphisms for RNN architecture complexity optimisation during evolution. The results show that the proposed method is capable of finding novel RNN architectures with comparable performance to state-of-the-art manually designed RNN architectures, but with reduced computational demand.

E2SCNet: Efficient Multiobjective Evolutionary Automatic Search for Remote Sensing Image Scene Classification Network Architecture.

Remote sensing image scene classification methods based on deep learning have been widely studied and discussed. However, most of the network architectures are directly reliant on natural image processing methods and are fixed. A few studies have focused on automatic search mechanisms, but they cannot weigh the interpretation accuracy and the parameter quantity for practical application. As a result, automatic global search methods based on multiobjective evolutionary computation have more advantages. However, in the ranking process, the network individuals with large parameter quantities are easy to eliminate, but a higher accuracy may be obtained after full training. In addition, evolutionary neural architecture search methods often take several days. In this article, in order to solve the above concerns, we propose an efficient multiobjective evolutionary automatic search framework for remote sensing image scene classification deep learning network architectures (E2SCNet). In E2SCNet, eight kinds of lightweight operators are used to build a diversified search space, and the coding connection mode is flexible. In the search process, a large model retention mechanism is implemented through two-step multiobjective modeling and evolutionary search, where one step involves the "parameter quantity and accuracy," and the other step involves the "parameter quantity and accuracy growth quantity." Moreover, a super network is constructed to share the weight in the process of individual network evaluation and promote the search speed. The effectiveness of E2SCNet is proven by comparison with several networks designed by human experts and networks obtained by gradient and evolutionary computing-based search methods.

A multi-objective optimization framework for determining optimal chemotherapy dosing and treatment duration

Traditional randomized clinical trials are regarded as the gold standard for assessing the efficacy of chemotherapy. However, this procedure has drawbacks such as high cost, time consumption, and limited patient exploration of treatment regimens. We develop a multi-objective optimization-based framework to address these limitations and determine the best chemotherapy dosing and treatment duration. The proposed framework uses patient-specific biological parameters to create a mathematical model of cell population dynamics in the patient’s body. The framework employs evolutionary heuristic search methods (simulated annealing and genetic algorithms) and a prescriptive analytics approach to optimize therapy sessions that transition from treatment to relaxation. We carefully adjust the chemotherapy dose during treatment to reduce tumor cells while preserving host cells (such as effector-immune cells). We strategically time the relaxation sessions to aid recovery, considering the ability of tumors and healthy cells to regenerate. We use a combined optimization method to determine the length of the session and the amount of drug to be administered. We compare quadratic and linear optimal control solvers for drug administration while genetic algorithms and simulated annealing are used to optimize session length. This approach is especially important in limited healthcare resources, ensuring efficient allocation while accurately identifying high-risk patients to optimize resource allocation and utilization.

An evolutionary neural architecture search method based on performance prediction and weight inheritance

Evolutionary Neural Architecture Search (ENAS) algorithms attract great attention since they can automatically search for appropriate network architectures for a given task. However, most ENAS algorithms suffer from a prohibitive computational burden. Moreover, some of these approaches directly use performance predictors for evaluations, which may introduce inaccurate assessments and harm the evolution. To overcome these shortcomings, we propose an efficient ENAS algorithm named EPPGA. EPPGA employs a predictor to pre-select potentially high-performing offspring, enhancing the performance and accelerating the evolution. As the offspring will be further accurately evaluated, even potentially inaccurate predictions will not adversely affect the evolution. Furthermore, a weight inheritance method is suggested to accelerate the evaluation, and new genetic operations are developed to produce offspring that share a substantial proportion of beneficial genetic materials with one parent, improving the performance predictor's effectiveness and promoting weight inheritance. Finally, a new efficient backbone block structure is designed to facilitate the search for lightweight networks. The experimental results demonstrate that EPPGA is a highly competitive algorithm on three benchmarks in terms of accuracy, model size, and computational cost, reveal the superiority of the proposed block structure, and confirm the effectiveness of the proposed performance predictor and weight inheritance method.

Ranking‐based architecture generation for surrogate‐assisted neural architecture search

AbstractArchitectures generation optimization has been received a lot of attention in neural architecture search (NAS) since its efficiency in generating architecture. By learning the architecture representation through unsupervised learning and constructing a latent space, the prediction process of predictors is simplified, leading to improved efficiency in architecture search. However, searching for architectures with top performance in complex and large NAS search spaces remains challenging. In this paper, an approach that combined a ranker and generative model is proposed to address this challenge through regularizing the latent space and identifying architectures with top rankings. We introduce the ranking error to gradually regulate the training of the generative model, making it easier to identify architecture representations in the latent space. Additionally, a surrogate‐assisted evolutionary search method that utilized neural network Bayesian optimization is proposed to efficiently explore promising architectures in the latent space. We demonstrate the benefits of our approach in optimizing architectures with top rankings, and our method outperforms state‐of‐the‐art techniques on various NAS benchmarks. The code is available at https://github.com/outofstyle/RAGS‐NAS.

A 3D-RRT-star algorithm for optimizing constrained mountain railway alignments

Railway alignment development in mountainous regions is a complex problem, especially when there are numerous obstacles in the study area. Obtaining a feasible solution that satisfies all the obstacle constraints requires considerable computing resources and time. To solve this problem, a graph-based shortest path method, i.e., three-dimensional rapidly exploring random tree star (3D-RRT-star), is customized. Three main innovations are included in this method: (1) A heuristic sampling process is proposed to avoid getting the RRT search trapped into local ranges and overlooking possible path alternatives by combining a specially designed RRT node sampling method and a railway spatially-reachable analysis. (2) A multi-level constraint discretization approach is proposed to describe the obstacles in the study area, while procedures are developed to tackle the obstacle constraints dynamically during the search process. (3) An evolutionary search method integrating a sampling strategy and constraint handling operator is devised for generating a set of dissimilar RRT paths, which are finally refined into railway alignments satisfying curve constraints. Ultimately, the proposed method is applied to a realistic railway case. The experimental results confirm that it can yield a better alignment than the best manually obtained solution. Furthermore, its search efficiency is compared to a previous optimization method for this problem and the results reveal significant improvement.

Colony-Based Search Algorithm for numerical optimization

The efficiency of an Evolutionary Algorithm is highly sensitive to the mutation and crossover operators it possesses, as well as to the strategy used for determining the direction of numerical evolution and the values of evolutionary step sizes. There is no analytical method to efficiently define the direction of numerical evolution and the value of evolutionary step size for EAs. The efficiency of EAs' search processes is also influenced by their ability to maintain numerical diversity within the population. This paper introduces the Colony-Based Search Algorithm (CSA). The development of CSA was motivated by the scientific and industrial need for a relatively more efficient EA. CSA possesses relatively more efficient artificial genetic operators and strategies for producing evolutionary direction and step size, and the ability to maintain numerical diversity. CSA generates the Clan Matrix containing the pattern vectors to be evolved in the current iteration by randomly selecting pattern vectors from the Colony Matrix at the beginning of each iteration. This makes it easier for CSA to maintain numerical diversity among pattern vectors for a long time. CSA's mutation method includes three randomly blended components with different properties. The problem-solving performance of CSA is statistically compared with the problem-solving performance of eight popular evolutionary search methods (i.e., SADE, SHADE, LSHADE, COBIDE, JADE, CK, GWO, and SFS) by using benchmark functions of CEC'2017 and CEC’2022. In the experiments, the 3D viewshed analysis was addressed as a real-world problem, employing the CSA. The statistical analyses conducted on the experimental results indicate that CSA performs relatively better than the compared methods to solve numerical problems.

Lightweight multi-objective evolutionary neural architecture search with low-cost proxy metrics

Multi-Objective Evolutionary Neural Architecture Search (MOENAS) methods employ evolutionary algorithms to approximate a set of architectures representing optimal trade-offs between network performance and complexity. Directly estimating network performance via error rates or losses incurs long runtimes due to the computationally expensive network training procedure. Instead, low-cost metrics that require no network training have been proposed as a proxy for network performance. However, these metrics might exhibit inconsistent correlations with network performance across different search spaces. The influences of training-based and training-free metrics on the effectiveness and efficiency of MOENAS are still under-explored.We introduce the Enhanced Training-Free MOENAS (E-TF-MOENAS) that employs the widely-used NSGA-II as the search algorithm and optimizes multiple training-free performance metrics as separate objectives. Experiments on NAS-Bench-101 and NAS-Bench-201 show that E-TF-MOENAS outperforms training-free methods that use a single training-free performance metric and could obtain comparable results to training-based methods but with approximately 30 times less computation cost. E-TF-MOENAS obtains architectures in NAS-Bench-201 with state-of-the-art mean accuracies of 94.37%, 73.50%, and 46.62% for CIFAR-10, CIFAR-100, and ImageNet16-120, respectively, within less than 3 GPU hours. It is beneficial to utilize multiple training-free proxy metrics simultaneously and E-TF-MOENAS provides a convenient framework for building such an efficient NAS approach. The source code can be found at https://github.com/ELO-Lab/E-TF-MOENAS.

Strength and manufacturability enhancement of a composite automotive component via an integrated finite element/artificial neural network multi-objective optimization approach

This study addresses the enhancement of an injection-molded fiber-reinforced plastic / metal hybrid automotive structure and its plastic injection molding process through the integration of the finite element method, artificial intelligence and evolutionary search methods. Experiments are conducted to validate the finite element models. The orthogonal array and Latin hypercube methods are employed to generate a database via finite element analysis. The database is then used to train artificial neural networks that accurately evaluate component distortion, manufacturing time, and structural strength. A genetic optimization algorithm is applied to identify optimal process parameters. The procedure was demonstrated to simultaneously reduce product warpage and manufacturing time by 10 and 62 %, respectively, when compared with the reference manufacturing process while strength is kept above the required levels with a reduced number of required data points. A more in-depth investigation into the causes of strength variation and deformation is also provided. The results contribute to the advance of robust composite automotive structures with superior quality, manufactured through efficient processes.

Seasonal forecast-informed reservoir operation. Potential benefits for a water-stressed Mediterranean basin

The increased seasonal demand for water puts pressure on Mediterranean water resources, which are often exploited in a non-renewable way. Besides, climate change can alter hydroclimatic patterns and exaggerate freshwater stress. Flexible operation of existing water reservoirs is one of the most cost-effective ways to mitigate water-related stress by storing water when it is abundant and releasing it when droughts persist. In this context, hydroclimatic forecasts can be central in properly informing reservoir operation. Nevertheless, the link between forecast skill and forecast value is neither easily predictable nor necessarily positive. Each system requires specific forecasts according to its characteristics, and the skill of existing forecast systems does not necessarily translate into a significant gain in system performance. In this work, we develop downscaled seasonal forecasts of reservoir inflow for the Faneromeni irrigation dam on Crete island. We then quantify the value of these seasonal forecasts in informing the reservoir operations. While the current operation of this reservoir is based on the available storage at the beginning of the irrigation season, we investigate alternatives by using the Evolutionary Multi Objectives Direct Policy Search method, which allows the design of flexible rules to cope with the variability of the hydrologic conditions as well as to include forecast information for conditioning operational decisions. Under historical climate conditions, results demonstrate a notable enhancement in performance solely by implementing more flexible operating policies. Incorporating perfect forecasts results in an additional improvement of 4% on average throughout the period from 1993 to 2019. However, when using actual forecasts, this gain diminishes to 1%. These outcomes support the exploration of potential trade-off solutions that effectively balance the competing demands within the region.

Efficient multi-objective evolutionary neural architecture search for U-Nets with diamond atrous convolution and Transformer for medical image segmentation

Deep encoder–decoder neural networks like U-Nets have made significant contributions to the development of computer vision applications such as image segmentation. Neural architecture search (NAS) has the potential to further automatically adjust the architectures of U-Nets for various medical image segmentation tasks. Most of the NAS techniques focus on optimizing segmentation accuracies of network architectures. In real-world medical image segmentation scenarios, two main challenges are poor medical image quality and diverse deployment devices with different computing capabilities. A large architecture designed only for the high segmentation accuracy is difficult to run on various deployment devices. To address these challenges, this paper proposes a multi-objective evolutionary neural architecture search method (CTU-NAS) for U-Nets with diamond atrous convolution and Transformer for medical image segmentation. A hybrid U-Net architecture (CTU-Net) with diamond atrous convolution and Transformer modules is designed as the supernet of CTU-NAS. Then a channel search strategy based on sorting and selection is applied to speed up the search for subnets by precisely selecting and training the most important channels more times. In addition, CTU-NAS employs a dual acceleration mechanism based on weight sharing and surrogate model to lower the cost of evaluations of subnets. CTU-NAS applies a multi-objective evolutionary algorithm to balance between the segmentation accuracy and the number of parameters. Experimental results on two medical segmentation datasets show that CTU-NAS is capable of quickly generating a group of excellent network architectures with different sizes and their performances also outperform or come close to those of the manually designed networks.

Evolutionary neural architecture search combining multi-branch ConvNet and improved transformer

Deep convolutional neural networks (CNNs) have achieved promising performance in the field of deep learning, but the manual design turns out to be very difficult due to the increasingly complex topologies of CNNs. Recently, neural architecture search (NAS) methods have been proposed to automatically design network architectures, which are superior to handcrafted counterparts. Unfortunately, most current NAS methods suffer from either highly computational complexity of generated architectures or limitations in the flexibility of architecture design. To address above issues, this article proposes an evolutionary neural architecture search (ENAS) method based on improved Transformer and multi-branch ConvNet. The multi-branch block enriches the feature space and enhances the representational capacity of a network by combining paths with different complexities. Since convolution is inherently a local operation, a simple yet powerful “batch-free normalization Transformer Block” (BFNTBlock) is proposed to leverage both local information and long-range feature dependencies. In particular, the design of batch-free normalization (BFN) and batch normalization (BN) mixed in the BFNTBlock blocks the accumulation of estimation shift ascribe to the stack of BN, which has favorable effects for performance improvement. The proposed method achieves remarkable accuracies, 97.24 % and 80.06 % on CIFAR10 and CIFAR100, respectively, with high computational efficiency, i.e. only 1.46 and 1.53 GPU days. To validate the universality of our method in application scenarios, the proposed algorithm is verified on two real-world applications, including the GTSRB and NEU-CLS dataset, and achieves a better performance than common methods.

Curiosity Creates Diversity in Policy Search

When searching for policies, reward-sparse environments often lack sufficient information about which behaviors to improve upon or avoid. In such environments, the policy search process is bound to blindly search for reward-yielding transitions and no early reward can bias this search in one direction or another. A way to overcome this is to use intrinsic motivation in order to explore new transitions until a reward is found. In this work, we use a recently proposed definition of intrinsic motivation, Curiosity, in an evolutionary policy search method. We propose Curiosity-ES, 1 an evolutionary strategy adapted to use Curiosity as a fitness metric. We compare Curiosity-ES with other evolutionary algorithms intended for exploration, as well as with Curiosity-based reinforcement learning, and find that Curiosity-ES can generate higher diversity without the need for an explicit diversity criterion and leads to more policies which find reward.

Efficient Self-learning Evolutionary Neural Architecture Search

The evolutionary algorithm has become a major method for neural architecture search recently. However, the fixed probability distribution employed by the traditional evolutionary algorithm may lead to structural complexity and redundancy due to its inability to control the size of individual architectures, and it cannot learn from empirical information gathered during the search process to guide the subsequent search more effectively and efficiently. Moreover, evaluating the performance of all the searched architectures requires significant computing resources and time overhead. To overcome these challenges, we present the Efficient Self-learning Evolutionary Neural Architecture Search (ESE-NAS) method. Firstly, we propose an Adaptive Learning Strategy for Mutation Sampling, composed of a Model Size Control module and a Credit Assignment method for Mutation Candidates, to guide the search process by learning from the model size information and evaluation results of the architectures and adjusting the probability distributions for evolution sampling accordingly. Additionally, we developed a neural architecture performance predictor to further improve the efficiency of NAS. Experiments on CIFAR-10 and CIFAR-100 datasets show that ESE-NAS significantly brings forward the first hitting time of the optimal architectures and reaches a competitive performance level with classic manual-designed and NAS models while maintaining structural simplicity and efficiency.

ASMEvoNAS: Adaptive segmented multi-objective evolutionary network architecture search

Network architecture search (NAS) has attracted much attention as an automatic design technique of network architecture. In particular, multi-objective evolutionary algorithms (MOEAs) have been a popular kind of optimizer in NAS due to their global optimization capability. However, as a population-based iterative search method, MOEAs are subject to the unbearable computational cost of individual evaluation on multiple objectives at each generation, which affects their generalization ability and transferability of MOEA-based NAS. Therefore, an adaptive segmented multi-objective evolutionary network architecture search (ASMEvoNAS) method is proposed in this paper. Firstly, an adaptive segmented evaluation strategy is designed to adaptively select different but more suitable objectives to efficiently assess the candidate architectures at different evolutionary stages, instead of evaluating them by all the considered objectives simultaneously. Thus, the computational cost and complexity of the search process can be controlled and reduced to some extent. Secondly, a preference-based pre-selection strategy is designed to filter out the initialized architectures with high parameter quantities to reduce the total parameter scale of the whole population and memory consumption. Last, a novel desirable gene reservation-based crossover and a directed connection-based mutation are proposed to produce offspring. Experimental results show that ASMEvoNAS shows promising performance on CIFAR-10, CIFAR-100, and ImageNet with error rates of 2.21%, 15.57%, and 24.43% top-1, respectively. The proposed method reduces the search cost to 0.36 GPU-Days on CIFAR-10 while maintaining competitive classification performance compared to state-of-the-art networks. In addition, ASMEvoNAS presents superior performance when dealing with the considered transfer tasks, as well as the benchmark dataset of NAS.

Multi-Lane Differential Variable Speed Limit Control via Deep Neural Networks Optimized by an Adaptive Evolutionary Strategy.

In advanced transportation-management systems, variable speed limits are a crucial application. Deep reinforcement learning methods have been shown to have superior performance in many applications, as they are an effective approach to learning environment dynamics for decision-making and control. However, they face two significant difficulties in traffic-control applications: reward engineering with delayed reward and brittle convergence properties with gradient descent. To address these challenges, evolutionary strategies are well suited as a class of black-box optimization techniques inspired by natural evolution. Additionally, the traditional deep reinforcement learning framework struggles to handle the delayed reward setting. This paper proposes a novel approach using covariance matrix adaptation evolution strategy (CMA-ES), a gradient-free global optimization method, to handle the task of multi-lane differential variable speed limit control. The proposed method uses a deep-learning-based method to dynamically learn optimal and distinct speed limits among lanes. The parameters of the neural network are sampled using a multivariate normal distribution, and the dependencies between the variables are represented by a covariance matrix that is optimized dynamically by CMA-ES based on the freeway's throughput. The proposed approach is tested on a freeway with simulated recurrent bottlenecks, and the experimental results show that it outperforms deep reinforcement learning-based approaches, traditional evolutionary search methods, and the no-control scenario. Our proposed method demonstrates a 23% improvement in average travel time and an average of a 4% improvement in CO, HC, and NOx emission.Furthermore, the proposed method produces explainable speed limits and has desirable generalization power.

Graph Neural Network Guided Evolutionary Search of Grain Boundaries in 2D Materials

Grain boundaries (GBs) in two-dimensional (2D) materials are known to dramatically impact material properties ranging from the physical, chemical, mechanical, electronic, and optical, to name a few. Predicting a range of physically realistic GB structures for 2D materials is critical to exercising control over their properties. This, however, is nontrivial given the vast structural and configurational (defect) search space between lateral 2D sheets with varying misfits. Here, in a departure from traditional evolutionary search methods, we introduce a workflow that combines the Graph Neural Network (GNN) and an evolutionary algorithm for the discovery and design of novel 2D lateral interfaces. We use a representative 2D material, blue phosphorene (BP), and identify 2D GB structures to test the efficacy of our GNN model. The GNN was trained with a computationally inexpensive machine learning bond order potential (Tersoff formalism) and density functional theory (DFT). Systematic downsampling of the training data sets indicates that our model can predict structural energy under 0.5% mean absolute error with sparse (<2000) DFT generated energy labels for training. We further couple the GNN model with a multiobjective genetic algorithm (MOGA) and demonstrate strong accuracy in the ability of the GNN to predict GBs. Our method is generalizable, is material agnostic, and is anticipated to accelerate the discovery of 2D GB structures.

PowerSynth 2: Physical Design Automation for High-Density 3-D Multichip Power Modules

Moving towards an electrified world requires ultra-high-density power converters. With the adoption of wide-bandgap semiconductors (e.g., SiC, GaN), the next-generation power converters are on the horizon. However, the complexity of compact and high-speed converters is beyond the current industry-standard design flow based on manual and iterative steps. Therefore, research on design automation and optimization has been identified as an emerging topic in the power electronics society. Among different components of the converter, the physical design of power modules has proven critical for achieving high power density and energy efficiency. We present the latest design automation flow for high-density (2D/2.5D/3D) and heterogeneous multi-chip power modules (MCPM) through PowerSynth 2 framework. In this work, we further demonstrate electro-thermal optimization for State-of-the-Art (SOTA) 3D packaging technologies. Using the latest PowerSynth 2 framework, electro-thermal design optimization is carried out on a sample 3D MCPM layout using both exhaustive and evolutionary search methods. The optimized design is hardware-validated against physical measurements. The measurement result has shown an order of magnitude productivity improvement within 10% accuracy compared to the SOTA industry design flow.