Discrete Search Space Research Articles

Feature Selection as Deep Sequential Generative Learning

Feature selection aims to identify the most pattern-discriminative feature subset. In prior literature, filter (e.g., backward elimination) and embedded (e.g., LASSO) methods have hyperparameters (e.g., top- k , score thresholding) and tie to specific models, thus, hard to generalize; wrapper methods search a feature subset in a huge discrete space and is computationally costly. To transform the way of feature selection, we regard a selected feature subset as a selection decision token sequence and reformulate feature selection as a deep sequential generative learning task that distills feature knowledge and generates decision sequences. Our method includes three steps: (1) We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores. (2) We leverage the trained feature subset utility evaluator as a gradient provider to guide the identification of the optimal feature subset embedding; (3) We decode the optimal feature subset embedding to autoregressively generate the best feature selection decision sequence with autostop. Extensive experimental results show this generative perspective is effective and generic, without large discrete search space and expert-specific hyperparameters. The code is available at http://tinyurl.com/FSDSGL .

Targeted nano-energetic material exploration through active learning algorithm implementation

This paper presents an active learning-based method designed to guide experiments towards user-defined specific regions, termed ”regions of interest,” within vast and multi-dimensional thermite design spaces. Thermites composed of metallic reactant coupled to an inorganic oxidizer are non-explosive energetic materials which stay inert and stable until subjected to a sufficiently strong thermal stimulus, after which they undergo fast burning with release of high amount of chemical energy (up to 16 kJ.cm−3). They represent an interesting class of nano-engineered energetic material because of their high adiabatic flame temperature (> 2600 °C) and customizable combustion properties. We introduced a new acquisition function combining linearly two factors with the usual standard deviation of a Gaussian Process Regression algorithm. A first factor guides the sampling towards the defined zone of interest, and a second, is an incentive function that encourages the exploration of under-sampled regions of the design space. We found that our algorithm effectively provides up to several tens of nanothermites that achieve specific desired properties well-distributed within the design space, after only 200 samplings, whereas, Latin Hypercube Sampling procedure samples less than 10 points of interest. Our framework is tailored for typical discrete search spaces involving multiple measured physical properties and when only a small dataset is available, as it is the case in thermite materials. But more generally, this research represents a significant advancement for large-scale problems in energetic materials science and engineering, where predicting the effect of feature modification is desired but limited simulations or experiments can be afforded due to their high cost.

An improved nutcracker optimization algorithm for discrete and continuous optimization problems: Design, comprehensive analysis, and engineering applications

This study is presented to examine the performance of a newly proposed metaheuristic algorithm within discrete and continuous search spaces. Therefore, the multithresholding image segmentation problem and parameter estimation problem of both the proton exchange membrane fuel cell (PEMFC) and photovoltaic (PV) models, which have different search spaces, are used to test and verify this algorithm. The traditional techniques could not find approximate solutions for those problems in a reasonable amount of time, so researchers have used metaheuristic algorithms to overcome those shortcomings. However, the majority of metaheuristic algorithms still suffer from slow convergence speed and stagnation into local minima problems, which makes them unsuitable for tackling these optimization problems. Therefore, this study proposes an improved nutcracker optimization algorithm (INOA) for better solving those problems in an acceptable amount of time. INOA is based on improving the performance of the standard algorithm using a newly proposed convergence improvement strategy that aims to improve the convergence speed and prevent stagnation in local minima. This algorithm is first applied to estimating the unknown parameters of the single-diode, double-diode, and triple-diode models for a PV module and a solar cell. Second, four PEMFC modules are used to further observe INOA's performance for the continuous optimization challenge. Finally, the performance of INOA is investigated for solving the multi-thresholding image segmentation problem to test its effectiveness in a discrete search space. Several test images with different threshold levels were used to validate its effectiveness, stability, and scalability. Comparison to several rival optimizers using various performance indicators, such as convergence curve, standard deviation, average fitness value, and Wilcoxon rank-sum test, demonstrates that INOA is an effective alternative for solving both discrete and continuous optimization problems. Quantitively, INOA could solve those problems better than the other rival optimizers, with improvement rates for final results ranging between 0.8355 % and 3.34 % for discrete problems and 4.97 % and 99.9 % for continuous problems.

Targeted materials discovery using Bayesian algorithm execution

Rapid discovery and synthesis of future materials requires intelligent data acquisition strategies to navigate large design spaces. A popular strategy is Bayesian optimization, which aims to find candidates that maximize material properties; however, materials design often requires finding specific subsets of the design space which meet more complex or specialized goals. We present a framework that captures experimental goals through straightforward user-defined filtering algorithms. These algorithms are automatically translated into one of three intelligent, parameter-free, sequential data collection strategies (SwitchBAX, InfoBAX, and MeanBAX), bypassing the time-consuming and difficult process of task-specific acquisition function design. Our framework is tailored for typical discrete search spaces involving multiple measured physical properties and short time-horizon decision making. We demonstrate this approach on datasets for TiO2 nanoparticle synthesis and magnetic materials characterization, and show that our methods are significantly more efficient than state-of-the-art approaches. Overall, our framework provides a practical solution for navigating the complexities of materials design, and helps lay groundwork for the accelerated development of advanced materials.

Balanced quantum neural architecture search

In the last decade, there has been continuously increasing attention on Neural Architecture Search (NAS). The design of network architecture is aimed at automatically generating efficient neural networks in the absence of prior knowledge. Most of current oneshot NAS methods are based on weight sharing. However, there are two main problems that lead to suboptimal search results. Firstly, the large scale supernet results in a complex and difficult search for optimal subnetwork structures in the large discrete search space. Secondly, weight sharing makes the internal weight parameters of the supernet cannot always vary towards the final weights required for the optimal network structure due to the influence of different structures. These problems are responsible for the fact that the performance of the subnetworks searched by the supernet cannot represent the real performance of the subnetworks trained from scratch. To solve these problems, we propose the method based on quantum evolution and balance pool, called BQNAS, which consists of two stages. The single path supernet is trained based on weight sharing and balance pool sampling method. Then, quantum parallelism is exploited for one-hot encoding and searching for the optimal subnetwork structure based on quantum evolutionary algorithms. It can greatly improve the search efficiency of oneshot approaches in subnetwork sampling and evaluate the real performance for searching for superior subnetwork structure. Extensive experiments on two datasets show that the proposed approach outperforms the state-of-the-art ones. Specifically, BQNAS achieves a top-1 accuracy of 97.27% on CIFAR-10 and 81.36% on CIFAR-100.

Jumping particle swarm optimization algorithm framework for content-based image retrieval system

Content-based image retrieval (CBIR) has been studied well in the last decades in numerous research fields such as medicine, journalism, and private life. Applications of CBIR have been widely employed in medical images due to their direct impact on human life. With continues growing of digital libraries, there is a need for an efficient method to retrieve images from large datasets. In this paper, a new method was developed for CBIR based on the jumping particle swarm optimization (JPSO) algorithm. The proposed algorithm represents a developed instant of particle swarm optimization (PSO). However, JPSO the approach does not consider the velocity components to guide particle movements in the problem space. Instead of relying on inertia and velocity, intermittently random jumps (moves) occur from one solution to another within the discrete search space. To test the performance of the proposed algorithm, three types of medical image databases were used in the experiment which are the endoscopy 100, dental 100, and 50 skull image databases. The results show that the proposed algorithm could achieve high accuracy in image extraction and retrieve the accurate image category compared with other research works.

NDARTS: A Differentiable Architecture Search Based on the Neumann Series

Neural architecture search (NAS) has shown great potential in discovering powerful and flexible network models, becoming an important branch of automatic machine learning (AutoML). Although search methods based on reinforcement learning and evolutionary algorithms can find high-performance architectures, these search methods typically require hundreds of GPU days. Unlike searching in a discrete search space based on reinforcement learning and evolutionary algorithms, the differentiable neural architecture search (DARTS) continuously relaxes the search space, allowing for optimization using gradient-based methods. Based on DARTS, we propose NDARTS in this article. The new algorithm uses the Implicit Function Theorem and the Neumann series to approximate the hyper-gradient, which obtains better results than DARTS. In the simulation experiment, an ablation experiment was carried out to study the influence of the different parameters on the NDARTS algorithm and to determine the optimal weight, then the best performance of the NDARTS algorithm was searched for in the DARTS search space and the NAS-BENCH-201 search space. Compared with other NAS algorithms, the results showed that NDARTS achieved excellent results on the CIFAR-10, CIFAR-100, and ImageNet datasets, and was an effective neural architecture search algorithm.

Bayesian and Multi-Armed Contextual Meta-Optimization for Efficient Wireless Radio Resource Management

Optimal resource allocation in modern communication networks calls for the optimization of objective functions that are only accessible via costly separate evaluations for each candidate solution. The conventional approach carries out the optimization of resource-allocation parameters for each system configuration, characterized, e.g., by topology and traffic statistics, using global search methods such as Bayesian optimization (BO). These methods tend to require a large number of iterations, and hence a large number of key performance indicator (KPI) evaluations. In this paper, we propose the use of meta-learning to transfer knowledge from data collected from related, but distinct, configurations in order to speed up optimization on new network configurations. Specifically, we combine meta-learning with BO, as well as with multi-armed bandit (MAB) optimization, with the latter having the potential advantage of operating directly on a discrete search space. Furthermore, we introduce novel contextual meta-BO and meta-MAB algorithms, in which transfer of knowledge across configurations occurs at the level of a mapping from graph-based contextual information to resource-allocation parameters. Experiments for the problem of open loop power control (OLPC) parameter optimization for the uplink of multi-cell multi-antenna systems provide insights into the potential benefits of meta-learning and contextual optimization.

An Improved Binary Quadratic Interpolation Optimization for 0-1 Knapsack Problems

This paper presents a new binary optimization technique for solving the 0–1 knapsack problem. This algorithm is based on converting the continuous search space of the recently proposed quadratic interpolation optimization (QIO) into discrete search space using various V-shaped and S-shaped transfer functions; this algorithm is abbreviated as BQIO. To further improve its performance, it is effectively integrated with a uniform crossover operator and a swap operator to explore the discrete binary search space more effectively. This improved variant is called BIQIO. Both BQIO and BIQIO are assessed using 20 well-known knapsack instances and compared to four recently published metaheuristic algorithms to reveal their effectiveness. The comparison among algorithms is based on three performance metrics: the mean fitness value, Friedman mean rank and computational cost. The first two metrics are used to observe the accuracy of the results, while the last metric is employed to show the efficiency of each algorithm. The results of this comparison reveal the superiority of BIQIO over the classical BQIO and four rival optimizers.

Role of transfer functions in PSO to select diagnostic attributes for chronic disease prediction: An experimental study

Particle Swarm Optimization (PSO) is a classic and popularly used meta-heuristic algorithm in many real-life optimization problems due to its less computational complexity and simplicity. The binary version of PSO, known as BPSO, is used to solve binary optimization problems, such as feature selection. Like other meta-heuristic optimization techniques designed on the continuous search space, PSO uses the transfer functions (TFs) to map the candidate solutions to the discrete search space in BPSO, and these TFs play a vital role to get the desired results. Over the years, many forms of TFs have been introduced in the literature, most of which fall under one of the five families - Linear, S-shaped, V-shaped, U-shaped, and Time-varying Mirrored S-shaped TFs. The goal of this study is to determine an appropriate setup constituting a TF and a classifier for feature selection from different types of clinical data. In this study, the impacts of the five TF families have been investigated, considering one from each family for the selection of attributes/features, while predicting disease using diagnosis or medical reports. The classification tasks are carried out using four standard classifiers: Support Vector Machine, Decision Tree, K-Nearest Neighbors, and Gaussian Naive Bayes. For experimental purposes, we have used four publicly available datasets namely, the UCI Heart Disease dataset, Wisconsin Breast Cancer dataset, UCI Chronic Kidney Disease dataset, and PIMA Indians Diabetes dataset. After an exhaustive set of experiments, we have obtained 96.72%, 99.82%, 100.00%, and 84.41% disease prediction scores in the best case for Heart disease, Breast Cancer, Chronic Kidney disease, and Diabetes, respectively. The obtained results are comparable to several state-of-the-art methods considered here for comparison. The present study helps in selecting a suitable BPSO setup (i.e., a TF and a classifier) to select important diagnostic attributes useful to design a computer-aided decision support system for the said diseases.

TA-DARTS: Temperature Annealing of Discrete Operator Distribution for Effective Differential Architecture Search

In the realm of machine learning, the optimization of hyperparameters and the design of neural architectures entail laborious and time-intensive endeavors. To address these challenges, considerable research effort has been directed towards Automated Machine Learning (AutoML), with a focus on enhancing these inherent inefficiencies. A pivotal facet of this pursuit is Neural Architecture Search (NAS), a domain dedicated to the automated formulation of neural network architectures. Given the pronounced impact of network architecture on neural network performance, NAS techniques strive to identify architectures that can manifest optimal performance outcomes. A prominent algorithm in this area is Differentiable Architecture Search (DARTS), which transforms discrete search spaces into continuous counterparts using gradient-based methodologies, thereby surpassing prior NAS methodologies. Notwithstanding DARTS’ achievements, a discrepancy between discrete and continuously encoded architectures persists. To ameliorate this disparity, we propose TA-DARTS in this study—a temperature annealing technique applied to the Softmax function, utilized for encoding the continuous search space. By leveraging temperature values, architectural weights are judiciously adjusted to alleviate biases in the search process or to align resulting architectures more closely with discrete values. Our findings exhibit advancements over the original DARTS methodology, evidenced by a 0.07%p enhancement in validation accuracy and a 0.16%p improvement in test accuracy on the CIFAR-100 dataset. Through systematic experimentation on benchmark datasets, we establish the superiority of TA-DARTS over the original mixed operator, thereby underscoring its efficacy in automating neural architecture design.

An improved binary African vultures optimization approach to solve the UC problem for power systems

Unit commitment (UC) is one of the most crucial problems in electrical power systems. It concerns finding the optimal schedule for generating units such that the production cost is minimized over a given period. This paper aims to solve the unit commitment problem by proposing a binary version of a recent metaheuristic algorithm, the African Vultures Optimization Algorithm (AVOA), which is a nature-inspired metaheuristic algorithm that mimics the hunting mechanism and behavior of African vultures. This algorithm has the advantage of creating various phase shift approaches to avoid premature convergence and local optimum trapping. AVOA uses two mechanisms for the exploration phase and four for the exploitation phase that ensure the algorithm's ability to diversify and intensify. The AVOA is a continuous algorithm that cannot tackle the mixed-integer nature of the UC. In this paper, a sigmoid transfer function is used to convert the algorithm into a binary algorithm to decide the on/off status of the generating units. To further enhance the performance of the proposed algorithm, new values of the controlling parameters are proposed. Finally, the performance of the proposed algorithm is tested on IEEE 30-, 14-, and 57-bus systems. The proposed Binary AVOA (BAVOA) has superiority over the other algorithms given in the case study. In addition, results revealed that the BAVOA gives better results in the discrete search space (DSS) compared to the continuous search space (CSS). For instance, the operation cost of an IEEE 30-bus system in the DSS is 12,768 dollars which is 7% less than the CSS.

Discrete grey wolf optimization algorithm for solving k-coverage problem in directional sensor networks with network lifetime maximization viewpoint

Unlike the traditional wireless sensor networks (WSNs) which apply omni-directional sensors, the directional sensor networks (DSNs) utilize directional sensor nodes. Each directional sensor in DSN is a self-configured in one direction and one coverage range at a same time. In critical industries, the target coverage problem in which every target should be observed by k different sensors increases the reliability and leads the fault tolerant observation. In this case, the huge amount of energy is consumed in comparison with one coverage problem. Since the business continuity in such industries strongly depends on the network lifetime, the energy management and lifetime maximization of such resource-limited networks are still of the most important challenges. To address the issue, this paper formulates the k-coverage challenge to a discrete optimization problem with the network lifespan expansion viewpoint which is an NP-Hard problem. To solve this combinatorial problem, a discrete grey wolf optimization algorithm (D-GWA) is presented. This D-GWA proposes abundant novel exploration and exploitation procedures which permutes discrete search space efficiently. In this regard, the new fitness function is also defined which steers solutions in the directions to engage sensors uniformly in different rounds. It potentially improves network lifetime because of uniform battery usage. During the course of optimization, it utilizes temperature concept of simulated annealing algorithm to running away from local trap. To verify the proposed algorithm in solving k-coverage problem in DSNs, twelve scenarios are conducted. The simulation results experimentally prove that the totally dominance of the proposed algorithm against state-of-the-arts HTOH, LA-based, GWA, and hybrid PSO-GA approaches respectively are the amount of 31.37%, 20.97%, 14.12%, and 7.93% improvement in term of network lifetime expansion. Furthermore, the behavior of D-GWA shows the high potential of scalability in the larger search space.

Solving wind‐integrated unit commitment problem by a modified African vultures optimization algorithm

AbstractUnit commitment (UC) stands out as a significant challenge in electrical power systems. With the rapid growth in power demand and the pressing issues of fossil fuel scarcity and global warming, it has become crucial to enhance the utilization of renewable energy sources. This study focuses on addressing the UC problem by incorporating a wind farm and proposes a modified version of the metaheuristic African vultures optimization algorithm (AVOA) in binary form, utilizing the sigmoid transfer function. The modified AVOA employs multiple phase‐shift tactics to overcome premature local optima. By determining the on/off status of generating units, the modified AVOA improves the algorithm's effectiveness. Additionally, the paper develops an auto‐regressive moving average model (ARMA) to forecast wind speeds, with the AVOA assisting in selecting the optimal orders (q and p) of the ARMA model. This is done using historical wind speed data to capture uncertainty in the wind speed. The wind power is then calculated using various models and integrated into the UC problem. The effectiveness of the modified AVOA is examined on the IEEE 30‐bus system. The binary AVOA (BAVOA) outperforms several algorithms presented in the case study, demonstrating its superiority. Furthermore, the results indicate that BAVOA delivers superior outcomes within the discrete search space when compared to the continuous search space.

Binary arithmetic optimization algorithm for feature selection.

Feature selection, widely used in data preprocessing, is a challenging problem as it involves hard combinatorial optimization. So far some meta-heuristic algorithms have shown effectiveness in solving hard combinatorial optimization problems. As the arithmetic optimization algorithm only performs well in dealing with continuous optimization problems, multiple binary arithmetic optimization algorithms (BAOAs) utilizing different strategies are proposed to perform feature selection. First, six algorithms are formed based on six different transfer functions by converting the continuous search space to the discrete search space. Second, in order to enhance the speed of searching and the ability of escaping from the local optima, six other algorithms are further developed by integrating the transfer functions and Lévy flight. Based on 20 common University of California Irvine (UCI) datasets, the performance of our proposed algorithms in feature selection is evaluated, and the results demonstrate that BAOA_S1LF is the most superior among all the proposed algorithms. Moreover, the performance of BAOA_S1LF is compared with other meta-heuristic algorithms on 26 UCI datasets, and the corresponding results show the superiority of BAOA_S1LF in feature selection. Source codes of BAOA_S1LF are publicly available at: https://www.mathworks.com/matlabcentral/fileexchange/124545-binary-arithmetic-optimization-algorithm.

Generation of Nonlinear Substitutions by Simulated Annealing Algorithm

The problem of nonlinear substitution generation (S-boxes) is investigated in many related works in symmetric key cryptography. In particular, the strength of symmetric ciphers to linear cryptanalysis is directly related to the nonlinearity of substitution. In addition to being highly nonlinear, S-boxes must be random, i.e., must not contain hidden mathematical constructs that facilitate algebraic cryptanalysis. The generation of such substitutions is a complex combinatorial optimization problem. Probabilistic algorithms are used to solve it, for instance the simulated annealing algorithm, which is well-fitted to a discrete search space. We propose a new cost function based on Walsh–Hadamard spectrum computation, and investigate the search efficiency of S-boxes using a simulated annealing algorithm. For this purpose, we conduct numerous experiments with different input parameters: initial temperature, cooling coefficient, number of internal and external loops. As the results of the research show, applying the new cost function allows for the rapid generation of nonlinear substitutions. To find 8-bit bijective S-boxes with nonlinearity 104, we need about 83,000 iterations. At the same time, the probability of finding the target result is 100%.

Discrete komodo algorithm for traveling salesman problem

This work presents a Discrete Komodo Algorithm (DKA) to solve the traveling salesman problem (TSP). Inspired by the Komodo Mlipir Algorithm, a swarm intelligence algorithm based on the behavior of komodo dragons, DKA introduces an enhanced edge-construction operator and a novel edge-destruction operator to explore the discrete search space of TSP. To prioritize the exploration of potential edges, DKA uses a Priority Queue that stores all possible edges with their weights, and its counts appear on tour as its priority. The edges with smaller weights and counts have higher probabilities of being selected for the tour. DKA is tested with 45 different TSP instances from TSPLIB. Experimental results show that DKA guarantees produced the best-known solution for small test problems for up to 24 nodes, and the best solutions are within 6.92% of the known optimal solutions. Compared with some of the previous algorithms, DKA outperforms the recent state-of-the-art algorithms and is significantly better than classical metaheuristic algorithms by achieving a lower relative error.

A Particle Swarm Optimization and Variable Neighborhood Search based multipopulation algorithm for Inter-Domain Path Computation problem

It is a common occurrence nowadays for a network to be gigantic in size and complex in architecture. Network navigation thus faces many new problems in terms of routing and resource utilization, one of which arises in multi-domain networks. This paper focuses on the Inter-Domain Path Computation under Node-defined Domain Uniqueness Constraint (IDPC-NDU) problem, whose purpose is to find the shortest path between two nodes in a network under a constraint that such a path is only allowed to traverse each domain at most once. Considering that this is an NP-Hard problem, an approximate approach is more practical than an exact one. Meanwhile, Particles Swarm Optimization Algorithm (PSO) has been long known for its powerful ability to discover near-optimal solutions in a reasonable time; however, its application in discrete search space is limited. Therefore, we decide to use the multi-population framework, a sub-branch of multitasking optimization which allows information exchange not only between problems but also between different optimization heuristics, to improve upon the basic PSO method. Specifically, this paper introduces a hybridization between PSO and Variable Neighborhood Search (VNS). The encoding and decoding method are created specifically for the IDPC-NDU problem and for the PSO algorithm, and VNS serves to enhance further the algorithm’s ability to escape local optima. Experiments are carried out to prove the new algorithm’s efficacy, especially in the context of similar multitasking methods.

Memetic Algorithms for Spatial Partitioning Problems

Spatial optimization problems (SOPs) are characterized by spatial relationships governing the decision variables, objectives, and/or constraint functions. In this article, we focus on a specific type of SOP called spatial partitioning, which is a combinatorial problem due to the presence of discrete spatial units. Exact optimization methods do not scale with the size of the problem, especially within practicable time limits. This motivated us to develop population-based metaheuristics for solving such SOPs. However, the search operators employed by these population-based methods are mostly designed for real-parameter continuous optimization problems. For adapting these methods to SOPs, we apply domain knowledge in designing spatially aware search operators for efficiently searching through the discrete search space while preserving the spatial constraints. To this end, we put forward a simple yet effective algorithm called s warm-based s p atial meme ti c al gorithm (SPATIAL) and test it on the school (re)districting problem. Detailed experimental investigations are performed on real-world datasets to evaluate the performance of SPATIAL. Besides, ablation studies are performed to understand the role of the individual components of SPATIAL. Additionally, we discuss how SPATIAL is helpful in the real-life planning process and its applicability to different scenarios and motivate future research directions.

AutoMA: Towards Automatic Model Augmentation for Transferable Adversarial Attacks

Recent adversarial attack works attempt to improve the transferability by applying various differentiable transformations on input images. Considering the differentiable transformations and the original model together as a new model, these methods can be regarded as model augmentation that effectively derives an ensemble of models from the single original model. Despite their impressive performance, the model augmentation policies used in these methods are manually designed by experimental attempts, leaving the design of model augmentation policy an open question. In this paper, we propose an Automatic Model Augmentation (AutoMA) approach to find a strong model augmentation policy for transferable adversarial attacks. Specifically, we design a discrete search space that contains various diffierentiable transformations with different parameters and adopt reinforcement learning to search for the strong augmentation policy. The sampled augmentation policies together with the rewards they obtain during the searching process reveal several valuable observations for designing more powerful attacks using model augmentation policy: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1) Augmentation transformations on color space are less effective; 2) The transformation type diversity matters; and 3) Using small distortion for geometric transformations while larger distortion for intensity transformations.</i> Extensive experiments show that the augmentation policy found by AutoMA achieves superior performance than existing manually designed policies in a wide range of cases.