High-dimensional Solution Space Research Articles

Multipopulation-Based Differential Evolution for Large-Scale Many-Objective Optimization.

In recent years, numerous efficient many-objective optimization evolutionary algorithms have been proposed to find well-converged and well-distributed nondominated optimal solutions. However, their scalability performance may deteriorate drastically to solve large-scale many-objective optimization problems (LSMaOPs). Encountering high-dimensional solution space with more than 100 decision variables, some of them may lose diversity and trap into local optima, while others may achieve poor convergence performance. This article proposes a multipopulation-based differential evolution algorithm, called LSMaODE, which can solve LSMaOPs efficiently and effectively. In order to exploit and explore the exponential decision space, the proposed algorithm divides the population into two groups of subpopulations, which are optimized with different strategies. First, the randomized coordinate descent technique is applied to 10% of individuals to exploit the decision variables independently. This subpopulation maintains diversity in the decision space to avoid premature convergence into local optimum. Second, the remaining 90% of individuals are optimized with the nondominated guided random interpolation strategy, which interpolates individual among three nondominated solutions randomly. The strategy can guide the population convergent toward the nondominated solutions quickly, meanwhile, maintain good distribution in objective space. Finally, the proposed LSMaODE is evaluated on the LSMOP test suites from the scalability in both decision and objective dimensions. The performance is compared against five state-of-the-art large-scale many-objective evolutionary algorithms. The experimental results show that LSMaODE provides highly competitive performance.

Learning-Based Cuckoo Search Algorithm to Schedule a Flexible Job Shop With Sequencing Flexibility.

This work considers an extended version of flexible job-shop problem from a postprinting or semiconductor manufacturing environment, which needs a directed acyclic graph rather than a linear order to describe the precedences among operations. To obtain its reliable and high-quality schedule in a reasonable time, a learning-based cuckoo search (LCS) algorithm is presented. In it, cuckoo search is selected as an optimizer. To produce promising solutions in a high-dimensional solution space, a sparse autoencoder is introduced to compress a high-dimensional solution into an informative low-dimensional one. It extends the application area of autoencoder-embedded evolutionary optimization methods into combinational optimization by developing an improved one-hot encoding method. Then, in order to reveal the linkages among decision variables and enhance the explore ability of the proposed method, a factorization machine (FM) is used, for the first time, to capture the relevant and complementary features of population. Hence, a parallel framework involving three co-evolved subpopulations is constructed. The first one is an autoencoder embedded subpopulation, the second one is assisted by an FM, and the last one undergoes a regular iteration process. To balance the exploration and exploitation of the proposed framework and avoid unnecessary computation, a reinforcement learning algorithm is used to adaptively adjust the proportion of subpopulations and tune parameters of each subpopulation iteratively. Numerical simulations with benchmarks are performed to compare it with CPLEX, some classical heuristics, and several recently developed methods. The results shows that it well outperforms them.

Multispace Evolutionary Search for Large-Scale Optimization With Applications to Recommender Systems

Large-scale optimization is vital in today’s artificial intelligence (AI) applications for extracting essential knowledge from huge volumes of data. In recent years, to improve the evolutionary algorithms used to solve optimization problems involving a large number of decision variables, many attempts have been made to simplify the problem solution space of a given problem for the evolutionary search. In the literature, the existing approaches can generally be categorized as decomposition-based methods and dimension-reduction-based methods. The former decomposes a large-scale problem into several smaller subproblems, while the latter transforms the original high-dimensional solution space into a low-dimensional space. However, it is worth noting that a given large-scale optimization problem may not always be decomposable, and it is also difficult to guarantee that the global optimum of the original problem is preserved in the reduced low-dimensional problem space. This article, thus, proposes a new search paradigm, namely the multispace evolutionary search, to enhance the existing evolutionary search methods for solving large-scale optimization problems. In contrast to existing approaches that perform an evolutionary search in a single search space, the proposed paradigm is designed to conduct a search in multiple solution spaces that are derived from the given problem, each possessing a unique landscape. The proposed paradigm makes no assumptions about the large-scale optimization problem of interest, such as that the problem is decomposable or that a certain relationship exists among the decision variables. To verify the efficacy of the proposed paradigm, comprehensive empirical studies in comparison to five state-of-the-art algorithms were conducted using the CEC2013 large-scale benchmark problems as well as an AI application in e-commerce, i.e., movie recommendation.

Elite Directed Particle Swarm Optimization with Historical Information for High-Dimensional Problems

High-dimensional optimization problems are ubiquitous in every field nowadays, which seriously challenge the optimization ability of existing optimizers. To solve this kind of optimization problems effectively, this paper proposes an elite-directed particle swarm optimization (EDPSO) with historical information to explore and exploit the high-dimensional solution space efficiently. Specifically, in EDPSO, the swarm is first separated into two exclusive sets based on the Pareto principle (80-20 rule), namely the elite set containing the top best 20% of particles and the non-elite set consisting of the remaining 80% of particles. Then, the non-elite set is further separated into two layers with the same size from the best to the worst. As a result, the swarm is divided into three layers. Subsequently, particles in the third layer learn from those in the first two layers, while particles in the second layer learn from those in the first layer, on the condition that particles in the first layer remain unchanged. In this way, the learning effectiveness and the learning diversity of particles could be largely promoted. To further enhance the learning diversity of particles, we maintain an additional archive to store obsolete elites, and use the predominant elites in the archive along with particles in the first two layers to direct the update of particles in the third layer. With these two mechanisms, the proposed EDPSO is expected to compromise search intensification and diversification well at the swarm level and the particle level, to explore and exploit the solution space. Extensive experiments are conducted on the widely used CEC’2010 and CEC’2013 high-dimensional benchmark problem sets to validate the effectiveness of the proposed EDPSO. Compared with several state-of-the-art large-scale algorithms, EDPSO is demonstrated to achieve highly competitive or even much better performance in tackling high-dimensional problems.

Efficient well placement optimization based on theory-guided convolutional neural network

Well placement optimization is important in reservoir management, but it is challenging to implement due to the high-dimensional solution space and large number of reservoir simulations required. Surrogate models may assist to alleviate the computational burden by efficiently approximating full-order models. Although deep learning has been proven to be effective for surrogate modeling, most deep learning surrogates are purely data-driven, and underlying physical principles or theories of subsurface flows are not considered. In this work, a theory-guided convolutional neural network (TgCNN) framework is extended as a surrogate for subsurface flows with position-varying sink/source terms (well locations), which is further utilized for well placement optimization. In TgCNN, the physical constraints are incorporated to guide the training process of the surrogate by adding the residual of governing equations (and boundary/initial conditions) into the loss function. Guided by theory, the TgCNN surrogate can achieve better accuracy and generalizability, even when trained with limited data. The trained TgCNN surrogate can be further used for well placement optimization by combining it with the genetic algorithm (GA). The TgCNN surrogate also achieves satisfactory extrapolation performance for scenarios with different well numbers, and thus joint optimization of well number and placement can also be implemented with the TgCNN surrogate. The performance of the proposed optimization strategy is compared with the optimization framework that uses the simulator directly, and the results verify the accuracy of the TgCNN surrogate-based GA. Moreover, using the TgCNN surrogate can improve the efficiency of optimization significantly compared with running the simulators repeatedly. The effect of geologic uncertainty for the optimization is also investigated, and the results demonstrate that the optimization results may deviate from the optimal well placements as the degree of uncertainty increases.

Design Of Fuzzy Rule-Based System Using Genetic Algorithms

The aim of this paper is to obtain a compact and optimal fuzzy rule-based model from observation data by utilizing the Genetic algorithm technique. The approach is optimized by applying Genetic Algorithms, owing to its capability of searching irregular and high dimensional solution spaces. Genetic Algorithms has been applied to learn consequent part of fuzzy rules and models with fixed number of rules. In the work we propose a Genetic algorithm approach to a non-linear air conditioning system for the construction of optimal fuzzy rules in two steps. First, fuzzy clustering is applied to obtain an initial rule based model having pre-calculated number of rules with antecedents only. In the second step, the regions of rule-consequents are obtained by a binary coded Genetic Algorithm which leads to the extraction of an optimal rule based model.

Design of a freeform imaging spectrometer based on a solution-diversified automatic design method.

A freeform imaging spectrometer design method is proposed. Only system specifications, including the slit length, numerical aperture, magnification, spectral range, and spectral resolution, are required as design inputs, and imaging spectrometer systems with various optical power distributions and structures are output. Using three-mirror imaging spectrometers as design examples, a series of initial solutions are obtained with a diffraction grating placed at the secondary mirror. In order to understand and explore the utilization of large number of output results, these initial solutions are visualized by mapping from high-dimensional solution space to three-dimensional space using the t-distributed symmetric neighbor embedding (t-SNE) dimensionality reduction algorithm; then a neural network is trained to fit the imaging qualities, showing good generalization performance, and can predict imaging quality for systems with unknown optical power distributions and structures.

Enhancing hierarchical surrogate-assisted evolutionary algorithm for high-dimensional expensive optimization via random projection

By remarkably reducing real fitness evaluations, surrogate-assisted evolutionary algorithms (SAEAs), especially hierarchical SAEAs, have been shown to be effective in solving computationally expensive optimization problems. The success of hierarchical SAEAs mainly profits from the potential benefit of their global surrogate models known as “blessing of uncertainty” and the high accuracy of local models. However, their performance leaves room for improvement on high-dimensional problems since now it is still challenging to build accurate enough local models due to the huge solution space. Directing against this issue, this study proposes a new hierarchical SAEA by training local surrogate models with the help of the random projection technique. Instead of executing training in the original high-dimensional solution space, the new algorithm first randomly projects training samples onto a set of low-dimensional subspaces, then trains a surrogate model in each subspace, and finally achieves evaluations of candidate solutions by averaging the resulting models. Experimental results on seven benchmark functions of 100 and 200 dimensions demonstrate that random projection can significantly improve the accuracy of local surrogate models and the new proposed hierarchical SAEA possesses an obvious edge over state-of-the-art SAEAs.

Adaptive signal control for bus service reliability with connected vehicle technology via reinforcement learning

This paper presents an adaptive signal controller for managing traffic delays and urban bus service reliability with fully adaptable acyclic timing plans. The signal controller is built upon a reinforcement learning framework that consists of a model-based and a data-driven component. The model-based component is represented by a hybrid kinematic wave traffic model that integrates macroscopic flow-based and microscopic vehicle-based state variables subject to stochastic demands and bus service status. To cope with the high dimensional solution space, the data-driven component is incorporated as a multi-layer artificial neural network and is used to approximate future traffic states and system performances with respect to prevailing control settings. Before the controller can be used, the neural network is to be trained through a series of realised dynamic state transitions via an on-policy temporal difference learning algorithm. The proposed control framework is tested over a real world corridor scenario in London, UK. The proposed controller is able to reduce both traffic delays and bus service variabilities subject to stochastic demands with acyclic timing plans that can be derived in short computational time. This study contributes to the design of adaptive network traffic control for multi-modal networks with connected vehicle technology and advanced learning-based optimisation techniques.

An Improved Vantage Point Bees Algorithm to Solve Combinatorial Optimization Problems from TSPLIB

AbstractThis paper presents an improved version of the Vantage Point Bees Algorithm (VPBA‐II), which is implemented to solve the Travelling Salesman Problem. The Vantage Point Tree has been used to produce initial tour solutions and also as a global search operator of the proposed algorithm to find the minimal Hamiltonian tour of the Travelling Salesman Problem. VPBA‐II is tested on 15 different benchmark datasets from TSPLIB, particularly for the high dimensional combinatorial solution spaces, and it outperformed the basic Bees Algorithm. The composition of the local search operators combined with Vantage Point Tours perform better except one dataset and achieved optimum results according to best‐known solutions of Travelling Salesman Problem as a best‐case scenario. The experiments prove that Vantage Point Tour construction could be used as initialization and global search operator to improve the basic Bees Algorithm performance on the combinatorial domains.

Image super-resolution using multi-granularity perception and pyramid attention networks

Recently, single image super-resolution (SISR) has been widely applied in the fields of multimedia and computer vision communities and obtained remarkable performance. However, most current methods ignore to utilize multi-granularity features of low-resolution (LR) image to further improve the SISR performance. And the channel and spatial features obtained from original LR images are treated equally, resulting in unnecessary computations for abundant uninformative features, thereby hindering the representational ability of super-resolution (SR) models. In this paper, we present a novel Multi-Granularity Pyramid Attention Network (MGPAN) which fully exploits the multi-granularity perception and attention mechanisms to improve the quality of reconstructed images. We design a multi-branch dilated convolution layer with varied kernels corresponding to receptive fields of different sizes to modulate multi-granularity features for adaptively capturing more important information. Moreover, a novel spatial pyramid pooling attention (SPPA) module is constructed to integrate the channel-wise and multi-scale spatial information, which is beneficial to compute the response values from the multi-scale regions of each neuron, and then establish the accurate mapping from low to high dimensional solution space. Besides, for long-short-term information preservation and information flow enhancement, we adopt the short, long, and global skip connection structures to concatenate and fuse the states of each module, which can improve the SR network performance effectively. Extensive experiments on several standard benchmark datasets show that the proposed MGPAN can provide state-of-the-art or even better performance in both quantitative and qualitative measurements.

Efficient trajectory optimization for curved running using a 3D musculoskeletal model with implicit dynamics

Trajectory optimization with musculoskeletal models can be used to reconstruct measured movements and to predict changes in movements in response to environmental changes. It enables an exhaustive analysis of joint angles, joint moments, ground reaction forces, and muscle forces, among others. However, its application is still limited to simplified problems in two dimensional space or straight motions. The simulation of movements with directional changes, e.g. curved running, requires detailed three dimensional models which lead to a high-dimensional solution space. We extended a full-body three dimensional musculoskeletal model to be specialized for running with directional changes. Model dynamics were implemented implicitly and trajectory optimization problems were solved with direct collocation to enable efficient computation. Standing, straight running, and curved running were simulated starting from a random initial guess to confirm the capabilities of our model and approach: efficacy, tracking and predictive power. Altogether the simulations required 1 h 17 min and corresponded well to the reference data. The prediction of curved running using straight running as tracking data revealed the necessity of avoiding interpenetration of body segments. In summary, the proposed formulation is able to efficiently predict a new motion task while preserving dynamic consistency. Hence, labor-intensive and thus costly experimental studies could be replaced by simulations for movement analysis and virtual product design.

Proper orthogonal decomposition method for multiscale elliptic PDEs with random coefficients

In this paper, we develop an efficient multiscale reduced basis method to solve elliptic PDEs with multiscale and random coefficients in a multi-query setting. Our method consists of offline and online stages. In the offline stage, a small number of reduced multiscale basis functions are constructed within each coarse grid block using the proper orthogonal decomposition (POD) method. Moreover, local tensor spaces are defined to approximate the solution space of the multiscale random PDEs. In the online stage, a weak formulation is derived and discretized using the Galerkin method to compute the solution. Since the multiscale reduced basis functions can efficiently approximate the high-dimensional solution space, our method is very efficient in solving multiscale elliptic PDEs with random coefficients. Convergence analysis of the proposed method is presented, which shows the dependence of the numerical error on the number of snapshots and the truncation threshold in the POD method. Finally, numerical results are presented to demonstrate the accuracy and efficiency of the proposed method for several multiscale problems with or without scale separation in the physical space.

Application of IPO: a heuristic neuro-fuzzy classifier

Heuristic methods are used to design an adaptive-network-based fuzzy inference system (ANFIS) neuro-fuzzy classifier. The reason is that these classifiers include diverse structures, each of which has a considerable effect on their performance. So, the designer of an ANFIS classifier confronts a high-dimensional solution space and heuristic methods are of high capability in solving such problems (finding the best optimum values of these parameters). Using an efficient method of accurate designing to achieve the best performance is considered as the main challenge in terms of these classifiers. In this paper, an intelligent method based on one of the newest heuristic methods called inclined planes system optimization algorithm (IPO) has been proposed and implemented for the first time so that automatic designing of a neuro-fuzzy classifier is performed. IPO method is inspired by the dynamics of spherical objects’ sliding motion along a set of frictionless inclined planes based on which objects in cooperation with each other move towards the best response to the problem according to Newton’s Second Law and equations of motion. The results obtained from repetitive tests performed on several well-known databases with various numbers of reference classes as well as different feature vector lengths with acceptable and certain complexities indicated capability of the proposed method compared to other heuristic methods for automatic design of a neuro-fuzzy classifier.

On the Optimal Decomposition of High-Dimensional Solution Spaces of Complex Systems

In the early development phase of complex technical systems, uncertainties caused by unknown design restrictions must be considered. In order to avoid premature design decisions, sets of good designs, i.e., designs which satisfy all design goals, are sought rather than one optimal design that may later turn out to be infeasible. A set of good designs is called a solution space and serves as target region for design variables, including those that quantify properties of components or subsystems. Often, the solution space is approximated, e.g., to enable independent development work. Algorithms that approximate the solution space as high-dimensional boxes are available, in which edges represent permissible intervals for single design variables. The box size is maximized to provide large target regions and facilitate design work. As a result of geometrical mismatch, however, boxes typically capture only a small portion of the complete solution space. To reduce this loss of solution space while still enabling independent development work, this paper presents a new approach that optimizes a set of permissible two-dimensional (2D) regions for pairs of design variables, so-called 2D-spaces. Each 2D-space is confined by polygons. The Cartesian product of all 2D-spaces forms a solution space for all design variables. An optimization problem is formulated that maximizes the size of the solution space, and is solved using an interior-point algorithm. The approach is applicable to arbitrary systems with performance measures that can be expressed or approximated as linear functions of their design variables. Its effectiveness is demonstrated in a chassis design problem.

Sidelobe reduction of large‐scale antenna array for 5G beamforming via hierarchical cuckoo search

Large-scale antenna array (LSAA) is one of the key factors as well as a fundamental technology in fifth generation wireless communication systems. Beampattern sidelobe (SL) suppressing is a complex nonlinear optimisation problem, especially for LSAAs, because of the high-dimensional solution space. A SL reduction method for LSAA based on a hierarchical cuckoo search (HCS) algorithm is proposed. HCS divides the candidate solutions into different levels to improve population utilisation, so that it is suitable for dealing with large-scale elements. The effectiveness of the proposed method for synthesis of the beampattern of LSAA is verified by theoretical and EM simulations.

Real-time modelling of diastolic filling of the heart using the proper orthogonal decomposition with interpolation

In this research, a reduced order method (ROM) called the Proper Orthogonal Decomposition with Interpolation (PODI) is used to drastically reduce computation time of highly complex and non-linear problems as encountered in simulating the heart. The idea behind this method is to first construct a database of pre-computed full-scale solutions using the Element Free Galerkin method (EFG) and then project a selected subset of these solutions to a low dimensional space. Using the Moving Least Square method an interpolation is carried out for the problem at hand, before the resulting coefficients are projected back to the original high dimensional solution space. Computations are carried out on a bi-ventricle model to investigate the performance and accuracy varying the material parameters and to determine the sensitivity with respect to the parametric values. The PODI calculations are completed within 1.25 s on a normal desktop machine with the relative ℓ2 error norm not exceeding 2.5×10−3. Hence, it is demonstrated that real-time modelling of the heart can be successfully carried out at acceptable error levels.

Multiple target localization in WSNs using compressed sensing reconstruction based on ABC algorithm

In allusion to the application of compressed sensing theory in multi-target localization in wireless sensor networks, the paper proposes a method of sparse signal reconstruction based on artificial bee colony algorithm and designs partial update strategy of location coding. A mode of disturbances mutation operation for food source exploitation is designed to improve the search efficiency in high-dimensional solution space; it also improves the global search efficiency of the algorithm. The comparing of simulation analysis with other algorithms verifies that the algorithm effectively solves problem of multi-target localization in wireless sensor networks and has better resistance to interference.

On multiscale methods in Petrov–Galerkin formulation

In this work we investigate the advantages of multiscale methods in Petrov–Galerkin (PG) formulation in a general framework. The framework is based on a localized orthogonal decomposition of a high dimensional solution space into a low dimensional multiscale space with good approximation properties and a high dimensional remainder space, which only contains negligible fine scale information. The multiscale space can then be used to obtain accurate Galerkin approximations. As a model problem we consider the Poisson equation. We prove that a Petrov–Galerkin formulation does not suffer from a significant loss of accuracy, and still preserve the convergence order of the original multiscale method. We also prove inf-sup stability of a PG continuous and a discontinuous Galerkin finite element multiscale method. Furthermore, we demonstrate that the Petrov–Galerkin method can decrease the computational complexity significantly, allowing for more efficient solution algorithms. As another application of the framework, we show how the Petrov–Galerkin framework can be used to construct a locally mass conservative solver for two-phase flow simulation that employs the Buckley–Leverett equation. To achieve this, we couple a PG discontinuous Galerkin finite element method with an upwind scheme for a hyperbolic conservation law.

An immune memory optimization algorithm based on the non-genetic information

In order to improve the ability to optimize artificial immune algorithm, the memory mechanism of non-genetic information is introduced into optimization algorithm. An immune memory optimization algorithm based on the non-genetic information is proposed. Emulating human society education and experiential inheritance mechanism, the algorithm takes, stores and uses non genetic information in the evolutionary process of the population. By setting up a separate memory base, the algorithm stores non genetic information, and guides the subsequent search process. The algorithm uses the short-term memory of the prior knowledge and guides the subsequent evolution, which can increase the intelligence of search and reduce the blind search and repeat the search. The immune memory optimization algorithm based on the non-genetic information includes key operators: mutation operator, crossover operator and complement operator. The mutation operator is able to efficiently use non genetic information of grandparents to search, which can speed up the local search efficiency. In addition, the threshold to control the search depth of single dimension can avoid falling into local optimal solution making the evolutionary standstill. Through calculating comprehensive information about contemporary populations of all antibodies, complementary operator produces new antibodies containing excellent gene fragment in the global solution space. With small probability rules, crossover operator happens in an interval of multi generation, choosing the optimal antibody and a random antibody to exchange information about a single dimension. Crossover operator and complement operator can both be conducive to jumping out of optimal location. In simulation experiment, the immune memory optimization algorithm based on the non-genetic information uses four standard test functions: Ackley function, Griewank function, Rastrigin function, and transformed Rastrigin function. In order to better compare with contrast algorithm, in the case of high dimension the values of dimension are 20 and 30, and the experiment tests the four functions to make the statistical analysis of the results. On the other hand, to further test optimal performance of the algorithm in a more global massive space, multiple random experiment is carried out in the case of dimension 100. Compared with other intelligent algorithm, the simulation experiment with standard test functions of high dimension indicates that the new algorithms are superior in convergence speed, convergence precision and robustness comparison algorithm. In addition, the simulation results in the super high dimension show that the new algorithm has the global searching ability in high-dimensional solution space.