True Pareto Set Research Articles

A dual-sampling based evolutionary algorithm for large-scale multi-objective optimization

The vast search space in large-scale multi-objective optimization represents a significant challenge for evolutionary algorithms to converge towards the Pareto Front. As an effective search strategy, direction-guided sampling technique could improve the search efficiency by exploring along the approximated directions to approach the Pareto set. However, the approximated directions may fail to interact with the true Pareto set and result in inefficient search. To address this issue, a dual-sampling method is proposed in this paper. In addition to the samples along the directions approximated by direction-guided sampling, fuzzy Gaussian sampling is applied to adjust the search direction and generate more accurate and evenly distributed solutions. Moreover, a convergence-and-diversity-based mating selection is introduced to balance the exploration and exploitation. The experiments on 72 test benchmarks with bi- and tri-objectives and 500 to 5000 decision variables show the superiority of the proposed algorithm compare with the state-of-the-art algorithms.

Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization

We consider the problem of black-box multi-objective optimization (MOO) using expensive function evaluations (also referred to as experiments), where the goal is to approximate the true Pareto set of solutions by minimizing the total resource cost of experiments. For example, in hardware design optimization, we need to find the designs that trade-off performance, energy, and area overhead using expensive computational simulations. The key challenge is to select the sequence of experiments to uncover high-quality solutions using minimal resources. In this paper, we propose a general framework for solving MOO problems based on the principle of output space entropy (OSE) search: select the experiment that maximizes the information gained per unit resource cost about the true Pareto front. We appropriately instantiate the principle of OSE search to derive efficient algorithms for the following four MOO problem settings: 1) The most basic single-fidelity setting, where experiments are expensive and accurate; 2) Handling black-box constraints which cannot be evaluated without performing experiments; 3) The discrete multi-fidelity setting, where experiments can vary in the amount of resources consumed and their evaluation accuracy; and 4) The continuous-fidelity setting, where continuous function approximations result in a huge space of experiments. Experiments on diverse synthetic and real-world benchmarks show that our OSE search based algorithms improve over state-of-the-art methods in terms of both computational-efficiency and accuracy of MOO solutions.

Guided Manta Ray foraging optimization using epsilon dominance for multi-objective optimization in engineering design

In recent decades, metaheuristics have proven their effectiveness in solving large-scale real-world problems with multiple objectives. However, we still need to design robust algorithms capable of converging and approximating efficiently the true Pareto set. In this paper, we extend the recently Manta Ray foraging optimization (MOMRFO) to the multiobjective case. MOMRFO uses a population archive to store the non-dominated solutions generated so far by the exploration process. The leader’s solutions are selected from the population archive to guide the Manta Rays population towards promising search regions. We use crowding distance and ε-dominance to provide a good compromise between diversity and convergence of the obtained potential Pareto set.The proposed algorithm is validated on five bi-objective test functions, seven three objective test functions, and is applied to structural design problems such as four-bar truss design, speed reduced design, welded beam design, and disk brake design. The algorithm is compared with four well-known multi-objective meta-heuristics. The experimental results show that the MOMRFO algorithm outperforms against the selected multiobjective meta-heuristics by providing better convergence behaviour with a better diversity of solutions.

Exact and efficient reliability and performance optimization of synchronous task graphs

SRAM-based FPGAs have found many applications in modern computer systems. In these systems, high-performance computing applications are executed as task graphs in which reliability and performance are crucial constraints. In this paper, an exact method is presented to efficiently optimize the reliability and performance of synchronous task graphs running on SRAM-based FPGAs in harsh environments. Solving this optimization problem leads to the generation of a true Pareto set of Fault Tolerance (FT) techniques. Each solution of this set determines FT techniques of the tasks and leads to specific reliability and makespan. Thus, this solution set trades off between reliability and makespan, and one of the solutions that best meets system requirements can be applied to the running tasks to optimize the reliability and performance. The proposed technique is novel as it obtains the true Pareto set of FT techniques of the whole task graph by partitioning the task graph into its segments, optimizing different segments separately, and joining the obtained solutions. This partitioning strategy leads to reduce the computation time significantly. In this paper, it is mathematically proved that the proposed partitioning strategy generates global optima from the local ones without losing any optimal solutions. The experiments show that the proposed technique improves the MTTF of real-world and random task graphs by 46.30% on average without any negative effects on the performance. Then, the efficiency of the computation time of the proposed technique is demonstrated by conducting several experiments on small-, medium-, and large-size synchronous task graphs and comparing the results with other exact and evolutionary optimization methods. Finally, supplementary experiments in dynamic environments show that the proposed technique outperforms adaptive state-of-the-art FT techniques in terms of reliability and makespan improvement.

Multi-Fidelity Multi-Objective Bayesian Optimization: An Output Space Entropy Search Approach

We study the novel problem of blackbox optimization of multiple objectives via multi-fidelity function evaluations that vary in the amount of resources consumed and their accuracy. The overall goal is to appromixate the true Pareto set of solutions by minimizing the resources consumed for function evaluations. For example, in power system design optimization, we need to find designs that trade-off cost, size, efficiency, and thermal tolerance using multi-fidelity simulators for design evaluations. In this paper, we propose a novel approach referred as Multi-Fidelity Output Space Entropy Search for Multi-objective Optimization (MF-OSEMO) to solve this problem. The key idea is to select the sequence of candidate input and fidelity-vector pairs that maximize the information gained about the true Pareto front per unit resource cost. Our experiments on several synthetic and real-world benchmark problems show that MF-OSEMO, with both approximations, significantly improves over the state-of-the-art single-fidelity algorithms for multi-objective optimization.
 Please note: A corrigendum was submitted for this paper on 24 September 2020.

Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimization

We consider the problem of multi-objective (MO) blackbox optimization using expensive function evaluations, where the goal is to approximate the true Pareto set of solutions while minimizing the number of function evaluations. For example, in hardware design optimization, we need to find the designs that trade-off performance, energy, and area overhead using expensive simulations. We propose a novel uncertainty-aware search framework referred to as USeMO to efficiently select the sequence of inputs for evaluation to solve this problem. The selection method of USeMO consists of solving a cheap MO optimization problem via surrogate models of the true functions to identify the most promising candidates and picking the best candidate based on a measure of uncertainty. We also provide theoretical analysis to characterize the efficacy of our approach. Our experiments on several synthetic and six diverse real-world benchmark problems show that USeMO consistently outperforms the state-of-the-art algorithms.

Techno-Economic Optimization of an Off-Grid Solar/Wind/Battery Hybrid System with a Novel Multi-Objective Differential Evolution Algorithm

Techno-economic optimization of a standalone solar/wind/battery hybrid system located in Xining, China, is the focus of this paper, and reliable and economic indicators are simultaneously employed to address the problem. To obtain a more precise Pareto set, a novel multi-objective differential evolution algorithm is proposed, where differential evolution with a parameter-adaptive mechanism is applied in the decomposition framework. The algorithm effectiveness is verified by performance comparisons on the benchmark test problems with two reference algorithms: a non-dominated sorting genetic algorithm and a multi-objective evolution algorithm based on decomposition. The applicability of the proposed algorithm for the capacity-optimization problem is also validated by comparisons with the same reference algorithms above, where the true Pareto set of the problem is approximated by combining of the three algorithms through the non-dominant relationship. The results show the proposed algorithm has the lowest inverted generational distance indicator and provides 85% of the true Pareto set. Analyses of the Pareto frontier show that it can produce significant economic benefits by reducing reliability requirements appropriately when loss of power supply probability is less than 0.5%. Furthermore, sensitivity analyses of the initial capital of wind turbine, photovoltaic panel and battery system are performed, and the results show that photovoltaic panel’s initial capital has the greatest impact on levelized cost of electricity, while the initial capital of wind turbine has the least impact.

A Convergence Indicator for Multi-Objective Optimisation Algorithms

The algorithms of multi-objective optimisation had a relative growth in the last years. Thereby, it's requires some way of comparing the results of these. In this sense, performance measures play a key role. In general, it's considered some properties of these algorithms such as capacity, convergence, diversity or convergence-diversity. There are some known measures such as generational distance (GD), inverted generational distance (IGD), hypervolume (HV), Spread($\Delta$), Averaged Hausdorff distance ($\Delta_p$), R2-indicator, among others. In this paper, we focuses on proposing a new indicator to measure convergence based on the traditional formula for Shannon entropy. The main features about this measure are: 1) It does not require tho know the true Pareto set and 2) Medium computational cost when compared with Hypervolume.

Optimal Computing Budget Allocation to Select the Nondominated Systems—A Large Deviations Perspective

We consider the optimal computing budget allocation problem to select the nondominated systems on finite sets under a stochastic multi-objective ranking and selection setting. This problem has been addressed in the settings of correct selection guarantee when all the systems have normally distributed objectives with no correlation within and between solutions. We revisit this problem from a large deviation perspective and present a mathematically robust formulation that maximizes the lower bound of the rate function of the probability of false selection ( $P(\text{FS})$ ) defined as the probability of not identifying the true Pareto set. The proposed formulation allows general distributions and explicitly characterizes the sampling correlations across performance measures. Three budget allocation strategies are proposed. One of the approaches is guaranteed to attain the global optimum of the lower bound of the rate function but has high computational cost. Therefore, a heuristic to approximate the global optimal strategy is proposed to save computational resources. Finally, for the case of normally distributed objectives, a computationally efficient procedure is proposed, which adopts an iterative algorithm to find the optimal budget allocation. Numerical experiments illustrate the significant improvements of the proposed strategies over others in the existing literature in terms of the rate function of $P(\text{FS})$ .

A decomposition-based reliability and makespan optimization technique for hardware task graphs

This paper presents an approach to optimize the reliability and makespan of hardware task graphs, running on FPGA-based reconfigurable computers, in space-mission computing applications with dynamic soft error rates (SERs). Thus, with rises and falls of the SER, the presented approach dynamically generates a set of solutions that apply redundancy-based fault tolerance (FT) techniques to the running tasks. The set of solutions is generated by decomposing the task graph into multiple subgraphs, applying a multi-objective optimization algorithm to the subgraphs separately, and finally combining and filtering out the obtained solutions of the subgraphs. In this regard, a heuristic has been proposed to decompose task graphs in such a way that a high coverage of the true Pareto set is attained. The experiments show that the presented approach covers 97.37% of the true Pareto set and improves the average computation time of generating the Pareto set from 6.29 h to 81.86 ms. In addition, it outperforms the NSGA-II algorithm in terms of the Pareto set coverage and computation time. Additional experiments demonstrate the advantages of the presented approach over the state-of-the-art adaptive FT techniques in dynamic environments.

New simulation-based frameworks for multi-objective reliability-based design optimization of structures

• Two new simulation-based frameworks are presented for MORBDO problems. • Only one simulation run is needed during entire multi-objective optimization process. • The Pareto optimal set is enhanced by iterative local search of the proposed NSWSM. • WSM evaluates reliability of design candidates to conduct NSGA-II to true Pareto-set. • New simulations are suitable for complex problems with discrete and continuous variables. In the present study, two new simulation-based frameworks are proposed for multi-objective reliability-based design optimization (MORBDO). The first is based on hybrid non-dominated sorting weighted simulation method (NSWSM) in conjunction with iterative local searches that is efficient for continuous MORBDO problems. According to NSWSM, uniform samples are generated within the design space and, then, the set of feasible samples are separated. Thereafter, the non-dominated sorting operator is employed to extract the approximated Pareto front. The iterative local sample generation is then performed in order to enhance the accuracy, diversity, and increase the extent of non-dominated solutions. In the second framework, a pseudo-double loop algorithm is presented based on hybrid weighted simulation method (WSM) and the Non-dominated Sorting Genetic Algorithm II (NSGA-II) that is efficient for problems including both discrete and continuous variables. According to hybrid WSM-NSGA-II, proper non-dominated solutions are produced in each generation of NSGA-II and, subsequently, WSM evaluates the reliability level of each candidate solution until the algorithm converges to the true Pareto solutions. The valuable characteristic of presented approaches is that only one simulation run is required for WSM during entire optimization process, even if solutions for different levels of reliability be desired. Illustrative examples indicate that NSWSM with the proposed local search strategy is more efficient for small dimension continuous problems. However, WSM-NSGA-II outperforms NSWSM in terms of solutions quality and computational efficiency, specifically for discrete MORBDOs. Employing global optimizer in WSM-NSGA-II provided more accurate results with lower samples than NSWSM.

An evaluation of non-redundant objective sets based on the spatial similarity ratio

Multi-objective optimization is a challenging task in many disciplines. Although a number of algorithms have simplified this problem by degenerating redundant objectives into low-dimensional sets, there is currently no consensus method for evaluating their performance. In this paper, we propose an evaluation method that uses a spatial similarity ratio (SSR) to determine the quality of non-redundant objective sets (NRSs). We consider the reduction of all NRSs of three functions from 5D to 2D or 3D using our SSR-based method, and compare the results to those given by an inverted generational distance-based method. The results demonstrate that our method is more accurate, as it takes information from both the non-redundant and redundant objective sets into consideration. In addition, using the proposed SSR-based approach, no prior knowledge of the true Pareto set is required. Therefore, we can conclude that our SSR-based method is feasible for the assessment of non-redundant objective sets.

A Comparative Benchmark Study of using Different Multi-Objective Optimization Algorithms for Restraint System Design

Vehicle restraint system design is a difficult optimization problem to solve because (1) the nature of the problem is highly nonlinear, non-convex, noisy, and discontinuous; (2) there are large numbers of discrete and continuous design variables; (3) a design has to meet safety performance requirements for multiple crash modes simultaneously, hence there are a large number of design constraints. Based on the above knowledge of the problem, it is understandable why design of experiment (DOE) does not produce a high-percentage of feasible solutions, and it is difficult for response surface methods (RSM) to capture the landscape of the problem. Furthermore, in order to keep the restraint system more robust, the complexity of restraint system content needs to be minimized in addition to minimizing the relative risk score to achieve New Car Assessment Program (NCAP) 5-star rating. These call for identifying the most appropriate multi-objective optimization algorithm to solve this type of vehicle restraint system design problem. In this paper, several advanced multi-objective optimization algorithms are employed to solve a large-scale restraint system design problem. The accuracy to the true Pareto set and percentage of feasible solutions are employed as the performance metrics to judge how well each algorithm performs. A detailed analysis of why some algorithms perform better than others will be investigated, and the experience gained from this benchmark study will offer some guidance as to how to choose multi-objective optimization algorithms for solving future restraint system design problems. Language: en

Stochastic Optimization over a Pareto Set Associated with a Stochastic Multi-Objective Optimization Problem

We deal with the problem of minimizing the expectation of a real valued random function over the weakly Pareto or Pareto set associated with a Stochastic Multi-objective Optimization Problem, whose objectives are expectations of random functions. Assuming that the closed form of these expectations is difficult to obtain, we apply the Sample Average Approximation method in order to approach this problem. We prove that the Hausdorff---Pompeiu distance between the weakly Pareto sets associated with the Sample Average Approximation problem and the true weakly Pareto set converges to zero almost surely as the sample size goes to infinity, assuming that our Stochastic Multi-objective Optimization Problem is strictly convex. Then we show that every cluster point of any sequence of optimal solutions of the Sample Average Approximation problems is almost surely a true optimal solution. To handle also the non-convex case, we assume that the real objective to be minimized over the Pareto set depends on the expectations of the objectives of the Stochastic Optimization Problem, i.e. we optimize over the image space of the Stochastic Optimization Problem. Then, without any convexity hypothesis, we obtain the same type of results for the Pareto sets in the image spaces. Thus we show that the sequence of optimal values of the Sample Average Approximation problems converges almost surely to the true optimal value as the sample size goes to infinity.

Study of self-stopping PDMOSA and performance measure in multiobjective optimization

The paper presents improvements in simulated annealing based multiobjective algorithms and the modifications in the metrics methods used for the performance measure in multiobjective optimization. Simulated annealing (SA) based multiobjective algorithms have been made self-stopping as they do not require a total number of iterations. The performance metrics have been improved such that they can be used for the performance measure of a multiobjective algorithm and two multiobjective algorithms without a known true Pareto set. Pareto dominant based multiobjective simulated annealing with self-stopping criterion (PDMOSA-I) has been compared with existing Suppapitnarm multiobjective simulated annealing (SMOSA) in detail. The computational cost with PDMOSA-I is less than that with SMOSA. With the help of metrics methods, it has been found that the quality, extent and diversification of a Pareto set of solutions produced by PDMOSA-I is better than that produced by SMOSA.

Managing approximation models in collaborative optimization

Collaborative optimization (CO), one of the multidisciplinary design optimization techniques, has been credited with guaranteeing disciplinary autonomy while maintaining interdisciplinary compatibility due to its bi-level optimization structure. However, a few difficulties caused by certain features of its architecture have been also reported. The architecture, with discipline-level optimizations nested in a system-level optimization, leads to considerably increased computational time. In addition, numerical difficulties such as the problem of slow convergence or unexpected nonlinearity of the compatibility constraint in the system-level optimization are known weaknesses of CO. This paper proposes the use of an approximation model in place of the disciplinary optimization in the system-level optimization in order to relieve the aforementioned difficulties. The disciplinary optimization result, the optimal discrepancy function value, is modeled as a function of the interdisciplinary target variables, and design variables of the system level. However, since this approach is hindered by the peculiar form of the compatibility constraint, it is hard to exploit well-developed conventional approximation methods. In this paper, neural network classification is employed as a classifier to determine whether a design point is feasible or not. Kriging is also combined with the classification to make up for the weakness that the classification cannot estimate the degree of infeasibility. In addition, for the purpose of enhancing the accuracy of the predicted optimum, this paper also employs two approximation management frameworks for single-objective and multi-objective optimization problem in the system-level optimization. The approximation is continuously updated using the information obtained from the optimization process. This can cut down the required number of disciplinary optimizations considerably and lead to a design (or Pareto set) near to the true optimum (or true Pareto set) of the system-level optimization.