Multi-objective Setting Research Articles

Dynamic preference inference network: Improving sample efficiency for multi-objective reinforcement learning by preference estimation

Multi-objective reinforcement learning (MORL) addresses the challenge of optimizing policies in environments with multiple conflicting objectives. Traditional approaches often rely on scalar utility functions, which require predefined preference weights, limiting their adaptability and efficiency. To overcome this, we propose the Dynamic Preference Inference Network (DPIN), a novel method designed to enhance sample efficiency by dynamically estimating the trajectory decision preference of the agent. DPIN leverages a neural network to predict the most favorable preference distribution for each trajectory, enabling more effective policy updates and improving overall performance in complex MORL tasks. Extensive experiments in various benchmark environments demonstrate that DPIN significantly outperforms existing state-of-the-art methods, achieving higher scalarized returns and hypervolume. Our findings highlight DPIN’s ability to adapt to varying preferences, reduce sample complexity, and provide robust solutions in multi-objective settings.

Physics-informed Reinforcement Learning optimization of PWR core loading pattern

The nuclear sector represents the largest single source of carbon-free energy in the United States. Nevertheless, keeping existing nuclear assets competitive on the grid is key to their ability to continue to provide dispatchable clean energy along side renewable generation. Optimizing the fuel cycle cost through the optimization of core loading patterns is one approach to addressing this lack of competitiveness. However, this optimization task involves multiple objectives and constraints, resulting in a vast number of candidate solutions that cannot be explicitly solved. While stochastic optimization (SO) methodologies are utilized by various nuclear utilities and vendors for fuel cycle reload design, manual design remains the preferred approach. To advance the state-of-the-art in core reload patterns, we have developed methods based on Deep Reinforcement Learning for single and multi-objective optimization. Previous research has laid the groundwork for these approaches and demonstrated its ability to discover high-quality patterns within a reasonable timeframe. Moreover, RL methods have shown superiority against SO in the multi-objective settings. In this paper, we introduce an innovative paradigm by incorporating multi-objective RL, interpretable AI, and physics knowledge to further improve the performance of the algorithms. When the problem increases in complexity, the classical single-objective sometimes failed to even found feasible solutions. Leveraging physic-information becomes pivotal to find high-quality and more diverse solutions faster. These findings motivate incorporation of more physics to further aid in the quest of surpassing human intelligence. Future work will focus on scaling this method and addressing fuel performance and control rod limits to demonstrate that these new designs can be considered for a real reactor.

Multi-Objective set point optimization control of grate cooler considering energy efficiency evaluation

The grate cooler is a key piece of equipment for cooling cement clinker, whose parameter adjustment has a notable influence on energy consumption and equipment operation. This article proposes a multi-objective set-point optimization control method for the grate cooler control system, which takes into account energy efficiency evaluation. Based on the energy efficiency analysis of the grate cooler, the secondary air temperature, tertiary air temperature, and outlet clinker temperature are used as energy efficiency evaluation indicators, and a grate cooler energy efficiency evaluation model is established to construct a multi-objective function. After that, the grate pressure set point and fan current set point are optimized by utilizing the MOEA/D algorithm, and the predictive control method is combined to perform rolling optimization of the control parameters. Eventually, a simulation experiment is conducted based on the actual production data, and the simulation results demonstrate the feasibility and effectiveness of the optimized control method.

ERGO-II: An Improved Bayesian Optimization Technique for Robust Design With Multiple Objectives, Failed Evaluations, and Stochastic Parameters

Abstract In this work, the efficient robust global optimization (ERGO) method is revisited with the aim of enhancing and expanding its existing capabilities. The original objective of ERGO was to address the computational challenges associated with optimization-under-uncertainty through the use of Bayesian optimization (BO). ERGO tackles robust optimization problems which are characterized by sensitivity in the objective function due to stochasticity in the design space. It does this by concurrently minimizing the mean and variance of the objective in a multi-objective setting. To handle the computational complexity arising from the uncertainty propagation, ERGO exploits the analytical expression of the surrogate model underlying BO. In this study, ERGO is extended to accommodate multiple objectives, incorporate an improved predictive error estimation approach, investigate the treatment of failed function evaluations, and explore the handling of stochastic parameters next to stochastic design variables. To evaluate the effectiveness of these improvements, the enhanced ERGO scheme is compared with the original method using an analytical test problem with varying dimensionality. Additionally, the novel optimization technique is applied to an aerodynamic design problem to validate its performance.

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

We derive efficient algorithms to compute weakly Pareto optimal solutions for smooth, convex and unconstrained multiobjective optimization problems in general Hilbert spaces. To this end, we define a novel inertial gradient-like dynamical system in the multiobjective setting, which trajectories converge weakly to Pareto optimal solutions. Discretization of this system yields an inertial multiobjective algorithm which generates sequences that converge weakly to Pareto optimal solutions. We employ Nesterov acceleration to define an algorithm with an improved convergence rate compared to the plain multiobjective steepest descent method (Algorithm 1). A further improvement in terms of efficiency is achieved by avoiding the solution of a quadratic subproblem to compute a common step direction for all objective functions, which is usually required in first-order methods. Using a different discretization of our inertial gradient-like dynamical system, we obtain an accelerated multiobjective gradient method that does not require the solution of a subproblem in each step (Algorithm 2). While this algorithm does not converge in general, it yields good results on test problems while being faster than standard steepest descent.

An adaptive multi-objective optimal forecast combination and its application for predicting intermittent demand

While time series forecasting models are generally trained by optimising certain forms of error, the end-user’s forecasting needs in a multi-objective setting can be broader, and often mutually conflicting. A production manager may prioritise high product fill rates and low average inventory resulting from a forecast over just low error. The conflict among multiple objectives is notably worrisome in intermittent demand forecasting, where error-minimising approaches can devalue the practitioner’s objectives. To address such forecasting problems, we propose an Adaptive Multi-objective Optimal Combination (AMOC) of forecasts which incorporates the end-user’s preferences across multiple objectives. We demonstrate the use of AMOC in a real-life application of intermittent demand forecasting for optimising four distinct inventory management objectives using five specialised forecasting methods across single-period and multi-period inventory handling scenarios. Additionally, we conduct a comprehensive experiment on a subset of M5 competition data to exhibit the robustness of the AMOC using 13 diverse forecasting methods and four statistical objectives.

Decomposition Is All You Need: Single-Objective to Multi-Objective Optimization towards Artificial General Intelligence

As a new abstract computational model in evolutionary transfer optimization (ETO), single-objective to multi-objective optimization (SMO) is conducted at the macroscopic level rather than the intermediate level for specific algorithms or the microscopic level for specific operators; this method aims to develop systems with a profound grasp of evolutionary dynamic and learning mechanism similar to human intelligence via a “decomposition” style (in the abstract of the well-known “Transformer” article “Attention is All You Need”, they use “attention” instead). To the best of our knowledge, it is the first work of SMO for discrete cases because we extend our conference paper and inherit its originality status. In this paper, by implementing the abstract SMO in specialized memetic algorithms, key knowledge from single-objective problems/tasks to the multi-objective core problem/task can be transferred or “gathered” for permutation flow shop scheduling problems, which will reduce the notorious complexity in combinatorial spaces for multi-objective settings in a straight method; this is because single-objective tasks are easier to complete than their multi-objective versions. Extensive experimental studies and theoretical results on benchmarks (1) emphasize our decomposition root in mathematical programming, such as Lagrangian relaxation and column generation; (2) provide two “where to go” strategies for both SMO and ETO; and (3) contribute to the mission of building safe and beneficial artificial general intelligence for manufacturing via evolutionary computation.

RETRACTED ARTICLE: Effectiveness of Mixed Fuzzy Time Window Multi-objective Allocation in E-Commerce Logistics Distribution Path

The study of logistics distribution network under e-commerce environment is conducive to the establishment of efficient logistics distribution system, but also to promote the further development of e-commerce and improve social benefits of great significance. This study considers multiple fuzzy factors and introduces a customer fuzzy time window with variable coefficients, establishes a multi-objective set allocation integrated multi-level location path planning model, and proposes an archive type multi-objective simulated annealing improvement algorithm based on master–slave parallel framework embedded taboo search to solve the model. Tabu search and large-scale neighborhood algorithm are used to solve the initial solutions of the first level network and the second level network respectively, and archival reception criterion is introduced to deal with the multi-objective problem. The results of the proposed algorithm for the two-level site-routing problem are less than 6% different from the internationally known optimal solution. The master–slave parallel computing framework improves the efficiency of the algorithm by about 6.38%. The experimental results prove the effectiveness and necessity of the improved optimization. In addition, this study simulates the site-routing problem model constructed by the study by extending the data of standard examples. The experimental results prove the correctness and reference significance of the multilevel site-routing problem model with multiple fuzzy factors.

Enhancing Branch-and-Bound for Multiobjective 0-1 Programming

In the biobjective branch-and-bound literature, a key ingredient is objective branching, that is, to create smaller and disjoint subproblems in the objective space, obtained from the partial dominance of the lower bound set by the upper bound set. When considering three or more objective functions, however, applying objective branching becomes more complex, and its benefit has so far been unclear. In this paper, we investigate several ingredients that allow us to better exploit objective branching in a multiobjective setting. We extend the idea of probing to multiple objectives for the 0-1 case, enhance it in several ways, and show that, when coupled with objective branching, it results in significant speedups in terms of CPU times. We also show how to adapt it to the general integer case. Furthermore, we investigate cut generation based on the objective branching constraints. Besides, we generalize the best bound idea for node selection to multiple objectives, and we show that the proposed rules outperform, in the multiobjective literature, the commonly employed depth-first and breadth-first strategies. We also analyze problem-specific branching rules. We test the proposed ideas on available benchmark instances for three problem classes with three and four objectives, namely, the capacitated facility location problem, the uncapacitated facility location problem, and the knapsack problem. Our enhanced multiobjective branch-and-bound algorithm outperforms the best existing branch-and-bound–based approach and is the first to obtain competitive and even slightly better results than a state-of-the-art objective space search method on a subset of the problem classes. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms – Discrete. Funding: This work was supported in whole, or in part, by the Austrian Science Fund [Grant P 31366]. Supplemental Material: The online supplement is available at https://doi.org/10.1287/ijoc.2022.0299 .

A Decomposition Algorithm for Single and Multiobjective Integrated Market Selection and Production Planning

We study an integrated market selection and production planning problem. There is a set of markets with deterministic demand, and each market has a certain revenue that is obtained if the market’s demand is satisfied throughout a planning horizon. The demand is satisfied with a production scheme that has a lot-sizing structure. The problem is to decide on which markets’ demand to satisfy and plan the production simultaneously. We consider both single and multiobjective settings. The single objective problem maximizes the profit, whereas the multiobjective problem includes the maximization of the revenue and the minimization of the production cost objectives. We develop a decomposition-based exact solution algorithm for the single objective setting and show how it can be used in a proposed three-phase algorithm for the multiobjective setting. The master problem chooses a subset of markets, and the subproblem calculates an optimal production plan to satisfy the selected markets’ demand. We investigate the subproblem from a cooperative game theory perspective to devise cuts and strengthen them based on lifting. We also propose a set of valid inequalities and preprocessing rules to improve the proposed algorithm. We test the efficacy of our solution method over a suite of problem instances and show that our algorithm substantially decreases solution times for all problem instances. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms – Discrete. Funding: This work was supported by TUBITAK [Grant 1059B191801782].

Co-evolution of neural architectures and features for stock market forecasting: A multi-objective decision perspective

In a multi-objective setting, a portfolio manager’s highly consequential decisions can benefit from assessing alternative forecasting models of stock index movement. The present investigation proposes a new approach to identify a set of non-dominated neural network models for further selection by the decision-maker. A new co-evolution approach is proposed to simultaneously select the features and topology of neural networks (collectively referred to as neural architecture), where the features are viewed from a topological perspective as input neurons. Further, the co-evolution is posed as a multi-criteria problem to evolve sparse and efficacious neural architectures. The well-known dominance and decomposition based multi-objective evolutionary algorithms are augmented with a non-geometric crossover operator to diversify and balance the search for neural architectures across conflicting criteria. Moreover, the co-evolution is augmented to accommodate the data-based implications of distinct market behaviors prior to and during the ongoing COVID-19 pandemic. A detailed comparative evaluation is carried out with the conventional sequential approach of feature selection followed by neural topology design, as well as a scalarized co-evolution approach. The results on three market indices (NASDAQ, NYSE, and S&P500) in pre- and peri-COVID time windows convincingly demonstrate that the proposed co-evolution approach can evolve a set of non-dominated neural forecasting models with better generalization capabilities.

Monte Carlo tree search algorithms for risk-aware and multi-objective reinforcement learning

In many risk-aware and multi-objective reinforcement learning settings, the utility of the user is derived from a single execution of a policy. In these settings, making decisions based on the average future returns is not suitable. For example, in a medical setting a patient may only have one opportunity to treat their illness. Making decisions using just the expected future returns–known in reinforcement learning as the value–cannot account for the potential range of adverse or positive outcomes a decision may have. Therefore, we should use the distribution over expected future returns differently to represent the critical information that the agent requires at decision time by taking both the future and accrued returns into consideration. In this paper, we propose two novel Monte Carlo tree search algorithms. Firstly, we present a Monte Carlo tree search algorithm that can compute policies for nonlinear utility functions (NLU-MCTS) by optimising the utility of the different possible returns attainable from individual policy executions, resulting in good policies for both risk-aware and multi-objective settings. Secondly, we propose a distributional Monte Carlo tree search algorithm (DMCTS) which extends NLU-MCTS. DMCTS computes an approximate posterior distribution over the utility of the returns, and utilises Thompson sampling during planning to compute policies in risk-aware and multi-objective settings. Both algorithms outperform the state-of-the-art in multi-objective reinforcement learning for the expected utility of the returns.

A Method Converting Multi-Properties Objective Reachability Problems to Multi-Objective Sets Reachability Problems over FKS

In this paper, we focus on the symmetrical relationship between multi-properties objective reachability problems and multi-objective sets reachability problems over FKS based on Zadeh logic. First, we give the formal definitions of those two problems. Then, we study their relationships and find that a multi-properties objective reachability problem and a special case of multi-objective sets reachability problems have symmetry. Finally, we give a polynomial time algorithm based on this symmetry to convert a multi-properties objective reachability problem to a multi-objective sets reachability problem. In addition, an illustrative example is listed to express some possible application methods based on our work.

Socially CompliAnt Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation

Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a “socially compliant” manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human-populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human-robot coexistence. To address this challenge, imitation learning is a promising framework, since it is easier for humans to demonstrate the task of social navigation rather than to formulate reward functions that accurately capture the complex multi-objective setting of social navigation. The use of imitation learning and inverse reinforcement learning to social navigation for mobile robots, however, is currently hindered by a lack of large-scale datasets that capture socially compliant robot navigation demonstrations in the wild. To fill this gap, we introduce Socially CompliAnt Navigation Dataset ( SCAND )–a large-scale, first-person-view dataset of socially compliant navigation demonstrations. Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human tele-operated driving demonstrations that comprises multi-modal data streams including 3D lidar, joystick commands, odometry, visual and inertial information, collected on two morphologically different mobile robots–a Boston Dynamics Spot and a Clearpath Jackal–by four different human demonstrators in both indoor and outdoor environments. We additionally perform preliminary analysis and validation through real-world robot experiments and show that navigation policies learned by imitation learning on SCAND generate socially compliant behaviors.

An integrated container terminal scheduling problem with different‐berth sizes via multiobjective hydrologic cycle optimization

Integrated berth and quay crane allocation problem (BQCAP) are two essential seaside operational problems in container terminal scheduling. Most existing works consider only one objective on operation and partition of quay into berths of the same lengths. In this study, BQCAP is modeled in a multiobjective setting that aims to minimize total equipment used and overall operational time and the quay is partitioned into berths of different lengths, to make the model practical in the real-world and complex quay layout setting. To solve the new BQCAP efficiently, a multiobjective hydrologic cycle optimization algorithm is devised considering problem characteristics and historical Pareto-optimal solutions. Specifically, the quay crane of the large vessel in all Pareto-optimal solutions is rearranged to increase the chance of finding a good solution. Besides, worse solutions are probabilistic retained to maintain diversity. The proposed algorithm is applied to a real-world terminal scheduling problem with different sizes from a container terminal company. Experimental results show that our algorithm generally outperforms the other well-known peer algorithms and its variants on solving BQCAP, especially in finding the Pareto-optimal solutions range.

SAMURAI: A New Asynchronous Bayesian Optimization Technique for Optimization-Under-Uncertainty

In this work, optimization-under-uncertainty (OUU) is treated by simultaneously minimizing the mean of the objective and its variance due to variability of design variables and/or parameters in a multi-objective setting, while simultaneously ensuring that the minimal probability of constraint failure is met. This allows the designer to choose its robustness level without the need to repeat the optimization as typically encountered when formulated as a single objective and ensuring that the system will not fail with a prescribed probability. To account for the computational cost that is often encountered in OUU problems, the problem is fitted in a Bayesian optimization framework. The use of surrogate modeling techniques to efficiently solve problems under uncertainty has effectively found its way in the optimization community leading to surrogate-assisted OUU schemes. The surrogates are often considered cheap-to-sample black-boxes and are sampled to obtain the desired quantities of interest. However, since the analytical formulation of the surrogates is known, the mean square predictive error of the quantities of interest can be derived. To obtain these quantities without sampling, an analytical uncertainty propagation and reliability analysis through the surrogate is presented. The multi-objective Bayesian optimization framework and the analytical uncertainty propagation and reliability analysis are linked together through the formulation of the reliability-based robust expected improvement. To further enhance the efficiency of the approach, the Bayesian optimization method is solved in an asynchronous manner. In doing so the novel Surrogate-assisted Asynchronous Multi-objective optimization under Uncertainty framework for Robust and reliable solutions to Applications in Industry (SAMURAI) scheme is defined. The method is applied to a number of case studies and the design of a low- airfoil for blended-wing–bodies, which proves the effectiveness of the novel methodology.

Optimization of Regional Industrial Structure Based on Multiobjective Optimization and Fuzzy Set

With the development of China’s economy, it is required to change the mode of economic growth, from extensive growth mode to intensive growth mode. Therefore, the research on industrial structure is of great significance. This study proposes a regional industrial structure optimization method based on multiobjective optimization and fuzzy set. Firstly, the multiobjective industrial structure evaluation model is constructed, then the evaluation model based on improved fuzzy industrial structure is proposed, and finally the application effect of improved fuzzy industrial structure evaluation model is analyzed. The results show that the first generation iteration speed of the improved fuzzy industrial structure evaluation model is fast, but it is slow in the second to eighth iterations. Compared with the traditional square method, the convergence of the fuzzy structure is improved to a certain extent. The improved evaluation model has also been improved. This is better than other algorithms. Using K-means and Manhattan distance to initialize the clustering method, the results are relatively stable. In terms of running time, Manhattan distance initialization method and K-means clustering method have less iterations and short time-consuming. Therefore, when the data sample is large, the application of K-means clustering algorithm will be more effective.

Design of a heuristic algorithm for the generalized multi-objective set covering problem

Design of a heuristic algorithm for the generalized multi-objective set covering problem

A multi-objective evolutionary algorithm based on niche selection in solving irregular Pareto fronts

The key characteristic of multi-objective evolutionary algorithm is that it can find a good approximate multi-objective optimal solution set when solving multi-objective optimization problems(MOPs). However, most multi-objective evolutionary algorithms perform well on regular multi-objective optimization problems, but their performance on irregular fronts deteriorates. In order to remedy this issue, this paper studies the existing algorithms and proposes a multi-objective evolutionary based on niche selection to deal with irregular Pareto fronts. In this paper, the crowding degree is calculated by the niche method in the process of selecting parents when the non-dominated solutions converge to the first front, which improves the the quality of offspring solutions and which is beneficial to local search. In addition, niche selection is adopted into the process of environmental selection through considering the number and the location of the individuals in its niche radius, which improve the diversity of population. Finally, experimental results on 23 benchmark problems including MaF and IMOP show that the proposed algorithm exhibits better performance than the compared MOEAs.

Exact and metaheuristic algorithms for the vehicle routing problem with a factory-in-a-box in multi-objective settings

Emergencies, such as pandemics (e.g., COVID-19), warrant urgent production and distribution of goods under disrupted supply chain conditions. An innovative logistics solution to meet the urgent demand during emergencies could be the factory-in-a-box manufacturing concept. The factory-in-a-box manufacturing concept deploys vehicles to transport containers that are used to install production modules (i.e., factories). The vehicles travel to customer locations and perform on-site production. Factory-in-a-box supply chain optimization is associated with a wide array of decisions. This study focuses on selection of vehicles for factory-in-a-box manufacturing and decisions regarding the optimal routes within the supply chain consisting of a depot, suppliers, manufacturers, and customers. Moreover, in order to contrast the options of factory-in-a-box manufacturing with those of conventional manufacturing, the location of the final production is determined for each customer (i.e., factory-in-a-box manufacturing with production at the customer location or conventional manufacturing with production at the manufacturer locations). A novel multi-objective optimization model is presented for the vehicle routing problem with a factory-in-a-box that aims to minimize the total cost associated with traversing the edges of the network and the total cost associated with visiting the nodes of the network. A customized multi-objective hybrid metaheuristic solution algorithm that directly considers problem-specific properties is designed as a solution approach. A case study is performed for a vaccination project involving factory-in-a-box manufacturing along with conventional manufacturing. The case study reveals that the developed solution method outperforms the ε-constraint method, which is a classical exact optimization method for multi-objective optimization problems, and several well-known metaheuristics.