Single-loop methods for bilevel parameter learning in inverse imaging

A Survey on Mixed-Integer Programming Techniques in Bilevel Optimization

Population-based optimization algorithms, such as evolutionary algorithms, have enjoyed a lot of attention in the past three decades in solving challenging search and optimization problems. In this chapter, we discuss recent population-based evolutionary algorithms for solving different types of bilevel optimization problems, as they pose numerous challenges to an optimization algorithm. Evolutionary bilevel optimization (EBO) algorithms are gaining attention due to their flexibility, implicit parallelism, and ability to customize for specific problem solving tasks. Starting with surrogate-based single-objective bilevel optimization problems, we discuss how EBO methods are designed for solving multi-objective bilevel problems. They show promise for handling various practicalities associated with bilevel problem solving. The chapter concludes with results on an agro-economic bilevel problem. The chapter also presents a number of challenging single and multi-objective bilevel optimization test problems, which should encourage further development of more efficient bilevel optimization algorithms.

Bilevel optimization is a very active field of applied mathematics. The main reason is that bilevel optimization problems can serve as a powerful tool for modeling hierarchical decision making processes. This ability, however, also makes the resulting problems challenging to solve—both in theory and practice. Fortunately, there have been significant algorithmic advances in the field of bilevel optimization so that we can solve much larger and also more complicated problems today compared to what was possible to solve two decades ago. This results in more and more challenging bilevel problems that researchers try to solve today. This survey gives a detailed overview of one of these more challenging classes of bilevel problems: bilevel optimization under uncertainty. We review the classic ways of addressing uncertainties in bilevel optimization using stochastic or robust techniques. Moreover, we highlight that the sources of uncertainty in bilevel optimization are much richer than for usual, i.e., single-level, problems since not only the problem’s data can be uncertain but also the (observation of the) decisions of the two players can be subject to uncertainty. We thus also review the field of bilevel optimization under limited observability, the area of problems considering only near-optimal decisions, and discuss intermediate solution concepts between the optimistic and pessimistic cases. Finally, we also review the rich literature on applications studied using uncertain bilevel problems such as in energy, for interdiction games and security applications, in management sciences, and networks.

A Co-evolutionary Decomposition-based Chemical Reaction Algorithm for Bi-level Combinatorial Optimization Problems

Presolving linear bilevel optimization problems

Bilevel optimization of block compressive sensing with perceptually nonlocal similarity

While bilevel optimization is gaining prominence across various domains, the field lacks standardized tools for generating test problems that can effectively evaluate and guide the development of efficient solution algorithms. We define the term "trivial bilevel optimization problem" as a bilevel problem whose high-point relaxation solution is also feasible. These easy-to-solve problems frequently arise in na�ve implementations of random bilevel optimization problem generators, significantly impacting the evaluation of bilevel solution algorithms. However, this problem has not been addressed in the literature to our best knowledge. This work introduces a non-trivial mixed-integer bilevel optimization problem generator, NT-BMIPGen, and a problem library, designed to eliminate the generation of trivial bilevel problems. Through analysis of the bilevel problem structure, we identify key factors contributing to problem triviality. Particularly, the upper to lower variable ratios and the number of decoupled lower-level continuous variables increasing two-level competition significantly impact problem triviality. The proposed generator allows users to customize problem structures while ensuring non-triviality by checking the bilevel feasibility of highpoint relaxation. Additionally, we present a library of 105 non-trivial problems to support algorithm development and testing. This work advances the understanding of bilevel problem complexity and provides essential tools for the systematic evaluation of bilevel optimization algorithms.

Author's Preface to the English Edition. Introduction. Part I: Adjoint Equations and Perturbation Theory. 1. Main and Adjoint Equations. Perturbation Theory. 2. Simple Main and Adjoint Equations in Mathematical Physics. 3. Nonlinear Equations. 4. Inverse Problems and Adjoint Equations. Part II: Problems of Environment and Optimization Methods on the Basis of Adjoint Equations. 5. Analysis of Mathematical Models in Environmental Problems. 6. Adjoint Equations, Optimization. 7. Adjoint Equations and Models of General Circulation of Atmosphere and Ocean. 8. Adjoint Equations in Data Processing Problems. Appendix I: Splitting Methods in the Solution of Global Problems. Appendix II: Difference Analogue of Nonstationary Heat Diffusion Equation in Atmosphere and Ocean. Bibliography. Index.

In recent years, research in the field of bilevel optimization has gathered pace and it is increasingly being used to solve problems in engineering, logistics, economics, transportation etc. Rapid increase in the size and complexity of the problems emerging from these domains has prompted active interest in the design of efficient algorithms for bilevel optimization. While Memetic Algorithms (MAs) have been quite successful in solving single level optimization problems, there have been very few studies exploring their application in bilevel problems. MAs essentially attempt to combine advantages of global and local search strategies to locate optimum solutions with low computational cost (function evaluations). In this paper, we present a new nested approach for solving bilevel optimization problems. The presented approach uses memetic algorithm at the upper level, while a global or a local search method is used in the lower level during various phases of the search. The performance of the proposed approach is compared with two established approaches, NBLEA and BLEAQ, using SMD benchmark problem set. The numerical experiments demonstrate the benefits of the proposed approach both in terms of accuracy and computational cost, establishing its potential for solving bilevel optimization problems.

A study of mixed discrete bilevel programs using semidefinite and semi-infinite programming

Real-life problems including transportation, planning and management often involve several decision makers whose actions depend on the interaction between each other. When involving two decision makers, such problems are classified as bi-level optimization problems. In terms of mathematical programming, a bi-level program can be described as two nested problems where the second decision problem is part of the first problem's constraints. Bi-level problems are NP-hard even if the two levels are linear. Since each solution implies the resolution of the second level to optimality, efficient algorithms at the first level are mandatory. In this work we propose BOBP, a Bayesian Optimization algorithm to solve Bi-level Problems, in order to limit the number of evaluations at the first level by extracting knowledge from the solutions which have been solved at the second level. Bayesian optimization for hyper parameter tuning has been intensively used in supervised learning (e.g., neural networks). Indeed, hyper parameter tuning problems can be considered as bi-level optimization problems where two levels of optimization are involved as well. The advantage of the bayesian approach to tackle multi-level problems over the BLEAQ algorithm, which is a reference in evolutionary bi-level optimization, is empirically demonstrated on a set of bi-level benchmarks.

Steady-state optimization of biochemical systems by bi-level programming

Optimization problems involving a leader decision maker with multiple follower decision makers are referred to as bi-level multi-follower programming problems (BMF-P). In this work, we present novel algorithms for the exact and global solution of two classes of bi-level programming problems, namely (i) bi-level multi-follower mixed-integer linear programming problems (BMF-MILP) and (ii) bi-level multi-follower mixed-integer convex quadratic programming problems (BMF-MIQP) containing both integer and continuous variables at all optimization levels. Based on multi-parametric programming theory, the main idea is to recast the lower level, follower, problems as multi-parametric programming problems, in which the optimization variables of the upper level, leader, problem are considered as parameters for the lower level problems. The resulting exact multi-parametric mixed-integer linear or quadratic solutions are then substituted into the upper level problem, which can be solved as a set of single-level, independent, deterministic mixed-integer optimization problems. The proposed algorithm is applied for the solution of the challenging problem of planning and scheduling integration.

t. This paper surveys some of the work our group has done in electrical impedance tomography.

It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to varepsilon -feasibility regarding its nonlinear constraints for an arbitrarily small but positive varepsilon , the obtained bilevel solution as well as its objective value may be arbitrarily far away from the actual bilevel solution and its actual objective value. This result even holds for bilevel problems for which the nonconvex lower level is uniquely solvable, for which the strict complementarity condition holds, for which the feasible set is convex, and for which Slater’s constraint qualification is satisfied for all feasible upper-level decisions. Since the consideration of varepsilon -feasibility cannot be avoided when solving nonconvex problems to global optimality, our result shows that computational bilevel optimization with continuous and nonconvex lower levels needs to be done with great care. Finally, we illustrate that the nonlinearities in the lower level are the key reason for the observed bad behavior by showing that linear bilevel problems behave much better at least on the level of feasible solutions.