Bilevel Optimization Problem Research Articles

Accurate model reduction of large hydropower systems with associated adaptive inflow

Simulation of sizeable hydro-thermal power systems, such as Northern Europe or larger, requires several extensive simplifications and model reductions to decrease simulation time. Such reductions for hydrosystems are often called Equivalent models. Their purpose is to mimic a more detailed hydropower model while decreasing computation time. Both aspects are vital for accurate and useable simulation results. Here, different Equivalent models for hydropower have been developed together with a new function for adaptive Equivalent inflow based on local inflows to the detailed system. The models were computed via a bilevel optimization problem factoring in the novel adaptive inflow. Based on this, the new function for adaptive inflow was calculated using regression. The Equivalents have then been evaluated in a case study of hydropower systems in Northern Sweden regarding accuracy in hourly and total power generation, revenue estimation, and relative computation time. For all Equivalents, the computation time is decreased by ¿96%. Further, the Equivalents demonstrate improved performances in hourly and total power production and revenue estimations. The best hourly power difference was 9.2%, and the best revenue estimation was 5.9%. Especially notable is the low total power production difference of ¡0.5% compared to the more detailed model.

Community Battery Storage Systems Planning for Voltage Regulation in Low Voltage Distribution Systems

The regulation of the grid voltage within operational limits becomes increasingly challenging as residential photovoltaic (PV) adoption rises. Therefore, this study proposes a method for the efficient planning of multiple community battery energy storage systems (BESS) in low voltage distribution systems embedded with high residential rooftop PV units. A bi-level optimization method based on a Neural Network Optimization Algorithm is developed to regulate the voltage in grid-connected solar PV. Since BESS characteristics are crucial for the reliable operation of the distribution networks, the objective of the bi-level optimization problem is to optimally place and operate BESS collectively at a distributed system level. The charging/discharging protocol of the batteries management system is obtained utilizing linear programming that minimizes the daily voltage signal. Simulations were carried out on a modified IEEE low voltage test feeder to examine the impact of PV integration and BESS installation on the voltage profile. Experimental results show the efficacy of the proposed method in enabling the utility to determine the optimal location, capacity, and number of BESS in the distribution system to keep the network voltage magnitude within acceptable bounds. In addition, results demonstrate that the network topology, load profiles, and amount of PV power highly influence the BESS characteristics.

Bilevel aggregator-prosumers' optimization problem in real-time: A convex optimization approach

This paper proposes a Real-Time Market (RTM) platform for an aggregator and its corresponding prosumers to participate in the electricity wholesale market. The proposed energy market platform is modeled as a bilevel optimization problem where the aggregator and the prosumers are considered as self-interested agents. We present a convex optimization problem which can capture a subset of the set of global optima of the bilevel problem as its optimal solution.

Bilevel hyperparameter optimization for support vector classification: theoretical analysis and a solution method

Support vector classification (SVC) is a classical and well-performed learning method for classification problems. A regularization parameter, which significantly affects the classification performance, has to be chosen and this is usually done by the cross-validation procedure. In this paper, we reformulate the hyperparameter selection problem for support vector classification as a bilevel optimization problem in which the upper-level problem minimizes the average number of misclassified data points over all the cross-validation folds, and the lower-level problems are the l_1-loss SVC problems, with each one for each fold in T-fold cross-validation. The resulting bilevel optimization model is then converted to a mathematical program with equilibrium constraints (MPEC). To solve this MPEC, we propose a global relaxation cross-validation algorithm (GR–CV) based on the well-know Sholtes-type global relaxation method (GRM). It is proven to converge to a C-stationary point. Moreover, we prove that the MPEC-tailored version of the Mangasarian–Fromovitz constraint qualification (MFCQ), which is a key property to guarantee the convergence of the GRM, automatically holds at each feasible point of this MPEC. Extensive numerical results verify the efficiency of the proposed approach. In particular, compared with other methods, our algorithm enjoys superior generalization performance over almost all the data sets used in this paper.

A Bi-Level Programming Model for the Integrated Problem of Low Carbon Supplier Selection and Transportation

In this paper, we investigate an integrated problem of low-carbon supplier selection and transportation. The supplier selection decision depends on the location and energy consumption level of batching plants at the manufacturing stage. Meanwhile, ready-mixed concrete is allocated and delivered to construction sites by concrete mixer trucks at the transportation stage. A bi-level programming model for the integrated problem is established. The bi-level optimization problem is transformed into a single-level problem by KKT (Karush–Kuhn–Tucker) optimality conditions. In order to validate the proposed model, a case study is conducted based on real-world problems. Experimental results show that the proposed method efficiently solves the integrated problem and the model can not only reduce carbon emissions but also optimize transportation time.

Product line optimization with multiples sites

We consider the problem faced by a retail chain that must select what mutual-substitute items to display in each one of its stores to maximize revenues. The number of items cannot exceed the limit space capacity of each store. Customers purchase the one product that maximizes their utility, which depends on the product price, travel cost to the store, and reservation price, known to the retailer. The retailer can set different price markdowns at different stores and products. The retailer considers the decisions of customers, and solves a mixed-integer bilevel optimization problem, which can be formulated as a single-level optimization problem by using optimality conditions for the lower level. We propose Branch and Cut and Cut and Branch methods and include a family of valid inequalities to solve the problem. We compare the results with those of a Benders decomposition method. Our computational results show that the proposed Cut and Branch method obtains the best performance and improves the current state of the art.

Necessary optimality conditions for nonsmooth multi-objective bilevel optimization problem under the optimistic perspective

Necessary optimality conditions for nonsmooth multi-objective bilevel optimization problem under the optimistic perspective

Android malware detection as a Bi-level problem

Malware detection is still a very challenging topic in the cybersecurity field. This is mainly due to the use of obfuscation techniques. To solve this issue, researchers proposed to extract frequent API (Application Programming Interface) call sequences and then use them as behavior indicators. Several methods aiming at generating malware detection rules have been proposed with the goal to come up with a set of rules that is able to accurately detect malicious code patterns. However, the rules generation process heavily depends on the training database content which will affect the detection rate of the model when confronted to new variants of malicious patterns. In order to assess a rule’s detection accuracy, we need to execute the rule on the whole malware database which makes the detection rule quality evaluation very sensitive to the database content. To solve this issue, we suggest in this paper to consider the detection rules generation process as a BLOP (Bi-Level Optimization Problem), where a lower-level optimization task is embedded within the upper-level one. The goal of the upper-level is to generate a set of detection rules in the form of: trees of combined patterns. Those rules are able to detect not only the real patterns from the base of examples but also the artificial patterns generated by the lower-level. The lower-level aims to generate a set of artificial malicious patterns that escape the rules of the upper-level. An efficient co-evolutionary algorithm is adopted as a search engine to ensure optimization at both levels. Such an automated competition between the two levels makes our new method BMD (Bi-level Malware Detection) able to produce effective detection rules that are capable of detecting new predictable malicious behaviors in addition to existing ones. Based on the statistical analysis of the experimental results, our BMD method has shown its merits when compared to several relevant state-of-the-art malware detection techniques on different Android malware datasets.

Levenberg–Marquardt method and partial exact penalty parameter selection in bilevel optimization

We consider the optimistic bilevel optimization problem, known to have a wide range of applications in engineering, that we transform into a single-level optimization problem by means of the lower-level optimal value function reformulation. Subsequently, based on the partial calmness concept, we build an equation system, which is parameterized by the corresponding partial exact penalization parameter. We then design and analyze a Levenberg–Marquardt method to solve this parametric system of equations. Considering the fact that the selection of the partial exact penalization parameter is a critical issue when numerically solving a bilevel optimization problem by means of the value function reformulation, we conduct a careful experimental study to this effect, in the context of the Levenberg–Marquardt method, while using the Bilevel Optimization LIBrary (BOLIB) series of test problems. This study enables the construction of some safeguarding mechanisms for practical robust convergence of the method and can also serve as base for the selection of the penalty parameter for other bilevel optimization algorithms. We also compare the Levenberg–Marquardt method introduced in this paper to other existing algorithms of similar nature.

Cross Entropy Covariance Matrix Adaptation Evolution Strategy for Solving the Bi-Level Bidding Optimization Problem in Local Energy Markets

The increased penetration of renewables in power distribution networks has motivated significant interest in local energy systems. One of the main goals of local energy markets is to promote the participation of small consumers in energy transactions. Such transactions in local energy markets can be modeled as a bi-level optimization problem in which players (e.g., consumers, prosumers, or producers) at the upper level try to maximize their profits, whereas a market mechanism at the lower level maximizes the energy transacted. However, the strategic bidding in local energy markets is a complex NP-hard problem, due to its inherently nonlinear and discontinued characteristics. Thus, this article proposes the application of a hybridized Cross Entropy Covariance Matrix Adaptation Evolution Strategy (CE-CMAES) to tackle such a complex bi-level problem. The proposed CE-CMAES uses cross entropy for global exploration of search space and covariance matrix adaptation evolution strategy for local exploitation. The CE-CMAES prevents premature convergence while efficiently exploring the search space, thanks to its adaptive step-size mechanism. The performance of the algorithm is tested through simulation in a practical distribution system with renewable energy penetration. The comparative analysis shows that CE-CMAES achieves superior results concerning overall cost, mean fitness, and Ranking Index (i.e., a metric used in the competition for evaluation) compared with state-of-the-art algorithms. Wilcoxon Signed-Rank Statistical test is also applied, demonstrating that CE-CMAES results are statistically different and superior from the other tested algorithms.

A Bi-Level Framework for Expansion Planning in Active Power Distribution Networks

This paper presents a new framework for multistage expansion planning in active power distribution networks, in which the distribution system operator (DSO) considers active network management by clearing the local energy market at the distribution level. The proposed model is formulated as a bi-level optimization problem, where the upper level minimizes the net present value of the total costs imposed to DSO associated with the investment and maintenance of the network assets as well as the network operation, while the lower level on clearing the local energy market captures the participation of distributed energy resource (DER) owners and demand aggregators to maximize the social welfare. The expansion plans consider the investments in DER owners’ assets as well as variety of network assets in which the profitability of DER owners’ investment is guaranteed. The Karush-Kuhn-Tucker optimality conditions and the strong duality theory are employed through which the model is converted to a mixed integer linear programming optimization problem. The implementation of the suggested model on the 24-bus and 86-bus distribution test systems validates its performance and efficacy in making cost-effective planning decisions.

Kernel Structure Design for Data-Driven Probabilistic Load Flow Studies

In power system analysis, probabilistic load flow (PLF) accounts for uncertainties stemming from power entities such as renewable generation systems and consumer demands. In this paper, we present a Gaussian Process (GP) emulator for enabling data-driven PLF on practical distribution networks (DNs) without requiring knowledge of underlying system parameters. The main novelty of our proposal lies in the kernel design process. An ideal kernel allows for greater efficiency during training and inferencing stages without being subject to common issues that include overfitting to the training dataset and poor accuracies. In this proposal, the kernel selection process is formulated as a bi-level optimization problem. This is a very difficult problem to solve as it is composed of discrete variables and non-convex constraints. We overcome these issues by proposing a best-response strategy refinement process to identify an efficient kernel configuration in an iterative manner. The convergence properties of this iterative algorithm and the approximating capabilities of the resulting GP emulator are established using potential game theoretic constructs, universal approximation theorem and representer theorem. The performance of the proposed emulator is then showcased via comprehensive simulations conducted on practical DNs and comparisons with the state-of-the-art.

Risk-Based Formulation of the Transit Priority Network Design

Demand uncertainties are inevitable in transportation networks. The transit priority network design problem over more than a decade of development has been solved under deterministic conditions. This paper proposes a model to find the optimal transit priority scheme in a multimodal transportation network under uncertain demand. This model is formulated as a risk-based bi-level optimization problem. At the upper-level, a risk measure of expected social cost is minimized subject to a chance constraint on total travel time with a user-specified confidence level and a budget constraint. At the lower-level, a mode choice, a traffic user equilibrium assignment, and a transit assignment are applied. An ant colony algorithm is utilized to solve this complex design problem. Numerical results using a real world middle-size city network empirically demonstrate that the demand uncertainty has a significant impact on the solution and the proposed model is applicable to realistic networks.

An exact projection-based algorithm for bilevel mixed-integer problems with nonlinearities

We propose an exact global solution method for bilevel mixed-integer optimization problems with lower-level integer variables and including nonlinear terms such as, e.g., products of upper-level and lower-level variables. Problems of this type are extremely challenging as a single-level reformulation suitable for off-the-shelf solvers is not available in general. In order to solve these problems to global optimality, we enhance an approximative projection-based algorithm for mixed-integer linear bilevel programming problems from the literature to become exact under one additional assumption. This assumption still allows for discrete and continuous leader and follower variables on both levels, but forbids continuous upper-level variables to appear in lower-level constraints and thus ensures that a bilevel optimum is attained. In addition, we extend our exact algorithm to make it applicable to a wider problem class. This setting allows nonlinear constraints and objective functions on both levels under certain assumptions, but still requires that the lower-level problem is convex in its continuous variables. We also discuss computational experiments on modified library instances.

A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse

In this paper, we propose a novel Hessian inverse free Fully Single Loop Algorithm (FSLA) for bilevel optimization problems. Classic algorithms for bilevel optimization admit a double loop structure which is computationally expensive. Recently, several single loop algorithms have been proposed with optimizing the inner and outer variable alternatively. However, these algorithms not yet achieve fully single loop. As they overlook the loop needed to evaluate the hyper-gradient for a given inner and outer state. In order to develop a fully single loop algorithm, we first study the structure of the hyper-gradient and identify a general approximation formulation of hyper-gradient computation that encompasses several previous common approaches, e.g. back-propagation through time, conjugate gradient, etc. Based on this formulation, we introduce a new state variable to maintain the historical hyper-gradient information. Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm. We theoretically show that the error generated by the new state can be bounded and our algorithm converges. Finally, we verify the efficacy our algorithm empirically through multiple bilevel optimization based machine learning tasks. A long version of this paper can be found in: https://arxiv.org/abs/2112.04660.

Distance-based time-dependent optimal ratio control scheme (TORCS) in congested mixed autonomy networks

Connected and automated vehicle technologies provide an opportunity for a more efficient distribution of traffic in urban networks through cooperative routing behavior. Controlling the route choice of a proportion of autonomous vehicles (AVs) to establish the system optimum (SO) condition has already been investigated in static networks. This study aims to develop a time-dependent optimal ratio control scheme (TORCS) in which an optimal ratio of AVs is identified and selected to seek SO routing. The objective of the control scheme is to achieve a reasonable compromise between the system efficiency (i.e., total travel time savings) and the control cost that is proportional to the total distance traveled by SO-seeking AVs. The proposed distance-based TORCS is formulated as a bi-level optimization problem. A mixed equilibrium simulation-based dynamic traffic assignment model (SBDTA) estimates time-dependent mixed traffic patterns in the lower level based on the time-dependent optimal ratios of SO-seeking AVs obtained from the upper level. We also introduce a new concept of frequency control within the TORCS application. The frequency-based TORCS enables the central agent to control the route choice of selected AVs only in the time periods with the most significant impact on system travel time reduction. This is formulated with a binary variable within the TORCS problem indicating an action/no action regarding controlling AVs at a given assignment interval. The applicability of the model is demonstrated on a real large-scale network of Melbourne, Australia.

Applying directional upper semi-regular convexificators in bilevel optimization

In this paper, following the optimistic approach in bilevel programming, we investigate necessary and sufficient optimality conditions for a bilevel optimization problem. Our approach consists of using the optimal value reformulation together with an exact penalization technique. Unlike Dempe and Pilecka (Journal of Global Optimization 61: 769–788, 2015), we reach our goal without resorting to convexificators; the reason is that we do not assume that the sets of all continuity directions are convex or closed. The obtained results are given in terms of directional upper semi-regular convexificators. Some examples are given to illustrate our results.

Economic withholding under high penetration of wind power production: A robust equilibrium method

This paper presents the behavior of bidding above marginal cost, called economic withholding, for price-maker generating companies under wind uncertainty. Economic withholding prevents some lower cost resources from supplying the demand, and compels the ISO to buy more costly resources to meet the load, therefore result in higher market clearing prices than what the competitive outcome would have been. We formulate the underlying economic withholding problem under uncertainty as a robust bilevel optimization problem, where the generating company makes a robust decision in the presence of continuous uncertain variables in order to maximize its overall profit and guarantee the revenue security in the worst-case scenario. This formulation provides an alternative distribution-free equilibrium concept. A practical solution methodology is proposed to drive the exact reformulation of the robust bilevel optimization resulted in Mixed Integer Linear Programming (MILP) problem which is scalable and can be easily implemented in practical portfolio optimization and operation. Simulation results confirm the effectiveness of the proposed framework to provide optimum economic withholding decision.

Comparative analysis of risk-aversion and strategic investment behavior in generation expansion

Both risk-averse behavior and strategic bidding may distort the outcome of energy and capacity markets. Although these two behavioral aspects may trigger similar bidding behavior and investments into generation capacity, they are fundamentally different.While in the literature risk-aversion and strategic behavior have been investigated separately, the presented research offers a comparative analysis to identify situations in which both types of behavior result in the same outcome. First, a framework is introduced in which both strategic and risk-averse behavior can be investigated for a generation expansion planning problem. The problem including the risk-averse investor is modeled as a Nash-equilibrium problem and solved via Alternating Direction Method of Multipliers, whereas the Stackelberg game between the strategic agent and the markets is formulated as a bi-level optimization problem and solved using disjunctive constraints.Using a set of 1920 problem instances we reveal that the distinguishability of the two types of behavior is highly technology-dependent and influenced by the way the capacity target is set. Implications of this work may support regulators in disentangling risk-aversion and strategic behavior based on observed market outcomes.

Joint design and compression of convolutional neural networks as a Bi-level optimization problem.

Over the last decade, deep neural networks have shown great success in the fields of machine learning and computer vision. Currently, the CNN (convolutional neural network) is one of the most successful networks, having been applied in a wide variety of application domains, including pattern recognition, medical diagnosis and signal processing. Despite CNNs’ impressive performance, their architectural design remains a significant challenge for researchers and practitioners. The problem of selecting hyperparameters is extremely important for these networks. The reason for this is that the search space grows exponentially in size as the number of layers increases. In fact, all existing classical and evolutionary pruning methods take as input an already pre-trained or designed architecture. None of them take pruning into account during the design process. However, to evaluate the quality and possible compactness of any generated architecture, filter pruning should be applied before the communication with the data set to compute the classification error. For instance, a medium-quality architecture in terms of classification could become a very light and accurate architecture after pruning, and vice versa. Many cases are possible, and the number of possibilities is huge. This motivated us to frame the whole process as a bi-level optimization problem where: (1) architecture generation is done at the upper level (with minimum NB and NNB) while (2) its filter pruning optimization is done at the lower level. Motivated by evolutionary algorithms’ (EAs) success in bi-level optimization, we use the newly suggested co-evolutionary migration-based algorithm (CEMBA) as a search engine in this research to address our bi-level architectural optimization problem. The performance of our suggested technique, called Bi-CNN-D-C (Bi-level convolution neural network design and compression), is evaluated using the widely used benchmark data sets for image classification, called CIFAR-10, CIFAR-100 and ImageNet. Our proposed approach is validated by means of a set of comparative experiments with respect to relevant state-of-the-art architectures.