Approximate Linear Programming Research Articles

Self-Guided Approximate Linear Programs: Randomized Multi-Shot Approximation of Discounted Cost Markov Decision Processes

Self-Guided Approximate Linear Programs: Randomized Multi-Shot Approximation of Discounted Cost Markov Decision Processes

Approximate linear programming for a queueing control problem

Admission decisions for loss systems accessed by multiple customer classes are a classical queueing control problem with a wide variety of applications. When a server is available, the decision is whether to admit an arriving customer and collect a lump-sum revenue. The system can be modeled as a continuous-time infinite-horizon dynamic program, but suffers from the curse of dimensionality when different customer classes have different service rates. We use approximate linear programming to solve the problem under three approximation architectures: affine, separable piecewise linear and finite affine. The finite affine approximation is a recently proposed generalization of the affine approximation, which allows for non-stationary parameters. For both affine and finite affine approximations, we derive equivalent, but more compact, formulations that can be efficiently solved. We propose a column generation algorithm for the separable piecewise linear approximation. Our numerical results show that the finite affine approximation can obtain the tightest bounds for 75% of the instances among the three approximations. Especially, when the number of servers is large and/or the load on the system is high, the finite affine approximation always achieves the tightest bounds. Regarding policy performance, the finite affine approximation has the best performance on average compared to the other two approximations and the achievable performance region method (Bertsimaset al., 1994, Kumar and Kumar, 1994). Furthermore, the finite affine approximation is 4 to 5 orders of magnitude faster than the achievable performance region method and the separable piecewise linear approximation for large-scale instances. Therefore, considering bounds, policy performance, and computational efficiency, the finite affine approximation emerges as a competitive approximation architecture for the class of problems studied here.

Self-Adapting Network Relaxations for Weakly Coupled Markov Decision Processes

High-dimensional weakly coupled Markov decision processes (WDPs) arise in dynamic decision making and reinforcement learning, decomposing into smaller Markov decision processes (MDPs) when linking constraints are relaxed. The Lagrangian relaxation of WDPs (LAG) exploits this property to compute policies and (optimistic) bounds efficiently; however, dualizing linking constraints averages away combinatorial information. We introduce feasibility network relaxations (FNRs), a new class of linear programming relaxations that exactly represents the linking constraints. We develop a procedure to obtain the unique minimally sized relaxation, which we refer to as self-adapting FNR, as its size automatically adjusts to the structure of the linking constraints. Our analysis informs model selection: (i) the self-adapting FNR provides (weakly) stronger bounds than LAG, is polynomially sized when linking constraints admit a tractable network representation, and can even be smaller than LAG, and (ii) self-adapting FNR provides bounds and policies that match the approximate linear programming (ALP) approach but is substantially smaller in size than the ALP formulation and a recent alternative Lagrangian that is equivalent to ALP. We perform numerical experiments on constrained dynamic assortment and preemptive maintenance applications. Our results show that self-adapting FNR significantly improves upon LAG in terms of policy performance and/or bounds, while being an order of magnitude faster than an alternative Lagrangian and ALP, which are unsolvable in several instances. This paper was accepted by Baris Ata, stochastic models and simulation. Supplemental Material: The electronic companion and data files are available at https://doi.org/10.1287/mnsc.2022.01108 .

Dynamic Home Care Routing and Scheduling with Uncertain Number of Visits per Referral

Despite the rapid growth of the home care industry, research on the scheduling and routing of home care visits in the presence of uncertainty is still limited. This paper investigates a dynamic version of this problem in which the number of referrals and their required number of visits are uncertain. We develop a Markov decision process (MDP) model for the single-nurse problem to minimize the expected weighted sum of the rejection, diversion, overtime, and travel time costs. Because optimally solving the MDP is intractable, we employ an approximate linear program (ALP) to obtain a feasible policy. The typical ALP approach can only solve very small-scale instances of the problem. We derive an intuitively explainable closed-form solution for the optimal ALP parameters in a special case of the problem. Inspired by this form, we provide two heuristic reduction techniques for the ALP model in the general problem to solve large-scale instances in an acceptable time. Numerical results show that the ALP policy outperforms a myopic policy that reflects current practice, and is better than a scenario-based policy in most instances considered. Funding: This work was supported by the Natural Sciences and Engineering Research Council of Canada [Grants RGPIN-2018-05225 and RGPIN-2020-210524] and by the Telfer School of Management SMRG Postdoctoral Research Fellowship Support [Grant 2020]. Supplemental Material: The electronic companion is available at https://doi.org/10.1287/trsc.2023.0120 .

Dynamic container slot allocation for a liner shipping service

In this paper, we study a dynamic container slot allocation problem (DCSAP) for a liner container shipping company that aims to make an acceptance or rejection decision to each dynamically arriving container slot booking request. To capture the dynamic arrival feature and real-time acceptance/rejection decision of the booking request, we formulate the DCSAP as a dynamic programming (DP) model with the objective of maximizing the total revenues generated by accepted container booking requests over the entire booking horizon. Due to the well-known curse of dimensionality of solving a DP model, we develop a series of models to transform the intractable DP model into a solvable approximate linear programming (ALP) model. We further propose a spatiotemporal-heterogeneity-based (STH-based) decomposition preprocessor by identifying the spatiotemporal property of the DCSAP before solving the ALP model. Extensive numerical experiments are conducted to assess the applicability of the developed research methodology.

Algorithms for a risk-averse Stackelberg game with multiple adversaries

We consider a Stackelberg game that arises in a security domain (SSG), where a defender can simultaneously protect m out of n targets from an adversary that observes the defense strategy before deciding on an utility maximizing attack. Given the high stakes in security settings, it is reasonable that the defender in this game is risk averse with respect to the attacker’s decisions.Here we focus on developing efficient solution algorithms for a specific SSG, where the defender uses an entropic risk measure to model risk aversion to the attacker’s strategies, and where multiple attackers select targets following logit quantal response equilibrium models. This problem can be formulated as a nonconvex nonlinear optimization problem. We propose two solution methods: (1) approximate the problem through convex mixed integer nonlinear programs (MINR) and (2) a general purpose methodology (CELL) to optimize nonconvex and nonseparable fractional problems through mixed integer linear programming approximations. Both methods provide arbitrarily good incumbents and lower bounds on SSG. We present cutting plane methods to solve these problems for large instances. Our computational experiments illustrate the advantages of introducing risk aversion into the defender’s behavior and show that MINR dominates CELL, producing in 2 h solutions that are within 2% of optimal on average.

Decision Rule Approaches for Pessimistic Bilevel Linear Programs Under Moment Ambiguity with Facility Location Applications

We study a pessimistic stochastic bilevel program in the context of sequential two-player games, where the leader makes a binary here-and-now decision, and the follower responds with a continuous wait-and-see decision after observing the leader’s action and revelation of uncertainty. We assume that only the information regarding the mean, covariance, and support is known. We formulate the problem as a distributionally robust (DR) two-stage problem. The pessimistic DR bilevel program is shown to be equivalent to a generic two-stage distributionally robust stochastic (nonlinear) program with both a random objective and random constraints under proper conditions of ambiguity sets. Under continuous distributions, using linear decision rule approaches, we construct upper bounds on the pessimistic DR bilevel program based on (1) a 0-1 semidefinite programming (SDP) approximation and (2) an exact 0-1 copositive programming reformulation. When the ambiguity set is restricted to discrete distributions, an exact 0-1 SDP reformulation is developed, and explicit construction of the worst-case distribution is derived. To further improve the computation of the proposed 0-1 SDPs, a cutting-plane framework is developed. Moreover, based on a mixed-integer linear programming approximation, another cutting-plane algorithm is proposed. Extensive numerical studies are conducted to demonstrate the effectiveness of the proposed approaches on a facility location problem. History: Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0168 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0168 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Optimizing bilinear multi-source water supply systems using mixed-integer linear programming approximations: An analysis of the Israeli seawater desalination array

Decision support systems (DSSs) in water resources management often use optimization models to make decisions. Therefore, the quality of these models is crucial for the success of DSSs. However, optimization models can struggle with reaching global optima, resulting in inconsistent and untrustworthy results for decision-makers. Additionally, current solution methods can be slow and inefficient, hindering real-time interaction with the DSS. This study addresses the optimal regional Multisource Water Supply Systems (MWSS) management using efficient global optimization techniques. The MWSS problem is formulated as a nonlinear optimization program due to using bilinear terms, which can result in multiple local optima. To overcome this challenge, four Mixed Integer Linear Program (MILP) formulations are proposed, which can be globally solved in a computationally effective manner using readily available MILP solvers. Our proposed formulations are tested on the Israeli Water Supply System (IWSS), which presents a unique challenge due to the inclusion of the seawater desalination array. In addition, this system must balance conventional and non-conventional water sources with varying qualities and characteristics. The results show that the proposed formulations provide a computationally efficient solution for the IWSS problem, allowing for the optimal utilization of different water sources in the IWSS.

IFlipper: Label Flipping for Individual Fairness

As machine learning becomes prevalent, mitigating any unfairness present in the training data becomes critical. Among the various notions of fairness, this paper focuses on the well-known individual fairness, which states that similar individuals should be treated similarly. While individual fairness can be improved when training a model (in-processing), we contend that fixing the data before model training (pre-processing) is a more fundamental solution. In particular, we show that label flipping is an effective pre-processing technique for improving individual fairness. Our system iFlipper solves the optimization problem of minimally flipping labels given a limit to the individual fairness violations, where a violation occurs when two similar examples in the training data have different labels. We first prove that the problem is NP-hard. We then propose an approximate linear programming algorithm and provide theoretical guarantees on how close its result is to the optimal solution in terms of the number of label flips. We also propose techniques for making the linear programming solution more optimal without exceeding the violations limit. Experiments on real datasets show that iFlipper significantly outperforms other pre-processing baselines in terms of individual fairness and accuracy on unseen test sets. In addition, iFlipper can be combined with in-processing techniques for even better results.

Approximation Algorithms and Linear Programming Relaxations for Scheduling Problems Related to Min-Sum Set Cover

We consider single-machine scheduling problems that are natural generalizations or variations of the min-sum set-cover problem. For these scheduling problems, we give new approximation algorithms. Some of these algorithms rely on time-indexed linear programming relaxations. We show how a variant of alpha-point scheduling leads to the best known approximation ratios, including a guarantee of four for an interesting special case of the so-called generalized min-sum set-cover problem. We also make explicit the connection between the greedy algorithm for min-sum set cover and the concept of Sidney decomposition for precedence-constrained single-machine scheduling and show how this leads to a 4-approximation algorithm for single-machine scheduling with so-called bipartite OR-precedence constraints. Funding: This work has been supported by the Alexander von Humboldt Foundation with funds from the German Federal Ministry of Education and Research.

Nonparametric Approximate Dynamic Programming via the Kernel Method

This paper presents a novel, non-parametric approximate dynamic programming (ADP) algorithm that enjoys dimension-independent approximation and sample complexity guarantees. We obtain this algorithm by “kernelizing” a recent mathematical program for ADP (the “smoothed approximate linear program”). Loosely, our guarantees show that we can exchange the importance of choosing a good approximation architecture a priori (as required by existing approaches) with sampling effort. We also present a simple active set algorithm for solving the resulting quadratic program, and prove the correctness of this method. Via a computational study on a controlled queueing network, we show that our approach is capable of outperforming parametric linear programming approaches to ADP, as well as non-trivial, tailored heuristics for the same network, even when employing generic, polynomial kernels.

Optimal Energy Management and Storage Sizing for Electric Vehicles With Dual Storage

Battery degradation reduces the performance and lifetime of electric vehicles (EVs). Using energy storage devices with different characteristics alongside the battery can minimize degradation while satisfying driving demands. However, this introduces the additional complexities of sizing multiple storage devices and controlling them in real time. In this brief, we first provide a computationally tractable method to manage power-sharing between dual energy storages using approximate linear programming (ALP), an approximation of infinite horizon dynamic programming (DP). We formulate a procedure to determine the optimal sizes of the two storages based on the solution to the energy management problem to account for the tradeoff between vehicle range, storage size, and weight. We validate our approaches on a numerical case study. Numerical results show that our controller shares power efficiently between the storages, and our sizing procedure provides a design with minimal cost.

Service expansion for chained business facilities under congestion and market competition

We study a service expansion problem for chained business facilities under endogenic facility congestion and exogenous market competition. More specifically, we consider a company that operates a chain of facilities and plans to expand service capacities with the objective of maximizing its profit, accounting for revenue and expansion costs. To estimate revenue, the company needs to anticipate customer behaviors. Due to the co-existence of competition and congestion, customer behaviors are explained as a two-stage process. In the first stage, customers make “channel” choices, i.e., they decide whether to seek services from the company. Such a choice reflects the market competition and is predicted by a discrete choice model. Subsequently, customers who select the company will choose one facility to patronize. Owing to congestion, the facility choice will induce “user equilibrium”, which in return affects the outcome of market competition. To facilitate the company’s decision-making in this complex business environment, we develop a generic modeling framework. Unfortunately, the proposed model is nonconvex. To solve it, we first design an approximate mixed-integer linear programming approach subject to adjustable approximation errors. We then propose a surrogate optimization framework for large-scale instances, which explores the hidden bilevel structure of the model and leverages a “learning-to-optimize” problem and a customer behavior estimation subroutine. Using extensive computational experiments, we demonstrate the effectiveness of the proposed approaches. Finally, we conduct sensitivity analysis and draw practical implications.

Reductions of non-separable approximate linear programs for network revenue management

We suggest a novel choice of non-separable basis functions for an approximate linear programming approach to the well-known network revenue management problem. Considering non-separability is particularly important when interdependencies between resources are large. Such a situation can be illustrated for example by a bus line, where different origin-destination pairs have many overlapping segments. Traditional separable approximation approaches tend to ignore the resulting interactions.We suggest to group resources into non-separable subnetworks. For each chosen subnetwork, basis functions either span the whole function space or consist of linear functions. Given this more general choice of basis functions, we extend existing reductions of approximate linear programs. If there is only one subnetwork, for which the basis functions span the whole function space, we prove the equivalence to a compact linear program of polynomial size. For the general case, we suggest an approximate reduction. Numerical examples illustrate our novel upper bounds for the maximum expected revenue and the corresponding competitive policies. In particular, we find that the added benefit of non-separability heavily depends on the network structure and the capacity.Our work helps to better understand the impact of assuming separability in network revenue management. The polynomial sized reductions make it possible to estimate the added average revenue resulting from incorporating interactions between resources. The theory we develop demonstrates how the interpretation of dual variables as state-action probabilities can be applied to reduce exponentially large approximate linear programs via variable aggregation.

Compact Approximate Linear Programs for Network Revenue Management: Reduction via Constraint Decomposition and Adaptation for Locally Separable Approximation

Compact Approximate Linear Programs for Network Revenue Management: Reduction via Constraint Decomposition and Adaptation for Locally Separable Approximation

Block coordinate type methods for optimization and learning

We study nonconvex (composite) optimization and learning problems where the decision variables can be split into blocks of variables. Random block projection is a popular technique to handle this kind of problem for its remarkable reduction of the computational cost from the projection. This powerful method has not been well proposed for the situation that first-order information is prohibited and only zeroth-order information is available. In this paper, we propose to develop different classes of zeroth-order stochastic block coordinate type methods. Zeroth-order block coordinate descent (ZS-BCD) is proposed for solving unconstrained nonconvex optimization problem. For composite optimization, we establish the zeroth-order stochastic block mirror descent (ZS-BMD) and its associated two-phase method to achieve the complexity bound for the composite optimization problem. Furthermore, we also establish zeroth-order stochastic block coordinate conditional gradient (ZS-BCCG) method for nonconvex (composite) optimization. By implementing ZS-BCCG method, in each iteration, only (approximate) linear programming subproblem needs to be solved on a random block instead of a rather costly projection subproblem on the whole decision space, in contrast to the existing traditional stochastic approximation methods. In what follows, an approximate ZS-BCCG method and corresponding two-phase ZS-BCCG method are proposed. This is also the first time that a two-phase BCCG method has been developed to carry out the complexity analysis of nonconvex composite optimization problem.

Efficient compact linear programs for network revenue management

We are concerned with computing bid prices in network revenue management using approximate linear programming. It is well-known that affine value function approximations yield bid prices which are not sensitive to remaining capacity. The analytic reduction to compact linear programs allows the efficient computation of such bid prices. On the other hand, capacity-dependent bid prices can be obtained using separable piecewise linear value function approximations. Even though compact linear programs have been derived for this case also, they are still computationally much more expensive compared to using affine functions. We propose compact linear programs requiring substantially smaller computing times while, simultaneously, significantly improving the performance of capacity-independent bid prices. This simplification is achieved by taking into account remaining capacity only if it becomes scarce. Although our proposed linear programs are relaxations of the unreduced approximate linear programs, we conjecture equivalence and provide according numerical support. We measure the quality of an approximation by the difference between the expected performance of an induced policy and the corresponding theoretical upper bound. Using this paradigm in numerical experiments, we demonstrate the competitiveness of our proposed linear programs.

Technical Note—Product-Based Approximate Linear Programs for Network Revenue Management

A Novel and Promising Approximation for Network Revenue Management In “Product-Based Approximate Linear Programs for Network Revenue Management,” Zhang, Samiedaluie, and Zhang propose a novel separable piecewise linear (SPL) approximation for the network revenue management problem. The coefficients of the proposed SPL approximation can be interpreted as each product’s revenue contribution to the value of each resource in a given period, which provides more granular information compared with the existing resource-based SPL approximation in the literature. The new approximation provides more flexibility for policy construction. Furthermore, the new approximation opens the opportunity to derive a set of valid inequalities to further improve the computational performance and achieve additional gains in the expected revenue. Computational experiments with instances of various network structures and parameters demonstrate its efficacy: the new approximation leads to bid-price policies generating higher expected revenues and demonstrates better performance in terms of both computational efficiency and numerical stability.

A robust omnichannel pricing and ordering optimization approach with return policies based on data-driven support vector clustering

This work considers an omnichannel retailer selling a product with market demand uncertainties to customers in different regions through an e-store and multiple brick-and-mortar stores. The retailer also manages online product returns by selecting appropriate return policies. A robust omnichannel pricing and ordering optimization model is proposed with two, i.e., a full-refund and a no-refund, return policies. The online demand and the offline demand depend on the prices of the e-store and the brick-and-mortar stores, where the online demand also depends on the refund of the e-store. A linearization technique is adopted to deal with nonlinearity of the model. A data-driven robust optimization approach is used to construct uncertainty sets based on available historical data using support vector clustering to handle demand uncertainties. Furthermore, the proposed model is transformed into an approximate mixed integer linear programming model which can be solved by using commercial software. An electronics retailer in China is used as a case study to illustrate the effectiveness and practicality of the proposed model and the solution method. A comparison with the box uncertainty set reveals that the data-driven uncertainty set is less conservative and performs better by obtaining higher profit for the retailer. Furthermore, sensitive analysis results indicate that return policies and the return rate of the product affect the optimal pricing and ordering decisions and the total profit.

Profit maximization via capacity control for distribution logistics problems

We consider a distribution logistics scenario where a shipping operator, managing a limited amount of resources, receives a stream of service requests, issued by a set of customers along a booking time-horizon, that are referred to a future operational period. The shipping operator must then decide about accepting or rejecting each incoming request at the time it is issued, accounting for revenues, but also considering resource consumptions. In this context, the decision process is based on dynamically finding the best trade-off between the immediate return of accepting the request and the convenience of preserving capacity to possibly exploit more valuable future requests. We give a dynamic formulation of the problem aimed at maximizing the operator revenues, accounting also for the operational distribution costs. Due to the “curse of dimensionality”, the dynamic program cannot be solved optimally. For this reason, we propose a mixed-integer linear programming approximation, whose exact or approximate solutions provide the relevant information to apply some commonplace revenue management policies in the real-time decision-making. Adopting a capacitated vehicle routing problem as an underlying distribution application, we analyze the computational behavior of the proposed techniques on a set of academic test problems.