New Class Of Policies Research Articles

Optimal Robust Policy for Feature-Based Newsvendor

We study policy optimization for the feature-based newsvendor, which seeks an end-to-end policy that renders an explicit mapping from features to ordering decisions. Most existing works restrict the policies to some parametric class that may suffer from suboptimality (such as affine class) or lack of interpretability (such as neural networks). Differently, we aim to optimize over all functions of features. In this case, the classic empirical risk minimization yields a policy that is not well-defined on unseen feature values. To avoid such degeneracy, we consider a Wasserstein distributionally robust framework. This leads to an adjustable robust optimization, whose optimal solutions are notoriously difficult to obtain except for a few notable cases. Perhaps surprisingly, we identify a new class of policies that are proven to be exactly optimal and can be computed efficiently. The optimal robust policy is obtained by extending an optimal robust in-sample policy to unobserved feature values in a particular way and can be interpreted as a Lipschitz regularized critical fractile of the empirical conditional demand distribution. We compare our method with several benchmarks using synthetic and real data and demonstrate its superior empirical performance. This paper was accepted by J. George Shanthikumar, data science. Supplemental Material: The online appendix and data are available at https://doi.org/10.1287/mnsc.2023.4810 .

Delay-Join the Shortest Queue Routing for a Parallel Queueing System with Removable Servers

We introduce a new class of policies called “delay-join the shortest queue (delay-JSQ)” for use in parallel processing networks with removable servers. When jobs arrive to the system while all servers are on, jobs should be routed to the shortest queue. However, when servers are off, they take a random time to turn back on, which we allow to occur only when the number of jobs in each of the nonempty queues exceeds a fixed threshold. This new class of policies balances the load among all servers that are currently on and balances the capacity by keeping servers off until they are needed. A detailed numerical study shows that at moderate loads (where server farms and increasingly manufacturing facilities operate), delay-JSQ outperforms JSQ by up to 80%. In addition, it does so without precise knowledge of the input parameters and even when the input process is nonstationary.

Warranty Matching in a Consumer Electronics Closed-Loop Supply Chain

Problem definition : We examine a dynamic assignment problem faced by a large wireless service provider (WSP) that is a Fortune 100 company. This company manages two warranties: (i) a customer warranty that the WSP offers its customers and (ii) an original equipment manufacturer (OEM) warranty that OEMs offer the WSP. The WSP uses devices refurbished by the OEM as replacement devices, and hence their warranty operation is a closed-loop supply chain. Depending on the assignment the WSP uses, the customer and OEM warranties might become misaligned for customer-device pairs, potentially incurring a cost for the WSP. Academic/practical relevance : We identify, model, and analyze a new dynamic assignment problem that emerges in this setting called the warranty matching problem. We introduce a new class of policies, called farsighted policies, which can perform better than myopic policies. We also propose a new heuristic assignment policy, the sampling policy, which leads to a near-optimal assignment. Our model and results are motivated by a real-world problem, and our theory-guided assignment policies can be used in practice; we validate our results using data from our industrial partner. Methodology : We formulate the problem of dynamically assigning devices to customers as a discrete-time stochastic dynamic programming problem. Because this problem suffers from the curse of dimensionality, we propose and analyze a set of reasonable classes of assignment policies. Results : The performance metric that we use for a given assignment policy is the average time that a replacement device under a customer warranty is uncovered by an OEM warranty. We show that our assignment policies reduce the average uncovered time and the expected number of out-of-OEM-warranty returns by more than 75% in comparison with our industrial partner’s current assignment policy. We also provide distribution-free bounds for the performance of a myopic assignment policy and of random assignment, which is a proxy for the WSP’s current policy. Managerial implications : Our results indicate that, in closed-loop supply chains, being completely farsighted might be better than being completely myopic. Also, policies that are effective in balancing short-term and long-term costs can be simple and effective, as illustrated by our sampling policy. We describe how the performance of myopic and farsighted policies depend on the size and length of inventory buildup.

Warranty Matching in a Consumer Electronics Closed-Loop Supply Chain

Problem definition: We examine a dynamic assignment problem faced by a large Wireless Service Provider (WSP) that is a Fortune 100 company. This company manages two warranties: (i) a customer warranty that WSP offers its customers, and (ii) an Original Equipment Manufacturer (OEM) warranty that OEMs offer the WSP. The WSP uses devices refurbished by the OEM as replacement devices and hence their warranty operation is a closed-loop supply chain. Depending on the assignment policy WSP uses, the customer and OEM warranties might become misaligned for customer-device pairs, potentially incurring a cost for WSP. Academic/practical relevance: We identify, model, and analyze a new dynamic assignment problem that emerges in this setting called the warranty matching problem. We introduce a new class of policies, called farsighted policies, which can perform better than myopic policies. We also propose a new heuristic assignment policy, the sampling policy, which leads to a near-optimal assignment. Our model and results are motivated by a real-world problem, and our theory-guided assignment policies can be used in practice; we validate our results using data from our industrial partner. Methodology: We formulate the problem of dynamically assigning devices to customers as a discrete-time stochastic dynamic programming problem. Because this problem suffers from the curse of dimensionality, we propose and analyze a set of reasonable classes of assignment policies. Results: The performance metric that we use for a given assignment policy is the average time that a replacement device under a customer warranty is uncovered by an OEM warranty. We show that our assignment policies reduce the average uncovered time and the expected number of out-of-OEM-warranty returns by more than 75% in comparison to our industrial partner's current assignment policy. We also provide distribution-free bounds for the performance of a myopic assignment policy, and of random assignment, which is a proxy for the current WSP policy. Managerial implications: Our results indicate that, in closed-loop supply chains, being completely far-sighted might be better than being completely myopic. Also, policies that are effective in balancing short-term and long-term costs can be simple and effective, as illustrated by our sampling policy. We describe how the performance of myopic and farsighted policies depend on the size and length of inventory build-up.

New strategies for stochastic resource-constrained project scheduling

We study the stochastic resource-constrained project scheduling problem or SRCPSP, where project activities have stochastic durations. A solution is a scheduling policy, and we propose a new class of policies that is a generalization of most of the classes described in the literature. A policy in this new class makes a number of a priori decisions in a preprocessing phase, while the remaining scheduling decisions are made online. A two-phase local search algorithm is proposed to optimize within the class. Our computational results show that the algorithm has been efficiently tuned toward finding high-quality solutions and that it outperforms all existing algorithms for large instances. The results also indicate that the optimality gap even within the larger class of elementary policies is very small.

Using Predictions in Online Optimization

We consider online convex optimization (OCO) problems with switching costs and noisy predictions. While the design of online algorithms for OCO problems has received considerable attention, the design of algorithms in the context of noisy predictions is largely open. To this point, two promising algorithms have been proposed: Receding Horizon Control (RHC) and Averaging Fixed Horizon Control (AFHC). The comparison of these policies is largely open. AFHC has been shown to provide better worst-case performance, while RHC outperforms AFHC in many realistic settings. In this paper, we introduce a new class of policies, Committed Horizon Control (CHC), that generalizes both RHC and AFHC. We provide average-case analysis and concentration results for CHC policies, yielding the first analysis of RHC for OCO problems with noisy predictions. Further, we provide explicit results characterizing the optimal CHC policy as a function of properties of the prediction noise, e.g., variance and correlation structure. Our results provide a characterization of when AFHC outperforms RHC and vice versa, as well as when other CHC policies outperform both RHC and AFHC.

New Benchmark Results for the Stochastic Resource-Constrained Project Scheduling Problem

We study the stochastic resource-constrained project scheduling problem or SRCPSP, where project activities have stochastic durations. A solution is a scheduling policy, and we propose a new class of policies that is a generalization of most of the classes described in the literature. A policy in this new class makes a number of a-priori decisions in a preprocessing phase while the remaining scheduling decisions are made online. A two-phase local search algorithm is proposed to optimize within the class. Our computational results show that the algorithm has been efficiently tuned towards finding high-quality solutions, and that it outperforms all existing algorithms for large instances. The results also indicate that the optimality gap even within the larger class of elementary policies is very small.

Delay-Optimal Opportunistic Scheduling and Approximations: The Log Rule

This paper considers the design of multiuser opportunistic packet schedulers for users sharing a time-varying wireless channel from performance and robustness points of view. For a simplified model falling in the classical Markov decision process framework, we numerically compute and characterize mean-delay-optimal scheduling policies. The computed policies exhibit radial sum-rate monotonicity: As users' queues grow linearly, the scheduler allocates service in a manner that deemphasizes the balancing of unequal queues in favor of maximizing current system throughput (being opportunistic). This is in sharp contrast to previously proposed throughput-optimal policies, e.g., Exp rule and MaxWeight (with any positive exponent of queue length). In order to meet performance and robustness objectives, we propose a new class of policies, called the Log rule, that are radial sum-rate monotone (RSM) and provably throughput-optimal. In fact, it can also be shown that an RSM policy minimizes the asymptotic probability of sum-queue overflow. We use extensive simulations to explore various possible design objectives for opportunistic schedulers. When users see heterogenous channels, we find that emphasizing queue balancing, e.g., Exp rule and MaxWeight, may excessively compromise the overall delay. Finally, we discuss approaches to implement the proposed policies for scheduling and resource allocation in OFDMA-based multichannel systems.

A Decomposition Approach for a Class of Capacitated Serial Systems

We study a class of two-echelon serial systems with identical ordering/production capacities or limits for both echelons. Demands are assumed to be integer valued. For the case where the lead time to the upstream echelon is one period, the optimality of state-dependent modified echelon base-stock policies is proved using a decomposition approach. For the case where the upstream lead time is two periods, we introduce a new class of policies called “two-tier base-stock policies,” and prove their optimality. Some insight about the inventory control problem in N echelon serial systems with identical capacities at all stages and arbitrary lead times everywhere is also provided. We argue that a generalization of two-tier base-stock policies, which we call “multitier base-stock policies,” are optimal for these systems; we also provide a bound on the number of parameters required to specify the optimal policy.

Adaptive and Distributed Algorithms for Vehicle Routing in a Stochastic and Dynamic Environment

In this paper we present distributed and adaptive algorithms for motion coordination of a group of m autonomous vehicles. The vehicles operate in a convex environment with bounded velocity and must service demands whose time of arrival, location and on-site service are stochastic; the objective is to minimize the expected system time (wait plus service) of the demands. The general problem is known as the m-vehicle Dynamic Traveling Repairman Problem (m-DTRP). The best previously known control algorithms rely on centralized a-priori task assignment and are not robust against changes in the environment, e.g. changes in load conditions; therefore, they are of limited applicability in scenarios involving ad-hoc networks of autonomous vehicles operating in a time-varying environment. First, we present a new class of policies for the 1-DTRP problem that: (i) are provably optimal both in light- and heavy-load condition, and (ii) are adaptive, in particular, they are robust against changes in load conditions. Second, we show that partitioning policies, whereby the environment is partitioned among the vehicles and each vehicle follows a certain set of rules in its own region, are optimal in heavy-load conditions. Finally, by combining the new class of algorithms for the 1-DTRP with suitable partitioning policies, we design distributed algorithms for the m-DTRP problem that (i) are spatially distributed, scalable to large networks, and adaptive to network changes, (ii) are within a constant-factor of optimal in heavy-load conditions and stabilize the system in any load condition. Simulation results are presented and discussed.

Restricted time-remembering policies for the inventory rationing problem

We analyse an ( s, Q) inventory model with unit Poisson demand, several demand classes and lost sales. When dealing with different demand classes the usual approach is to control the inventory by critical levels at which stock is reserved for demand of higher priority. We focus on time-remembering policies where the critical levels are allowed to depend on the time since the actual outstanding order (if any) was issued. In particular, we introduce a new class of policies called restricted time-remembering (RTR) policies. We divide the lead time in m intervals of time and restrict the critical levels of the RTR policy to be constant over each interval of time. This policy is much easier to implement than the optimal rationing policy, where the critical levels can change continuously over time, and in a numerical study we show that the performance of the RTR policy is close to that of the optimal policy, even when the number of intervals is 2 or 3.

Linear preselective policies for stochastic project scheduling

In the context of stochastic resource-constrained project scheduling we introduce a novel class of scheduling policies, the linear preselective policies. They combine the benefits of preselective policies and priority policies; two classes that are well known from both deterministic and stochastic scheduling. We study several properties of this new class of policies which indicate its usefulness for computational purposes. Based on a new representation of preselective policies as and/or precedence constraints we derive efficient algorithms for computing earliest job start times and state a necessary and sufficient dominance criterion for preselective policies. A computational experiment based on 480 instances empirically validates the theoretical findings.

Note. Periodic Review (s, S) Policies for Joint Replenishment Inventory Systems

This note deals with the joint replenishment problem with stochastic demands. A new class of policies called P(s, S) policy is proposed for this problem. The proposed policy uses independent, periodic review (s, S) policies for each item. Results on test problems show that the proposed policy is superior to other classes of policies for the problem.

A New Class of Policies in Vector-Valued Markov Decision Processes

For a vector-valued Markov decision process with discounted reward criterion, we introduce a new class of policies called the semi-stationary policies and show that an optimal semi-stationary policy that attains the extreme points of the set of rewards induced by all policies can be described as a combination of optimal stationary policies.

A 98%-Effective Lot-Sizing Rule for a Multi-Product, Multi-Stage Production / Inventory System

We study a multi-product, multi-stage production inventory system in continuous time. In-process and finished goods are referred to as products and inventories of a single item held at different locations are treated as different products. External demand can occur for any or all of the products at a constant, product-dependent rate. When an order is placed for a product it is instantly delivered. The set-up cost is a function of the set of products being ordered at that point in time. The goal is to schedule orders for each of the products over an infinite horizon so as to minimize the long-run average cost. We define a new class of policies in which each product uses a stationary interval of time between successive orders, and the ratio of the order intervals of any two products is an integer power of two. We show that there is always a policy in this class that is within 2% of optimal. This extends a similar result for the one-warehouse, multi-retailer system. For assembly systems we show how to compute such a policy in 0(N log N) time. For general systems a sequence of max-flow, min-cut computations is required. The algorithm is efficient for very large systems.

98%-Effective Integer-Ratio Lot-Sizing for One-Warehouse Multi-Retailer Systems

A warehouse supplies N retailers. Constant external demand occurs at each retailer, and shortages are not allowed. There are linear holding costs and fixed costs for ordering at the warehouse and at each retailer. The goal is to minimize the long-run average cost over an infinite time horizon. We define a new class of policies for this problem whose simple structure facilitates both computation and implementation. The cost of a policy that is optimal within this class is shown to be within 2% of the cost of an optimal policy for the original problem, in the worst case. Such a policy can be computed in O(N log N) time.

A Random Walk Production-Inventory Policy: Rationale and Implementation

The smoothing of fluctuations in production and inventory has been handled successfully as a problem in servomechanisms: an optimal linear filter (production scheduling policy) balances costs of inventory fluctuation and costs of production fluctuation. This approach requires the system to respond (by making production rate changes) either continuously or at regular intervals: the usual assumption is that the cost of responding is proportional to the size of the response. This paper explores some reasons why the frequency of production changes may have to be controlled, and offers a new class of policies, called random walk policies, to accomplish this. One of these, the (a, b, c) policy, is investigated for stationary operating characteristics, and a specific example is provided to illustrate the technique of finding optimal values for policy parameters. Some of the characteristics of systems in which random walk inventory policies may be useful are discussed; and more general techniques for identifying the form of the optimal control policy are mentioned.