Stochastic Resource Allocation Problems Research Articles

Stochastic Resource Allocation Problems: Minmax and Maxmin Solutions

Problem definition: Min-max or max-min objective criteria naturally arise in many resource allocation applications, especially when the attendant goal involves distribution of a limited resource across multiple subsystems, locations, or activities. Each has a different consumption rate of the resource. In this setting, the decision maker is interested in optimizing the system performance, which can be measured in terms of the smallest or largest order statistic across subsystems. Examples include the allocation of health resources to maximize the first runout time or to minimize the maximum delay. Methodology/results: In the literature, there are no unified theoretical and computational results on the optimal allocations for these problems. Thus, in this paper, we develop key theoretical properties of the attendant stochastic resource allocation problems, providing a technical condition for the objective functions to have favorable properties, upon which we derive novel optimal allocation rules with clear interpretation. Managerial implications: The proportional allocation rule can be interpreted as a robust solution when the full distribution of the interarrival time is unknown. It generally has poor performance, especially when the resource consumption rates vary across locations, which happens regularly in practice. Our allocation rule is exact and should be used for the allocation of scarce resources—that is, when the number of resources per location is limited, which is often encountered in healthcare settings.

Combining optimisation and simulation using logic-based Benders decomposition

Operations Research practitioners often want to model complicated functions that are difficult to encode in their underlying optimisation framework. A common approach is to solve an approximate model, and then use a simulation to evaluate the true objective value of one or more solutions. We propose a new approach to integrating simulation into the optimisation model itself. The idea is to run the simulation at each incumbent solution to a master problem. The simulation results are then used to guide the trajectory of the optimisation model itself using logic-based Benders cuts. We test the approach on a class of stochastic resource allocation problems with monotonic performance measures. We derive strong novel Benders cuts that are provably valid for all problems of the given form. We consider two concrete examples: a nursing home shift scheduling problem, and an airport check in counter allocation problem. While previous papers on these applications could only approximately solve realistic instances, we are able to solve them exactly within a reasonable amount of time. Moreover, while those papers account for the inherent variance of the problem by including estimates of the underlying random variables as model parameters, we are able to compute sample-average approximations to optimality with up to 100 scenarios.

The Risk-Averse Static Stochastic Knapsack Problem

This research proposes and analyzes new models for a stochastic resource allocation problem that arises in a variety of operations contexts. One of the primary contributions of the paper lies in providing a succinct, robust, and general model that can address a range of different risk-based objectives and cost assumptions under uncertainty. Although the model expression is relatively simple, it embeds a reasonably high degree of underlying complexity, as the analysis shows. In addition, in-depth analysis of the model, both in its general form and under various specific risk measures, uncovers some interesting and powerful insights regarding the problem trade-offs. Furthermore, this analysis leads to a highly efficient class of heuristic algorithms for solving the problem, which we demonstrate via numerical experimentation to provide close-to-optimal solutions. This computational benefit is a critical element for solving a class of broadly applicable larger problems for which our problem arises as a subproblem that requires repeated solution.

An Efficient Elite-Based Simulation–Optimization Approach for Stochastic Resource Allocation Problems in Manufacturing and Service Systems

Stochastic resource allocation problems (SRAPs) involve determining the optimal configuration of a limited resource to achieve an objective function under given constraints and random effects in manufacturing systems (MSs) and service systems (SSs). The problems are traditionally solved by determining the optimal solution. It is generally preferable to determine as many global optima as possible, or at least a small set of diverse but good candidates, to help the decision-maker rapidly adopt alternative solutions from the set if one solution is unsuitable. However, many local or global optima occur in SRAPs in MSs and SSs due to the interaction between random system factors, such as processing time uncertainty and machine failure rates. Thus, enhancing the searching efficiency of algorithms for SRAPs is a challenge. This study proposes an efficient simulation–optimization approach, called elite-based particle swarm optimization (EPSO), using an optimal replication allocation strategy (ORAS) (i.e., EPSO[Formula: see text], to address three types of SRAPs from the literature. Three simulation models were constructed to evaluate the system performance under random factors. We developed a novel EPSO to explore and exploit the solution space. We created an elite group (EG) that includes multiple solutions, and each solution of the EG has a statistically nonsignificant difference from the current optimal solution. The new feature of EPSO updates the velocity and position of the particles in the design space based on multiple global optima from the EG to enhance diversity and prevent premature convergence. We propose an ORAS to allocate a limited number of replications to each solution. Three numerical experiments were performed to verify the effectiveness and efficiency of EPSO[Formula: see text] compared with other simulation–optimization approaches, namely particle swarm optimization (PSO) and the genetic algorithm (GA) with both optimal computing budget allocation (OCBA) and the ORAS. The experimental results reveal that the solution quality of EPSO improved compared with that of PSO and GA, and the ORAS provides a more efficient allocation of the number of replications compared with the OCBA in the three experiments. Finally, the proposed approach also provides an elite set at the end of the algorithm, instead of a single optimal solution, to support decision-making.

Approximate dynamic programming for stochastic resource allocation problems

A stochastic resource allocation model, based on the principles of Markov decision processes &#x0028 MDPs &#x0029 , is proposed in this paper. In particular, a general-purpose framework is developed, which takes into account resource requests for both instant and future needs. The considered framework can handle two types of reservations &#x0028 i.e., specified and unspecified time interval reservation requests &#x0029 , and implement an overbooking business strategy to further increase business revenues. The resulting dynamic pricing problems can be regarded as sequential decision-making problems under uncertainty, which is solved by means of stochastic dynamic programming &#x0028 DP &#x0029 based algorithms. In this regard, Bellman’s backward principle of optimality is exploited in order to provide all the implementation mechanisms for the proposed reservation pricing algorithm. The curse of dimensionality, as the inevitable issue of the DP both for instant resource requests and future resource reservations, occurs. In particular, an approximate dynamic programming &#x0028 ADP &#x0029 technique based on linear function approximations is applied to solve such scalability issues. Several examples are provided to show the effectiveness of the proposed approach.

Solving bi-objective uncertain stochastic resource allocation problems by the CVaR-based risk measure and decomposition-based multi-objective evolutionary algorithms

This paper investigates the uncertain stochastic resource allocation problem in which the results of a given allocation of resources are described as probabilities and these probabilities are considered to be uncertain from practical aspects. Here uncertainties are introduced by assuming that these probabilities depend on random parameters which are impacted by various factors. The redundancy allocation problem (RAP) and the multi-stage weapon-target assignment (MWTA) problem are special cases of stochastic resource allocation problems. Bi-objective models for the uncertain RAP and MWTA problem in which the conditional value-at-risk measure is used to control the risk brought by uncertainties are presented in this paper. The bi-objective formulation covers the objectives of minimizing the risk of failure of completing activities and the resulting cost of resources. With the aim of determining referenced Pareto fronts, a linearized formulation and an approximated linear formulation are put forward for RAPs and MWTA problems based on problem-specific characteristics, respectively. Two state-of-the-art decomposition-based multi-objective evolutionary algorithms (i.e., MOEA/D-AWA and DMOEA- $$\varepsilon \hbox {C}$$ ) are used to solve the formulated bi-objective problem. In view of differences between MOEA/D-AWA and DMOEA- $$\varepsilon \hbox {C}$$ , two matching schemes inspired by DMOEA- $$\varepsilon \hbox {C}$$ are proposed and embedded in MOEA/D-AWA. Numerical experiments have been performed on a set of uncertain RAP and MWTA instances. Experimental results demonstrate that DMOEA- $$\varepsilon \hbox {C}$$ outperforms MOEA/D-AWA on the majority of test instances and the superiority of DMOEA- $$\varepsilon \hbox {C}$$ can be ascribed to the $$\varepsilon $$ -constraint framework.

Real-time stochastic optimal scheduling of large-scale electric vehicles: A multidimensional approximate dynamic programming approach

This paper studies the real-time charging optimization (RTCO) of large-scale electric vehicles (EVs), which is a multistage and multidimensional stochastic resource allocation problem. In order to handle the complex RTCO of large-scale EVs, a multidimensional approximate dynamic programming (ADP-RTCO) is designed for the sequent optimal decision makings. The hierarchy of ADP-RTCO contains two layers. In the upper layer, RTCO of large-scale EVs is formulated as a multidimensional energy storage problem by classifying EVs into several virtual EV clusters (EVCs). Then, temporal difference learning based policy iteration method is used to obtain the value function approximation for each EVC. In the lower layer, priority based reallocation algorithm is employed to obtain the detailed charging power for connected EVs. ADP-RTCO is designed with on-line learning ability to enhance its adaptability and robustness to ambiguous environment. Comprehensive simulation results demonstrate the optimality and robustness of ADP-RTCO in both cost-saving and load-flattening problems. Furthermore, ADP-RTCO can obtain higher-quality solution compared with other algorithms and is applicable for RTCO of large-scale EVs under uncertainties due to its high computation efficiency.

Network Resource Allocation via Stochastic Subgradient Descent: Convergence Rate

This paper considers a general stochastic resource allocation problem that arises widely in wireless networks, cognitive radio, networks, smart-grid communications, and cross-layer design. The problem formulation involves expectations with respect to a collection of random variables with unknown distributions, representing exogenous quantities such as channel gain, user density, or spectrum occupancy. We consider the constant step-size stochastic dual subgradient descent (SDSD) method that has been widely used for online resource allocation in networks. The problem is solved in dual domain, which results in a primal resource allocation subproblem at each time instant. The goal here is to characterize the non-asymptotic behavior of such stochastic resource allocations in an almost sure sense. It is well known that with a step size of $\epsilon $ , SDSD converges to an $\mathcal {O}(\epsilon)$ -sized neighborhood of the optimum. In practice, however, there exists a trade-off between the rate of convergence and the choice of $\epsilon $ . This paper establishes a convergence rate result for the SDSD algorithm that precisely characterizes this trade-off. Toward this end, a novel stochastic bound on the gap between the objective function and the optimum is developed. The asymptotic behavior of the stochastic term is characterized in an almost sure sense, thereby generalizing the existing results for the stochastic subgradient methods. For the stochastic resource allocation problem at hand, the result explicates the rate with which the allocated resources become near-optimal. As an application, the power and user-allocation problem in device-to-device networks is formulated and solved using the SDSD algorithm. Further intuition on the rate results is obtained from the verification of the regularity conditions and accompanying simulation results.

Novel hybrid approach with elite group optimal computing budget allocation for the stochastic multimodal problem

The stochastic multimodal problem commonly arises in the search for efficiency and solution quality in practice. In this study, a hybrid approach is developed for the stochastic multimodal problem. The proposed approach comprises particle swarm optimization (PSO), constriction factor PSO (CPSO), and elite group optimal computing budget allocation (EGOCBA). CPSO or PSO is applied to determine the correct direction in the design space, and EGOCBA is adopted to allocate the appropriate number of samples to each alternative and provide reliable evaluations and identifications to rank particles in the CPSO or PSO procedure. This work improves the searching efficiency of optimal computing budget allocation (OCBA) in the stochastic multimodal problem. Several alternatives referred to as the “elite group,” the performance of which is close to that of the current best solution in each swarm, absorb most of the computing budget of OCBA. However, distinguishing the best solution from the elite group is time consuming because of the various local and global optima in the stochastic multimodal problem. Therefore, this study proposes EGOCBA that avoids extra computing costs for the elite group by implementing a new selection procedure. This EGOCBA reserves all the solutions of the elite group and filters them to form an optimal set that contains all the best solutions having an equal mean performance without a significant difference. These tasks are achieved through a confidence interval test in the end of the algorithm. The optimal set can provide more than one best solution to support multiple decisions depending on different decision variables. Two experiments are conducted on stochastic multimodal optimization problem and stochastic resource allocation problem. Experimental results reveal the efficiency and effectiveness of the proposed approach in deriving multiple optimal solutions in a multimodal class and stochastic environment.

Static and dynamic resource allocation models for single-leg transportation markets with service disruptions

This work considers the stochastic resource allocation problem for single-leg transportation markets with service disruptions. The single-period version of the problem is formulated as a stochastic model with arbitrarily distributed resource capacity. We then completely characterize the optimal solution to the stochastic model. The multi-period version of the problem is formulated as a dynamic programming model. We characterize the monotone structure of the optimal solution for the dynamic model under uniform resource consumption rates. For the case with general resource consumption rates, a counterexample is provided to show that there exist cases where the optimal solution is not monotone.

Scenario-based stochastic resource allocation with uncertain probability parameters

The stochastic resource allocation (SRA) problem is an extensive class of combinatorial optimization problems widely existing in complex systems such as communication networks and unmanned systems. In SRA, the ability of a resource to complete a task is described by certain probability, and the objective is to maximize the reward by appropriately assigning available resources to different tasks. This paper is aimed at an important branch of SRA, that is, stochastic SRA (SSRA) for which the probability for resources to complete tasks is also uncertain. Firstly, a general SSRA model with multiple independent uncertain parameters (GSSRA-MIUP) is built to formulate the problem. Then, a scenario-based reformulation which can address multi-source uncertainties is proposed to facilitate the problem-solving process. Secondly, in view of the superiority of the differential evolution algorithm in real-valued optimization, a discrete version of this algorithm was originally proposed and further combined with a specialized local search to create an efficient hybrid optimizer. The hybrid algorithm is compared with the discrete differential evolution algorithm, a pure random sampling method, as well as a restart local search method. Experimental results show that the proposed hybrid optimizer has obvious advantages in solving GSSRA-MIUP problems.

Dynamic Control of Birth-and-Death Restless Bandits: Application to Resource-Allocation Problems

We develop a unifying framework to obtain efficient index policies for restless multi-armed bandit problems with birth-and-death state evolution. This is a broad class of stochastic resource allocation problems whose objective is to determine efficient policies to share resources among competing projects. In a seminal work, Whittle developed a methodology to derive well-performing (Whittle’s) index policies that are obtained by solving a relaxed version of the original problem. Our first main contribution is the derivation of a closed-form expression for Whittle’s index as a function of the steady-state probabilities. In some particular cases, qualitative insights can be obtained from its expression; nevertheless, it requires several technical conditions to be verified. We, therefore, formulate a fluid version of the relaxed optimization problem, and in our second main contribution, we develop a fluid index policy. The latter does provide qualitative insights and it is equivalent to Whittle’s index policy in the light-traffic regime. The applicability of our approach is illustrated by two important problems: optimal class selection and optimal load balancing. Allowing state-dependent capacities, we can model important phenomena, e.g., power-aware server-farms and opportunistic scheduling in wireless systems. Whittle’s index and our fluid index policy show remarkably good performance in numerical simulations.

Stochastic resource allocation for containerized cargo transportation networks when capacities are uncertain

We consider the stochastic resource allocation problem for containerized cargo transportation with uncertain capacities and network effects, in which a freight operator needs to allocate a certain amount of capacity to each product to maximize the expected profit. We formulate the problem as a constrained stochastic programming model and provide theoretical results that completely characterize the optimal solution to the model under a special case. Under a general case, we build an approximation model of the problem and propose a sampling based algorithm to solve the approximation model. A number of numerical experiments are offered to test the algorithm.

A hybrid particle swarm optimization with local search for stochastic resource allocation problem

Discrete and stochastic resource allocation problems are difficult to solve because of the combinatorial explosion of feasible search space. Resource management is important area and a significant challenge is encountered when considering the relationship between uncertainty factors and inputs and outputs of processes in the service and manufacturing systems. These problems are unavailable in closed-form expressions for objective function. In this paper, we propose $$\hbox {PSO}_{\mathrm{OTL}}$$ , a new approach of the hybrid simulation optimization structure, to achieve a near optimal solution with few simulation replications. The basic search algorithm of particle swarm optimization (PSO) is applied for proper exploration and exploitation. Optimal computing budget allocation combined with PSO is used to reduce simulation replications and provide reliable evaluations and identifications for ranking particles of the PSO procedure. Two-sample t tests were used to reserve good particles and maintain the diversity of the swarm. Finally, trapping in local optimum in the design space was overcome by using the local search method to generate new diverse particles when a similar particle exists in the swarm. This study proposed intelligent manufacturing technology, called the $$\hbox {PSO}_{\mathrm{OTL}}$$ , and compared it with four algorithms. The results obtained demonstrate the superiority of $$\hbox {PSO}_{\mathrm{OTL}}$$ in terms of search quality and computational cost reduction.

Stochastic resource allocation using a predictor-based heuristic for optimization via simulation

Some combinatorial stochastic resource allocation problems lack algebraically defined objective functions and hence require optimization via simulation as a mechanism for obtaining good solutions. For this class of problems, we propose a new predictor-based heuristic that uses a distance criterion to perform the solution search. To demonstrate our solution approach, we apply this heuristic to the problem of selecting the proper design configuration of an unmanned aerial system (UAS) fleet so as to maximize mission effectiveness. We compare our approach to black box optimization via simulation approaches (two tabu search-based procedures and a greedy heuristic) and glean both methodological and practical insights.

STOCHASTIC RESOURCE ALLOCATION IN MULTIAGENT ENVIRONMENTS: AN APPROACH BASED ON DISTRIBUTED Q-VALUES AND BOUNDED REAL-TIME DYNAMIC PROGRAMMING

This paper contributes to solve effectively stochastic resource allocation problems in multiagent environments. To address it, a distributed Q-values approach is proposed when the resources are distributed among agents a priori, but the actions made by an agent may influence the reward obtained by at least another agent. This distributed Q-values approach allows to coordinate agents' reward and thus permits to reduce the set of states and actions to consider. On the other hand, when the resources are available to all agents, no distributed Q-values is possible and tight lower and upper bounds are proposed for existing heuristic search algorithms. Our experimental results demonstrate the efficiency of our distributed Q-values in terms of planning time as well as our tight bounds in terms of fast convergence and reduction of backups.

Optimal allocation of stock levels and stochastic customer demands to a capacitated resource

This paper considers a new class of stochastic resource allocation problems that requires simultaneously determining the customers that a capacitated resource must serve and the stock levels of multiple items that may be used in meeting these customers’ demands. Our model considers a reward (revenue) for serving each assigned customer, a variable cost for allocating each item to the resource, and a shortage cost for each unit of unsatisfied customer demand in a single-period context. The model maximizes the expected profit resulting from the assignment of customers and items to the resource while obeying the resource capacity constraint. We provide an exact solution method for this mixed integer nonlinear optimization problem using a Generalized Benders Decomposition approach. This decomposition approach uses Lagrangian relaxation to solve a constrained multi-item newsvendor subproblem and uses CPLEX to solve a mixed-integer linear master problem. We generate Benders cuts for the master problem by obtaining a series of subgradients of the subproblem’s convex objective function. In addition, we present a family of heuristic solution approaches and compare our methods with several MINLP (Mixed-Integer Nonlinear Programming) commercial solvers in order to benchmark their efficiency and quality.

A dynamic programming implemented resource competition game theoretic model

Resource competition is commonly occurred in animal populations and studied intensively by researchers. Previous studies have applied game theoretic model by finding Nash equilibrium to investigate this phenomenon. However computation of the Nash equilibrium requires an understanding of the payoff matrix that allocates the rewards received by players when they adopt each of the strategies in the game. In our study we present a dynamic programming implemented framework to compute 2 × 2 intraspecific finite resource allocation game's payoff matrix explicitly. We assume that two distinct types of individuals, aggressive and non-aggressive, are in the population. Then we divide the entire animal development period into three different stages: initialization, quasilinear growth and termination. Each stage for each type of players is specified with their own development coefficient, which determines how resource consumption could convert into strength as reward. Each player has equal and finite resource at the beginning of their development and fights against other players in the population to maximize its own potential reward. Based on these assumptions it is reasonable to use backward induction dynamic programming to compute payoff matrix. We present numerical examples for three different types of aggressive individuals and compute the payoff matrices correspondingly. Then we use the derived payoff matrices to determine the Nash equilibrium and Evolutionary Stable Strategy. Our research provide a framework for future quantitative studies on animal resource competition problems and could be expanded to n-players interspecific stochastic asymmetric resource allocation problem by changing some settings of dynamic programming formulation.

Model Predictive Control for Stochastic Resource Allocation

In this paper, we consider a class of stochastic resource allocation problems where resources assigned to a task may fail probabilistically to complete assigned tasks. Failures to complete a task are observed before new resource allocations are selected. The resulting temporal resource allocation problem is a stochastic control problem, with a discrete state space and control space that grow in cardinality exponentially with the number of tasks. We modify this optimal control problem by expanding the admissible control space, and show that the resulting control problem can be solved exactly by efficient algorithms in time that grows nearly linear with the number of tasks. The approximate control problem also provides a bound on the achievable performance for the original control problem. The approximation is used as part of a model predictive control (MPC) algorithm to generate resource allocations over time in response to information on task completion status. We show in computational experiments that, for single resource class problems, the resulting MPC algorithm achieves nearly the same performance as the optimal dynamic programming algorithm while reducing computation time by over four orders of magnitude. In multiple resource class experiments involving 1000 tasks, the model predictive control performance is within 4% of the performance bound obtained by the solution of the expanded control space problem.

A single-resource allocation problem with Poisson resource requirements

We consider a stochastic resource allocation problem that generalizes the knapsack problem to account for random item weights that follow a Poisson distribution. When the sum of realized weights exceeds capacity, a penalty cost is incurred. We wish to select the items that maximize expected profit. We provide an effective solution method and illustrate the advantages of this approach via computational experiments.