Stochastic Resource Research Articles

Unit Commitment Towards Decarbonized Network Facing Fixed and Stochastic Resources Applying Water Cycle Optimization

This paper presents a trustworthy unit commitment study to schedule both Renewable Energy Resources (RERs) with conventional power plants to potentially decarbonize the electrical network. The study has employed a system with three IEEE thermal (coal-fired) power plants as dispatchable distributed generators, one wind plant, one solar plant as stochastic distributed generators, and Plug-in Electric Vehicles (PEVs) which can work either loads or generators based on their charging schedule. This paper investigates the unit commitment scheduling objective to minimize the Combined Economic Emission Dispatch (CEED). To reduce combined emission costs, integrating more renewable energy resources (RER) and PEVs, there is an essential need to decarbonize the existing system. Decarbonizing the system means reducing the percentage of CO2 emissions. The uncertain behavior of wind and solar energies causes imbalance penalty costs. PEVs are proposed to overcome the intermittent nature of wind and solar energies. It is important to optimally integrate and schedule stochastic resources including the wind and solar energies, and PEVs charge and discharge processes with dispatched resources; the three IEEE thermal (coal-fired) power plants. The Water Cycle Optimization Algorithm (WCOA) is an efficient and intelligent meta-heuristic technique employed to solve the economically emission dispatch problem for both scheduling dispatchable and stochastic resources. The goal of this study is to obtain the solution for unit commitment to minimize the combined cost function including CO2 emission costs applying the Water Cycle Optimization Algorithm (WCOA). To validate the WCOA technique, the results are compared with the results obtained from applying the Dynamic Programming (DP) algorithm, which is considered as a conventional numerical technique, and with the Genetic Algorithm (GA) as a meta-heuristic technique.

Network Resource Allocation via Stochastic Subgradient Descent: Convergence Rate

This paper considers a general stochastic resource allocation problem that arises widely in wireless networks, cognitive radio, networks, smart-grid communications, and cross-layer design. The problem formulation involves expectations with respect to a collection of random variables with unknown distributions, representing exogenous quantities such as channel gain, user density, or spectrum occupancy. We consider the constant step-size stochastic dual subgradient descent (SDSD) method that has been widely used for online resource allocation in networks. The problem is solved in dual domain, which results in a primal resource allocation subproblem at each time instant. The goal here is to characterize the non-asymptotic behavior of such stochastic resource allocations in an almost sure sense. It is well known that with a step size of $\epsilon $ , SDSD converges to an $\mathcal {O}(\epsilon)$ -sized neighborhood of the optimum. In practice, however, there exists a trade-off between the rate of convergence and the choice of $\epsilon $ . This paper establishes a convergence rate result for the SDSD algorithm that precisely characterizes this trade-off. Toward this end, a novel stochastic bound on the gap between the objective function and the optimum is developed. The asymptotic behavior of the stochastic term is characterized in an almost sure sense, thereby generalizing the existing results for the stochastic subgradient methods. For the stochastic resource allocation problem at hand, the result explicates the rate with which the allocated resources become near-optimal. As an application, the power and user-allocation problem in device-to-device networks is formulated and solved using the SDSD algorithm. Further intuition on the rate results is obtained from the verification of the regularity conditions and accompanying simulation results.

Preallocation and Planning Under Stochastic Resource Constraints

Resource constraints frequently complicate multi-agent planning problems. Existing algorithms for resource-constrained, multi-agent planning problems rely on the assumption that the constraints are deterministic. However, frequently resource constraints are themselves subject to uncertainty from external influences. Uncertainty about constraints is especially challenging when agents must execute in an environment where communication is unreliable, making on-line coordination difficult. In those cases, it is a significant challenge to find coordinated allocations at plan time depending on availability at run time. To address these limitations, we propose to extend algorithms for constrained multi-agent planning problems to handle stochastic resource constraints. We show how to factorize resource limit uncertainty and use this to develop novel algorithms to plan policies for stochastic constraints. We evaluate the algorithms on a search-and-rescue problem and on a power-constrained planning domain where the resource constraints are decided by nature. We show that plans taking into account all potential realizations of the constraint obtain significantly better utility than planning for the expectation, while causing fewer constraint violations.

The Role of Data-Driven Priors in Multi-Agent Crowd Trajectory Estimation

Resource constraints frequently complicate multi-agent planning problems. Existing algorithms for resource-constrained, multi-agent planning problems rely on the assumption that the constraints are deterministic. However, frequently resource constraints are themselves subject to uncertainty from external influences. Uncertainty about constraints is especially challenging when agents must execute in an environment where communication is unreliable, making on-line coordination difficult. In those cases, it is a significant challenge to find coordinated allocations at plan time depending on availability at run time. To address these limitations, we propose to extend algorithms for constrained multi-agent planning problems to handle stochastic resource constraints. We show how to factorize resource limit uncertainty and use this to develop novel algorithms to plan policies for stochastic constraints. We evaluate the algorithms on a search-and-rescue problem and on a power-constrained planning domain where the resource constraints are decided by nature. We show that plans taking into account all potential realizations of the constraint obtain significantly better utility than planning for the expectation, while causing fewer constraint violations.

An adaptive overload threshold selection process using Markov decision processes of virtual machine in cloud data center

In Cloud Computing (CC), the cost for computation and energy is less by current cloud data centers because it exploits virtualization for an effective resource management. The Virtual Machine (VM) migration authorizes virtualization because it mitigates the difficulties of dynamic workload by repositioning VMs within cloud data centers. Through VM migration many goals of resource management are attained like load balancing, power management, fault tolerance, and system maintenance. The overload threshold is one of the key criterions to determine whether a host is overloaded or not. Achieving desired balance in guaranteeing quality of service, improving resource utilization and degrading energy consumption in data centers is the expected results of any overload threshold selection strategies. But, it is difficult due to the stochastic resource demands of VMs. In this paper, to address this problem, the overload threshold selection is modelled as a Markov decision process. With the solution of the improved Bellman optimality equation by the value iteration method, the optimization model is resolved, and the optimum overload threshold is adaptively selected. The hybrid processes are summarized as the Markov decision processes based adaptive overload threshold selection algorithm. Validations and comparisons are performed to illustrate its effectiveness and efficiency.

Pricing contracts and planning stochastic resources in brand display advertising

Publishers, advertisers, and users are the three players in online display advertising, where the traded object is an impression (i.e., a user's visit). The publisher usually sells impressions months in advance of their physical appearance by signing contracts on prices and quantities with advertisers. With the introduction of ad exchanges, the publisher can also sell impressions that are unsold in the contracting market in a spot market. We build a bi-objective optimization problem to maximize the publisher's expected revenue and the advertisers’ fairness of impression allocation in the presence of a spot market and impression supply uncertainty. Here, the publisher decides on the contract prices and creates a plan for impression allocations. The problem is a non-convex program, solved by integrating local and global heuristic methods. We also set a lower bound and an upper bound to facilitate and justify the solutions. Numerical examples indicate that ignoring supply uncertainty may over-estimate the expected revenue, and that fairness may be achieved by sacrificing a small portion of revenue. However, too much fairness may reduce the revenue. It is also shown that the heuristic algorithms are computationally effective.

On the performance of priority rules for the stochastic resource constrained multi-project scheduling problem

Investigate the performance of priority rules on the stochastic version of RCMPSP.New robustness measures are proposed and the trade-off relationship with quality measures is studied.SP (Serial/Parallel) is used to generate the multi-project instances.Best recommendation is given considering both the quality and robustness measures. The majority of research studies the resource constrained multi-project scheduling problem in a deterministic environment, regardless of the uncertainty nature of the environment. In this paper, we assume that the activity duration is a stochastic variable, and propose two new robustness measures to analyse the performance of priority rules under a stochastic environment. A full factorial experiment is designed to solve the problem and investigate the relationship between project characteristics and the performance of priority rules. Furthermore, a trade-off relationship between the quality and robustness is investigated and the best priority rules are recommended from both a project and portfolio managers perspective.

Applying hybrid Monte Carlo Tree Search methods to Risk-Aware Project Scheduling Problem

In this paper we investigate an application of hybrid Monte Carlo Tree Search (MCTS) based algorithms to solving dynamic decision making problems.We employ UCT (the most popular MCTS approach) in combination with well-known Resource Constrained Project Scheduling Problem (RCPSP) and Stochastic Resource Constrained Project Scheduling Problem (SRCPSP) solvers to devise strategies for a generic and highly dynamic version of RCPSP, which we call Risk-Aware Project Scheduling Problem (RAPSP). We compare these strategies’ performance with results of both pure MCTS approach and non-MCTS solvers for projects of varied characteristics. We reach a conclusion that proposed hybrid simulation-heuristic methods are promising approaches to dynamic decision making problems, RAPSP in particular. Consequently, we argue that more research effort should be directed to applications of MCTS algorithm outside the domain of game-playing, with which it is commonly associated.At the same time, to the best of our knowledge, this paper is the first attempt at defining generalized SRCPSP model encompassing arbitrary risks and risk response / mitigation strategies as an optimization problem and applying Computational Intelligence methods to build fully-automated decision making systems. We strongly believe it to be a research direction worth further investigation, combining project scheduling, risk management and metaheuristic optimization techniques into a well-defined platform allowing direct comparisons of different strategies.

Stochastic Resource Provisioning for Containerized Multi-Tier Web Services in Clouds

Under today's bursty web traffic, the fine-grained per-container control promises more efficient resource provisioning for web services and better resource utilization in cloud datacenters. In this paper, we present Two -stage S tochastic P rogramming R esource A llocator (2SPRA). It optimizes resource provisioning for containerized n-tier web services in accordance with fluctuations of incoming workload to accommodate predefined SLOs on response latency. In particular, 2SPRA is capable of minimizing resource over-provisioning by addressing dynamics of web traffic as workload uncertainty in a native stochastic optimization model. Using special-purpose OpenOpt optimization framework, we fully implement 2SPRA in Python and evaluate it against three other existing allocation schemes, in a Docker-based CoreOS Linux VMs on Amazon EC2. We generate workloads based on four real-world web traces of various traffic variations: AOL, WorldCup98, ClarkNet, and NASA. Our experimental results demonstrate that 2SPRA achieves the minimum resource over-provisioning outperforming other schemes. In particular, 2SPRA allocates only 6.16 percent more than application's actual demand on average and at most 7.75 percent in the worst case. It achieves 3x further reduction in total resources provisioned compared to other schemes delivering overall cost-savings of 53.6 percent on average and up to 66.8 percent. Furthermore, 2SPRA demonstrates consistency in its provisioning decisions and robust responsiveness against workload fluctuations.

Stochastic Averaging for Constrained Optimization With Application to Online Resource Allocation

Existing approaches to resource allocation for nowadays stochastic networks are challenged to meet fast convergence and tolerable delay requirements. The present paper leverages online learning advances to facilitate stochastic resource allocation tasks. By recognizing the central role of Lagrange multipliers, the underlying constrained optimization problem is formulated as a machine learning task involving both training and operational modes, with the goal of learning the sought multipliers in a fast and efficient manner. To this end, an order-optimal offline learning approach is developed first for batch training, and it is then generalized to the online setting with a procedure termed learn-and-adapt. The novel resource allocation protocol permeates benefits of stochastic approximation and statistical learning to obtain low-complexity online updates with learning errors close to the statistical accuracy limits, while still preserving adaptation performance, which in the stochastic network optimization context guarantees queue stability. Analysis and simulated tests demonstrate that the proposed data-driven approach improves the delay and convergence performance of existing resource allocation schemes.

Novel hybrid approach with elite group optimal computing budget allocation for the stochastic multimodal problem

The stochastic multimodal problem commonly arises in the search for efficiency and solution quality in practice. In this study, a hybrid approach is developed for the stochastic multimodal problem. The proposed approach comprises particle swarm optimization (PSO), constriction factor PSO (CPSO), and elite group optimal computing budget allocation (EGOCBA). CPSO or PSO is applied to determine the correct direction in the design space, and EGOCBA is adopted to allocate the appropriate number of samples to each alternative and provide reliable evaluations and identifications to rank particles in the CPSO or PSO procedure. This work improves the searching efficiency of optimal computing budget allocation (OCBA) in the stochastic multimodal problem. Several alternatives referred to as the “elite group,” the performance of which is close to that of the current best solution in each swarm, absorb most of the computing budget of OCBA. However, distinguishing the best solution from the elite group is time consuming because of the various local and global optima in the stochastic multimodal problem. Therefore, this study proposes EGOCBA that avoids extra computing costs for the elite group by implementing a new selection procedure. This EGOCBA reserves all the solutions of the elite group and filters them to form an optimal set that contains all the best solutions having an equal mean performance without a significant difference. These tasks are achieved through a confidence interval test in the end of the algorithm. The optimal set can provide more than one best solution to support multiple decisions depending on different decision variables. Two experiments are conducted on stochastic multimodal optimization problem and stochastic resource allocation problem. Experimental results reveal the efficiency and effectiveness of the proposed approach in deriving multiple optimal solutions in a multimodal class and stochastic environment.

A mixed-integer linear programming model for solving fuzzy stochastic resource constrained project scheduling problem

This paper addresses resource-constrained project scheduling problem with mixed uncertainty of randomness and fuzziness (FS-RCPSP). The activity durations are considered to be fuzzy random variables. A resource flow network based mathematical model with fuzzy random variables is presented. Then, this model is transformed into a mixed-integer linear programming model with crisp variables. The CPLEX 12.6.0.1 solver in AIMMS (2014) is employed for applying the proposed model to solve 960 benchmark instances generated from the well-known sets J30 and J60 in PSPLIB. The computational results are encouraging and indicate the ability of the proposed model to handle the FS-RCPSP.

Static and dynamic resource allocation models for single-leg transportation markets with service disruptions

This work considers the stochastic resource allocation problem for single-leg transportation markets with service disruptions. The single-period version of the problem is formulated as a stochastic model with arbitrarily distributed resource capacity. We then completely characterize the optimal solution to the stochastic model. The multi-period version of the problem is formulated as a dynamic programming model. We characterize the monotone structure of the optimal solution for the dynamic model under uniform resource consumption rates. For the case with general resource consumption rates, a counterexample is provided to show that there exist cases where the optimal solution is not monotone.

Stochastic and Periodic Resource Scheduling Problem in Home Health Care

Stochastic and Periodic Resource Scheduling Problem in Home Health Care

A Market Mechanism for Sustainable and Efficient Resource Use Under Uncertainty

Sustainability and efficiency are potentially conflicting social objectives in natural resource management. We propose a market mechanism to allocate use rights over a stochastic resource to private managers. The mechanism endogenously determines the maximal tenure length guaranteeing that the sustainability goal is obeyed for sure over the entire period. In addition, the mechanism achieves efficiency, i.e. it maximizes the expected present value of resource rents that accrue to society. Potential applications include improved fishing agreements between developing countries and distant-water fishing fleets.

Multi-agent credit assignment in stochastic resource management games

AbstractMulti-agent systems (MASs) are a form of distributed intelligence, where multiple autonomous agents act in a common environment. Numerous complex, real world systems have been successfully optimized using multi-agent reinforcement learning (MARL) in conjunction with the MAS framework. In MARL agents learn by maximizing a scalar reward signal from the environment, and thus the design of the reward function directly affects the policies learned. In this work, we address the issue of appropriate multi-agent credit assignment in stochastic resource management games. We propose two new stochastic games to serve as testbeds for MARL research into resource management problems: the tragic commons domain and the shepherd problem domain. Our empirical work evaluates the performance of two commonly used reward shaping techniques: potential-based reward shaping and difference rewards. Experimental results demonstrate that systems using appropriate reward shaping techniques for multi-agent credit assignment can achieve near-optimal performance in stochastic resource management games, outperforming systems learning using unshaped local or global evaluations. We also present the first empirical investigations into the effect of expressing the same heuristic knowledge in state- or action-based formats, therefore developing insights into the design of multi-agent potential functions that will inform future work.

Scenario-based stochastic resource allocation with uncertain probability parameters

The stochastic resource allocation (SRA) problem is an extensive class of combinatorial optimization problems widely existing in complex systems such as communication networks and unmanned systems. In SRA, the ability of a resource to complete a task is described by certain probability, and the objective is to maximize the reward by appropriately assigning available resources to different tasks. This paper is aimed at an important branch of SRA, that is, stochastic SRA (SSRA) for which the probability for resources to complete tasks is also uncertain. Firstly, a general SSRA model with multiple independent uncertain parameters (GSSRA-MIUP) is built to formulate the problem. Then, a scenario-based reformulation which can address multi-source uncertainties is proposed to facilitate the problem-solving process. Secondly, in view of the superiority of the differential evolution algorithm in real-valued optimization, a discrete version of this algorithm was originally proposed and further combined with a specialized local search to create an efficient hybrid optimizer. The hybrid algorithm is compared with the discrete differential evolution algorithm, a pure random sampling method, as well as a restart local search method. Experimental results show that the proposed hybrid optimizer has obvious advantages in solving GSSRA-MIUP problems.

Dynamic Control of Birth-and-Death Restless Bandits: Application to Resource-Allocation Problems

We develop a unifying framework to obtain efficient index policies for restless multi-armed bandit problems with birth-and-death state evolution. This is a broad class of stochastic resource allocation problems whose objective is to determine efficient policies to share resources among competing projects. In a seminal work, Whittle developed a methodology to derive well-performing (Whittle’s) index policies that are obtained by solving a relaxed version of the original problem. Our first main contribution is the derivation of a closed-form expression for Whittle’s index as a function of the steady-state probabilities. In some particular cases, qualitative insights can be obtained from its expression; nevertheless, it requires several technical conditions to be verified. We, therefore, formulate a fluid version of the relaxed optimization problem, and in our second main contribution, we develop a fluid index policy. The latter does provide qualitative insights and it is equivalent to Whittle’s index policy in the light-traffic regime. The applicability of our approach is illustrated by two important problems: optimal class selection and optimal load balancing. Allowing state-dependent capacities, we can model important phenomena, e.g., power-aware server-farms and opportunistic scheduling in wireless systems. Whittle’s index and our fluid index policy show remarkably good performance in numerical simulations.

Stochastic resource allocation for containerized cargo transportation networks when capacities are uncertain

We consider the stochastic resource allocation problem for containerized cargo transportation with uncertain capacities and network effects, in which a freight operator needs to allocate a certain amount of capacity to each product to maximize the expected profit. We formulate the problem as a constrained stochastic programming model and provide theoretical results that completely characterize the optimal solution to the model under a special case. Under a general case, we build an approximation model of the problem and propose a sampling based algorithm to solve the approximation model. A number of numerical experiments are offered to test the algorithm.

Achieving Low-Delay and Fast-Convergence in Stochastic Network Optimization

Due to the rapid growth of mobile data demands, there have been significant interests in stochastic resource control and optimization for wireless networks. Although significant advances have been made in stochastic network optimization theory, to date, most of the existing approaches are plagued by either slow convergence or unsatisfactory delay performances. To address these challenges, in this paper, we develop a new stochastic network optimization framework inspired by the Nesterov accelerated gradient method. We show that our proposed Nesterovian approach offers utility-optimality, fast-convergence, and significant delay reduction in stochastic network optimization. Our contributions in this paper are three-fold: i) we propose a Nesterovian joint congestion control and routing/scheduling framework for both single-hop and multi-hop wireless networks; ii) we establish the utility optimality and queueing stability of the proposed Nesterovian method, and analytically characterize its delay reduction and convergence speed; and iii) we show that the proposed Nesterovian approach offers a three-way performance control between utility-optimality, delay, and convergence.