Stochastic Processing Networks Research Articles

Design and Analysis of Stochastic Processing and Matching Networks

Stochastic Processing Networks (SPNs) [24] are ubiquitous in engineering with applications in Data Centers (eg. packet routing), Telecommunication (eg. call centers), Transportation, etc. Stochastic Matching Networks (SMNs) are newly emerging variation of SPNs where 'jobs' and 'servers' arrive in the system and are matched using a decision rule. SMNs have applications in Online matching markets (eg. uber platform), Healthcare (eg. kidney transplant), Quantum Networks, Assemble-to-order (eg. manufacturing plants) and many other crucial problems. The aim of my research is to design provably efficient decision-making policies for Stochastic Processing Networks (SPNs) and Stochastic Matching Networks (SMNs) and develop analytical tools to obtain performance guarantees on their throughput and delay.

Logarithmic heavy traffic error bounds in generalized switch and load balancing systems

AbstractMotivated by applications to wireless networks, cloud computing, data centers, etc., stochastic processing networks have been studied in the literature under various asymptotic regimes. In the heavy traffic regime, the steady-state mean queue length is proved to be $\Theta({1}/{\epsilon})$ , where $\epsilon$ is the heavy traffic parameter (which goes to zero in the limit). The focus of this paper is on obtaining queue length bounds on pre-limit systems, thus establishing the rate of convergence to heavy traffic. For the generalized switch, operating under the MaxWeight algorithm, we show that the mean queue length is within $\textrm{O}({\log}({1}/{\epsilon}))$ of its heavy traffic limit. This result holds regardless of the complete resource pooling (CRP) condition being satisfied. Furthermore, when the CRP condition is satisfied, we show that the mean queue length under the MaxWeight algorithm is within $\textrm{O}({\log}({1}/{\epsilon}))$ of the optimal scheduling policy. Finally, we obtain similar results for the rate of convergence to heavy traffic of the total queue length in load balancing systems operating under the ‘join the shortest queue’ routeing algorithm.

Asymptotically Optimal Control of a Stochastic Processing Network with DAG Jobs

Asymptotically Optimal Control of a Stochastic Processing Network with DAG Jobs

Diffusion Approximation for Fair Resource Control—Interchange of Limits Under a Moment Condition

In a prior study [Ye HQ, Yao DD (2016) Diffusion limit of fair resource control–Stationary and interchange of limits. Math. Oper. Res. 41(4):1161–1207.] focusing on a class of stochastic processing network with fair resource control, we justified the diffusion approximation (in the context of the interchange of limits) provided that the pth moment of the workloads are bounded. To this end, we introduced the so-called bounded workload condition, which requires the workload process be bounded by a free process plus the initial workload. This condition is for a derived process, the workload, as opposed to primitives such as arrival processes and service requirements; as such, it could be difficult to verify. In this paper, we establish the interchange of limits under a moment condition of suitable order on the primitives directly: the required order is [Formula: see text] on the moments of the primitive processes so as to bound the pth moment of the workload. This moment condition is trivial to verify, and indeed automatically holds in networks where the primitives have moments of all orders, for instance, renewal arrivals with phase-type interarrival times and independent and identically distributed phase-type service times.

Heavy-traffic Analysis of the Generalized Switch under Multidimensional State Space Collapse

Stochastic Processing Networks that model wired and wireless networks, and other queueing systems, have been studied in heavytraffic limit under the so-called Complete Resource Pooling (CRP) condition. When the CRP condition is not satisfied, heavy-traffic results are known only in the special case of an input-queued switch and bandwidth-sharing network. In this paper, we consider a very general queueing system called the 'generalized switch' that includes wireless networks under fading, data center networks, input-queued switch, etc. The primary contribution of this paper is to present the exact value of the steadystate mean of certain linear combinations of queue lengths in the heavy-traffic limit under MaxWeight scheduling algorithm. We use the Drift method, and we also present a negative result that it is not possible to obtain the remaining linear combinations (and consequently all the individual mean queue lengths) using this method. We do this by presenting an alternate view of the Drift method in terms of an (under-determined) system of linear equations. Finally, we use this system of equations to obtain upper and lower bounds on all linear combinations of queue lengths.

Dynamic SDN Control Plane Request Assignment in NFV Datacenters

Owing to powerful programmability, software defined networking (SDN) is well aligned with the requirements of NFV datacenters. Although the mainstream SDN frameworks can effectively respond to the requests of control plane, the load distribution among controllers is not balanced due to the request dynamics and request diversity. The existing solutions balance the load of control plane by migrating switches between controllers. However, these solutions are based on the binding between switches and controllers, so they are difficult to adapt to the dynamic and diverse requests. In this paper, we propose a new framework to decouple the binding. The new framework performs modular management for request queues. A complete request queue is provided for each type of request between each switch controller pair, so that the assignment between requests is independent of each other. Based on the proposed framework, we transform the request assignment problem into a variant of the scheduling problem in a Stochastic Processing Network (SPN), and propose a Maximum Pressure Policy (MPP) which can provide runtime guarantees on request throughput and response latency. To fit with the constraints inherent to large-scale deployment, we propose a distributed version of MPP, named DMPP. DMPP runs in local state on each switch and performs scheduling logic for batch requests. We implement a protosystem of our solution and evaluate it on the settings representing real-world scenarios. The results show that our solution can provide guarantees on request throughput and response latency, and significantly outperforms state-of-the-art solutions through a more efficient resource usage

Information and Memory in Dynamic Resource Allocation

We propose a general framework, dubbed stochastic processing under imperfect information (SPII), to study the impact of information constraints and memories on dynamic resource allocation. The framework involves a stochastic processing network (SPN) scheduling problem in which the scheduler may access the system state only through a noisy channel, and resource allocation decisions must be carried out through the interaction between an encoding policy (that observes the state) and an allocation policy (that chooses the allocation). Applications in the management of large-scale data centers and human-in-the-loop service systems are among our chief motivations. We quantify the degree to which information constraints reduce the size of the capacity region in general SPNs and how such reduction depends on the amount of memories available to the encoding and allocation policies. Using a novel metric, capacity factor, our main theorem characterizes the reduction in capacity region (under “optimal” policies) for all nondegenerate channels and across almost all combinations of memory sizes. Notably, the theorem demonstrates, in substantial generality, that (1) the presence of a noisy channel always reduces capacity, (2) more memory for the allocation policy always improves capacity, and (3) more memory for the encoding policy has little to no effect on capacity. Finally, all of our positive (achievability) results are established through constructive, implementable policies.

Heavy-traffic Analysis of the Generalized Switch under Multidimensional State Space Collapse

Stochastic Processing Networks that model wired and wireless networks, and other queueing systems, have been studied in heavytraffic limit under the so-called Complete Resource Pooling (CRP) condition. When the CRP condition is not satisfied, heavy-traffic results are known only in the special case of an input-queued switch and bandwidth-sharing network. In this paper, we consider a very general queueing system called the 'generalized switch' that includes wireless networks under fading, data center networks, input-queued switch, etc. The primary contribution of this paper is to present the exact value of the steadystate mean of certain linear combinations of queue lengths in the heavy-traffic limit under MaxWeight scheduling algorithm. We use the Drift method, and we also present a negative result that it is not possible to obtain the remaining linear combinations (and consequently all the individual mean queue lengths) using this method. We do this by presenting an alternate view of the Drift method in terms of an (under-determined) system of linear equations. Finally, we use this system of equations to obtain upper and lower bounds on all linear combinations of queue lengths.

Dynamic Dispatch and Centralized Relocation of Cars in Ride-Hailing Platforms

We consider a ride-hailing platform that seeks to maximize its profit by dynamically dispatching cars to pick up customers and centrally relocating cars from one area to another. We model the ride-hailing platform as a closed stochastic processing network. Because the problem appears intractable, we resort to an approximate analysis in the heavy-traffic regime and consider the resulting Brownian control problem. This problem is simplified considerably and reduced to a lower-dimensional singular control problem called the workload formulation. We develop a novel algorithm to solve the workload problem numerically. We apply this algorithm to the workload problem derived from the New York City taxi data set. The solution helps us derive a dynamic control policy for the New York City application. In doing so, we prescribe the ride-hailing platform to first solve an offline linear program, whose optimal solution can be interpreted as the optimal static control policy. This solution helps partition the areas of the city into pools of areas. The platform only uses the information on the fraction of cars in the various pools, which reduces the state space dimension significantly, making the problem computationally tractable. When the distribution of cars among the pools is balanced, the platform follows the optimal static control policy. Otherwise, the platform intervenes to move the system to a more balanced state by either dropping demand or using a dispatch or relocation activity that is not used under the optimal static control policy. We demonstrate the effectiveness of the proposed dynamic control policy for the New York City application using a simulation study.

Information, Memory and Capacity in Dynamic Resource Allocation

We propose a general framework, dubbed Stochastic Processing under Imperfect Information (SPII), to study the impact of information constraints and memories on dynamic resource allocation. The framework involves a Stochastic Processing Network (SPN) scheduling problem in which the decision maker may access the system state only through a noisy channel, and resource allocation decisions must be carried out through the interaction between an encoding policy (who observes the state) and allocation policy (who chooses the allocation). Applications in the management of large-scale data centers and human-in-the-loop service systems are among our chief motivations. We quantify the degree to which information constraints reduce the size of the capacity region in general SPNs, and how such reduction depends on the amount of memories available. Using a novel metric, capacity factor, our main theorem characterizes the reduction in capacity region (under "optimal" policies) for all nondegenerate channels, and across almost all combinations of memory sizes. Notably, the theorem demonstrates, in substantial generality, that (1) the presence of a noisy channel always reduces capacity, (2) more memories for the allocation policy always improve capacity, and (3) more memories for the encoding policy have little to no effect on capacity. Finally, all of our positive (achievability) results are established through constructive, implementable policies. Our proof program involves the development of a host of new techniques by combining ideas from information theory, learning and queueing theory. We create a simple yet powerful generalization of the Max-Weight policy, in which individual Markov chains are selected dynamically, in a manner analogous to how schedules are used in a conventional Max-Weight policy.

Joint routing and scheduling control in a two-class network with a flexible server

In this study we analyze a queueing model with a Gurvich structure. In such a network, the controller may route incoming jobs to different classes, but they are routed to the same server. This structure, although it falls into the general class of stochastic processing networks, is somewhat unconventional. We focus on a single-server two-class version of a Gurvich network in this paper. For a Poisson arrival stream and exponential service rates, we develop a Markov decision process representation of the system and prove structural results on optimal routing and scheduling controls. We show that the optimal policy uses $$c\mu $$ scheduling and switching curve routing. We also investigate the fluid model and perturbation expansions thereof, which are useful in deriving near-optimal policies in the original network.

Heavy Traffic Approximation for the Stationary Distribution of a Generalized Jackson Network: The BAR Approach

In the seminal paper of Gamarnik and Zeevi [17], the authors justify the steady-state diffusion approximation of a generalized Jackson network (GJN) in heavy traffic. Their approach involves the so-called limit interchange argument, which has since become a popular tool employed by many others who study diffusion approximations. In this paper we illustrate a novel approach by using it to justify the steady-state approximation of a GJN in heavy traffic. Our approach involves working directly with the basic adjoint relationship (BAR), an integral equation that characterizes the stationary distribution of a Markov process. As we will show, the BAR approach is a more natural choice than the limit interchange approach for justifying steady-state approximations, and can potentially be applied to the study of other stochastic processing networks such as multiclass queueing networks.

Heavy traffic approximation for the stationary distribution of a generalized Jackson network: The BAR approach

In the seminal paper of Gamarnik and Zeevi (2006), the authors justify the steady-state diffusion approximation of a generalized Jackson network (GJN) in heavy traffic. Their approach involves the so-called limit interchange argument, which has since become a popular tool employed by many others who study diffusion approximations. In this paper we illustrate a novel approach by using it to justify the steady-state approximation of a GJN in heavy traffic. Our approach involves working directly with the basic adjoint relationship (BAR), an integral equation that characterizes the stationary distribution of a Markov process. As we will show, the BAR approach is a more natural choice than the limit interchange approach for justifying steady-state approximations, and can potentially be applied to the study of other stochastic processing networks such as multiclass queueing networks.

Construction of Asymptotically Optimal Control for Crisscross Network from a Free Boundary Problem

An asymptotic framework for optimal control of multiclass stochastic processing networks, using formal diffusion approximations under suitable temporal and spatial scaling, by Brownian control problems (BCP) and their equivalent workload formulations (EWF), has been developed by Harrison (1988). This framework has been implemented in many works for constructing asymptotically optimal control policies for a broad range of stochastic network models. To date all asymptotic optimality results for such networks correspond to settings where the solution of the EWF is a reflected Brownian motion in ℝ+ or a wedge in [Formula: see text]. In this work we consider a well studied stochastic network which is perhaps the simplest example of a model with more than one dimensional workload process. In the regime considered here, the singular control problem corresponding to the EWF does not have a simple form explicit solution. However, by considering an associated free boundary problem one can give a representation for an optimal controlled process as a two dimensional reflected Brownian motion in a Lipschitz domain whose boundary is determined by the solution of the free boundary problem. Using the form of the optimal solution we propose a sequence of control policies, given in terms of suitable thresholds, for the scaled stochastic network control problems and prove that this sequence of policies is asymptotically optimal. As suggested by the solution of the EWF, the policy we propose requires a server to idle under certain conditions which are specified in terms of thresholds determined from the free boundary.

Contextuality Scenarios Arising from Networks of Stochastic Processes

An empirical model is a generalization of a probability space. It consists of a simplicial complex of subsets of a class 𝒳 of random variables such that each simplex has an associated probability distribution. The ensuing marginalizations are coherent, in the sense that the distribution on a face of a simplex coincides with the marginal of the distribution over the entire simplex. An empirical model is called contextual if its distributions cannot be obtained by marginalizing a joint distribution over 𝒳. Contextual empirical models arise naturally in quantum theory, giving rise to some of its counter -intuitive statistical consequences. In this paper, we present a different and classical source of contextual empirical models: the interaction among many stochastic processes. We attach an empirical model to the ensuing network in which each node represents an open stochastic process with input and output random variables. The statistical behaviour of the network in the long run makes the empirical model generically contextual and even strongly contextual.

Stochastic Processing Networks

Stochastic processing networks arise as models in manufacturing, telecommunications, transportation, computer systems, the customer service industry, and biochemical reaction networks. Common characteristics of these networks are that they have entities—such as jobs, packets, vehicles, customers, or molecules—that move along routes, wait in buffers, receive processing from various resources, and are subject to the effects of stochastic variability through such quantities as arrival times, processing times, and routing protocols. The mathematical theory of queueing aims to understand, analyze, and control congestion in stochastic processing networks. In this article, we begin by summarizing some of the highlights in the development of the theory of queueing prior to 1990; this includes some exact analysis and development of approximate models for certain queueing networks. We then describe some surprises of the early 1990s and ensuing developments of the past 25 years related to the use of approximate models for analyzing the stability and performance of multiclass queueing networks. We conclude with a description of recent developments for more general stochastic processing networks and point to some open problems.

Construction of asymptotically optimal control for crisscross network from a free boundary problem

An asymptotic framework for optimal control of multiclass stochastic processing networks, using formal diffusion approximations under suitable temporal and spatial scaling, by Brownian control prob...

Directed Information Graphs

We propose a graphical model for representing networks of stochastic processes, the minimal generative model graph. It is based on reduced factorizations of the joint distribution over time. We show that under appropriate conditions, it is unique and consistent with another type of graphical model, the directed information graph, which is based on a generalization of Granger causality. We demonstrate how directed information quantifies Granger causality in a particular sequential prediction setting. We also develop efficient methods to estimate the topological structure from data that obviate estimating the joint statistics. One algorithm assumes upper bounds on the degrees and uses the minimal dimension statistics necessary. In the event that the upper bounds are not valid, the resulting graph is nonetheless an optimal approximation in terms of Kullback-Leibler (KL) divergence. Another algorithm uses near-minimal dimension statistics when no bounds are known, but the distribution satisfies a certain criterion. Analogous to how structure learning algorithms for undirected graphical models use mutual information estimates, these algorithms use directed information estimates. We characterize the sample-complexity of two plug-in directed information estimators and obtain confidence intervals. For the setting when point estimates are unreliable, we propose an algorithm that uses confidence intervals to identify the best approximation that is robust to estimation error. Last, we demonstrate the effectiveness of the proposed algorithms through the analysis of both synthetic data and real data from the Twitter network. In the latter case, we identify which news sources influence users in the network by merely analyzing tweet times.

Models and Insights for Hospital Inpatient Operations: Time-Dependent ED Boarding Time

One key factor contributing to emergency department (ED) overcrowding is prolonged waiting time for admission to inpatient wards, also known as ED boarding time. To gain insights into reducing this waiting time, we study operations in the inpatient wards and their interface with the ED. We focus on understanding the effect of inpatient discharge policies and other operational policies on the time-of-day waiting time performance, such as the fraction of patients waiting longer than six hours in the ED before being admitted. Based on an empirical study at a Singaporean hospital, we propose a novel stochastic processing network with the following characteristics to model inpatient operations: (1) A patient’s service time in the inpatient wards depends on that patient’s admission and discharge times and length of stay. The service times capture a two-time-scale phenomenon and are not independent and identically distributed. (2) Pre- and post-allocation delays model the extra amount of waiting caused by secondary bottlenecks other than bed unavailability, such as nurse shortage. (3) Patients waiting for a bed can overflow to a nonprimary ward when the waiting time reaches a threshold, where the threshold is time dependent. We show, via simulation studies, that our model is able to capture the inpatient flow dynamics at hourly resolution and can evaluate the impact of operational policies on both the daily and time-of-day waiting time performance. In particular, our model predicts that implementing a hypothetical policy can eliminate excessive waiting for those patients who request beds in mornings. This policy incorporates the following components: a discharge distribution with the first discharge peak between 8 a.m. and 9 a.m. and 26% of patients discharging before noon, and constant-mean allocation delays throughout the day. The insights gained from our model can help hospital managers to choose among different policies to implement depending on the choice of objective, such as to reduce the peak waiting in the morning or to reduce daily waiting time statistics. This paper was accepted by Assaf Zeevi, stochastic models and simulation.

Justifying Diffusion Approximations for Stochastic Processing Networks Under a Moment Condition

Stochastic processing networks (SPN) are, in general, difficult objects to study analytically. The diffusion approximation refers to using the stationary distribution of the diffusion limit as an approximation of the diffusion-scaled process (say, the workload) in the original SPN. To validate such an approximation amounts to justifying the interchange of two limits, t→∞ and k→∞, with t being the time index and k, the scaling parameter. Here, we show this interchange of limits is justified for a broad class of SPN under a p*-th moment condition on the primitive data, interarrival and service times; and we provide an explicit characterization of the required order (p*), which depends naturally on the desired order of convergence of the workload process. To illustrate the generality of this moment condition, we first use it to establish the justification for resource sharing networks, where, to be processed each job needs to concurrently occupy multiple resources (servers), whereas each resource is shared among different job classes following a so-called proportional fair allocation scheme. We then show the same approach applies to most multi-class queueing networks that are known to have diffusion limits.