Diffusion Control Problem Research Articles

Enhancing Make-to-Order Manufacturing Agility: When Flexible Capacity Meets Dynamic Pricing

The rise of online marketplaces has raised customer expectations regarding customization and lead time. It poses significant challenges to manufacturing firms and prompts a move from make-to-stock to a more flexible make-to-order system. Compared to make-to-stock settings, make-to-order systems cannot smooth fluctuations in demand using available stock. While viewing dynamic pricing as a useful strategy to balance supply with demand, many manufacturing firms can also create capacity flexibility. In that scenario, system costs could be cut by managing capacity and demand simultaneously. In this paper, we consider a make-to-order production environment with base and surge capacity as well as the ability to adjust product pricing. Our main focus is on operational decision-making, assuming that the base capacity and surge capacity are fixed, but activating the surge capacity incurs a setup cost. Initially, we propose a stochastic control model to reflect this complex decision problem. However, our initial model leads to an intractable dynamic programming problem. To overcome this, we convert the problem to a more tractable diffusion control problem. This approach helps to reveal the conditions under which utilizing flexible capacity is more advantageous than relying solely on fixed capacity. When flexible capacity is advantageous, we provide a solution to the diffusion control problem that can guide optimal capacity and price adjustments. We discover an interesting interplay between capacity adjustment and dynamic pricing. In particular, we find that the price, which aims at reducing congestion, may not monotonically increase with the congestion level when capacity adjustments incur a fixed cost.

A numerical method for ergodic optimal control of switching diffusions with reflection

The ergodic or long run average cost control problem for diffusions is one of the classical problems of stochastic control that still eludes a completely satisfactory treatment. This is certainly true for the setting in which the systems to be controlled is modeled by the solution of a switching stochastic differential equation with reflection (SSDER). In this paper, we advance in this rather difficult problem by setting forth preliminary results for the unidimensional case. Besides carving out a numerical method, our treatment of the ergodic control in this scenario straddles issues of existence and uniqueness of solution of the SSDER and a verification theorem for the associated HJB equation. We conclude by illustrating the effectiveness of the method considering the control of energy consumption in a large parallel processing computer system composed of one queue and several processing stations.

Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation

This work considers the infinite horizon discounted risk-sensitive optimal control problem for the switching diffusions with a compact control space and controlled through the drift; thus, the the generator of the switching diffusions also depends on the controls. Note that the running cost of interest can be unbounded, so a decent estimation on the value function is obtained, under suitable conditions. To solve such a risk-sensitive optimal control problem, we adopt the viscosity solution methods and propose a numerical approximation scheme. We can verify that the value function of the optimal control problem solves the optimality equation as the unique viscosity solution. The optimality equation is also called the Hamilton–Jacobi–Bellman (HJB) equation, which is a second-order partial differential equation (PDE). Since, the explicit solutions to such PDEs are usually difficult to obtain, the finite difference approximation scheme is derived to approximate the value function. As a byproduct, the ϵ-optimal control of finite difference type is also obtained.

A Note on Asymptotics Between Singular and Constrained Control Problems of One-Dimensional Diffusions

We study the asymptotic relations between certain singular and constrained control problems for one-dimensional diffusions with both discounted and ergodic objectives. In the constrained control problems the controlling is allowed only at independent Poisson arrival times. We show that when the underlying diffusion is recurrent, the solutions of the discounted problems converge in Abelian sense to those of their ergodic counterparts. Moreover, we show that the solutions of the constrained problems converge to those of their singular counterparts when the Poisson rate tends to infinity. We illustrate the results with drifted Brownian motion and Ornstein-Uhlenbeck process.

State-Dependent Temperature Control for Langevin Diffusions

We study the temperature control problem for Langevin diffusions in the context of non-convex optimization. The classical optimal control of such a problem is of the bang-bang type, which is overly sensitive to errors. A remedy is to allow the diffusions to explore other temperature values and hence smooth out the bang-bang control. We accomplish this by a stochastic relaxed control formulation incorporating randomization of the temperature control and regularizing its entropy. We derive a state-dependent, truncated exponential distribution, which can be used to sample temperatures in a Langevin algorithm, in terms of the solution to an HJB partial differential equation. We carry out a numerical experiment on a one-dimensional baseline example, in which the HJB equation can be easily solved, to compare the performance of the algorithm with three other available algorithms in search of a global optimum.

Approximations and Optimal Control for State-Dependent Limited Processor Sharing Queues

The paper studies approximations and control of a processor sharing (PS) server where the service rate depends on the number of jobs occupying the server. The control of such a system is implemented by imposing a limit on the number of jobs that can share the server concurrently, with the rest of the jobs waiting in a first-in-first-out (FIFO) buffer. A desirable control scheme should strike the right balance between efficiency (operating at a high service rate) and parallelism (preventing small jobs from getting stuck behind large ones). We use the framework of heavy-traffic diffusion analysis to devise near optimal control heuristics for such a queueing system. However, although the literature on diffusion control of state-dependent queueing systems begins with a sequence of systems and an exogenously defined drift function, we begin with a finite discrete PS server and propose an axiomatic recipe to explicitly construct a sequence of state-dependent PS servers that then yields a drift function. We establish diffusion approximations and use them to obtain insightful and closed-form approximations for the original system under a static concurrency limit control policy. We extend our study to control policies that dynamically adjust the concurrency limit. We provide two novel numerical algorithms to solve the associated diffusion control problem. Our algorithms can be viewed as “average cost” iteration: The first algorithm uses binary-search on the average cost, while the second faster algorithm uses Newton-Raphson method for root finding. Numerical experiments demonstrate the accuracy of our approximation for choosing optimal or near-optimal static and dynamic concurrency control heuristics.

Minimizing the Maximum Expected Waiting Time in a Periodic Single-Server Queue with a Service-Rate Control

We consider a single-server queue with unlimited waiting space, the first-come, first-served discipline, a periodic arrival-rate function, and independent and identically distributed service requirements, where the service-rate function is subject to control. We previously showed that a rate-matching control, whereby the service rate is made proportional to the arrival rate, stabilizes the queue-length process but not the (virtual) waiting-time process. To minimize the maximum expected waiting time (and stabilize the expected waiting time), we now consider a modification of the service-rate control involving two parameters: a time lag and a damping factor. We develop an efficient simulation search algorithm to find the best time lag and damping factor. That simulation algorithm is an extension of our recent rare-event simulation algorithm for the GIt/GI/1 queue to the GIt/GIt/1 queue, allowing the time-varying service rate. To gain insight into these controls, we establish a heavy-traffic limit with periodicity in the fluid scale. This produces a diffusion control problem for the stabilization, which we solve numerically by the simulation search in the scaled family of systems with ρ ↑ 1. The state space collapse in that theorem shows that there is a time-varying Little’s law in heavy traffic, implying that the queue length and waiting time cannot be simultaneously stabilized in this limit. We conduct simulation experiments showing that the new control is effective for stabilizing the expected waiting time for a wide range of model parameters, but we also show that it cannot stabilize the expected waiting time perfectly.

Beyond Heavy-Traffic Regimes: Universal Bounds and Controls for the Single-Server Queue

Central-limit (Brownian) approximations are widely used for the performance analysis and optimization of queueing networks because of their tractability relative to the original queueing models. The stationary distributions of the approximations are used as proxies for those of the queues. The convergence of suitably scaled and centered processes provides mathematical support for the use of these Brownian models. As with the central limit theorem, to establish convergence, one must impose assumptions directly on the primitives or indirectly on the parameters of a related optimization problem. These assumptions reflect an interpretation of the underlying parameters—a classification into so-called heavy-traffic regimes that specify a scaling relationship between the utilization and the arrival rate. Here, it matters whether a utilization of 90% in a queue with an arrival rate of λ = 100 is read as [Formula: see text] or as ρ(λ) ≡ 0.9, because different interpretations lead to different limits and, in turn, to different approximations. However, from a heuristic point of view, there is an immediate Brownian (i.e., normal) analogue of the queueing model that is derived directly from the primitives and requires no scaling interpretation of the parameters. In this model, the drift is that of the original queue, and the noise term is replaced by a Brownian motion with the same variance. This is intuitive and appealing as a tool, but it lacks mathematical justification. In this paper, we prove that for the fundamental M/GI/1 + GI queue, this direct intuitive approach works: the Brownian model is accurate uniformly over a family of patience distributions and universally in the heavy-traffic regime. The validity of this approach extends to dynamic control in that the solution of the directly derived diffusion control problem is universally accurate. To build mathematical support for the accuracy of this model, we introduce a framework built around “queue families” that allows us to treat various patience distributions simultaneously, and it uncovers the role of a concentration property of the queue. The e-companion is available at https://doi.org/10.1287/opre.2017.1715 .

Strict monotonicity of principal eigenvalues of elliptic operators in [formula omitted] and risk-sensitive control

This paper studies the eigenvalue problem on Rd for a class of second order, elliptic operators of the form Lf=aij∂xi∂xj+bi∂xi+f, associated with non-degenerate diffusions. We show that strict monotonicity of the principal eigenvalue of the operator with respect to the potential function f fully characterizes the ergodic properties of the associated ground state diffusion, and the unicity of the ground state, and we present a comprehensive study of the eigenvalue problem from this point of view. This allows us to extend or strengthen various results in the literature for a class of viscous Hamilton–Jacobi equations of ergodic type with smooth coefficients to equations with measurable drift and potential. In addition, we establish the strong duality for the equivalent infinite dimensional linear programming formulation of these ergodic control problems. We also apply these results to the study of the infinite horizon risk-sensitive control problem for diffusions, and establish existence of optimal Markov controls, verification of optimality results, and the continuity of the controlled principal eigenvalue with respect to stationary Markov controls.

Diffusion approximations for controlled weakly interacting large finite state systems with simultaneous jumps

We consider a rate control problem for an $N$-particle weakly interacting finite state Markov process. The process models the state evolution of a large collection of particles and allows for multiple particles to change state simultaneously. Such models have been proposed for large communication systems (e.g., ad hoc wireless networks) but are also suitable for other settings such as chemical-reaction networks. An associated diffusion control problem is presented and we show that the value function of the $N$-particle controlled system converges to the value function of the limit diffusion control problem as $N\to\infty$. The diffusion coefficient in the limit model is typically degenerate; however, under suitable conditions there is an equivalent formulation in terms of a controlled diffusion with a uniformly nondegenerate diffusion coefficient. Using this equivalence, we show that near optimal continuous feedback controls exist for the diffusion control problem. We then construct near asymptotically optimal control policies for the $N$-particle system based on such continuous feedback controls. Results from some numerical experiments are presented.

Optimal Control of Diffusion Coefficients via Decoupling Fields

We consider a diffusion control problem where the controller totally determines the state's diffusion coefficient but has no influence on the state's drift rate. By using the Pontryagin maximum principle we characterize an optimal control in terms of the adjoint forward-backward stochastic differential equation (FBSDE), turning out to be fully coupled. We use the method of decoupling fields for proving that the adjoint FBSDE possesses a solution.

An approximating diffusion control problem for dynamic admission and service rate control in a [formula omitted] queue

We study a diffusion control problem that is motivated by the dynamic admission and service rate control problem for a G∕M∕N+G queue. The objective is to minimize long run average cost. Because the original queueing control problem is not tractable, we solve the approximating diffusion control problem that arises under the QED heavy traffic regime and show that its optimal solution has two components: (1) a threshold control that regulates the diffusion and (2) a feedback-type drift rate control.

Diffusion control for a tempered anomalous diffusion system using fractional-order PI controllers

This paper is concerned with diffusion control problem of a tempered anomalous diffusion system based on fractional-order PI controllers. The contribution of this paper is to introduce fractional-order PI controllers into the tempered anomalous diffusion system for mobile actuators motion and spraying control. For the proposed control force, convergence analysis of the system described by mobile actuator dynamical equations is presented based on Lyapunov stability arguments. Moreover, a new Centroidal Voronoi Tessellation (CVT) algorithm based on fractional-order PI controllers, henceforth called FOPI-based CVT algorithm, is provided together with a modified simulation platform called Fractional-Order Diffusion Mobile Actuator-Sensor 2-Dimension Fractional-Order Proportional Integral (FO-Diff-MAS2D-FOPI). Finally, extensive numerical simulations for the tempered anomalous diffusion process are presented to verify the effectiveness of our proposed fractional-order PI controllers.

Novel Ebullated Bed Residue Hydrocracking Process

Residue hydrocracking has been attracting more and more attention to the refining industry in recent years, and one of the best approaches is ebullated bed residue hydrocracking (EBRH). STRONG ebullated bed residue hydrocracking uses a new type of reactor, and a 50 KTA demonstration unit has been put into operation now. Crucial points of STRONG residue hydrocracking are proposed physical and chemical properties of a solid catalyst, flow mechanism, efficient separation of gas–liquid–solid in the reactor, and reaction performance within various scales of the pilot tests. The results showed that the STRONG reactor with a microspherical catalyst inside a canceled recycling pump inside and outside overcame mass transfer difficulties, solved diffusion control problems, and created high catalyst efficiency to reduce catalyst consumption. To prevent the catalyst from being entrained from the reactor, a triphase separator was used and the amount of catalyst carried off was less than 1 μg/g.

Ergodic diffusion control of multiclass multi-pool networks in the Halfin–Whitt regime

We consider Markovian multiclass multi-pool networks with heterogeneous server pools, each consisting of many statistically identical parallel servers, where the bipartite graph of customer classes and server pools forms a tree. Customers form their own queue and are served in the first-come first-served discipline, and can abandon while waiting in queue. Service rates are both class and pool dependent. The objective is to study the limiting diffusion control problems under the long run average (ergodic) cost criteria in the Halfin–Whitt regime. Two formulations of ergodic diffusion control problems are considered: (i) both queueing and idleness costs are minimized, and (ii) only the queueing cost is minimized while a constraint is imposed upon the idleness of all server pools. We develop a recursive leaf elimination algorithm that enables us to obtain an explicit representation of the drift for the controlled diffusions. Consequently, we show that for the limiting controlled diffusions, there always exists a stationary Markov control under which the diffusion process is geometrically ergodic. The framework developed in [Ann. Appl. Probab. 25 (2015) 3511–3570] is extended to address a broad class of ergodic diffusion control problems with constraints. We show that the unconstrained and constrained problems are well posed, and we characterize the optimal stationary Markov controls via HJB equations.

Dynamic Scheduling in a Many-Server Multi-Class System: The Role of Customer Impatience in Large Systems

We study optimal scheduling of customers in service systems, such as call centers. In such systems, customers typically hang up and abandon the system after waiting for a long time for their service to commence. Such abandonments are detrimental for the system, and so managers typically use scheduling as a tool to mitigate it. In this paper we study the interplay between customer impatience and scheduling decisions when managing heterogeneous customer classes. Specifically, our focus is on general patience distributions, in which a customer's instantaneous propensity to abandon may change over time. We find that incorporating these temporal changes into the scheduling decisions can lead to a significant improvement in the system performance. Formally, we solve the underlying Diffusion Control Problem for these systems, and characterize near-optimal scheduling policies. One of our main results is that for a class of parameters, we establish sufficient conditions for both the optimality and non-optimality of threshold policies, which are widely used due to their ease of implementation.

Dynamic Scheduling for Parallel Server Systems in Heavy Traffic: Graphical Structure, Decoupled Workload Matrix and some Sufficient Conditions for Solvability of the Brownian Control Problem

We consider a dynamic scheduling problem for parallel server systems. J. M. Harrison has proposed a scheme for using diffusion control problems to approximately solve such control problems for heavily loaded systems. This approach has been very successfully used in the special case when the diffusion control problem can be reduced to an equivalent one for a one-dimensional workload process. However, it remains a challenging open problem to make substantial progress on using Harrison’s scheme when the workload process is more than one-dimensional. Here we present some new structural results concerning the diffusion control problem for parallel server systems with arbitrary workload dimension. Specifically, we prove that a certain server-buffer graph associated with a parallel server system is a forest of trees. We then exploit this graphical structure to prove that there exists a matrix, used to define the workload process, that has a block diagonal-like structure. An important feature of this matrix is that, except when the workload is one-dimensional, this matrix is frequently different from a choice of workload matrix proposed by Harrison. We demonstrate that our workload matrix simplifies the structure of the control problem for the workload process by proving that when the original diffusion control problem has linear holding costs, the equivalent workload formulation also has a linear cost function. We also use this simplification to give sufficient conditions for a certain least control process to be an optimal control for the diffusion control problem with linear holding costs. Under these conditions, we propose a continuous review threshold-type control policy for the original parallel server system that exploits pooling of servers within trees in the server-buffer graph and uses non-basic activities connecting different trees in a critical manner. We call this partial pooling. We conjecture that this threshold policy is asymptotically optimal in the heavy traffic limit. We illustrate the solution of the diffusion control problem and our proposed threshold control policy for a three-buffer, three-server example.

Dynamic Scheduling in a Many-Server, Multiclass System: The Role of Customer Impatience in Large Systems

Problem definition: We study optimal scheduling of customers in service systems, such as call centers. In such systems, customers typically hang up and abandon the system if their wait for service is too long. Such abandonments are detrimental for the system, and so managers typically use scheduling as a tool to mitigate it. In this paper, we study the interplay between customer impatience and scheduling decisions when managing heterogeneous customer classes. Academic/practical relevance: Call centers constitute a large industry that has a global spending of around $300 billion and employs more than 15 million people worldwide. Our work focuses on improving call center operations, which can reduce costs and improve customer satisfaction. Mathematically, customer patience is typically modeled as exponentially distributed for tractability. Our work makes inroads into relaxing this restrictive assumption to allow modeling more realistic call center situations. Methodology: We use heavy traffic–motivated asymptotic queueing machinery that provides us the traction to successfully capture and incorporate the customer impatience distribution into the scheduling problem. In our approach, the scheduling problem reduces to a diffusion control problem, which we solve to propose near-optimal scheduling policies. Results: We propose near-optimal scheduling policies that can be implemented by call centers to improve their quality-of-service metrics. One of our main results is that, for a class of parameters, we establish sufficient conditions for both the optimality and nonoptimality of threshold policies. Managerial implications: Threshold policies are widely used for scheduling. Our work provides additional insight into whether these may be suboptimal. Our work provides an easy-to-implement alternative that can reduce customer abandonments considerably; for instance, our numerical results indicate that for a system with two customer types, the abandonment rate of one class can be lowered by 30% by using our policy relative to the best threshold policy. The online appendix is available at https://doi.org/10.1287/msom.2017.0642 .

Dynamic scheduling for parallel server systems in heavy traffic: Graphical structure, decoupled workload matrix and some sufficient conditions for solvability of the Brownian control problem

We consider a dynamic scheduling problem for parallel server systems. J. M. Harrison has proposed a scheme for using diffusion control problems to approximately solve such control problems for heavily loaded systems. This approach has been very successfully used in the special case when the diffusion control problem can be reduced to an equivalent one for a one-dimensional workload process. However, it remains a challenging open problem to make substantial progress on using Harrison’s scheme when the workload process is more than one-dimensional. Here we present some new structural results concerning the diffusion control problem for parallel server systems with arbitrary workload dimension. Specifically, we prove that a certain server-buffer graph associated with a parallel server system is a forest of trees. We then exploit this graphical structure to prove that there exists a matrix, used to define the workload process, that has a block diagonal-like structure. An important feature of this matrix is that, except when the workload is one-dimensional, this matrix is frequently different from a choice of workload matrix proposed by Harrison. We demonstrate that our workload matrix simplifies the structure of the control problem for the workload process by proving that when the original diffusion control problem has linear holding costs, the equivalent workload formulation also has a linear cost function. We also use this simplification to give sufficient conditions for a certain least control process to be an optimal control for the diffusion control problem with linear holding costs. Under these conditions, we propose a continuous review threshold-type control policy for the original parallel server system that exploits pooling of servers within trees in the server-buffer graph and uses non-basic activities connecting different trees in a critical manner. We call this partial pooling. We conjecture that this threshold policy is asymptotically optimal in the heavy traffic limit. We illustrate the solution of the diffusion control problem and our proposed threshold control policy for a three-buffer, three-server example.

Ergodic control of multi-class $M/M/N+M$ queues in the Halfin–Whitt regime

We study a dynamic scheduling problem for a multi-class queueing network with a large pool of statistically identical servers. The arrival processes are Poisson, and service times and patience times are assumed to be exponentially distributed and class dependent. The optimization criterion is the expected long time average (ergodic) of a general (nonlinear) running cost function of the queue lengths. We consider this control problem in the Halfin–Whitt (QED) regime, that is, the number of servers $n$ and the total offered load $\mathbf{r}$ scale like $n\approx\mathbf{r}+\hat{\rho}\sqrt{\mathbf{r}}$ for some constant $\hat{\rho}$. This problem was proposed in [Ann. Appl. Probab. 14 (2004) 1084–1134, Section 5.2]. The optimal solution of this control problem can be approximated by that of the corresponding ergodic diffusion control problem in the limit. We introduce a broad class of ergodic control problems for controlled diffusions, which includes a large class of queueing models in the diffusion approximation, and establish a complete characterization of optimality via the study of the associated HJB equation. We also prove the asymptotic convergence of the values for the multi-class queueing control problem to the value of the associated ergodic diffusion control problem. The proof relies on an approximation method by spatial truncation for the ergodic control of diffusion processes, where the Markov policies follow a fixed priority policy outside a fixed compact set.