Infinitesimal Perturbation Analysis Research Articles

Effects of Parametric Uncertainties in Cascaded Open Quantum Harmonic Oscillators and Robust Generation of Gaussian Invariant States

This paper is concerned with the generation of Gaussian invariant states in cascades of open quantum harmonic oscillators governed by linear quantum stochastic differential equations. We carry out infinitesimal perturbation analysis of the covariance matrix for the invariant Gaussian state of such a system and the related purity functional subject to inaccuracies in the energy and coupling matrices of the subsystems. This leads to the problem of balancing the state-space realizations of the component oscillators through symplectic similarity transformations in order to minimize the mean square sensitivity of the purity functional to small random perturbations of the parameters. This results in a quadratic optimization problem with an effective solution in the case of cascaded one-mode oscillators, which is demonstrated by a numerical example. The findings of the paper are applicable to robust state generation in quantum stochastic networks.

Optimal Event-Driven Multiagent Persistent Monitoring of a Finite Set of Data Sources

We consider the problem of controlling the movement of multiple cooperating agents so as to minimize an uncertainty metric associated with a finite number of data sources. In a one-dimensional (1-D) mission space, we adopt an optimal control framework and show that the solution can be reduced to a simpler parametric optimization problem: Determining a sequence of locations where each agent may dwell for a finite amount of time and then switch direction. This amounts to a hybrid system which we analyze using the infinitesimal perturbation analysis (IPA) to obtain a complete online solution through an event-driven gradient-based algorithm which is also robust with respect to the uncertainty model used. The resulting controller depends on observing the events required to excite the gradient-based algorithm, which cannot be guaranteed. We solve this problem by introducing a new metric for the objective function which creates a potential field guaranteeing that gradient values are nonzero. This approach is compared to an alternative graph-based target-visit scheduling and dwell times optimization algorithm. The simulation examples are included to demonstrate the proposed methods.

Event-Driven Trajectory Optimization for Data Harvesting in Multiagent Systems

We propose a new event-driven method for online trajectory optimization to solve the data harvesting problem: in a 2-D mission space, $N$ mobile agents are tasked with the collection of data generated at $M$ stationary sources and delivery to a base with the goal of minimizing expected collection and delivery delays. We define a new performance measure that addresses the event excitation problem in event-driven controllers and formulate an optimal control problem. The solution of this problem provides some insight on its structure, but it is computationally intractable, especially in the case where the data generating processes are stochastic. We propose an agent trajectory parameterization in terms of general function families, which can be subsequently optimized online through the use of infinitesimal perturbation analysis. Properties of the solutions are identified, including robustness with respect to the stochastic data generation process and scalability in the size of the event set characterizing the underlying hybrid dynamical system. Explicit results are provided for the case of elliptical and Fourier series trajectories, and comparisons with a state-of-the-art graph-based algorithm are given.

Optimal control approach to persistent monitoring problem based on monitoring index

A class of persistent monitoring problems in one-dimensional space with the aid of multi-agent systems, which is formulated as an optimisation problem of minimising a novel performance metric is studied. The concept of the monitoring index is introduced in the performance metric to describe the importance of different targets in the space due to the requirements of the monitoring task. Based on a given class of moving trajectories, an optimal control approach is used to minimise the performance metric of monitoring space. After some model transformation, the infinitesimal perturbation analysis method is used to design the gradient-based algorithm, aiming to find the optimal trajectories, where the monitoring index exhibits the effect on the monitoring time for different targets. Numerical simulations are represented to demonstrate the performance of the algorithm and reveal the effectiveness of the monitoring index.

A New Unbiased Stochastic Derivative Estimator for Discontinuous Sample Performances with Structural Parameters

In this paper, we propose a new unbiased stochastic derivative estimator in a framework that can handle discontinuous sample performances with structural parameters. This work extends the three most popular unbiased stochastic derivative estimators: (1) infinitesimal perturbation analysis (IPA), (2) the likelihood ratio (LR) method, and (3) the weak derivative method, to a setting where they did not previously apply. Examples in probability constraints, control charts, and financial derivatives demonstrate the broad applicability of the proposed framework. The new estimator preserves the single-run efficiency of the classic IPA-LR estimators in applications, which is substantiated by numerical experiments. The online appendix is available at https://doi.org/10.1287/opre.2017.1674 .

Stochastic Flow Models with Delays and Applications to Multi-Intersection Traffic Light Control

We extend Stochastic Flow Models (SFMs), used for a large class of discrete event and hybrid systems, by including the delays which typically arise in flow movement. We apply this framework to the multi-intersection traffic light control problem by including transit delays for vehicles moving from one intersection to the next. Using Infinitesimal Perturbation Analysis (IPA) for this SFM with delays, we derive new on-line gradient estimates of several congestion cost metrics with respect to the controllable green and red cycle lengths. The IPA estimators are used to iteratively adjust light cycle lengths to improve performance and, in conjunction with a standard gradient-based algorithm, to obtain optimal values which adapt to changing traffic conditions. We introduce two new cost metrics to better capture congestion and show that the inclusion of delays in our analysis leads to improved performance relative to models that ignore delays.

On-line Optimization of Power Efficiency in 3D Multicore Processors

This paper considers the problem of maximizing power efficiency in multi-core processors by dynamic voltage frequency scaling. We formulate the problem in an on-line setting and apply to it a gradient-based stochastic approximation algorithm. We use a cycle-level, full-system simulation platform capable of realistic modeling of state-of-the-art microarchitectures executing industry-standard benchmarks (Splash 2). The optimization algorithm computes the gradients by the Infinitesimal Perturbation Analysis (IPA) sample-based sensitivity analysis technique. Despite the fact that the system’s characteristics vary widely with time and hence the optimum point varies as well, the simulation experiments exhibit a tracking of the optimal values. To our knowledge this is the first application of IPA to co-optimizing throughput performance and power in computer processors.

Perturbation analysis: A framework for data-driven control and optimization of discrete event and hybrid systems

The history of Perturbation Analysis (PA) is intimately related to that of Discrete Event Dynamic Systems (DEDS), starting with a solution of a long-standing problem in the late 1970s and continuing today with the control and optimization of Hybrid Systems and the emergence of event-driven control methods. We review the origins of the PA theory and how it became part of a broader framework for modelling, control and optimization of DEDS. We then discuss the theoretical underpinnings of Infinitesimal Perturbation Analysis (IPA) as a data-driven stochastic gradient estimation method and how it has been applied over the past few decades. We explain how IPA offers a basis for general-purpose stochastic optimization of Markovian systems through the notion of the performance potential and how it has evolved beyond DEDS and now provides a framework for control and optimization of Hybrid Systems and, more generally, event-driven methodologies.

Joint optimisation of capacity and safety stock allocation

We study the joint optimisation of capacity and safety stock allocation in assembly systems. Particularly, we consider capacitated systems with base-stock policies and periodic review. Capacity allocation is restricted by budget constraints, which can connect multiple systems. Our objective is to minimise overall inventory holding costs while satisfying service level as well as budget constraints. We propose an algorithm to jointly approximate optimal capacity allocation and base-stock levels. To this end, we introduce a set of convex approximations for this non-convex optimisation problem. In order to solve the resulting convex programmes, we analytically compute sample path derivatives via infinitesimal perturbation analysis. By iteratively adapting the approximations, we achieve good capacity allocations and base-stock levels for the original problem. Furthermore, we introduce a heuristic to allocate capacity, which originates from link capacity allocation in communication networks and use it as a benchmark. The algorithm is applied to small illustrative examples as well as cases motivated by the semiconductor manufacturing process at IBM Systems. It turns out that particularly for high utilisation levels, our algorithm can achieve significant improvements compared to the capacity allocation heuristic.

Optimal Event-Driven Multi-Agent Persistent Monitoring with Graph-Limited Mobility

We address the persistent monitoring problem in a 2D mission space where agents are restricted to move on a graph topology consisting of multiple intersecting 1D line segments (subspaces) on such a graph. The objective is to control the movement of multiple cooperating agents allocated over these line segments in order to minimize an uncertainty metric associated with a finite set of data sources/targets. Previously, we have shown that in 1D the optimal control solution is for each agent to move at maximum speed and switch directions at a sequence of locations, possibly waiting for some time at a target. We show that this optimal control structure can be extended to the 2D setting considered here and that we can parameterize the optimal solution to determine each agent’s switching points and dwell times in each linear segment over a given finite time horizon. We use Infinitesimal Perturbation Analysis (IPA) to obtain a complete on-line solution through an event-driven gradient-based algorithm. We also solve the optimal assignment problem of agents over subspaces. Simulation examples are included to demonstrate the combined optimal agent allocation and trajectory design.

Perturbation Analysis: A Framework for Data-Driven Control and Optimization of Discrete Event and Hybrid Systems

The history of Perturbation Analysis (PA) is intimately related to that of Discrete Event Dynamic Systems (DEDS), starting with a solution of a long-standing problem in the late 1970s and continuing today with the control and optimization of Hybrid Systems and the emergence of event-driven control methods. We review the origins of the PA theory and how it became part of a broader framework for models, control and optimization of DEDS. We then discuss the theoretical underpinnings of Infinitesimal Perturbation Analysis (IPA) as a data-driven stochastic gradient estimation method and how it has been applied over the past few decades. We explain how IPA offers a basis for general-purpose stochastic optimization of Markov systems through the notion of the performance potential and how it has evolved beyond DEDS and now provides a framework for control and optimization of Hybrid Systems and, more generally, event-driven methodologies.

Inventory control policy for a periodic review system with expediting

• The inventory control problem of an inventory system with expediting is studied. • We propose (S, e) policy that allows to expedite a part of order whenever needed. • An effective simulation optimization approach is designed to find the best (S, e). • (S, e) policy outperforms similar policies and is applicable to various areas. In this work, we examine a single stage, periodic review inventory system where two modes, namely regular mode and expediting mode are available for a firm to obtain its replenishment. The firm can choose expediting mode with shorter lead time at a higher cost when necessary. A two-replenishment-mode model, with random expediting points is established and an innovative ordering policy ( S, e ) which replenishes the inventory level to S in every cycle and expedites a part of the order using fast mode when the inventory level drops to or below a certain level e , is proposed. A simulation-based optimization approach is employed to solve the problem. To be specific, an infinitesimal perturbation analysis (IPA) method is applied to estimate the gradients of the objective function and a gradient search algorithm is then used to find the best ( S, e ). Numerical experiments have validated the IPA estimate of ( S, e ) and shown that our new ( S, e ) policy outperforms the pure order-up-to policy and two single review moment policies. Sensitivity analysis has been conducted to evaluate the impact of system parameters of interest and managerial insights have been drawn.

Performance regulation in discrete event and hybrid dynamical systems using IPA

This paper considers a performance regulation technique for a class of timed discrete-event and hybrid dynamical systems, aimed at an output tracking of a given setpoint reference. The regulator, designed for simplicity and efficiency of computation, consists of a variable-gain integrator that yields effective tracking despite abrupt changes to the system. The gain is computed in real time by infinitesimal perturbation analysis, and the resultant control system displays wide stability margins as well as considerable robustness to modeling and computing errors. In particular, we consider errors resulting from approximations that are made to the controller’s gains in order to speed up their computations. Previous analysis was carried out under assumptions of differentiable sample-performance functions which serve to approximate a steady-state (expected-valued) function. We dispense with these two assumptions which often are unrealistic in time-varying systems. The paper proves convergence of the regulation technique and demonstrates the results on simulation examples.

Optimization of Manufacturing Supply Chain with Stochastic Demand and Planned Delivery Time

In this paper, we study a manufacturing system composed by a single-product machine, a buffer and a constant demand. A stochastic fluid model is adopted to describe the system and to take into account delivery times and returned products from the customers (withdraw right). The objective of this paper is to determine the optimal buffer level taken into account machine failures, delivery times and withdrawal right. This optimal buffer allows minimizing the sum of inventory, transportation, lost sales and returned products costs. Infinitesimal perturbation analysis method is used for optimizing the proposed system. Using the stochastic fluid model, the trajectories of buffer level are studied and the infinitesimal perturbation analysis estimators are evaluated. These estimators are shown to be unbiased and then they are implanted in an optimization algorithm which determines the optimal buffer level in the presence of delivery time and withdrawal right.

Simulation‐Based Optimization of Capacitated Assembly Systems under Beta‐Service Level Constraints

ABSTRACTWe study a class of capacitated assembly systems operated by base‐stock policies and address the problem of finding base‐stock levels that minimize holding costs under beta‐service level (fill rate) constraints over an infinite horizon. To solve this nonconvex constrained optimization problem, we develop a new simulation‐based optimization approach using the constrained level method (CLM) and infinitesimal perturbation analysis. The key idea of the algorithm is a novel family of convex approximations of the beta‐service level, which is iteratively refined during the course of the algorithm. The algorithm can easily handle integrality requirements for the base‐stock levels by combining the CLM with a cutting plane approach without significantly increasing the solution time on typical instances. We apply our approach to a comprehensive multi‐echelon assembly benchmark system from the literature and study the algorithm's behavior for different target service levels, capacity configurations, and simulation horizons. A comparison with a state‐of‐the‐art interior point algorithm shows that for realistic capacity constraints, our algorithm is on average 8%–20% better. Compared with the guaranteed service model, our approach reduces costs by 10%–15% while keeping the same service level.

Optimal design of personalized prostate cancer therapy using Infinitesimal Perturbation Analysis

The standard treatment for advanced prostate cancer is hormone therapy in the form of continuous androgen suppression (CAS), which unfortunately frequently leads to resistance and relapse. An alternative scheme is intermittent androgen suppression (IAS), in which patients are submitted to cycles of treatment (in the form of androgen deprivation) and off-treatment periods in an alternating manner. In spite of extensive recent clinical experience with IAS, the design of ideal protocols for any given patient remains a challenge. The level of prostate specific antigen (PSA) is frequently monitored to determine when patients will be taken off therapy and when therapy will resume. In this work, we propose a threshold-based policy for optimal IAS therapy design that is parameterized by lower and upper PSA threshold values and is associated with a cost metric that combines clinically relevant measures of therapy success. We use a Stochastic Hybrid Automaton (SHA) model of prostate cancer evolution under IAS and perform Infinitesimal Perturbation Analysis (IPA) to adaptively adjust PSA threshold values so as to improve therapy outcomes. We also apply this methodology to clinical data from real patients, and obtain promising results and valuable insights for personalized IAS therapy design.

Quantile sensitivity estimation for dependent sequences

AbstractIn this paper we estimate quantile sensitivities for dependent sequences via infinitesimal perturbation analysis, and prove asymptotic unbiasedness, weak consistency, and a central limit theorem for the estimators under some mild conditions. Two common cases, the regenerative setting and ϕ-mixing, are analyzed further, and a new batched estimator is constructed based on regenerative cycles for regenerative processes. Two numerical examples, the G/G/1 queue and the Ornstein–Uhlenbeck process, are given to show the effectiveness of the estimator.

Unreliable manufacturing supply chain optimisation based on an infinitesimal perturbation analysis

ABSTRACTThis paper studies the optimisation of a manufacturing/remanufacturing system, which produces one type of product and is composed of a serviceable inventory, two parallel machines, a recovery inventory, two quality inspection centres and customers who demand a constant quantity of product. The proposed system takes into account the withdrawn products and the return of the used products from the market. The manufacturing and remanufacturing products are stored in the serviceable inventory. The used products are inspected then disposed of or stored in the recovery inventory for remanufacturing. The withdrawn products are inspected and stored in the serviceable inventory for selling or stored in the recovery inventory for remanufacturing. The objective of this work is to find the optimal serviceable inventory level that minimises the cost function, then to study the influence of withdrawn products and used products on the value of the optimal serviceable inventory level. Infinitesimal perturbation analysis method is used to derive gradient estimators of the total cost with respect to inventory levels. These estimators are used in an optimisation algorithm that computes the optimal serviceable inventory level. Numerical results and influence analysis are presented to illustrate the usefulness of the proposed approach.

Optimization of integrated preventive maintenance based on infinitesimal perturbation analysis

This paper considers jointly the production control and preventive maintenance problem in a manufacturing system. This system is composed of a single unreliable machine that produces one or two types of products with constant and different demand, a customer and a buffer between the machine and the customer. Machine failures are time-dependent, i.e. the machine can fail at any time, so a block-type preventive maintenance policy is proposed to increase the system life. This policy is coupled with a hedging point policy which controls the machine production speed. Between each element of the system, constant transportation delays are considered, i.e. a conveyor between the machine and its downstream buffer and a transport between the buffer and the customer. Then, a continuous-flow model is proposed to represent the overall dynamics of the system. This model allows us to take into account explicitly the transportation delays. The main objective of this work is to find the optimal preventive maintenance period by means of infinitesimal perturbation analysis technique as well as the optimal hedging point in order to minimize the expected average cost function. For this, we find the gradient estimators of the cost function from a theoretical sample path study. Then we integrate them in a simulation algorithm to find a numerical estimation of the solution. Simulation results highlight our theoretical results.

Adaptive Quasi-Dynamic Traffic Light Control

We consider the traffic light control problem for a single intersection modeled as a stochastic hybrid system. We study a quasi-dynamic policy based on partial state information defined by detecting whether vehicle backlogs are above or below certain thresholds. The policy is parameterized by green and red cycle lengths as well as the road content thresholds. Using infinitesimal perturbation analysis, we derive online gradient estimators of a cost metric with respect to the controllable light cycles and threshold parameters and use these estimators to iteratively adjust all the controllable parameters through an online gradient-based algorithm so as to improve the overall system performance under various traffic conditions. The results obtained by applying this methodology to a simulated urban setting are also included.