Infinitesimal Perturbation Analysis Research Articles

Estimating Posterior Sensitivities with Application to Structural Analysis of Bayesian Vector Autoregressions

The inherent feature of Bayesian empirical analysis is the dependence of posterior inference on prior parameters, which researchers typically specify. However, quantifying the magnitude of this dependence remains difficult. This article extends Infinitesimal Perturbation Analysis, widely used in classical simulation, to compute asymptotically unbiased and consistent sensitivities of posterior statistics with respect to prior parameters from Markov chain Monte Carlo inference via Gibbs sampling. The method demonstrates the possibility of efficiently computing the complete set of prior sensitivities for a wide range of posterior statistics, alongside the estimation algorithm using Automatic Differentiation. The method’s application is exemplified in Bayesian Vector Autoregression analysis of fiscal policy in U.S. macroeconomic time series data. The analysis assesses the sensitivities of posterior estimates, including the Impulse response functions and Forecast error variance decompositions, to prior parameters under common Minnesota shrinkage priors. The findings illuminate the significant and intricate influence of prior specification on the posterior distribution. This effect is particularly notable in crucial posterior statistics, such as the substantial absolute eigenvalue of the companion matrix, ultimately shaping the structural analysis.

Production policy based on echelon base stocks for reverse logistics and industrial symbiosis: an IPA approach

Last years, the implementation of reverse loops in logistics and the deployment of industrial symbiosis becomes more important. However, these new activities management isn’t easy and it’s necessary to propose methodologies to facilitate the actors work. However, the existing studies are more at strategic level, aiming with implementation or cooperation. At the tactical/operational level, solutions are poorly effective and/or expensive. Our work targets an optimal production policy definition based on the base stock strategy adapted for an industrial symbiosis. The system is composed of two kinds of warehouses and three types of enterprises. The recovering depends on the kind of collected used finished products. All demands are uncertain such as the number of collected used finished products and generated waste. Then, the main objective is to minimize the sum of all economic, environmental and social costs by identifying the level of the base stocks. To do this, an infinitesimal perturbation analysis study is conducted to evaluate the gradient estimators of the objective function subject to an echelon base stock production policy. This result is then used in a simulation based-optimization algorithm to determine these stock levels and highlight our theoretical results by comparing with other replenishment strategy and mathematical programming.

A Transverse Hamiltonian Approach to Infinitesimal Perturbation Analysis of Quantum Stochastic Systems.

This paper is concerned with variational methods for open quantum systems with Markovian dynamics governed by Hudson-Parthasarathy quantum stochastic differential equations. These QSDEs are driven by quantum Wiener processes of the external bosonic fields and are specified by the system Hamiltonian and system-field coupling operators. We consider the system response to perturbations in these operators and introduce a transverse Hamiltonian which encodes the propagation of the perturbations through the unitary system-field evolution. This approach provides an infinitesimal perturbation analysis tool which can be used for the development of optimality conditions in quantum control and filtering problems. As performance criteria, such settings employ quadratic (or more complicated) cost functionals of the system and field variables to be minimized over the energy and coupling parameters of system interconnections. We demonstrate an application of the transverse Hamiltonian variational approach to a mean square optimal coherent quantum filtering problem for a measurement-free field-mediated cascade connection of a quantum system with a quantum observer.

On comparison of steady-state infinitesimal perturbation analysis and likelihood ratio derivative estimates

On comparison of steady-state infinitesimal perturbation analysis and likelihood ratio derivative estimates

Copula sensitivity analysis for portfolio credit derivatives

Modeling dependence among input random variables is often critical for performance evaluation of stochastic systems, and copulas provide one approach to model such dependence. In the financial industry, the dependence among default times of different assets drives the pricing of portfolio credit derivatives, including basket default swaps and collateralized debt obligations. Copula sensitivities provide information about how changes in the dependence level affect the output. We study sensitivity analysis of elliptical copulas and Archimedean copulas using infinitesimal perturbation analysis, and an unbiased estimators is derived using conditional Monte Carlo (CMC) to address discontinuities that arise in portfolio credit derivatives. In simulation experiments, the new estimators have smaller variance than other applicable methods.

Variance comparison between infinitesimal perturbation analysis and likelihood ratio estimators to stochastic gradient

We theoretically compare variances between the Infinitesimal Perturbation Analysis (IPA) estimator and the Likelihood Ratio (LR) estimator to Monte Carlo gradient for stochastic systems. The results presented in Cui et al. (2020) [2] on variance comparison between these two estimators are substantially improved. We also prove a practically interesting result that the IPA estimators to European vanilla and arithmetic Asian options' Delta, respectively, have smaller variance when the underlying asset's return process is independent with the initial price and square integrable.

Stochastic derivative estimation for max-stable random fields

We consider expected performances based on max-stable random fields and we are interested in their derivatives with respect to the spatial dependence parameters of those fields. Max-stable fields, such as the Brown–Resnick and Smith fields, are very popular in spatial extremes. We focus on the two most popular unbiased stochastic derivative estimation approaches: the likelihood ratio method (LRM) and the infinitesimal perturbation analysis (IPA). LRM requires the multivariate density of the max-stable field to be explicit, and IPA necessitates the computation of the derivative with respect to the parameters for each simulated value. We propose convenient and tractable conditions ensuring the validity of LRM and IPA in the cases of the Brown–Resnick and Smith field, respectively. Obtaining such conditions is intricate owing to the very structure of max-stable fields. Then we focus on risk and dependence measures, which constitute one of the several frameworks where our theoretical results can be useful. We perform a simulation study which shows that both LRM and IPA perform well in various configurations, and provide a real case study that is valuable for the insurance industry.

Effective Online Order Acceptance Policies for Omnichannel Fulfillment

Problem definition: Omnichannel retailing has led to the use of traditional stores as fulfillment centers for online orders. Omnichannel fulfillment problems have two components: (1) accepting a certain number of online orders prior to seeing store demands and (2) satisfying (or filling) some of these accepted online demands as efficiently as possible with any leftover inventory after store demands have been met. Hence, there is a fundamental trade-off between store cancellations of accepted online orders and potentially increased profits because of more acceptances of online orders. We study this joint problem of online order acceptance and fulfillment (including cancellations) to minimize total costs, including shipping charges and cancellation penalties in single-period and limited multiperiod settings. Academic/practical relevance: Despite the growing importance of omnichannel fulfillment via online orders, our work provides the first study incorporating cancellation penalties along with fulfillment costs. Methodology: We build a two-stage stochastic model. In the first stage, the retailer sets a policy specifying which online orders it will accept. The second stage represents the process of fulfilling online orders after the uncertain quantities of in-store purchases are revealed. We analyze threshold policies that accept online orders as long as the inventories are above a global threshold, a local threshold per region, or a hybrid. Results: For a single period, total costs are unimodal as a function of the global threshold and unimodal as a function of a single local threshold holding all other local thresholds at constant values, motivating a gradient search algorithm. Reformulating as an appropriate linear program with network flow structure, we estimate the derivative (using infinitesimal perturbation analysis) of the total cost as a function of the thresholds. We validate the performance of the threshold policies empirically using data from a high-end North American retailer. Our two-location experiments demonstrate that local thresholds perform better than global thresholds in a wide variety of settings. Conversely, in a narrow region with negatively correlated online demand between locations and very low shipping costs, global threshold outperforms local thresholds. A hybrid policy only marginally improves on the better of the two. In multiple periods, we study one- and two-location models and provide insights into effective solution methods for the general case. Managerial implications: Our methods provide effective algorithms to manage fulfillment costs for online orders, demonstrating a significant reduction over policies that treat each location separately and reflecting the significant advantage of incorporating shipping in computing thresholds. Numerical studies provide insights as to why local thresholds perform well in a wide variety of situations.

Sensitivity estimation of conditional value at risk using randomized quasi-Monte Carlo

Conditional value at risk (CVaR) is a popular measure for quantifying portfolio risk. Sensitivity analysis of CVaR is common in risk management and gradient-based optimization algorithms. In this paper, we study the infinitesimal perturbation analysis estimator for CVaR sensitivity using randomized quasi-Monte Carlo (RQMC) simulation. RQMC has proved valuable in financial option pricing with a better rate of convergence compared to Monte Carlo sampling, but theoretical guarantees for this new application of RQMC shall be studied. To this end, we first prove that the RQMC-based estimator is strongly consistent under very mild conditions. Under some technical conditions, RQMC yields a mean error rate of O(n−1/2−1/(4d−2)+ϵ) for arbitrarily small ϵ>0, where d represents the dimension of RQMC points and n is the sample size. Some typical applications of CVaR sensitivity estimation are conducted to both show how the theoretical results can be applied, as well as to provide numerical results documenting the superiority of the RQMC estimator.

A New Likelihood Ratio Method for Training Artificial Neural Networks

We investigate a new approach to compute the gradients of artificial neural networks (ANNs), based on the so-called push-out likelihood ratio method. Unlike the widely used backpropagation (BP) method that requires continuity of the loss function and the activation function, our approach bypasses this requirement by injecting artificial noises into the signals passed along the neurons. We show how this approach has a similar computational complexity as BP, and moreover is more advantageous in terms of removing the backward recursion and eliciting transparent formulas. We also formalize the connection between BP, a pivotal technique for training ANNs, and infinitesimal perturbation analysis, a classic path-wise derivative estimation approach, so that both our new proposed methods and BP can be better understood in the context of stochastic gradient estimation. Our approach allows efficient training for ANNs with more flexibility on the loss and activation functions, and shows empirical improvements on the robustness of ANNs under adversarial attacks and corruptions of natural noises. Summary of Contribution: Stochastic gradient estimation has been studied actively in simulation for decades and becomes more important in the era of machine learning and artificial intelligence. The stochastic gradient descent is a standard technique for training the artificial neural networks (ANNs), a pivotal problem in deep learning. The most popular stochastic gradient estimation technique is the backpropagation method. We find that the backpropagation method lies in the family of infinitesimal perturbation analysis, a path-wise gradient estimation technique in simulation. Moreover, we develop a new likelihood ratio-based method, another popular family of gradient estimation technique in simulation, for training more general ANNs, and demonstrate that the new training method can improve the robustness of the ANN.

Buffer allocation design for unreliable production lines using genetic algorithm and finite perturbation analysis

The buffer allocation problem in production lines is an NP-hard combinatorial optimisation problem. This paper proposes a new hybrid optimisation approach (using simulation) relying on genetic algorithm (GA) and finite perturbation analysis (FPA). Unlike the infinitesimal perturbation analysis, which deals with small (infinitesimal variation) perturbations for estimating gradients of the performance measure, FPA deals with larger (finite) or more lasting perturbations. It is an extension specifically dedicated to discrete decision variables and applicable to most discrete-event dynamic systems. The proposed method allows a global search using GA, with refinement in specific solution-space regions using FPA. The main objective is to maximise the average production rate of a production line with unreliable machines, by allocating the total buffer capacity in locations between machines. Extensive numerical experiments show that: (1) the proposed hybrid GA-FPA method clearly outperforms the state-of-the-art methods from the literature; (2) combining FPA and GA is beneficial when compared to employing GA or FPA independently.

AN IMPROVEMENT OF MARKOVIAN INTEGRATION BY PARTS FORMULA AND APPLICATION TO SENSITIVITY COMPUTATION

This paper establishes a new version of integration by parts formula of Markov chains for sensitivity computation, under much lower restrictions than the existing researches. Our approach is more fundamental and applicable without using Girsanov theorem or Malliavin calculus as did by past papers. Numerically, we apply this formula to compute sensitivity regarding the transition rate matrix and compare with a recent research by an IPA (infinitesimal perturbation analysis) method and other approaches.

Variance Comparison between Infinitesimal Perturbation Analysis and Likelihood Ratio Estimators to Stochastic Gradient

We theoretically compare variances between the Infinitesimal Perturbation Analysis (IPA) estimator and the Likelihood Ratio (LR) estimator to Monte Carlo gradient for stochastic systems. The conditions proposed in [Cui et al., 2020] when the IPA estimator has a smaller variance can yield sharper inequalities or be further relaxed. We also prove a practically interesting result that the IPA estimators to European vanilla and arithmetic Asian options' Delta, respectively, have smaller variance when the underlying asset's return process is independent of the initial price.

The Price of Decentralization: Event-Driven Optimization for Multiagent Persistent Monitoring Tasks

In persistent monitoring tasks, the objective is to control the movements of cooperating agents in order to minimize an uncertainty metric associated with a finite number of targets. We formulate an optimal control problem and show that the optimal solution can be reduced to or approximated by parametric agent trajectory families. The behavior of agents and targets under optimal control can be described by a hybrid system. This enables the use of infinitesimal perturbation analysis to obtain an online centralized solution through a gradient-based algorithm. We identify conditions under which this centralized solution to the parametric optimization problems can be recovered in a decentralized and event-driven manner. In the decentralized scheme, each agent optimizes its performance based on local information, except for one type of nonlocal event requiring communication from a nonneighbor agent, giving rise to a quantifiable “price of decentralization.” Simulation examples are included to illustrate the effectiveness of this “almost decentralized” optimization algorithm and compare it to its fully decentralized counterpart where the aforementioned nonlocal event is ignored.

Collision-free and crossing-free trajectory design for second-order agents persistent monitoring

In this paper, we utilize second-order agents to address a one-dimensional persistent monitoring problem with potential agent collision and boundary-crossing problems. The objective is to minimize a sum of accumulated performance metric associated with targets over a finite time horizon by controlling the movements of agents. Different from the existing work where a collision avoidance algorithm is designed for avoiding collisions and boundary crossings, we consider these problems at the beginning of the problem formulation and further utilize Exterior Point Method (EPM) to propose a novel combinatorial objective function which guarantees a collision-free and crossing-free solution by penalizing all possible agents collisions and boundary crossings. According to Pontryagin Minimum Principle (PMP), we show that the optimal agent trajectories can be fully described by some parameters. Based on Infinitesimal Perturbation Analysis (IPA), a standard gradient descent method is proposed to obtain the optimal parameters. Our collision-free and crossing-free approach is computationally efficient as compared to the existing algorithm. Numerical examples are included to demonstrate the effectiveness of the proposed methods.

Stochastic flow models with delays, blocking and applications to multi-intersection traffic light control

We extend Stochastic Flow Models (SFMs), used for a large class of discrete event and hybrid systems, by including the delays which typically arise in flow movements, as well as blocking effects due to space constraints. We apply this framework to the multi-intersection traffic light control problem by including transit delays for vehicles moving from one intersection to the next and possible blocking between two intersections. Using Infinitesimal Perturbation Analysis (IPA) for this SFM with delays and possible blocking, we derive new on-line gradient estimates of several congestion cost metrics with respect to the controllable green and red cycle lengths. The IPA estimators are used to iteratively adjust light cycle lengths to improve performance and, in conjunction with a standard gradient-based algorithm, to obtain optimal values which adapt to changing traffic conditions. We introduce two new cost metrics to better capture congestions and show that the inclusion of delays and possible blocking in our analysis lead to improved performance relative to models that ignore delays and/or blocking effects.

Collision-Free Trajectory Design for 2-D Persistent Monitoring Using Second-Order Agents

This article considers a 2-D persistent monitoring problem by controlling movements of second-order agents to minimize some uncertainty metric associated with targets in a dynamic environment. In contrast to common sensing models depending only on the distance from a target, we introduce an active sensing model, which considers the distance between an agent and a target, as well as the agent's velocity. We propose an objective function, which can achieve a collision-free agent trajectory by penalizing all possible collisions. Applying structural properties of the optimal control derived from the Hamiltonian analysis, we limit agent trajectories to a simpler parametric form under a family of 2-D curves depending on the problem setting, e.g., ellipses and Fourier trajectories. Our collision-free trajectories are optimized through an event-driven infinitesimal perturbation analysis and gradient descent method. Although the solution is generally locally optimal, this method is computationally efficient and offers an alternative to other traditional time-driven methods. Finally, simulation examples are provided to demonstrate our proposed results.

A Monte Carlo Method for Estimating Sensitivities of Reflected Diffusions in Convex Polyhedral Domains

We develop an effective Monte Carlo method for estimating sensitivities or gradients of expectations of sufficiently smooth functionals of a reflected diffusion in a convex polyhedral domain with respect to its defining parameters, namely its initial condition, drift, and diffusion coefficients and directions of reflection. Our method, which falls into the class of infinitesimal perturbation analysis (IPA) methods, uses a probabilistic representation for such sensitivities as the expectation of a functional of the reflected diffusion and its associated derivative process. The latter process is the unique solution to a constrained linear stochastic differential equation with jumps whose coefficients, domain, and directions of reflection are modulated by the reflected diffusion. We propose a consistent estimator for such sensitivities using an Euler approximation of the reflected diffusion and its associated derivative process. Proving that the Euler approximation converges is challenging because the derivative process jumps whenever the reflected diffusion hits the boundary of the domain. A key step in the proof is establishing a continuity property of the related derivative map, which is of independent interest. We compare the performance of our IPA estimator with a standard likelihood ratio estimator (whenever the latter is applicable) and provide numerical evidence that the variance of the former is substantially smaller than that of the latter. We illustrate our method with an example of a rank-based interacting diffusion model of equity markets. Interestingly, we show that estimating certain sensitivities of the rank-based interacting diffusion model using our method for a reflected Brownian motion description of the model outperforms a finite difference method for a stochastic differential equation description of the model.

Stochastic Gradient Estimation for Artificial Neural Networks

We investigate a new approach to compute the gradients of artificial neural networks (ANNs), based on the so-called push-out likelihood ratio method. Unlike the widely used backpropagation (BP) method that requires continuity of the loss function and the activation function, our approach bypasses this requirement by injecting artificial noises into the signals passed along the neurons. We show how this approach has a similar computational complexity as BP, and moreover is more advantageous in terms of removing the backward recursion and eliciting transparent formulas. We also formalize the connection between BP, a pivotal technique for training ANNs, and infinitesimal perturbation analysis, a classic path-wise derivative estimation approach, so that both our new proposed methods and BP can be better understood in the context of stochastic gradient estimation. Our approach allows efficient training for ANNs with more flexibility on the loss and activation functions, and shows empirical improvements on the robustness of ANNs under adversarial attacks and corruptions of natural noises.

Automated Sensitivity Computations for Bayesian Markov Chain Monte Carlo Inference: A New Approach for Prior Robustness and Convergence Analysis

Infinitesimal perturbation analysis is a widely used approach to assess the input sensitivities of stochas- tic dynamic systems in the classical simulation context. In this paper, we introduce an efficient nu- merical approach to undertake Infinitesimal perturbation analysis in the context of dependent Markov Chain Monte Carlo simulations widely used in Bayesian inference. Building on recent developments, we develop a scheme based on automatic differentiation to compute posterior sensitivities based on exact (up to computer floating point error) first-order derivatives of draws (Jacobians) with respect to prior input parameters alongside the estimation algorithm. Assessing such local robustness of posterior inference poses a challenge for existing methods such as finite differencing, symbolic differentiation, and likelihood ratio methods due to the complex stochastic dependence structure and computational intensity of dependent sample draws. The proposed methods allow for a comprehensive and exact local sensitivity analysis of Markov Chain Monte Carlo output with respect to all input parameters, i.e., prior hyper-parameters (prior robustness) and chain starting values (convergence). Our imple- mentation focuses on methods for Gibbs-based algorithms widely used for inference and forecasting in complex high-dimensional settings. We illustrate how the methods can help practitioners assess convergence and prior robustness in an application of Bayesian Vector Autoregression with shrinkage priors for US macroeconomic time series data and forecasting.