Stochastic Learning and Optimization - A Sensitivity-Based Approach

Abstract

We introduce a sensitivity-based view to the area of learning and optimization of stochastic dynamic systems. We show that this sensitivity-based view provides a unified framework for many different disciplines in this area, including perturbation analysis, Markov decision processes, reinforcement learning, and identification and adaptive control. Many results can be simply derived and intuitively explained by using two performance sensitivity formulas. In addition, we show that this sensitivity-based view opens up new directions for future research. For example, the event-based optimization, which has advantages over the state-based approaches, may be developed with this sensitivity-based view.