Improved empirical methods in reinforcement-learning evaluation

Abstract

OF THE DISSERTATION IMPROVED EMPIRICAL METHODS IN REINFORCEMENT-LEARNING EVALUATION by VUKOSI N. MARIVATE Dissertation Director: Michael L. Littman The central question addressed in this research is ”can we define evaluation methodologies that encourage reinforcement-learning (RL) algorithms to work effectively with real-life data?” First, we address the problem of overfitting. RL algorithms are often tweaked and tuned to specific environments when applied, calling into question whether learning algorithms that work for one environment will work for others. We propose a methodology to evaluate algorithms on distributions of environments, as opposed to a single environment. We also develop a formal framework for characterizing the ”capacity” of a space of parameterized RL algorithms and bound the generalization error of a set of algorithms on a distribution of RL environments given a sample of environments. Second, we develop a method for evaluating RL algorithms offline using a static collection of data. Our motivation is that real-life applications of