Algorithms for Learning and Decision Making

Abstract

Abstract : We have investigated learning algorithms for inference and decision making, by using exact and approximate optimization methods. Most of our research has been in approximate dynamic programming/reinforcement learning methods, with a focus on Markovian Decision Problems with a very large number of states. Much of our work is related to a fundamental algorithm, Q-learning, and related new methods that relate to exact and approximate policy iteration. In particular, we have investigated, convergence issues, error bounds, policy oscillation, exploration-enhanced methods, and issues of decision making in a multi-agent environment. Another research area is large-scale convex optimization methods, with a focus on problems whose cost function involves a sum of a large number of component functions. This includes a unifying framework for polyhedral approximation recently proposed by the principal investigator, incremental gradient and subgradient methods, which are currently at the forefront of algorithmic machine learning research, as well as a new incremental version of the proximal minimization algorithm. We have developed new polyhedral approximation algorithms, including a simplicial decomposition method that applies to large-scale conic programming problems.