Learning to Populate Geospatial Databases via Markov Processes

Abstract

Abstract : This report describes a 2 year project on learning to recognize an object using Markov decision processes. The underlying premise is that although the field of computer vision has made a great deal of progress during the last 20 years producing algorithms for specific subtasks (e.g., edge detection, model matching, stereo), it has produced very few end to end applications. This project investigated whether Markov decision processes and reinforcement learning might be used to automatically sequence and control vision algorithms to achieve specific tasks. In particular, we focus on learning object specific recognition strategies using reinforcement learning, where the vision algorithms to be controlled are a set of 11 commonly available 2-D vision algorithms. Although the work reported here is only a first attempt at a complex problem we were able to automatically learn to recognize two different classes of objects (buildings and maintenance rails) in aerial images of Fort Hood. We also were able to learn to distinguish one style of house from four other styles of houses in the residential section of Fort Hood. With a slightly higher error rate, we were able to distinguish all five types of houses from each other, demonstrating an ability to learn to identify similar yet different classes of objects. Finally, we were able to show that for these tasks, the dynamic control policies learned by reinforcement learning were better than any possible fixed sequence of algorithms.