LEARNING TASK-SPECIFIC SENSING, CONTROL AND MEMORY POLICIES

Abstract

AI agents and robots that can adapt and handle multiple tasks in real time promise to be a powerful tool. To address the control challenges involved in such systems, the underlying control approach has to take into account the important sensory information. Modern sensors, however, can generate huge amounts of data, rendering the processing and representation of all sensor data in real time computationally intractable. This issue can be addressed by developing task-specific focus of attention strategies that limit the sensory data that is processed at any point in time to the data relevant for the given task. Alone, however, this mechanism is not adequate for solving complex tasks since the robot also has to maintain selected pieces of past information. This necessitates AI agents and robots to have the capability to remember significant past events that are required for task completion. This paper presents an approach that considers focus of attention as a problem of selecting controller and feature pairs to be processed at any given point in time to optimize system performance. This approach is further extended by incorporating short term memory and a learned memory management policy. The result is a system that learns control, sensing, and memory policies that are task-specific and adaptable to real world situations using feedback from the world in a reinforcement learning framework. The approach is illustrated using table cleaning, sorting, stacking, and copying tasks in the blocks world domain.