Reducing Online Model-Development Time by Agents Using Constraints Between Shared Observations

Abstract

A situated agent must determine aspects of its environment in order to make appropriate decisions. This determination must be done quickly, as performance can suffer until each agent develops a sufficiently accurate model of its environment. We introduce a two-phase model-development approach called pre-deployment learning and situated model-development in agents (PLASMA) that leads to a significant reduction in the online (post-deployment) time required to determine environmental models. During the pre-deployment phase, an incompletely specified, site-independent model of an agent's environment is developed, with the site-dependent features represented as parameters in the model. This pre-deployment model is then completed during the post-deployment phase by determining the model parameters using constraints between local and shared observations. In this article, we use the PLASMA approach in developing an environmental model for potential solar visibility and panel collection characteristics by each agent in a power-aware wireless sensor network. We show that, by using temporal and spatial constraints between shared observations, agents can reduce online model-development time by as much as 80% relative to a standard multi-agent reinforcement learning algorithm. Furthermore, we show that using constraints between shared observations can further reduce post-deployment parameter determination time by as much as 67% relative to sharing only local variable values in a distributed variable satisfaction post-deployment technique. In all but the most unlikely of environmental conditions, using the PLASMA-based approach allows individual agent-harvesting models to be completed using only the first and second day observations when compared with 10 days of observations required by the power management algorithm of Kansal et al.