Recognizing intentions, interactions, and causes of plan failures

Abstract

Natural language systems for the description of image sequences have been developed (e.g. Neumann and Novak, 1986; Herzog et al., 1989). Even though these systems were used to verbalize the behaviour of human agents, they were limited in that they could only describe the purely visual, i.e. spatio-temporal, properties of the behaviour observed. For many applications of such systems (e.g. co-driver systems in traffic, expert systems in high-performance sports, tutorial systems that give “apprentices” instructions in construction tasks, etc.), it seems useful to extend their capabilities to cover a greater part of the performance of a human observer and thus make the system more helpful to the user. In particular, an interpretation process ought to be modelled that yields hypotheses about intentional entities from spatio-temporal information about agents. Its results should be verbalized in natural language.