Improving Object Disambiguation from Natural Language using Empirical Models

Abstract

Robots, virtual assistants, and other intelligent agents need effectively interpret verbal references environmental objects in order successfully interact and collaborate with humans in complex tasks. However, object disambiguation can be a challenging task due ambiguities in natural language. To reduce uncertainty when describing an object, humans often use a combination of unique object features and locative prepositions --prepositional phrases that describe where an object is located relative other features (i.e., reference objects) in a scene. We present a new system for object disambiguation in cluttered environments based on probabilistic models of unique object features and spatial relationships. Our work extends prior models of spatial relationship semantics by collecting and encoding empirical data from a series of crowdsourced studies better understand how and when people use locative prepositions, how reference objects are chosen, and how model prepositional geometry in 3D space (e.g., capturing distinctions between next to and beside). Our approach also introduces new techniques for responding compound locative phrases of arbitrary complexity and proposes a new metric for disambiguation confidence. An experimental validation revealed our method can improve object disambiguation accuracy and performance over past approaches.