S-AvE: Semantic Active Vision Exploration and Mapping of Indoor Environments for Mobile Robots

Vincenzo Suriani,Sandeep R Sabbella,Daniele Nardi,Sara Kaszuba,Francesco Riccio

doi:10.1109/ecmr50962.2021.9568806

Abstract

In order to operate and to understand human commands, robots must be provided with a knowledge representation integrating both geometric and symbolic knowledge. In the literature, such a representation is referred to as a semantic map that enables the robot to interpret user commands by grounding them to its sensory observations. However, even though a semantic map is key to enable cognition and high-level reasoning, it is a complex challenge to address due to generalization to various scenarios. As a consequence, commonly used techniques do not always guarantee rich and accurate representations of the environment and of the objects therein. In this paper, we set aside from previous approaches by attacking the problem of semantic mapping from a different perspective. While proposed approaches mainly focus on generating a reliable map starting from sensory observations often collected with a human user teleoperating the mobile platform, in this paper, we argue that the process of semantic mapping starts at the data gathering phase and it is a combination of both perception and motion. To tackle these issues, we design a new family of approaches to semantic mapping that exploit both active vision and domain knowledge to improve the overall mapping performance with respect to other map-exploration methodologies.

Highlights

Simultaneous exploration and map building are fundamental skills for mobile robots
Autonomous map building processes rely upon map exploration techniques that provide robots with an effective strategy to visit unknown portions of the environment
To improve robot capabilities in exhaustively exploring the environment at the semantic level, we introduce S-AVE (Semantic-Based Active Vision Exploration), a new map exploration technique

Summary

Introduction

Simultaneous exploration and map building are fundamental skills for mobile robots. building a comprehensive map of the environment – spanning from raw sensory observation to high-level semantic concepts [12] – is an extremely difficult task [13]. Semantic mapping [6, 9] enhances geometric, metric and topological knowledge about the environment by means of semantic concepts, enabling improved robot cognition In this context, classic exploration techniques are still generally used, resulting in inaccurate and incomplete semantic maps. Robot executes aims at maximizing the amount of visited portions of the environment disregarding objects therein Such approaches, do not exploit semantic labels to influence the map exploration strategy and often result in incomplete and poorly detailed maps, which limit robots’ autonomy and abilities. S-AVE, instead, explicitly uses detected objects to drive exploration by means of active vision [2, 3, 1], and it combines semantic information to state-of-the-art map exploration techniques [8] to improve the 3D representation of the environment. We release the source code for both S-AVE2, as well as the data collected during our user study

Semantic Active-Vision Exploration

Experimental Evaluation

Conclusion