Semantic maps from multiple visual cues

Abstract

Future service robots targeted to operate in domestic or industrial environment and in close collaboration with humans should possess the ability to produce meaningful internal perceptual representations of their own surroundings, enabling them to fulfill a variety of real-world tasks. For this purpose, we present here a semantic mapping framework featuring geometrical and semasiological attributes that reveal the relationships between objects and places in a real-life environment. The geometrical component consists of a 3D metric map, onto which a topological map is deployed. The semasiological part is realized by putting together a place recognition algorithm and an object recognition one. The categorization of the different places relies on the resolution of appearance-based consistency histograms, while for the recognition of objects in the scene, a hierarchical temporal memory (HTM) network boosted by a saliency attentional model, is utilized. These semantic attributes are then deposited on the topological map to augment it with the belief distributions regarding the visited places, enabling thus the agent to act in an intelligent manner in human populated environments. Thus, the proposed framework outlines a proficient system in the construction of human conceivable environment representations, which has been successfully assessed on real-world scenarios, proving its ability to provide a consistent solution to the emerging problem of the human-robot cohabitation.