Visual and physical segmentation of novel objects

Abstract

The ability to recognize novel items in complex environment is fundamental to robot intelligence and manipulation. In this work, we consider the issue of segmenting visual and physical aspects of novel objects of interest, specifically ones that are being seen for the first time and located in the proximity of unidentified obstacles. Illumination at different wavelengths and angles are projected by a robot, thus acquiring additional information about the scene and exploiting it for successful novel objects segmentation. By analyzing shades and reflections of a scene's objects, we were able to form and identify true edges and visually separate items of interest. In order to recognize the physical characteristics of segmented objects, we introduce a novel material classification technique which utilizes static electricity charge sensing. Our proposed methods do not require predefined models of target objects and assume no previously assigned targets' pose. Using extracted object visual and physical aspects, an autonomous robot manipulator successfully performed a trash separation task.