Abstract
Deep convolutional neural networks have shown outstanding performance in the task of semantically segmenting images. Applying the same methods on 3D data still poses challenges due to the heavy memory requirements and the lack of structured data. Here, we propose LatticeNet, a novel approach for 3D semantic segmentation, which takes raw point clouds as input. A PointNet describes the local geometry which we embed into a sparse permutohedral lattice. The lattice allows for fast convolutions while keeping a low memory footprint. Further, we introduce DeformSlice, a novel learned data-dependent interpolation for projecting lattice features back onto the point cloud. We present results of 3D segmentation on multiple datasets where our method achieves state-of-the-art performance. We also extend and evaluate our network for instance and dynamic object segmentation.
Highlights
Environment understanding is a crucial ability for autonomous agents
We propose LatticeNet, a novel approach for point cloud segmentation which alleviates the previously mentioned problems
We propose four new operators on the permutohedral lattice which are more suitable for CNNs and dense prediction tasks
Summary
Perceiving the geometrical structure of the scene and distinguishing between different classes of objects therein enables tasks like manipulation and interaction that were previously not possible Within this field, semantic segmentation of 2D images is a mature research area, showing outstanding success in dense per pixel categorization on images (Long et al 2015; Chen et al 2017; Lin et al 2017). This is one of the several papers published in Autonomous Robotscomprising the Special Issue on Robotics: Science and Systems 2020. – A network architecture capable of processing temporal information in order to improve semantic segmentation and to distinguish between dynamic and static objects within the scene
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have