Generalization of the shortest path approach for superpixel segmentation of omnidirectional images

Abstract

With the growing use of image capture devices using wide angles and the need for fast and accurate image analysis in computer vision, there is a demand for dedicated under-representation methods. Most decomposition methods segment an image into a small number of irregular homogeneous regions, called superpixels. Nevertheless, these approaches are generally designed to process natural 2D planar images, i.e., captured with a 90o angle view without distortion. In this work, we present SphSPS, a new general decomposition method (for Spherical Shortest Path-based Superpixels)11Available code at: https://github.com/rgiraud/sphsps, that is dedicated to wide 360o omnidirectional or spherical images. The produced superpixels respect both the geometry of the 3D spherical acquisition space, and the boundaries of image objects. To fastly extract relevant clustering features, we generalize the shortest path approach between a pixel and a superpixel center. We demonstrate that considering the geometry of the acquisition space to compute the shortest path enables to jointly improve the segmentation accuracy and the shape regularity of superpixels. To evaluate this regularity property, we propose a generalization of a standard 2D regularity metric to the spherical space, addressing the limitations of the only existing spherical compactness measure. Finally, SphSPS is validated on reference 360o images from the PSD (Panorama Segmentation Dataset) and also on synthetic road omnidirectional images. Our method significantly outperforms both planar and spherical state-of-the-art approaches in terms of segmentation accuracy, robustness to noise and regularity, providing a very interesting tool for superpixel-based applications on 360o images.