Panoptic Segmentation Using Multispectral Worldview-3 Images in Beach Areas

Abstract

Panoptic segmentation combines instance and semantic seg-mentation, enabling the classification of objects and back-grounds. It is still a method little explored in the remote sensing field, mostly due to the difficulty of generating the data. Moreover, the beach areas have great interest due to many objects and elements that may guide public policies. In this regard, we propose the first study on beach areas using panop-tic segmentation and the first panoptic segmentation study using multispectral data. We used the Gram-Schmidt pan-sharpening method for the multispectral bands and created a dataset with 850 samples with 128x 128 dimensions in the COCO panoptic annotation format. To evaluate the dataset, the Panoptic-FPN was used with modifications in the input (changing from three to eight channels). Results show 59.43 Panoptic Quality (PQ), 77.96 Segmentation Quality (SQ), and 75.07 Recognition Quality (RQ).