A sharper definition of alignment for Panoptic Quality

Abstract

The Panoptic Quality metric, developed by Kirillov et al. in 2019, makes object-level precision, recall and F1 measures available for evaluating image segmentation, and more generally any partitioning task, against a gold standard. Panoptic Quality is based on partial isomorphisms between hypothesized and true segmentations. Kirillov et al. desire that functions defining these one-to-one matchings should be simple, interpretable and effectively computable. They show that for t and h, true and hypothesized segments, the condition stating that there are more correct than wrongly predicted pixels, formalized as IoU(t,h)>.5 or equivalently as |t∩h|>.5|t∪h| has these properties. We show that a weaker function, requiring that more than half of the pixels in the hypothesized segment are in the true segment and vice-versa, formalized as |t∩h|>.5|t| and |t∩h|>.5|h|, is not only sufficient but also necessary. With a small proviso, every function defining a partial isomorphism satisfies this condition. We theoretically and empirically compare the two conditions.