Combining Classical and Neural Approaches for Image Segmentation

Abstract

In addition to detecting the 2D bounding boxes of objects in the image, instance segmentation determines on a pixel level, which pixel corresponds to the object. Although recent deep learning based instance segmentation methods are currently state of the art in accuracy, they are relatively slow, use massive computational resources, or are flat-out incapable of running in real-time on cheaper hardware. Classical algorithms can perform background segmentation on images made by infrastructure mounted cameras using far less computational power, however are less robust to changes in scene lighting. They are sensitive to shadows, and cannot perform pixel-level classification of detected objects. In this work, we compare the state of the art of deep learning based methods to classical solutions, and we present a hybrid CNN and classical algorithm based segmentation method, that is capable of running up to 71 frames per second on a standard PC. The method uses Yolov7 for object detection and performs segmentation based on background subtraction only within detected bounding boxes of objects. The background model is created by a classical background segmentation algorithm. The method works on an infrastructure mounted camera, and is robust versus changes in lighting, and even performs well on infrared images.