Dynamic dense CRF inference for video segmentation and semantic SLAM

Abstract

The dense conditional random field (dense CRF) is an effective post-processing tool for image/video segmentation and semantic SLAM. In this paper, we extend the traditional dense CRF inference algorithm to incremental sensor data modelling. The algorithm efficiently infers the maximum a posteriori probability (MAP) solution for a dynamically changing dense CRF model that is applied to incremental multi-class video segmentation and semantic SLAM. The computational cost is roughly proportional to the total change in the Gaussian pairwise edges of the dense CRF. In our system, with an increase in the number of frames of the sensor data, MAP calculations take approximately the same time to compute the overall three-dimensional dense CRF modelled for the entire video. Compared with the traditional dense CRF for video segmentation, this method is more suitable for incremental (in-line) video segmentation and robot semantic SLAM. The results of experiments show that if part of a pairwise edge is altered, our dynamic algorithm is significantly faster than the widely known standard dense CRF algorithm. In addition, the accuracy of its inference does not change. Several multi-class video segmentation tests confirmed the efficiency of inference of the algorithm. In another application, we used the dynamic dense CRF to incrementally integrate robot SLAM and video segmentation. The results show that an accurate SLAM can improve the accuracy of video segmentation, and the computational cost of the dense CRF MAP can be constrained over a constant range. The application of our algorithm is not limited to video segmentation: It is generic, and can be used to yield similar improvements in many optimization solutions for MAP in dynamically changing models.