Semantic segmentation guided feature point classification and seam fusion for image stitching

Abstract

Image stitching can be employed to stitch images taken from different times, perspectives, or devices into a panorama with a wider view. However, the imaging specification of images to be stitched is strict. If the imaging specification is not satisfied, artefacts caused by inaccurate alignment and unnatural distortion will occur. Semantic segmentation can solve the classification problem at the pixel level; however, image stitching significantly depends on the accuracy of feature points. Therefore, this paper proposes an image stitching algorithm based on semantic segmentation to guide feature point classification and seam fusion. First, the images are recognized by a cascade semantic segmentation network, and the image feature points are classified. Thereafter, the corresponding homography transformations are calculated using different class feature points, and the best homography mapping for the entire target image is selected. Finally, a seam-cutting algorithm based on semantic segmentation is used to compute the seam, and a feathering Poisson fusion with distance transformation is used to eliminate artefacts and light differences. Experiments show that the algorithm can generate transitional natural and perceptual stitching results even under the influence of perspective and light differences.