Optimal seamline detection for multiple image mosaicking via graph cuts

Abstract

While mosaicking images, especially captured from the scenes of large depth differences with respective to cameras at varying locations, the detection of seamlines within overlap regions is a key issue for creating seamless and pleasant image mosaics. In this paper, we propose a novel algorithm to efficiently detect optimal seamlines for mosaicking aerial images captured from different viewpoints and for mosaicking street-view panoramic images without a precisely common center in a graph cuts energy minimization framework. To effectively ensure that the seamlines are optimally detected in the laterally continuous regions with high image similarity and low object dislocation to magnificently conceal the parallax between images, we fuse the information of image color, gradient magnitude, and texture complexity into the data and smooth energy terms in graph cuts. Different from the traditional frame-to-frame optimization for sequentially detecting seamlines for mosaicking multiple images, our method applies a novel multi-frame joint optimization strategy to find seamlines within multi-overlapped images at one time. In addition, we propose simple but effective strategies to semi-automatically guide the seamlines by exploiting simple human–computer interaction strongly constraining the image regions that the seamlines will or won’t pass through, which is often ignored by many existing seamline detection methods. Experimental results on a large set of aerial, oblique and street-view panoramic images show that the proposed method is capable of creating high-quality seamlines for multiple image mosaicking, while not crossing majority of visually obvious foreground objects and most of overlap regions with low image similarity to effectively conceal the image parallax at different extents.