Optimal Seamline Research Articles

A Segmentation-Based Optimal Seamline Generation Method for SAR Image Mosaic

In the mosaic creation of multiple high-resolution synthetic aperture radar (SAR) images, achieving an optimal seamline in overlapping areas is crucial for seamless and visually satisfactory results. Many existing seamline generation methods are designed primarily for optical remote sensing images, but due to the differing characteristics of SAR images and optical images, applying these methods directly to SAR images poses challenges in finding the optimal seamline. In response, this paper proposes a segmentation-based optimal seamline generation (SOSG) method for SAR image mosaics. The SOSG method involves a multi-step process. First, SAR image joint segmentation is performed within the overlapping areas. Subsequently, homogeneous areas are identified based on the segmentation results. Following this, a pixel cost matrix is constructed, incorporating homogeneous areas and intensity differences. Finally, the minimum path cost from the starting pixel to the end pixel is computed using the Dijkstra algorithm to determine the optimal path. To assess the feasibility and effectiveness of the proposed method, experiments are conducted using multiple SAR images from the Chinese Gaofen-3 01 satellite as datasets. The experimental results demonstrate that the proposed method yields seamless mosaic images when compared to other methods, while delivering satisfactory outcomes. This indicates the potential of the proposed method in addressing the unique challenges posed by SAR images and enhancing the quality of SAR image mosaics.

Research on seamless image stitching based on fast marching method

AbstractImage stitching is an important way to achieve large‐field high‐resolution imaging. The inconsistencies in brightness and structure and defects in ghosting, blurring and misalignment between images, which are inevitable and difficult to eliminate, make a challenge to image stitching, due to the external lighting environment and changes in camera pose and parameters. Here, a novel method is proposed to search for the optimal seamline based on the fast marching method, which can stitch large parallax images with high quality. A feature weight map is first formed based on the similarity in colour, edge, texture and saliency of the images. Then it is used as the cost value of the seamline to search for the optimal seamline by fast marching method. The results show that this new method is more efficient to reduce defects, such as ghosting, misalignment and chromatic aberration, and realize high quality image stitching compared with traditional stitching tools and methods, which provides a new perspective for image stitching technology.

A Fast Globally Optimal Seamline Detection Method for High-Resolution Remote Sensing Images

Seamline detection is one of the most important issues in mosaicking high resolution remote sensing images (HRRSI). However, it is difficult to make a balance between efficiency and accuracy. On that account, a shortest matrix path-based dynamic programming (SMP-DP) algorithm is proposed to find the optimal seamline for HRRSI mosaicking. Firstly, a pixel cost matrix defined by intensity difference, gradient similarity, and geometric difference is constructed in the overlapping area. Secondly, the least average path cost from the starting pixel to each pixel is calculated and the SMP-DP algorithm is applied to find the optimal path. Experimental results on HRRSI prove that the proposed method detects high-quality seamline and crosses much fewer objects with the highest computational efficiency, compared with state-of-the-art method and commercial software.

An Upscaling–Downscaling Optimal Seamline Detection Algorithm for Very Large Remote Sensing Image Mosaicking

For the mosaicking of multiple remote sensing images, obtaining the optimal stitching line in the overlapping region is a key step in creating a seamless mosaic image. However, for very large remote sensing images, the computation of finding seamlines involves a huge amount of image pixels. To handle this issue, we propose a stepwise strategy to obtain pixel-level optimal stitching lines for large remote sensing images via an upscaling–downscaling image sampling procedure. First, the resolution of the image is reduced and the graph cut algorithm is applied to find an energy-optimal seamline in the reduced image. Then, a stripe along the preliminary seamline is identified from the overlap area to remove the other inefficient nodes. Finally, the graph cut algorithm is applied nested within the identified stripe to seek the pixel-level optimal seamline of the original image. Compared to the existing algorithms, the proposed method produces fewer spectral differences between stitching lines and less-crossed features in the experiments. For a wide range of remote sensing images involving large data, the new method uses less than 10 percent of the time needed by the SLIC+ graph cut method.

Optimizing Local Alignment along the Seamline for Parallax-Tolerant Orthoimage Mosaicking

Orthoimage mosaicking with obvious parallax caused by geometric misalignment is a challenging problem in the field of remote sensing. Because the obvious objects are not included in the digital terrain model (DTM), large parallax exists in these objects. A common strategy is to search an optimal seamline between orthoimages, avoiding the majority of obvious objects. However, stitching artifacts may remain because (1) the seamline may still cross several obvious objects and (2) the orthoimages may not be precisely aligned in geometry when the accuracy of the DTM is low. While applying general image warping methods to orthoimages can improve the local geometric consistency of adjacent images, these methods usually significantly modify the geometric properties of orthophoto maps. To the best of our knowledge, no approach has been proposed in the field of remote sensing to solve the problem of local geometric misalignments after orthoimage mosaicking with obvious parallax. In this paper, we creatively propose a method to optimize local alignment along the seamline after seamline detection. It consists of the following main processes. First, we locate regions with geometric misalignments along the seamline based on the similarity measure. Second, for any one region, we find one-dimensional (1D) feature matches along the seamline using a semi-global matching approach. The deformation vectors are calculated for these matches. Third, these deformation vectors are robustly and smoothly propagated into the buffer region centered on the seamline by minimizing the associated energy function. Finally, we directly warp the orthoimages to eliminate the local parallax under the guidance of dense deformation vectors. The experimental results on several groups of orthoimages show that our proposed approach is capable of eliminating the local parallax existing in the seamline while preserving most geometric properties of digital orthophoto maps, and that it outperforms state-of-the-art approaches in terms of both visual quality and quantitative metrics.

Hyperspectral Image Stitching via Optimal Seamline Detection

Hyperspectral images (HSIs) with both spatial and spectral information have found broad applications. Since most cameras have narrow viewing angle, generating panoramic images is essential to show a large-range view of the environment. So far, there are few studies on HSI stitching, and the stitching result still suffers from some problems, such as blurring and ghosting, geometric misalignment, visible seam, and spectral distortion. Hence, to address the above disadvantages, we propose a novel HSI stitching strategy using optimal seamline detection approach in this letter. First, we use a fast and robust seam estimation method to determine the seamline in each single band of HSI. This method works in RGB images, and we have modified it to be used in a single-band gray-scale image of HSI. Then, to guarantee the integrity of spatial and spectral information of hundreds of bands of HSI, we propose to apply the structural similarity (SSIM) index to select the optimal one among all band candidate seamlines and use the selected optimal seamline to stitch all the remaining bands. The experimental results demonstrate that our proposed approach outperforms traditional HSI stitching approach in both spatial and spectral performances.

Underwater Terrain Image Stitching Based on Spatial Gradient Feature Block

At present, underwater terrain images are all strip-shaped small fragment images preprocessed by the side-scan sonar imaging system. However, the processed underwater terrain images have inconspicuous and few feature points. In order to better realize the stitching of underwater terrain images and solve the problems of slow traditional image stitching speed, we proposed an improved algorithm for underwater terrain image stitching based on spatial gradient feature block. First, the spatial gradient fuzzy C-Means algorithm is used to divide the underwater terrain image into feature blocks with the fusion of spatial gradient information. The accelerated-KAZE (A-KAZE) algorithm is used to combine the feature block information to match the reference image and the target image. Then, the random sample consensus (RANSAC) is applied to optimize the matching results. Finally, image fusion is performed with the global homography and the optimal seam-line method to improve the accuracy of image overlay fusion. The experimental results show that the proposed method in this paper effectively divides images into feature blocks by combining spatial information and gradient information, which not only solves the problem of stitching failure of underwater terrain images due to unobvious features, and further reduces the sensitivity to noise, but also effectively reduces the iterative calculation in the feature point matching process of the traditional method, and improves the stitching speed. Ghosting and shape warping are significantly eliminated by re-optimizing the overlap of the image.

Dynamic mosaicking: combining A* algorithm with fractional Brownian motion for an optimal seamline detection

Image mosaicking is a combination of algorithms that use two or several images to create a single image. The resulting mosaic is a representation of a scene of the used images with a larger field of vision. However, since dynamic objects can exist in the overlap regions of these images, ghosting and parallax effects appear, therefore poor results are obtained. To overcome these unwanted effects and to achieve better results, a new method is presented in this paper. This approach uses a new way to detect dynamic objects in the common areas by using a fractional Brownian motion with a predetermined similarity function instead of a noise function, the Zero Normalized Cross Correlation. Thus, it will ensure that a map is created with each pixel having a unique value based on their surroundings even in homogeneous areas. Furthermore, this new approach combines the previously computed map with the machine learning algorithm A* for a fast and efficient way to find an optimal seamline. Consequently, the obtained experimental results were compared with different methods and better results were obtained as can be seen by a better quality seamline measure, a result mosaic without any artifacts and a faster computation time.

Underwater Image Enhancement and Mosaicking System Based on A-KAZE Feature Matching

Feature extraction and matching is a key component in image stitching and a critical step in advancing image reconstructions, machine vision and robotic perception algorithms. This paper presents a fast and robust underwater image mosaicking system based on (2D)2PCA and A-KAZE key-points extraction and optimal seam-line methods. The system utilizes image enhancement as a preprocessing step to improve quality and allow for greater keyframe extraction and matching performance, leading to better quality mosaicking. The application focus of this paper is underwater imaging and it demonstrates the suitability of the developed system in advanced underwater reconstructions. The results show that the proposed method can address the problems of noise, mismatching and quality issues which are typically found in underwater image datasets. The results demonstrate the proposed method as scale-invariant and show improvements in terms of processing speed and system robustness over other methods found in the literature.

Research on key techniques of multi-resolution coastline image fusion based on optimal seam-line

The coastline can be used as the demarcation line between the ocean and the continent, which plays a crucial role in the development of marine economy. The purpose of this paper is to achieve stitching of coastline images from different areas automatically and create panoramic coast images. To be specific, we proposed a multi-resolution image fusion method which is based on the optimal seam-line method to solve the huge shadow and unnatural image connection problems which caused by the process of the fused region of coastline splicing. In this paper, we adopted the multi-resolution wavelet fusion method by using multi-resolution wavelet fusion method in searching the optimal seam-line. At the same time, it is found in the contrast experiment that the image fused by the proposed algorithm has no obvious shadow situation and can be better fused in vision.

Dynamic mosaicking: region-based method using edge detection for an optimal seamline

Image mosaicking is a process of assembling multiple images to create an image with a larger field of view. It is used in different studies, but some errors, like ghosting or parallax effects, could occur when the images contain dynamic elements. To avoid the failure of the mosaic and to solve these errors, a new method that searches an optimal seamline for dynamic mosaicking is presented. By finding regions that are similar between the images, and regions that are not alike, the seamline was computed by going through the similar regions and by avoiding the not common regions. To achieve this, a combination of Canny edge detector, and the outliers and inliers from the RANSAC method were used to identify these regions. Then, the regions were incorporated in an intensity difference to create a map that reveals them. Thus, the optimal seamline was computed by going through the similar regions and by avoiding the unalike regions. The experimental results show that the proposed approach is capable of generating robust mosaics against ghosting and parallax effects.

A dynamic mosaicking method for finding an optimal seamline with Canny edge detector

Image mosaicking is a process of assembling multiple images to create an image with a larger field of view. Therefore, to avoid failures and to solve errors that could occur during this process, a new method for dynamic mosaicking is presented. This method finds regions that are similar between the images and regions that are not alike, then an optimal seamline is computed by going through the similar regions as much as possible and by avoiding the not common regions. In fact, this is done by combining Canny edge detector and the outliers and inliers from the RANSAC method to identify these areas. In addition, by highlighting these regions in an intensity difference, a map that reveals which areas to avoid and to frequent was created. Thus, an optimal seamline was found. The experimental results show that the proposed approach is capable of generating robust mosaics against ghosting and parallax effects.

Seamline network generation based on foreground segmentation for orthoimage mosaicking

For multiple orthoimages mosaicking, the detection of an optimal seamline in an overlapped region and the generation of a seamline network are two key issues for creating a seamless and pleasant large-scale digital orthophoto map. In this paper, a novel system is proposed to generate the large-scale orthophoto by mosaicking multiple orthoimages via Graph cuts. The proposed system is comprised of two parts. In the first part, to ensure that the detected seamline avoids crossing the obvious objects, a novel foreground segmentation-based approach is proposed to detect the optimal seamline for two adjacent images. The foreground objects are segmented from the overlapped region at the superpixel level followed by the pixel-level seamline optimization. In the second part, we propose a novel seamline network generation approach to produce the large-scale orthophoto by mosaicking multiple orthoimages. The pairwise and junction regions extracted from the initial network are refined using two-label and multi-label Graph cuts, respectively. The key advantage of our proposed seamline network is that junction points can be automatically and optimally found using the multi-label Graph cuts. The experimental results on two groups of orthoimages show that our proposed system can generate high-quality seamline networks with less artifacts, and that it outperforms the state-of-the-art algorithm and the commercial software based on visual comparison and statistical evaluation.

Automatic Seam-Line Detection in UAV Remote Sensing Image Mosaicking by Use of Graph Cuts

Image mosaicking is one of the key technologies in data processing in the field of computer vision and digital photogrammetry. For the existing problems of seam, pixel aliasing, and ghosting in mosaic images, this paper proposes and implements an optimal seam-line search method based on graph cuts for unmanned aerial vehicle (UAV) remote sensing image mosaicking. This paper first uses a mature and accurate image matching method to register the pre-mosaicked UAV images, and then it marks the source of each pixel in the overlapped area of adjacent images and calculates the energy value contributed by the marker by using the target energy function of graph cuts constructed in this paper. Finally, the optimal seam-line can be obtained by solving the minimum value of target energy function based on graph cuts. The experimental results show that our method can realize seamless UAV image mosaicking, and the image mosaic area transitions naturally.

Optimal Seamline Detection for Orthoimage Mosaicking Based on DSM and Improved JPS Algorithm

Based on the digital surface model (DSM) and jump point search (JPS) algorithm, this study proposed a novel approach to detect the optimal seamline for orthoimage mosaicking. By threshold segmentation, DSM was first identified as ground regions and obstacle regions (e.g., buildings, trees, and cars). Then, the mathematical morphology method was used to make the edge of obstacles more prominent. Subsequently, the processed DSM was considered as a uniform-cost grid map, and the JPS algorithm was improved and employed to search for key jump points in the map. Meanwhile, the jump points would be evaluated according to an optimized function, finally generating a minimum cost path as the optimal seamline. Furthermore, the search strategy was modified to avoid search failure when the search map was completely blocked by obstacles in the search direction. Comparison of the proposed method and the Dijkstra’s algorithm was carried out based on two groups of image data with different characteristics. Results showed the following: (1) the proposed method could detect better seamlines near the centerlines of the overlap regions, crossing far fewer ground objects; (2) the efficiency and resource consumption were greatly improved since the improved JPS algorithm skips many image pixels without them being explicitly evaluated. In general, based on DSM, the proposed method combining threshold segmentation, mathematical morphology, and improved JPS algorithms was helpful for detecting the optimal seamline for orthoimage mosaicking.

Edge-Enhanced Optimal Seamline Detection for Orthoimage Mosaicking

In this letter, we propose a novel algorithm to detect the optimal seamline by using the edge-enhanced energy function for orthoimage mosaicking. First, we generate the energy cost map by using traditional intensity and gradient difference. Then, to ensure that the detected optimal seamlines avoid crossing the obvious objects, the energy map is enhanced in the regions of valid edge segments around the obvious objects. The edge segments are separately detected from input images, and the invalid edge segments are removed by using texture complexity index, image difference, and consistency constraint. Finally, we detect the optimal seamline from this edge-enhanced energy cost map via graph cuts. Experimental results on several groups of challenging orthoimages show that the proposed algorithm is capable of creating high-quality seamlines for orthoimage mosaicking and outperforms state-of-the-art algorithms and the commercial software based on the visual comparison and statistical evaluation.

Improved Seam-Line Searching Algorithm for UAV Image Mosaic with Optical Flow.

Ghosting and seams are two major challenges in creating unmanned aerial vehicle (UAV) image mosaic. In response to these problems, this paper proposes an improved method for UAV image seam-line searching. First, an image matching algorithm is used to extract and match the features of adjacent images, so that they can be transformed into the same coordinate system. Then, the gray scale difference, the gradient minimum, and the optical flow value of pixels in adjacent image overlapped area in a neighborhood are calculated, which can be applied to creating an energy function for seam-line searching. Based on that, an improved dynamic programming algorithm is proposed to search the optimal seam-lines to complete the UAV image mosaic. This algorithm adopts a more adaptive energy aggregation and traversal strategy, which can find a more ideal splicing path for adjacent UAV images and avoid the ground objects better. The experimental results show that the proposed method can effectively solve the problems of ghosting and seams in the panoramic UAV images.

Superpixel-based optimal seamline detection in the gradient domain via graph cuts for orthoimage mosaicking

ABSTRACTThis paper presents an optimal seamline detection method for orthoimage mosaicking. To ensure that the detected optimal seamlines avoid crossing many obvious objects, we first design a simple but effective criterion in the gradient domain in lieu of the traditionally used intensity domain to measure the visibility of the seam. Thereafter, we fuse this new criterion into the graph cuts energy minimisation framework to globally find the last optimal seamlines. Instead of finding the optimal solutions of seamlines in overlap regions via graph cuts among the entire set of pixels, we first find them among superpixels created from input images and then refine them in the pixel level, which greatly improves the efficiency of the global graph cuts energy optimisation because the number of elements in graph cuts dramatically decreases. Experimental results on orthoimages show that our proposed method is capable of finding high-quality seamlines for orthoimage mosaicking, and outperforms state-of-the-art algorithms and software.

Seam-line determination via minimal connected area searching and minimum spanning tree for UAV image mosaicking

ABSTRACTSeam-line-based image fusion technology is widely used in unmanned aerial vehicle (UAV) image mosaicking for its superiority to effectively avoid the ghosting phenomenon. Existing algorithms typically determine a seam-line by minimizing the sum of the mismatching values on the line. However, the strategy of directly summing the mismatch scores is inadequate because a seam-line segment with significant mismatches, even if it is short, is more visible than a lengthy one with small differences. Seeking the minimal solution also results in these methods having large computation cost. To this end, we propose a novel seam-line determination method based on minimal connected area searching and minimum spanning tree to eliminate observable visual discontinuity and improve the time efficiency. The proposed method first finds the minimal connected area by eliminating occlusions across the overlapping area. It then applies the minimum spanning tree algorithm to find the optimal seam-line. Finally, it searches the seam-line around the equidistant lines to reduce the minimum connected area, thus reducing the computation. The experimental results on various types of UAV images demonstrate that the proposed method achieves excellent performance and competitive computational efficiency relative to the state-of-the-art methods for image mosaicking.

A dynamic mosaicking method based on histogram equalization for an improved seamline

Image mosaicking is a process of combining multiple images to obtain an image with a larger field of view. Errors, like ghosting and parallax effects, could appear in the mosaic if the images contain dynamic objects. This paper introduces a new idea based on histogram equalization that was used to find an optimal seamline to avoid dynamic aspects in the images. This was achieved by firstly applying a color transfer algorithm to adjust the lightness between the images in the lab color space. After that a registration was done with SIFT and RANSAC. And finally, before searching the seamline, the overall intensity of the images were fixed by changing the intensity of the underexposed and overexposed areas with a histogram equalization method. As a result, the path of the seamline was improved. The present algorithm gives a fast result and a better detection of the dynamic objects, thus a better a mosaic result.