Seamline Determination Research Articles

Triangulated Irregular Network based Seamline Determination for Fast Image Stitching of Multiple UAV Images

Abstract. To maintain the efficiency of UAV (unmanned aerial vehicle) remote sensing, rapid image stitching is essential to make multiple UAV images into a seamless mosaic image. Relief displacement introduces variations in object appearance for each image, causing mismatch errors at mosaic seamlines. Traditional approaches involve orthorectifying images using DSMs (digital surface models). While these approaches allow for accurate image stitching, they do not cope with the advantages of UAVs due to their time consumption. In contrast, fast image stitching techniques that do not use orthorectification are well suited for UAV image processing. Related researches have attempted to optimize seamlines to eliminate the errors caused by relief displacement without the use of DSMs. We propose to utilize a TIN (triangular irregular network) of tiepoints to effectively eliminate errors caused by relief displacement while maintaining the fast speed of image stitching. In this study, a TIN is constructed based image tiepoints whose ground coordinates have been obtained through bundle adjustment. The edges of the TIN are used to generate seamlines for image stitching, and the facets of the TIN are used to select minimal images for image stitching and to optimize seamlines. Image stitching results of our proposed method had small error of 1–2 pixels and the processing time of less than 10 minutes for 97 UAV images. This study showed that the proposed method could stitch multiple images while maintaining stable quality using only geometric clues of a TIN of tiepoints.

SEAMLINE OPTIMIZATION FOR UAV IMAGE MOSAICKING USING GEOMETRY OF TRIANGULATED IRREGULAR NETWORK

Abstract. For efficient UAV (Unmanned Aerial Vehicle) image monitoring, it is essential to mosaic multiple UAV images into one seamless image. In a mosaicked image, relief displacement of terrain is a major source of error. It is difficult to form seamlines that avoid all areas of relief displacements. A seamline determination method alone is limited in reducing the mismatch error on mosaicked image. In this study, we constructed a TIN (Triangulated Irregular Network) using tiepoints generated by rigorous bundle adjustments. We detected the regions where relief displacement occurred using the slope of TIN facets. It was found that the error of mosaicked image were mostly on TIN facets with high slopes. Our method generated a mosaicked image after elimination of error-prone region and showed that the distortions were effectively removed. This study showed that the proposed method could produce mosaicked images with stable quality using geometric clues of TIN. We expected that our method can be used for UAV image mosaicking robust to mismatching factors.

True orthophoto mosaic generation: a simple and fast method

UAV-based images are now widely used to generate large-scale orthophoto mosaics. The generation of an orthophoto mosaic is divided into two stages: image registration and image stitching. Using the DSM associated with each image, orthophotos are generated throughout the image registration. The single orthophotos generated are then joined to each other step by step in the second stage, utilizing various methods of image stitching. Image stitching methods depend on the complex and challenging processes of image matching and seamline determination. In addition, the registration and stitching of images in UAV-based mapping projects with a significant number of images is a time-consuming process. In this study, a straightforward method is provided for creating large-scale true orthophoto mosaics from UAV-based images without the requirement to generate single orthophotos, image registration and seamline network determination. Instead of registrating the images and then stitching them step-by-step, this method processes the DSM of the entire area and all of the images simultaneously. First, for each DSM point, the optimal image is determined from among all the visible images based on an optimization procedure. Optimization is based on two criteria: distance from nadir point and distance from the projection center. Using the determined optimal images, the differential rectification procedure is then run, and the orthophoto mosaic cells are filled. The results of this investigation demonstrated that the proposed method yielded a mosaic with minimal changes along the seamline. In addition, the proposed method is compared with the conventional orthophoto mosaic production method, which is based on image matching and determination of seamlines. Evaluations indicate that the proposed method is able to increase the production rate of orthophoto mosaic by 39% and 45% in dataset 1 and 2 respectively. Additionally, the geometric accuracy calculated using the checkpoints in the orthophoto mosaic generated by the suggested method has decreased by an average of 2 cm, indicating more precise results.

Automatic Seamline Determination for Urban Image Mosaicking Based on Road Probability Map from the D-LinkNet Neural Network.

Image mosaicking which is a process of constructing multiple orthoimages into a single seamless composite orthoimage, is one of the key steps for the production of large-scale digital orthophoto maps (DOM). Seamline determination is one of the most difficult technologies in the automatic mosaicking of orthoimages. The seamlines that follow the centerlines of roads where no significant differences exist are beneficial to improve the quality of image mosaicking. Based on this idea, this paper proposes a novel method of seamline determination based on road probability map from the D-LinkNet neural network for urban image mosaicking. This method optimizes the seamlines at both the semantic and pixel level as follows. First, the road probability map is obtained with the D-LinkNet neural network and related post processing. Second, the preferred road areas (PRAs) are determined by binarizing the road probability map of the overlapping area in the left and right image. The PRAs are the priority areas in which the seamlines cross. Finally, the final seamlines are determined by Dijkstra’s shortest path algorithm implemented with binary min-heap at the pixel level. The experimental results of three group data sets show the advantages of the proposed method. Compared with two previous methods, the seamlines obtained by the proposed method pass through the less obvious objects and mainly follow the roads. In terms of the computational efficiency, the proposed method also has a high efficiency.

Efficient seamline determination for UAV image mosaicking using edge detection

ABSTRACTImage mosaicking of data from individual high-resolution unmanned aerial vehicle (UAV) images is required to obtain sufficient coverage over extensive roads. During the mosaicking process, seamlines may be generated due to differences in illumination or projection between individual images, or the presence of moving objects. This study presents an efficient seamline determination technique based on edge detection for UAV road surface images. The algorithm can be divided into three main steps. First, we detect the edges in the overlapping intensity image within the road boundary. Next, we obtain an automatic seamline passing through regions of non-attraction in areas of overlap. Finally, we adjust the values of the overlapping region using the values of the corresponding individual images by following the coordinates of the seamline detected in the second step, ultimately creating an image mosaic. The experiment using UAV images of a road surface demonstrates that the proposed method produces a satisfactory result. The proposed method can be applied for quick mosaicking of UAV images intended for maintaining road safety.

Seam-line determination via minimal connected area searching and minimum spanning tree for UAV image mosaicking

ABSTRACTSeam-line-based image fusion technology is widely used in unmanned aerial vehicle (UAV) image mosaicking for its superiority to effectively avoid the ghosting phenomenon. Existing algorithms typically determine a seam-line by minimizing the sum of the mismatching values on the line. However, the strategy of directly summing the mismatch scores is inadequate because a seam-line segment with significant mismatches, even if it is short, is more visible than a lengthy one with small differences. Seeking the minimal solution also results in these methods having large computation cost. To this end, we propose a novel seam-line determination method based on minimal connected area searching and minimum spanning tree to eliminate observable visual discontinuity and improve the time efficiency. The proposed method first finds the minimal connected area by eliminating occlusions across the overlapping area. It then applies the minimum spanning tree algorithm to find the optimal seam-line. Finally, it searches the seam-line around the equidistant lines to reduce the minimum connected area, thus reducing the computation. The experimental results on various types of UAV images demonstrate that the proposed method achieves excellent performance and competitive computational efficiency relative to the state-of-the-art methods for image mosaicking.

Seamline Determination from Images and Digital Maps for Image Mosaicking

Seamline Determination from Images and Digital Maps for Image Mosaicking

파노라믹 리어뷰를 위한 동적 정합선 결정

파노라믹 리어뷰를 위한 동적 정합선 결정

Seamline Determination Based on PKGC Segmentation for Remote Sensing Image Mosaicking.

This paper presents a novel method of seamline determination for remote sensing image mosaicking. A two-level optimization strategy is applied to determine the seamline. Object-level optimization is executed firstly. Background regions (BRs) and obvious regions (ORs) are extracted based on the results of parametric kernel graph cuts (PKGC) segmentation. The global cost map which consists of color difference, a multi-scale morphological gradient (MSMG) constraint, and texture difference is weighted by BRs. Finally, the seamline is determined in the weighted cost from the start point to the end point. Dijkstra’s shortest path algorithm is adopted for pixel-level optimization to determine the positions of seamline. Meanwhile, a new seamline optimization strategy is proposed for image mosaicking with multi-image overlapping regions. The experimental results show the better performance than the conventional method based on mean-shift segmentation. Seamlines based on the proposed method bypass the obvious objects and take less time in execution. This new method is efficient and superior for seamline determination in remote sensing image mosaicking.

Automatic seam-line determination for orthoimage mosaics using edge-tracking based on a DSM

ABSTRACTA novel seam-line determination method was developed via edge tracking based on Digital Surface Model (DSM). The whole workflow includes 4 steps: generation of initial network with standard Voronoi Diagram, generation of edge diagram with DSM, relocation of vertices in the initial network based on edge diagram, and optimization of seam-lines using edge tracking algorithm based on edge diagram. This method is not sensitive to terrain since the edge diagram is detected by height gradient based on DSM. The edge tracking algorithm is also efficient since none shortest path searching algorithm (such as Dijkstra’s algorithm) is used. All the seam-lines are optimized based on a single edge diagram which has the same size as the DSM data. Preliminary results have shown that the proposed method produces acceptable results, from quality and efficiency perspectives.

TOWARDS FAST MORPHOLOGICAL MOSAICKING OF HIGH-RESOLUTION MULTI-SPECTRAL PRODUCTS – ON IMPROVEMENTS OF SEAMLINES

Abstract. The complex process of fully automatically establishing seamlines for the fast production of high-quality mosaics with high-amount of high-resolution multi-spectral images is detailed and improved in this paper. The algorithm is analyzed and a quasi-linear runtime in the number of considered pixels is proven for all situations. For typical situations the storage is even essentially smaller from a complexity theoretical perspective. Improvements from algorithm practical perspective are specified, too. The influence of different methods for the determination of seamlines based on gradients is investigated in detail for three Sentinel-2 products. The studied techniques cover well-known ones normally based on a single band. But also more sophisticated techniques based on multiple bands or even taking additional external geo-information data are taken into account. Based on the results a larger area covered by Image2006 orthorectified products with data of the Resourcesat-1 mission is regarded. The feasibility of applying advanced subordinated methods for improving the mosaic such as radiometric harmonization is examined. This also illustrates the robustness of the improved seamline determination approaches.

TOWARDS FAST MORPHOLOGICAL MOSAICKING OF HIGH-RESOLUTION MULTI-SPECTRAL PRODUCTS – ON IMPROVEMENTS OF SEAMLINES

The complex process of fully automatically establishing seamlines for the fast production of high-quality mosaics with high-amount of high-resolution multi-spectral images is detailed and improved in this paper. The algorithm is analyzed and a quasi-linear runtime in the number of considered pixels is proven for all situations. For typical situations the storage is even essentially smaller from a complexity theoretical perspective. Improvements from algorithm practical perspective are specified, too. The influence of different methods for the determination of seamlines based on gradients is investigated in detail for three Sentinel-2 products. The studied techniques cover well-known ones normally based on a single band. But also more sophisticated techniques based on multiple bands or even taking additional external geo-information data are taken into account. Based on the results a larger area covered by Image2006 orthorectified products with data of the Resourcesat-1 mission is regarded. The feasibility of applying advanced subordinated methods for improving the mosaic such as radiometric harmonization is examined. This also illustrates the robustness of the improved seamline determination approaches.

Seamline Determination for High Resolution Orthoimage Mosaicking Using Watershed Segmentation

Image mosaicking is a process during which multiple orthoimages are combined into a single seamless composite orthoimage. One of the most difficult steps in the automatic mosaicking of orthoimages is the seamline determination. This paper presents a novel algorithm that selects seamlines based on marker-based watershed segmentation. A representative seamline is extracted at the object level and the pixel level as follows. First, a watershed segmentation is performed to obtain the objects. To avoid over-segmentation, a regional adaptive marker-based watershed segmentation is proposed. Second, the object difference estimated by the correlation coefficient of each object is calculated, and the region adjacency matrix is built. Third, a technique for minimizing the maximum object cost is adopted to determine the objects through which the seamlines pass. Finally, pixel-level optimization is performed using Dijkstra's algorithm with a binary min-heap to determine the final seamlines. The experimental results on digital aerial orthoimages in different areas demonstrate the feasibility and effectiveness of the proposed method compared with other algorithms.

SGM-based seamline determination for urban orthophoto mosaicking

Mosaicking is a key step in the production of digital orthophoto maps (DOMs), especially for large-scale urban orthophotos. During this step, manual intervention is commonly involved to avoid the case where the seamline crosses obvious objects (e.g., buildings), which causes geometric discontinuities on the DOMs. How to guide the seamline to avoid crossing obvious objects has become a popular topic in the field of photogrammetry and remote sensing. Thus, a new semi-global matching (SGM)-based method to guide seamline determination is proposed for urban orthophoto mosaicking in this study, which can greatly eliminate geometric discontinuities. The approximate epipolar geometry of the orthophoto pairs is first derived and proven, and the approximate epipolar image pair is then generated by rotating the two orthorectified images according to the parallax direction. A SGM algorithm is applied to their overlaps to obtain the corresponding pixel-wise disparity. According to a predefined disparity threshold, the overlap area is identified as the obstacle and non-obstacle areas. For the non-obstacle regions, Hilditch thinning algorithm is used to obtain the skeleton line, followed by Dijkstra’s algorithm to search for the optimal path on the skeleton network as the seamline between two orthophotos. A whole seamline network is constructed based on the strip information recorded in flight. In the experimental section, the approximate epipolar geometric theory of the orthophoto is first analyzed and verified, and the effectiveness of the proposed method is then validated by comparing its results with the results of the geometry-based, OrthoVista, and orthoimage elevation synchronous model (OESM)-based methods.

Region change rate-driven seamline determination method

This paper presents a novel method of seamline determination using region change rate (RCR) for orthoimage mosaicking. RCR is the change rate of the segmented region and is calculated based on the percentage of changed pixels in the segmented region. The presented method contains a two-level seamline optimization procedure to determine the final seamline using RCR. First, an object-level optimization based on image segmentation and change detection is executed to determine the allowed regions, i.e., the connected regions with minimized maximum RCR value through which seamlines are allowed to pass. This optimization is used to limit the maximum RCR along the seamline. Second, to determine the optimized seamline, pixel-level optimization is performed using Dijkstra’s algorithm based on differential cost. Finally, experimental results from digital aerial orthoimages and comparisons with other methods demonstrate the potential of the presented method for seamline determination.

Seamline Determination Based on Semantic Segmentation for Aerial Image Mosaicking

We propose a novel method for seamline determination based on semantic segmentation for aerial image mosaicking. First, we train a convolutional neural network (CNN) for pixel labeling to extract building regions. Using the trained CNN, we create a building probability map from an input aerial image with no pre-processing. We then use Dijkstra’s algorithm to find the optimal seamline as a shortest path on the map. We evaluate the quality of the seamlines produced by our method on actual aerial images. Finally, we show that our seamlines never pass through any buildings and compare the effectiveness with the conventional mean-shift segmentation-based method.

Seamline Determination Based on Segmentation for Urban Image Mosaicking

This letter presents a method of seamline determination based on segmentation for orthoimage mosaicking in an urban area. Image segmentation is used to achieve regions of objects. First, preferred regions through which seamlines are inclined to be passed are determined by spans of segmented regions. Second, pixel-level optimization is carried out using Dijkstra's algorithm based on differential cost to find the optimized seamline. The experimental results on digital aerial orthoimages in the urban area prove that the new method is promising for the seamline determination in mosaicking.

Seam-line determination for image mosaicking: A technique minimizing the maximum local mismatch and the global cost

This paper presents a novel algorithm that selects seam-lines for mosaicking image patches. This technique uses Dijkstra’s algorithm to find a seam-line with the minimal objective function. Since a segment of seam-line with significant mismatch, even if it is short, is more visible than a lengthy one with small differences, a direct summation of the mismatch scores is inadequate. Limiting the level of the maximum difference along a seam-line should be part of the objective in the seam-line selection process. Our technique first determines this desired level of maximum difference, then applies Dijkstra’s algorithm to find the best seam-line. A quantitative measure to evaluate a seam-line is proposed. The measure is defined as the sum of a fixed number of top mismatch scores. The proposed algorithm is compared with other techniques quantitatively and visually about various types of images.