Stitching Results Research Articles

Stitching interferometry of cylindrical surface

High resolution and high precision are advantages of measuring cylindrical objects employing stitching interferometry. However, due to errors in the cylinder position in a measurement procedure, it is difficult to ensure the accuracy of a macroscopic shape contour and surface waviness contour of the workpiece of the stitching results. To overcome this problem, we proposed a new strategy of stitching interferometry for cylindricity calibration. The measuring process is divided into two parts - to measure the macroscopic shape and waviness contour, respectively. Finally, the two measurement results are merged to produce an accurate and complete 360° form map. Besides, we build a two-dimensional Gaussian filter weighting function for a cylindrical profile according to its special shape characteristic. To express the stitching results more precisely, we provide the uncertainty map and power spectral density of the stitching results. An experiment is carried out by taking a ceramic plug gage as the workpiece. The results demonstrate the excellent performance of our method in analyzing the characteristic of the cylindrical profile and show good agreement with such by using cylindricity measuring instruments.

UAV Image Stitching Based on Optimal Seam and Half-Projective Warp

This paper introduces an Unmanned Aerial Vehicle (UAV) image stitching method, based on the optimal seam algorithm and half-projective warp, that can effectively retain the original information of the image and obtain the ideal stitching effect. The existing seam stitching algorithms can eliminate the ghosting and blurring problems on the stitched images, but the deformation and angle distortion caused by image registration will remain in the stitching results. To overcome this situation, we propose a stitching strategy based on optimal seam and half-projective warp. Firstly, we define a new difference matrix in the overlapping region of the aligned image, which includes the color, structural and line difference information. Then, we constrain the search range of the seam by the minimum energy, and propose a seam search algorithm based on the global minimum energy to obtain the seam. Finally, combined with the seam position and half-projective warp, the shape of the stitched image is rectified to keep more regions in their original shape. The experimental results of several groups of UAV images show that our method has a superior stitching effect.

A Robust UAV Hyperspectral Image Stitching Method Based on Deep Feature Matching

Unmanned aerial vehicle (UAV) hyperspectral imaging has been extensively applied in various fields. However, due to the limited imaging width, hyperspectral images (HSIs) captured by UAV need to be stitched, so as to effectively cover the study area. In this article, an effective seamless stitching method with deep feature matching and elastic warp is proposed for HSIs, which consists of the following major steps. First, for each input HSI, a single-band gray-scale image is obtained by fusing the bands corresponding to the red, green, and blue wavelengths. Second, the feature points of each HSI are obtained with a robust VGG-style network and matched with a graph neural network. After point pairs are obtained, the next step is to estimate the transformation matrix of adjacent images, and a spectral correction method based on intrinsic decomposition is proposed to ensure the spectral consistency of adjacent images. In the final stage, a seam-cutting and multiscale blending strategy is adopted to ensure the spatial consistency of the stitching results. Experimental results on real HSIs show that the proposed method is superior to six representative image stitching approaches.

Content-Preserving Image Stitching With Piecewise Rectangular Boundary Constraints.

This article proposes an approach to content-preserving image stitching with regular boundary constraints, which aims to stitch multiple images to generate a panoramic image with piecewise rectangular boundaries. Existing methods treat image stitching and rectangling as two separate steps, which may result in suboptimal results as the stitching process is not aware of the further warping needs for rectangling. We address these limitations by formulating image stitching with regular boundaries in a unified optimization framework. Starting from the initial stitching result produced by traditional warping-based optimization, we obtain the irregular boundary from the warped meshes by polygon Boolean operations which robustly handle arbitrary mesh compositions. By analyzing the irregular boundary, we construct a piecewise rectangular boundary. Based on this, we further incorporate line and regular boundary preservation constraints into the image stitching framework, and conduct iterative optimizations to obtain an optimal piecewise rectangular boundary. Thus we can make the boundary of the stitching result as close as possible to a rectangle, while reducing unwanted distortions. We further extend our method to video stitching, by integrating the temporal coherence into the optimization. Experiments show that our method efficiently produces visually pleasing panoramas with regular boundaries and unnoticeable distortions.

Edge-guided Composition Network for Image Stitching

Panorama creation is still challenging in consumer-level photography because of varying conditions of image capturing. A long-standing problem is the presence of artifacts caused by structure inconsistent image transitions. Since it is difficult to achieve perfect alignment in unconstrained shooting environment especially with parallax and object movements, image composition becomes a crucial step to produce artifact-free stitching results. Current energy-based seam-cutting image composition approaches are limited by the hand-crafted features, which are not discriminative and adaptive enough to robustly create structure consistent image transitions. In this paper, we present the first end-to-end deep learning framework named Edge Guided Composition Network (EGCNet) for the composition stage in image stitching. We cast the whole composition stage as an image blending problem, and aims to regress the blending weights to seamlessly produce the stitched image. To better preserve the structure consistency, we exploit perceptual edges to guide the network with additional geometric prior. Specifically, we introduce a perceptual edge branch to integrate edge features into the model and propose two edge-aware losses for edge guidance. Meanwhile, we gathered a general-purpose dataset for image stitching training and evaluation (namely, RISD). Extensive experiments demonstrate that our EGCNet produces plausible results with less running time, and outperforms traditional methods especially under the circumstances of parallax and object motions.

Ghost Elimination via Multi-Component Collaboration for Unmanned Aerial Vehicle Remote Sensing Image Stitching

Ghosts are a common phenomenon widely present in unmanned aerial vehicle (UAV) remote sensing image stitching that seriously affect the naturalness of stitching results. In order to effectively remove ghosts and produce visually natural stitching results, we propose a novel image stitching method that can identify and eliminate ghosts through multi-component collaboration without object distortion, segmentation or repetition. Specifically, our main contributions are as follows: first, we propose a ghost identification component to locate a potential ghost in the stitching area; and detect significantly moving objects in the two stitched images. In particular, due to the characteristics of UAV shooting, the objects in UAV remote sensing images are small and the image quality is poor. We propose a mesh-based image difference comparison method to identify ghosts; and use an object tracking algorithm to accurately correspond to each ghost pair. Second, we design an image information source selection strategy to generate the ghost replacement region, which can replace the located ghost and avoid object distortion, segmentation and repetition. Third, we find that the process of ghost elimination can produce natural mosaic images by eliminating the ghost caused by initial blending with selected image information source. We validate the proposed method on VIVID data set and compare our method with Homo, ELA, SPW and APAP using the peak signal to noise ratio (PSNR) evaluation indicator.

Image stitching via deep homography estimation

Image stitching is a well-studied problem and has many applications in a variety of fields. Traditional feature based methods rely heavily on accurate localization or even distribution of hand-crafted features, and may fail for some difficult cases. Although there are robust deep learning based homography estimation or semantic alignment methods, their accuracies are not high enough for image stitching problem. In this paper, we present a deep neural network that estimates homography accurately enough for image stitching of images with small parallax. The key components of our network are feature maps with progressively increased resolution and matching cost volumes constructed in hybrid manner. Both of these designs are illustrated to be helpful for performance improvement. We also propose a new stitching oriented loss function that takes image contents into consideration. To train our network, we prepare a synthesized training dataset, the image pairs in which are more nature and similar to those of real world image stitching problem. Experimental results demonstrate that our method outperforms existing deep learning based methods and traditional feature based method in term of quantitative evaluation, visual stitching result and robustness.

Microstructured surface stitching based on wavelet transform three-dimensional data matching

We proposed a subaperture stitching method for concave–convex microstructured surfaces. The method is mainly based on wavelet transform (WT) and three-dimensional (3D) data matching. WT is applied to separate the high-frequency information from the low-frequency information, then the speeded-up robust feature-random sample consensus algorithm is used to match 3D surface data and calculate lateral displacement between adjacent subapertures. Finally, the pose is adjusted to obtain the stitching result. Simulations and experiments show that the proposed method can effectively stitch concave–convex microstructured surfaces.

UAV Image Stitching Based on Mesh-Guided Deformation and Ground Constraint

This article introduces a drone image stitching based on mesh-guided deformation and ground constraint, which can closely match the characteristics of images and achieve precise registration and acquire ideal stitching effect. The traditional methods use the homography model to align the image, which causes artifacts in the result of stitching the images with parallax. To overcome this situation, the image is divided into meshes and the mesh vertices of the target image are used to guide the warping. A new energy function is designed to represent the deformation characteristics of the image. We propose a new alignment term by using local homography and a local scale term by using the edge information of the mesh. The established mesh-guided deformation model can overcome image parallax caused by some external factors and eliminate the ghostly parts of the result. Moreover, imaged scene is not effectively planar and some fluctuations exist in the scene of the images, which will distort the stitching result. We propose a ground constraint with the ground plane as the main plane to reduce projection distortions in non-overlapping areas between images. Finally, the method of creating groundtruth is proposed, which can evaluate the naturalness of results and make comparison more reasonable. Several sets of challenging drone images are tested, and the experimental results show that our stitching system has good results.

Artefact‐free image stitching via a better normed seam‐cutting energy function

Image stitching, as the important field of computer graphics and vision, has received much attention in recent years. Image stitching techniques are generally decomposed into two phases: image alignment, which aligns target images with the reference images; and image composition, which fixes ghosting and visual artefacts. This work aims to propose a new strategy for the seam-cutting method which provides visually appealing result. Seam-cutting is one of the most influential methods in image composition, which can relieve artefacts and produce plausible results. However, it is observed that the state-of-the-art seam-cutting approaches usually lead to undesirable seams in some challenging scenes. Here, the authors put forward a novel seam-cutting method by defining a new energy function. This method uses 5 / 2 power of L 1 norm as a colour difference which can magnify the weight of colour distinction to avoid undesirable seams and artefacts. The proposed method can be easily implemented. The test images are collected from the public available challenging datasets and taken by ourselves. Experiments demonstrate that the proposed method can create comparable or even better stitching results compared to other state-of-the-art seam-cutting approaches.

The Influence of Control Point Distribution of Unmanned Aerial Vehicles for the Results of Aerial Stitching

Unmanned aerial vehicles are an important development direction for today’s aviation telemetry technology, and have gradually become an essential technology for three-dimensional space acquisition. To date, the flight stability of unmanned aerial vehicles has been greatly improved, and space positioning is miniaturized, which ensures the UAV to play the most role in battery life, image real-time transmission, high-risk area detection and other occasions. In the past, natural disasters relied on traditional technology to obtain local conditions, thus, it was impossible to obtain immediate on-site information. However, the immediacy and convenience of unmanned aerial vehicles have been used to grasp the immediate situation of disaster relief sites, providing information for substantial benefits to disaster relief related units. In this study, Pix4Dmapper image processing software is used to quickly create orthophotos, thus, three-dimensional terrain elevation models of stereoscopic view and terrain data can be constructed. The captured images are spliced through the software, and the measurement accuracy of the aerial stitching images is discussed according to the different distribution positions of the control points. The results show that the number of control points is too small to improve accuracy. Regarding the study area, the optimal distribution route of 20 control points is the best. In more complex terrain, the control points should be selected at locations that contain various elevation changes. If it is not possible to evenly arrange the control points, it is recommended to select the circular route method.

Three-dimensional image coordinate-based missing region of interest area detection and damage localization for bridge visual inspection using unmanned aerial vehicles

In this study, the three-phase missing region of interest area detection and damage localization methodology based on three-dimensional image coordinates was proposed. In Phase 1, the coordinate transformation is performed by the position and attitude information of the unmanned aerial vehicles and camera, and the coordinates of the center point of each acquired image are obtained with the distance information between the camera and the target surface. For Phase 2, the size of the field of view of every acquired image is calculated using the focal length and working distance of the camera. Finally, in Phase 3, the missing part of the region of interest area can be identified and any damage detected at the individual image level can also be localized on the whole inspection region using information about the sizes of the field of view in all images calculated in the previous phase. In order to demonstrate the proposed methodology, experimental validation was performed on the actual bridge pier and deck as well as the lab-scale concrete shear wall. In the tests, the missing area detection and damage localization results were compared with image stitching and human visual inspection results, respectively. Experimental validation results have shown that the proposed methodology identifies missing areas and damage locations within reasonable accuracy of 10 cm.

Image stitching with positional relationship constraints of feature points and lines

The elimination of parallax and the processing of natural issue in complex scenes are challenging tasks for image stitching. In this paper, an image stitching method with positional relationship constraints of feature points and lines, which can accomplish accurate alignment and reduce projection distortion, is proposed. At first, to reduce the computational cost and the number of outliers on subsequent feature matching, we combine the template matching to propose a quick way for detecting overlapping regions. Then, the appropriate reference image is determined to mitigate the projection distortion that the image warping is in the cases of non-planar geometry of the scenes. Furthermore, a local mesh model based on dual features is established to guide the mesh deformation. And an energy function is designed to refine alignment. In addition to alignment error terms, a novel positional relationship constraint term is proposed to improve quality of naturalness of final stitching results. Finally, experimental results demonstrate that our approach is superior to the existing image stitching algorithms in improving quality of the stitched image naturalness.

Null test of large convex aspheres by subaperture stitching with replaceable holograms

Null test is preferred for measuring surface error of aspheres. However for large convex aspheres, it is challenging to fabricate larger null optics. Subaperture stitching is hence introduced but requires different null optics for subapertures of different off-axis distances. It gets easier by using replaceable computer generated holograms (CGHs). In this paper, a convex high-order even asphere is taken as an example and multiple CGHs are designed for null test of different subapertures. The same transmission flat and tilt carrier are used to facilitate alignment of CGHs. Disturbance orders of diffraction are confirmed to be separated by the tilt carrier without ghost interference. A configuration-based stitching algorithm is applied to successfully retrieve the full aperture surface error. The CGH substrate error is calibrated out before stitching optimization by measuring its transmitted wavefront with a high precision reference flat. Finally, the stitching result is verified by through-the-back null test of the same asphere.

Bayesian uncertainty evaluation of stitching interferometry for cylindrical surface

Uncertainty evaluation is one of the most important concepts in metrological characterizations. This paper proposes a method for uncertainty evaluation of measurements of cylindrical surfaces by means of stitching interferometry. The proposed method is based on the Bayesian statistical analysis. The prior deviation for a tested surface is determined by both calculation of standard deviation through multiple measurements of a single sub-aperture and calibration of the optical system error. The probability distribution functions (PDF) of the prior and real misalignment error and measured data of the aperture are derived according to the stitching model. Simulated observations are approximated with the Gibbs Sampler using the Markov Chain Monte Carlo (MCMC) method. Our proposed method is a promising alternative to the ordinary least square technique (LST) with the maximum likelihood estimation (MLE) for obtaining the uncertainty of the misalignment and measurement error in the stitching results. Moreover, it can take the inevitable environment errors into account. The prediction of the uncertainty interval makes our method more attractive in industrial applications.

A New Stitching Method for Dark-Field Surface Defects Inspection Based on Simplified Target-Tracking and Path Correction.

A camera-based dark-field imaging system can effectively detect defects of microns on large optics by scanning and stitching sub-apertures with a small field of view. However, conventional stitching methods encounter problems of mismatches and location deviations, since few defects exist on the tested fine surface. In this paper, a highly efficient stitching method is proposed, based on a simplified target-tracking and adaptive scanning path correction. By increasing the number of sub-apertures and switching to camera perspective, the defects can be regarded as moving targets. A target-tracking procedure is firstly performed to obtain the marked targets. Then, the scanning path is corrected by minimizing the sum of deviations. The final stitching results are updated by re-using the target-tracking method. An experiment was carried out on an inspection of our specially designed testing sample. Subsequently, 118 defects were identified out of 120 truly existing defects, without stitching mismatches. The experiment results show that this method can help to reduce mismatches and location deviations of defects, and it was also effective in increasing the detectability for weak defects.

Panorama construction using binary trees

We present a novel algorithm for computing a panorama from a video sequence, which is based on AVL binary trees and is called PCBT. The main characteristic of PCBT is to compute automatically the reference frame, which is used to project the rest of frames on a planar projection. We use the homography as the camera motion model, it is computed only frame to frame, and the rest of the homographies needed for compositing the panorama are automatically computed through the binary tree using the previously computed homographies. The experiments show that PCBT improves the results of Autopano Giga and Image Composite Editor, two well-known applications, and the results of Stitching_Detailed sample of the OpenCV library, for computing panoramas under a planar projection.

A Novel Projective-Consistent Plane Based Image Stitching Method

When different target surfaces, in three-dimensional space, are mapped onto an image plane, they have different projections. These projections vary with the viewpoint. These local differences have influence on the accuracy of image stitching. Most of the existing image stitching methods divide an input image into a number of fixed-size cells, and the pixels within the same cell are then warped using the same local transformation model for the alignment. These methods are based on the hypothesis that the transformation models in one cell are consistent. However, this hypothesis does not hold in general. In this paper, we propose a novel projective-consistent plane based image stitching method (termed PCPS). It divides the overlapping regions of an input image into some projective-consistent planes according to the normal vectors’ orientations of local regions and the reprojection errors of aligned images. The local projective transformation model is estimated for each projective-consistent plane. And then, a hybrid warping model is estimated. For the pixels in overlapping regions, the local projective transformation models are adopted to achieve a better alignment. While for the pixels in non-overlapping regions, a global projective transformation model is estimated by using the inliers uniformly distributed in the projective-consistent planes to avoid distortion. Compared with the state-of-the-art image stitching methods, the experimental results on a number of challenging image sequences show that the projective transformation model estimated by the proposed PCPS method for each projective-consistent plane is more accurate, and the achieved stitching results have less seams and projective distortion.

Single-Perspective Warps in Natural Image Stitching.

Results of image stitching can be perceptually divided into single-perspective and multiple-perspective. Compared to the multiple-perspective result, the single-perspective result excels in perspective consistency but suffers from projective distortion. In this paper, we propose two single-perspective warps for natural image stitching. The first one is a parametric warp, which is an incremental combination of the dual-feature-based as-projective-as-possible warp and the quasi-homography warp. The second one is a mesh-based warp, which is determined by optimizing a total energy function that simultaneously emphasizes different characteristics of the single-perspective warp, including alignment, distortion and saliency. A comprehensive evaluation demonstrates that the proposed warp outperforms some state-of-the-art warps in urban scenes, including APAP, AutoStitch, SPHP and GSP.

Fast Color Blending for Seamless Image Stitching

In this letter, we propose a fast and robust method for stitching overlapped images captured by the unmanned aerial vehicle. First, we apply the shape-preserving half-projective method to precisely and stably align a pair of partially overlapped input images. Then, an optimal stitching line is searched to remove ghosts caused by the moving objects in the overlapped area. We subsequently propose a color blending method to eliminate all the color inconsistencies in the prealigned image. In accordance with the color differences of the pixels on the optimal stitching seam, we utilize weighted value coordinate interpolation algorithms to compute accurate color changes for all the pixels in the target image. The calculated color changes are then added to the target image to remove the color inconsistency. Furthermore, we introduce the superpixel segmentation to divide the target image into a reduced number of superpixels, and we assign each superpixel the same color change value. Such a superpixel level operation can greatly reduce the computational complexity. Experiments show that our method is promising to achieve effective and efficient stitching results.