Projective Transformation Model Research Articles

Multi-Modal Image Registration Based on Local Self-Similarity and Bidirectional Matching

The registration of images from infrared sensors and visible color sensors is a quite difficult problem due to their different phenomena. In this paper, we propose a new method to register visible and infrared images. The proposed approach consists of three main steps. In the first step, SURF (Speeded-Up Robust Features) algorithm is applied for local feature extraction. In the second step, LSS (local self-similarity descriptor) is computed for each extracted feature. Finally, a cross matching process followed by a consistency check in the projective transformation model is performed for feature correspondence and mismatch elimination. Experimental results show the proposed method achieves better accuracy for registering visible and infrared images as compared to state-of-the-art approaches.

Refined Calibration Model for Improving the Orientation Precision of Electron Backscatter Diffraction Maps.

For the precise determination of orientations in polycrystalline materials, electron backscatter diffraction (EBSD) requires a consistent calibration of the diffraction geometry in the scanning electron microscope (SEM). In the present paper, the variation of the projection center for the Kikuchi diffraction patterns which are measured by EBSD is calibrated using a projective transformation model for the SEM beam scan positions on the sample. Based on a full pattern matching approach between simulated and experimental Kikuchi patterns, individual projection center estimates are determined on a subgrid of the EBSD map, from which least-square fits to affine and projective transformations can be obtained. Reference measurements on single-crystalline silicon are used to quantify the orientation errors which result from different calibration models for the variation of the projection center.

PyTrx: A Python-Based Monoscopic Terrestrial Photogrammetry Toolset for Glaciology

Terrestrial time-lapse photogrammetry is a rapidly growing method for deriving measurements from glacial environments because it provides high spatio-temporal resolution records of change. Currently, however, the potential usefulness of time-lapse data is limited by the unavailability of user-friendly photogrammetry toolsets. Without prior knowledge in photogrammetry and computer coding, such data are used primarily to calculate ice flow velocities or to serve as qualitative records. PyTrx (available at https://github.com/PennyHow/PyTrx) is presented here as a Python-alternative toolset to widen the range of monoscopic photogrammetry (i.e. from a single viewpoint) toolsets on offer to the glaciology community. The toolset holds core photogrammetric functions for template generation, feature-tracking, image registration, and georectification (using a planar projective transformation model). In addition, PyTrx facilitates areal and line measurements, which can be detected from imagery using either an automated or manual approach. Examples of PyTrx's applications are demonstrated using time-lapse imagery from Kronebreen and Tunabreen, two tidewater glaciers in Svalbard. Products from these applications include ice flow velocities, surface areas of supraglacial lakes and meltwater plumes, and glacier terminus profiles.

A Novel Projective-Consistent Plane Based Image Stitching Method

When different target surfaces, in three-dimensional space, are mapped onto an image plane, they have different projections. These projections vary with the viewpoint. These local differences have influence on the accuracy of image stitching. Most of the existing image stitching methods divide an input image into a number of fixed-size cells, and the pixels within the same cell are then warped using the same local transformation model for the alignment. These methods are based on the hypothesis that the transformation models in one cell are consistent. However, this hypothesis does not hold in general. In this paper, we propose a novel projective-consistent plane based image stitching method (termed PCPS). It divides the overlapping regions of an input image into some projective-consistent planes according to the normal vectors’ orientations of local regions and the reprojection errors of aligned images. The local projective transformation model is estimated for each projective-consistent plane. And then, a hybrid warping model is estimated. For the pixels in overlapping regions, the local projective transformation models are adopted to achieve a better alignment. While for the pixels in non-overlapping regions, a global projective transformation model is estimated by using the inliers uniformly distributed in the projective-consistent planes to avoid distortion. Compared with the state-of-the-art image stitching methods, the experimental results on a number of challenging image sequences show that the projective transformation model estimated by the proposed PCPS method for each projective-consistent plane is more accurate, and the achieved stitching results have less seams and projective distortion.

Robust registration for remote sensing images by combining and localizing feature- and area-based methods

Highly accurate registration is one of the essential requirements for numerous applications of remote sensing images. Toward this end, we have developed a robust algorithm by combining and localizing feature- and area-based methods. A block-weighted projective (BWP) transformation model is first employed to map the local geometric relationship with weighted feature points in the feature-based stage, for which the weight is determined by an inverse distance weighted (IDW) function. Subsequently, the outlier-insensitive (OIS) model aims to further optimize the registration in the area-based stage. Considering the inevitable outliers (e.g., cloud, noise, land-cover change), OIS integrates Huber estimation with the structure tensor (ST), which is an approach that is robust to residual errors and outliers while preserving edges. Four pairs of remote sensing images with varied terrain features were tested in the experiments. Compared with the-state-of-art algorithms, the proposed algorithm is more effective, in terms of both visual quality and quantitative evaluation.

Mapping of local lattice parameter ratios by projective Kikuchi pattern matching

We describe a lattice-based crystallographic approximation for the analysis of distorted crystal structures via Electron Backscatter Diffraction (EBSD) in the scanning electron microscope. EBSD patterns are closely linked to local lattice parameter ratios via Kikuchi bands that indicate geometrical lattice plane projections. Based on the transformation properties of points and lines in the real projective plane, we can obtain continuous estimations of the local lattice distortion based on projectively transformed Kikuchi diffraction simulations for a reference structure. By quantitative image matching to a projective transformation model of the lattice distortion in the full solid angle of possible scattering directions, we enforce a crystallographically consistent approximation in the fitting procedure of distorted simulations to the experimentally observed diffraction patterns. As an application example, we map the locally varying tetragonality in martensite grains of steel.

Band-to-band registration and ortho-rectification of multilens/multispectral imagery: A case study of MiniMCA-12 acquired by a fixed-wing UAS

MiniMCA (Miniature Multiple Camera Array) is a lightweight, frame-based, and multilens composed multispectral sensor, which is suitable to mount on an unmanned aerial systems (UAS) to acquire high spatial and temporal resolution imagery for various remote sensing applications. Since MiniMCA has significant band misregistration effect, an automatic and precise band-to-band registration (BBR) method is proposed in this study. Based on the principle of sensor plane-to-plane projection, a modified projective transformation (MPT) model is developed. It is to estimate all coefficients of MPT from indoor camera calibration, together with two systematic errors correction. Therefore, we can transfer all bands into the same image space. Quantitative error analysis shows that the proposed BBR scheme is scene independent and can achieve 0.33 pixels of accuracy, which demonstrating the proposed method is accurate and reliable. Meanwhile, it is difficult to mark ground control points (GCPs) on the MiniMCA images, as its spatial resolution is low when the flight height is higher than 400m. In this study, a higher resolution RGB camera is adopted to produce digital surface model (DSM) and assist MiniMCA ortho-image generation. After precise BBR, only one reference band of MiniMCA image is necessary for aerial triangulation because all bands have same exterior and interior orientation parameters. It means that all the MiniMCA imagery can be ortho-rectified through the same exterior and interior orientation parameters of the reference band. The result of the proposed ortho-rectification procedure shows the co-registration errors between MiniMCA reference band and the RGB ortho-images is less than 0.6 pixels.

Co-registering satellite images and LIDAR DEMs through straight lines

Co-registration of optical and range data is a crucial step before they could be merged. Trustworthy correspondence between both data is determined via image registration models. These models are mainly built with points; however, with the aid of automated extraction tools for linear objects, photogrammetrists have been developing transformation models with lines. In this study, we propose to co-register satellite images and light detection and ranging (LIDAR) digital elevation models (DEMs) with straight lines. We developed a line extraction algorithm that was capable of extracting straight lines in both data sets. These lines were then utilised to register the LIDAR DEM with several high-resolution satellite images. We were able to reach one-pixel accuracy through the parallel projection transformation model and the direct linear transformation (DLT) model when geometric constraints among the lines were enforced. Our findings are similar to those reported with points and highlight the leverage of linear features in registering satellite imageries and LIDAR data.

Distinctive Order Based Self-Similarity descriptor for multi-sensor remote sensing image matching

Robust, well-distributed and accurate feature matching in multi-sensor remote sensing image is a difficult task duo to significant geometric and illumination differences. In this paper, a robust and effective image matching approach is presented for multi-sensor remote sensing images. The proposed approach consists of three main steps. In the first step, UR-SIFT (Uniform robust scale invariant feature transform) algorithm is applied for uniform and dense local feature extraction. In the second step, a novel descriptor namely Distinctive Order Based Self Similarity descriptor, DOBSS descriptor, is computed for each extracted feature. Finally, a cross matching process followed by a consistency check in the projective transformation model is performed for feature correspondence and mismatch elimination. The proposed method was successfully applied for matching various multi-sensor satellite images as: ETM+, SPOT 4, SPOT 5, ASTER, IRS, SPOT 6, QuickBird, GeoEye and Worldview sensors, and the results demonstrate its robustness and capability compared to common image matching techniques such as SIFT, PIIFD, GLOH, LIOP and LSS.

A MODIFIED PROJECTIVE TRANSFORMATION SCHEME FOR MOSAICKING MULTI-CAMERA IMAGING SYSTEM EQUIPPED ON A LARGE PAYLOAD FIXED-WING UAS

Abstract. In recent years, Unmanned Aerial System (UAS) has been applied to collect aerial images for mapping, disaster investigation, vegetation monitoring and etc. It is a higher mobility and lower risk platform for human operation, but the low payload and short operation time reduce the image collection efficiency. In this study, one nadir and four oblique consumer grade DSLR cameras composed multiple camera system is equipped on a large payload UAS, which is designed to collect large ground coverage images in an effective way. The field of view (FOV) is increased to 127 degree, which is thus suitable to collect disaster images in mountainous area. The synthetic acquired five images are registered and mosaicked as larger format virtual image for reducing the number of images, post processing time, and for easier stereo plotting. Instead of traditional image matching and applying bundle adjustment method to estimate transformation parameters, the IOPs and ROPs of multiple cameras are calibrated and derived the coefficients of modified projective transformation (MPT) model for image mosaicking. However, there are some uncertainty of indoor calibrated IOPs and ROPs since the different environment conditions as well as the vibration of UAS, which will cause misregistration effect of initial MPT results. Remaining residuals are analysed through tie points matching on overlapping area of initial MPT results, in which displacement and scale difference are introduced and corrected to modify the ROPs and IOPs for finer registration results. In this experiment, the internal accuracy of mosaic image is better than 0.5 pixels after correcting the systematic errors. Comparison between separate cameras and mosaic images through rigorous aerial triangulation are conducted, in which the RMSE of 5 control and 9 check points is less than 5 cm and 10 cm in planimetric and vertical directions, respectively, for all cases. It proves that the designed imaging system and the proposed scheme have potential to create large scale topographic map.

FPGA Implementation of Stereoscopic Image Processing Architecture Base on the Gray-Scale Projection Algorithm

이미지 처리와 컴퓨터 영상과 함께 두 개의 카메라를 갖는 스테레오 시스템은 인간의 시각과 비슷한 깊이 정보 능력 때문에 많은 분야에서 관심을 갖고 연구되고 있다. 깊이 맵 알고리즘은 카메라 시스템의 깊이를 측정하는데 많은 연산량을 필요로 한다. 하지만 병렬 연산 특성으로 인하여 높은 연산속도를 갖는 하드웨어로 구현이 가능하다. 본 논문은 그레이 스케일 영상에 Affine 변환을 적용한 누적데이터를 이용하는 DIS(Digital Image Stabilization) 알고리즘을 위한 높은 성능의 스테레오 영상처리가 가능한 하드웨어 구조를 제안한다. 제안하는 하드웨어는 Verilog HDL을 이용하여 설계하였고 FPGA를 이용하여 합성하였다. 제안하는 하드웨어는 C++을 이용한 소프트웨어 처리 결과와 유사한 결과를 얻었으며(PSNR), SAD(Sum of Absolute Difference)를 기반의 블록 매칭 알고리즘을 이용한 하드웨어 구조 보다 연산 속도 면에서 우수한 결과를 얻었다.

Detection of irregular ground area by single camera on mobile vehicles

For moving robots of outdoor use, it is essential to detect the movable ground area. In this paper, we propose a new method of making distinction between points on the ground surface (on-ground points) and not on the ground surface (off-ground points). These points are detected as feature points by SURF algorithm. In the model building phase, we build the projection transformation model of the ground seen from the camera angle. In the implementation phase, the camera detects feature points by SURF, and then makes correspondence of feature points between the consecutive frames. By inverse projection, the points can be classified into on-ground and off-ground points. This phase is executed online. We propose the principle of this theory and show an example to validate the theory.

Uniform Robust Scale-Invariant Feature Matching for Optical Remote Sensing Images

Extracting well-distributed, reliable, and precisely aligned point pairs for accurate image registration is a difficult task, particularly for multisource remote sensing images that have significant illumination, rotation, and scene differences. The scale-invariant feature transform (SIFT) approach, as a well-known feature-based image matching algorithm, has been successfully applied in a number of automatic registration of remote sensing images. Regardless of its distinctiveness and robustness, the SIFT algorithm suffers from some problems in the quality, quantity, and distribution of extracted features particularly in multisource remote sensing imageries. In this paper, an improved SIFT algorithm is introduced that is fully automated and applicable to various kinds of optical remote sensing images, even with those that are five times the difference in scale. The main key of the proposed approach is a selection strategy of SIFT features in the full distribution of location and scale where the feature qualities are quarantined based on the stability and distinctiveness constraints. Then, the extracted features are introduced to an initial cross-matching process followed by a consistency check in the projective transformation model. Comprehensive evaluation of efficiency, distribution quality, and positional accuracy of the extracted point pairs proves the capabilities of the proposed matching algorithm on a variety of optical remote sensing images.

Least-squares Matching with Advanced Geometric Transformation Models

The least-squares matching algorithm (LSM) for area-based image matching is a well known technique in photogrammetry and computer vision since more than two decades. Differences between two or more images can be modelled by estimating geometric and radiometric transformation functions within the functional model. Commonly the affine transformation is used as geometric transformation. Since this approach is not strict in terms of the projective imaging model, it is worthwhile to investigate alternative transformation models. This paper presents an advanced least-squares matching algorithm that uses the projective transformation model and polynomial transformations to handle geometric distortions between the images. The projective approach is geometrically strict as long as object surface and image sensors are planes. The polynomial approach is supposed to be geometrically strict for plane image sensors and non-plane object surfaces. The possibility of this kind of expansions has been mentioned in several papers but up to now no publicly available investigation is known. First results of the new geometric model have been published by the authors in 2008, showing promising effects on non-planar object patches. * Corresponding author