Perspective Warping Research Articles

S$^{2}$ Loop: A Lightweight Spectral-Spatio Loop Closure Detector for Resource-Constrained Platforms

Visual loop closure detection is an essential backend task for long-term vSLAM applications. However, prior works cannot simultaneously meet the requirements of high recall and low computing and memory overhead, which prohibits their applicability to resource-constrained platforms. In this work we propose S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop, a novel spectral-spatio loop detector that explores an efficient loop searching method based on spectral image analysis and direct image alignment. S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop digests an image by its Fourier spectrum which provides a compact image hash value for fast candidate selection. The spectra also enable a convenient method of image alignment based on Fourier phase correction. The aligned images are further matched by an optical-flow process at the pixel granularity, which precisely estimates the frame dissimilarity against the turbulence caused by the perspective warping of random 3D scenes. According to our benchmark experiments on various datasets, S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop shows two orders of magnitude of less computing and memory overhead with comparable precision and recall over the prior DBoW method on a desktop PC. It is also much faster than the prior Fern method that has low computing complexity, and exhibits a much higher recall. While querying loops in 1000 runtime keyframes, S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop deployed onto a low-end STM32F4 MCU achieves 64fps.

Study and Performance Evaluation Binary Robust Invariant Scalable Keypoints (BRISK) for Underwater Image Stitching

In this paper, performance evaluation of Binary Robust Invariant Scalable Key points (BRISK) for underwater image stitching is presented. Underwater image stitching is quite challenging because usually images produced by underwater cameras have few features and noise such as motion blur and underexposure. In this paper, we proposed a robust algorithm for the underwater image stitching which consists of several stages: taking an image frame, getting a key point using BRISK, feature matching using Random Sample Consensus (RANSAC), homography estimation, and perspective warping. The experimental result shows that the proposed algorithm can be implemented for underwater image and achieve better matching results even the less detected keypoints.

Visual Pose Estimation of Rescue Unmanned Surface Vehicle From Unmanned Aerial System.

This article addresses the problem of how to visually estimate the pose of a rescue unmanned surface vehicle (USV) using an unmanned aerial system (UAS) in marine mass casualty events. A UAS visually navigating the USV can help solve problems with teleoperation and manpower requirements. The solution has to estimate full pose (both position and orientation) and has to work in an outdoor environment from oblique view angle (up to 85° from nadir) at large distances (180 m) in real-time (5 Hz) and assume both moving UAS (up to 22 m s−1) and moving object (up to 10 m s−1). None of the 58 reviewed studies satisfied all those requirements. This article presents two algorithms for visual position estimation using the object's hue (thresholding and histogramming) and four techniques for visual orientation estimation using the object's shape while satisfying those requirements. Four physical experiments were performed to validate the feasibility and compare the thresholding and histogramming algorithms. The histogramming had statistically significantly lower position estimation error compared to thresholding for all four trials (p-value ranged from ~0 to 8.23263 × 10−29), but it only had statistically significantly lower orientation estimation error for two of the trials (p-values 3.51852 × 10−39 and 1.32762 × 10−46). The mean position estimation error ranged from 7 to 43 px while the mean orientation estimation error ranged from 0.134 to 0.480 rad. The histogramming algorithm demonstrated feasibility for variations in environmental conditions and physical settings while requiring fewer parameters than thresholding. However, three problems were identified. The orientation estimation error was quite large for both algorithms, both algorithms required manual tuning before each trial, and both algorithms were not robust enough to recover from significant changes in illumination conditions. To reduce the orientation estimation error, inverse perspective warping will be necessary to reduce the perspective distortion. To eliminate the necessity for tuning and increase the robustness, a machine learning approach to pose estimation might ultimately be a better solution.

Multiple-image super-resolution on mobile devices: an image warping approach

This paper discusses a super-resolution (SR) system implemented on a mobile device. We utilized an Android device’s camera to take successive shots and applied a classical multiple-image super-resolution (SR) technique that utilized a set of low-resolution (LR) images. Images taken from the mobile device are subjected to our proposed filtering scheme wherein images that have noticeable presence of blur are discarded to avoid outliers from affecting the produced high-resolution (HR) image. The remaining subset of images are subjected to non-local means denoising, then feature-matched against the first reference LR image. Successive images are then aligned with respect to the first image via affine and perspective warping transformations. The LR images are then upsampled using bicubic interpolation. An L2-norm minimization approach, which is essentially taking the pixel-wise mean of the aligned images, is performed to produce the final HR image.Our study shows that our proposed method performs better than the bicubic interpolation, which makes its implementation in a mobile device quite feasible. We have also proven in our experiments that there are substantial differences from images captured using burst mode that can be utilized by an SR algorithm to create an HR image.

Feature Coverage Indexes for Dual Homography Estimation in Constructing Panorama Image

Enlarged images can be obtained by various methods. Stitching is one of the efficient methods. It can produce panoramic images by stitching adjacent images which contain overlapping regions even though they are obtained through separate image sensors. Images that contain multiple different planes are hard to be stitched together because each plane has a different homography matrix for perspective warping. For this, a dual homography was proposed. However its performance varies depending on feature detectors which are used to find matching feature points between images. In this paper, we propose three feature coverage indexes which evaluate the stitching performance of feature detectors and predict the outcomes of the stitching. We evaluate four well-known feature detectors by the proposed indexes by applying them to the image stitching process and show that the prediction by the index values coincides with the stitching results.

Non-uniform image deblurring using an optical computing system

Removing non-uniform blur caused by camera shaking is troublesome because of its high computational cost. We analyze the efficiency bottlenecks of a non-uniform deblurring algorithm and propose an efficient optical computation deblurring framework that implements the time-consuming and repeatedly required modules, i.e., non-uniform convolution and perspective warping, by light transportation. Specifically, the non-uniform convolution and perspective warping are optically computed by a hybrid system that is composed of an off-the-shelf projector and a camera mounted on a programmable motion platform. Benefitting from the high speed and parallelism of optical computation, our system has the potential to accelerate existing non-uniform motion deblurring algorithms significantly. To validate the effectiveness of the proposed approach, we also develop a prototype system that is incorporated into an iterative deblurring framework to effectively address the image blur of planar scenes that is caused by 3D camera rotation around the x-, y- and z-axes. The results show that the proposed approach has a high efficiency while obtaining a promising accuracy and has a high generalizability to more complex camera motions.

Camera space shadow maps for large virtual environments

This paper presents a new single-pass shadow mapping technique that achieves better quality than the approaches based on perspective warping, such as perspective, light-space, and trapezoidal shadow maps. The proposed technique is appropriate for real-time rendering of large virtual environments that include dynamic objects. By performing operations in camera space, this solution successfully handles the general and the dueling frustum cases and produces high-quality shadows even for extremely large scenes. This paper also presents a fast nonlinear projection technique for shadow map stretching that enables complete utilization of the shadow map by eliminating wastage. The application of stretching results in a significant reduction in unwanted perspective aliasing, commonly found in all shadow mapping techniques. Technique is compared with other shadow mapping techniques, and the benefits of the proposed method are presented. The proposed shadow mapping technique is simple and flexible enough to handle most of the special scenarios. An API for a generic shadow mapping solution is presented. This API simplifies the generation of fast and high-quality shadows.