Homography Matrices Research Articles

Comparative Study: Outlier Elimination through Fundamental and Homography Matrices

Comparative Study: Outlier Elimination through Fundamental and Homography Matrices

The Detection System for a Danger State of a Collision between Construction Equipment and Workers Using Fixed CCTV on Construction Sites.

According to data from the Ministry of Employment and Labor in Korea, a significant portion of fatal accidents on construction sites occur due to collisions between construction workers and equipment, with many of these collisions being attributed to worker negligence. This study introduces a method for accurately localizing construction equipment and workers on-site, delineating areas prone to collisions as 'a danger area of a collision', and defining collision risk states. Utilizing advanced deep learning models which specialize in object detection, video footage obtained from strategically placed closed-circuit television (CCTV) cameras across the construction site is analyzed. The positions of each detected object are determined using transformation or homography matrices representing the conversion relationship between a sufficiently flat reference plane and image coordinates. Additionally, 'a danger area of a collision' is proposed for evaluating equipment collision risk based on the moving equipment's speed, and the validity of this area is verified. Through this, the paper presents a system designed to preemptively identify potential collision risks, particularly when workers are located within the 'danger area of a collision', thereby mitigating accident risks on construction sites.

A target-free video structural motion estimation method based on multi-path optimization

The vibration data are quite important for structural health monitoring (SHM). This paper proposed a novel method, to adaptively estimate video motions of the structure in subpixel accuracy, without attaching any targets. The proposed method includes three steps. In the first step, to remove outliers and simultaneously preserve feature points, the Gaussian range kernel is used along with the Gaussian spatial kernel, and calculated by the polynomial fitting and recursive integrals computing. In the second step, to calculate video pixel motions varied with spatial coordinates in the region of interest (ROI) for testing, the ROI is divided into multiple grid cells. Motions in each grid cell are modeled as local spatially-variant homography matrices, and their spatial consistency are enhanced by a shape-preserving constraint. The third step is to enhance both spatial and temporal correlations of the calculated homography matrices, achieved by the data term and the smoothness term in both space and time domains. The superiority of the proposed method over traditional methods was validated in several case studies for analyzing structural motions. Among the comparisons, the proposed method can produce image denoising, camera motions, structural motions, and structural modal information in subpixel accuracy, and with the best accuracy.

UAV image stitching by estimating orthograph with RGB cameras

In the field of image stitching, cases with large camera optical center movement and large parallax have been the virgin territory of research. The goal of image stitching is to overcome the parallax and stitch a natural image. We look into this problem in the context of ultra-low altitude flight of a UAV. We model the 3D world in this scenario and quickly estimate orthographic projection by pairs of homography matrices. Our stitching method can achieve precise alignment since it takes parallax well into consideration. The stitching results are natural and the extra time consumed is short.

Displacement Measurement Based on UAV Images Using SURF-Enhanced Camera Calibration Algorithm

Displacement is an important parameter in the assessment of the integrity of infrastructure; thus, its measurement is required in a multitude of guidelines or codes for structural health monitoring in most countries. To develop a low-cost and remote displacement measurement technique, a novel method based on an unmanned aerial vehicle (UAV) and digital image correlation (DIC) is presented in this study. First, an auxiliary reference image that meets the requirements is fabricated using the selected first image. Then, the speeded-up robust features (SURF) algorithm is introduced to track the feature points in the fixed areas. The least square algorithm is then employed to resolve the homography matrix of the auxiliary reference image and target images; then, the acquired homography matrices are utilized to calibrate the deviation caused by the UAV wobble. Finally, the integral pixel and sub-pixel matching of the DIC algorithm is employed to calculate the displacement of the target object. The numerical simulation results show that the proposed method has higher calculation accuracy and stability. The outdoor experiment results show that the proposed method has definite practicability.

Calibration of a multiview display with a tilted parallax barrier for reduction of nonlinear extrinsic crosstalk.

Multiview displays, which are capable of displaying different view-images for viewers at different positions, can suffer from undesirable view-image mixing, called crosstalk. In this paper, we propose a method to minimize a class of crosstalk that occurs due to misalignment of flat panel displays and parallax barriers that constitute multiview displays. More specifically, our method is aimed at cases where flat panel displays and parallax barriers are non-parallel. First, we introduce our framework, which associates homography matrices to sets of pixels visible through parallax barriers. Then, we propose an iterative method for finding pixels that are visible for a view-point. By using a stripe test pattern, the homography matrix that corresponds to a set of visible pixels can be found by monitoring the deformation of the stripe test pattern. Last, we experimentally check the validity of the iterative method and demonstrate that the proposed method successfully corrects the geometric distortion of a 3D image.

Accurate Uncalibrated Rectification Using Feature Matching Re-Selection

Rectification is a core programming process that involves multiple views. In this study, we focus on uncalibrated cases that neglect intrinsic and extrinsic camera information. Existing uncalibrated rectification methods use feature-matching techniques to form relationships between different views and use those features to estimate optimized homography matrices. However, outliers are inevitable in feature matching. Using these outliers in a rectification process produces vertical disparity errors and unwanted geometric distortion. To tackle the problem, we propose a novel method that can learn from the rectification results, re-select the matching pairs, and find superior solutions. The proposed method introduces a novel workflow for uncalibrated rectification that incorporates three cores: field of view (FoV) neutralization, rectification, and feature matching re-selection (FMR). While the FoV neutralization module handles FoV differences among views, a combination of the rectification module and FMR module results in the optimal homography matrices. The rectification module takes neutral correspondences and estimates the optimized rectified matrices. Applying the results from the rectification module, the FMR module optimizes the correspondences and return them to the rectification module. For the rectification module, we apply adaptive geometric constraints and our updated optimization strategy to secure satisfied vertical disparity errors while maintaining low distortion levels. Multidisciplinary experiments are performed to analyze the capabilities of the proposed method. We combine existing datasets with extra samples, including various outdoor environments, to gauge the performance better. Compared with existing methods, the proposed method produces fewer rectification errors and keeps the rectified images under low distortion rates.

Alternative Inverse Perspective Mapping Homography Matrix Computation for ADAS Systems Using Camera Intrinsic and Extrinsic Calibration Parameters

This paper presents a simple novel IPM matrix computation method, which does not require any additional solutions to calculate homography matrices. This method only requires determining the distance from camera to projecting surface. It’s simplicity allows execution even on very small energy-efficient embedded device processors without use of complex libraries such as OpenCV, that cannot be deployed there. Furthermore, it has strictly analytical solution, which allows avoidance of false optimums that are probable in heuristic methods for optimization. We provide formulas, that are easy transferable to any device and can be calculated using standard math libraries.

MultiLevel Feature Pooling Network for Uncalibrated Stereo Rectification in Autonomous Vehicles

Stereo vision systems in autonomous vehicles can come with camera sensors with different models and lenses to detect objects at long and short ranges. The intrinsic/extrinsic parameters of the camera sensors, such as the focal length and distortion parameters used for calibration, are not provided. In this article, we present a novel rectification method for uncalibrated stereo cameras based on their corresponding feature points. To obtain reliable key/feature points and descriptors from stereo images, we propose a deep neural network, called a multilevel feature pooling network. The model processes images individually to learn both local and semiglobal features, from which the location and descriptor of each pixel in the image are estimated. We also introduce a method to balance the zoom effect caused by the different focal lengths of the camera lenses and rectify the images by estimating homography matrices for stereo images based on their refined corresponding feature points.

The equivalence of two definitions of compatible homography matrices

In many computer vision applications, one acquires images of planar surfaces from two different vantage points. One can use a projective transformation to map pixel coordinates associated with a particular planar surface from one image to another. The transformation, called a homography, can be represented by a unique, to within a scale factor, 3 × 3 matrix. One requires a different homography matrix, scale differences apart, for each planar surface whose two images one wants to relate. However, a collection of homography matrices forms a valid set only if the matrices satisfy consistency constraints implied by the rigidity of the motion and the scene. We explore what it means for a set of homography matrices to be compatible and show that two seemingly disparate definitions are in fact equivalent. Our insight lays the theoretical foundations upon which the derivation of various sets of homography consistency constraints can proceed.

Image Registration Algorithm Based on Convolutional Neural Network and Local Homography Transformation

In order to overcome the poor robustness of traditional image registration algorithms in illuminating and solving the problem of low accuracy of a learning-based image homography matrix estimation algorithm, an image registration algorithm based on convolutional neural network (CNN) and local homography transformation is proposed. Firstly, to ensure the diversity of samples, a sample and label generation method based on moving direct linear transformation (MDLT) is designed. The generated samples and labels can effectively reflect the local characteristics of images and are suitable for training the CNN model with which multiple pairs of local matching points between two images to be registered can be calculated. Then, the local homography matrices between the two images are estimated by using the MDLT and finally the image registration can be realized. The experimental results show that the proposed image registration algorithm achieves higher accuracy than other commonly used algorithms such as the SIFT, ORB, ECC, and APAP algorithms, as well as another two learning-based algorithms, and it has good robustness for different types of illumination imaging.

A Robust and Rapid Camera Calibration Method by One Captured Image

A robust and rapid camera calibration method is essential in vision measurement. This paper proposes a camera calibration method using a 2.5-D coding target that requires just a single captured image and is capable of identifying the intrinsic and extrinsic parameters of a camera system simultaneously. The calibration target consists of four fixed planes to which coded patterns are attached. The fixed planes of the target prevent the ill condition that arises in Zhang’s method, and they provide four different homography matrices at the same time. The extrinsic parameters of the cameras are calculated using the geometric constraints of the feature circles on each plane. The coded pattern allows the four planes to be distinguished automatically. The proposed method is first verified by comparison experiments with three classical calibration methods. The results confirm that the proposed method has the greatest accuracy and smallest variance of errors. The influences on the calibration results of geometric parameters such as the dihedral angle and the area ratio of the feature circles are then investigated by systematic experiments. This proposed camera calibration method using a single captured image represents a breakthrough in a camera calibration technology and has a broad potential for application to machine vision and vision measurement.

Self-Supervised deep homography estimation with invertibility constraints

Remarkable performance of the homography estimation has been achieved by the deep CNN based approaches. These homography estimation methods, more often than not, are supervised methods and rely too much on the ground truth annotations as they aim to learn the mapping between image pairs and homography. On the other hand, the inherent invertibility of homography is helpful to avoid over-fitting and improve the performance, which however is ignored by previous homography estimation methods. In this paper, we propose a novel homography estimation approach, named “Self-Supervised Regression Network(SSR-Net)”, which relaxes the need of ground truth annotations and takes advantage of invertibility constraints. We utilize spatial pyramid pooling modules to improve the quality of extracted features in each image by exploiting context information. To employ the invertibility constraints, we adopt the matrix representation of the homography rather than the commonly used 4-point parameterization in other methods. Our proposed SSR-Net produce homography matrices and synthetic images in a cycled way. The network are trained in a self-supervised way by minimizing the combination of photometric loss and invertibility loss. Experiments on the synthetic dataset generated from MSCOCO dataset show that our proposed method outperforms several state-of-the-art approaches.

Real-Time Surround-View System for Mobile Robotic System

The paper is devoted to the problem of increasing the adequacy of perception of the environment by the mobile robot operator using remote control. A variant of a real-time surround-view system for mobile robot based on the multiple cameras with fisheye lenses and overlapping fields of view is proposed. A prototype of the circular view system has been developed. In this paper, the key features of the architecture and software of the surround-view system are considered. The algorithms for determining the internal parameters of cameras and distortion correction are researched. New models for describing the distortions of wide-angle and fisheye lens are used. Algorithms for finding the external parameters of cameras, as well as homography matrices using invariant descriptors, are implemented. Static homography matrices are used during stitching images into panorama. Various image-stitching techniques based on the overlapped images region blending are investigated and implemented. The methods of projective geometry and augmented reality were studied to obtain a perspective third-person view. A surface variant for projecting panorama of a surround view is proposed. For the implementation of the software selected cross-platform game engine "Unity". Directions for further research are identified.

SC-RANSAC: Spatial consistency on RANSAC

The goal of robust parameter estimation is developing a model which can properly fit to data. Parameter estimation of a geometric model, in presence of noise and error, is an important step in many image processing and computer vision applications. As the random sample consensus (RANSAC) algorithm is one of the most well-known algorithms in this field, there have been several attempts to improve its performance. In this paper, after giving a short review on existing methods, a robust and efficient method that detects the gross outliers to increase the inlier to outlier ratio in a reduced set of corresponding image points is proposed. It has a new hypothesis and verification scheme which utilizes spatial relations between extracted corresponding points in two images. It can also be considered as a preprocessing step for RANSAC to improve the accuracy as well as the runtime of RANSAC in estimating the parameters of a geometric model (such as fundamental and homography matrices). Obviously, like almost all previous works for enhancing RANSAC's runtime, the proposed method does not use heavy and compilicated processes. Performance analysis is performed on a variety of standard challenging datasets for estimating the homography and fundamental matrix (as an applicable case used in the literature, especially in the state-of-the-art methods). The performance is also compared quantitatively to RANSAC, PROSAC, and SCRAMSAC robust estimators to demonstrate its superiority. Experimental results show that the proposed method removes about 50% of outliers in most cases and hence extremely reduces the required runtime of RANSAC, while improving its accuracy.

Global homography calibration for monocular vision-based pose measurement of mobile robots

Compared with the size of a calibration board, the inspected area within the camera field of view is much bigger in many applications such as visual surveillance systems. In this paper, we present a global image-to-ground homography calibration method to obtain the mapping between the image and the planar scene lying in the whole camera field of view. The calibration approach is proposed by fusing multiple local homography matrices, with each of them only reflecting the relationship between a small calibration board and its corresponding small image part. By further calibrating a height-related homography, visual measurement can be extended for arbitrary planar surfaces with known heights. The proposed method presents such merits as no requirement for camera internal parameters, high calibration accuracy, and ease of implementation. Experimental results show that (1) the proposed global homography method achieves more accurate measurement results than the local homography method; (2) the height-related homography is calibrated with high accuracy; (3) the proposed approach can be used for mobile robot localization with accuracy close to the performance limit of a monocular camera.

Homographic p-norms: Metrics of homographic image transformation

Images often need to be aligned in a single coordinate system, and homography is one of the most efficient geometric models to align images. This paper presents homographic p-norms, scalar metrics of homographic image transformation, and to the best of our knowledge these are the most rigorous definition of scalar metrics quantifying homographic image transformations. We first define a metric between two homography matrices, and show it satisfies metric properties. Then we propose metrics of a single homography matrix for a general planar region, and ones for a usual rectangle image. For use of the proposed metrics, we provide useful homographic 2-norm expressions derived from the definition of the metrics, and compare the approximation errors of the metrics with respect to the exact metric. As a result, the discrete version of the metric obtained by pixel-wise computation is greatly close to the exact metric. The proposed metrics can be applied to evaluation of transformation magnitude, image closeness estimation, evaluation of camera pose difference, selection of image pair in stereo vision, panoramic image mosaic, and deblurring. Experimental results show the efficacy of the proposed metrics.

Enforcing consistency constraints in uncalibrated multiple homography estimation using latent variables

An approach is presented for estimating a set of interdependent homography matrices linked together by latent variables. The approach allows enforcement of all underlying consistency constraints while accounting for the arbitrariness of the scale of each individual matrix. The input data is assumed to be in the form of a set of homography matrices individually estimated from image data with no regard to the consistency constraints, appended by a set of error covariances, each characterising the uncertainty of a corresponding homography matrix. A statistically motivated cost function is introduced for upgrading, via optimisation, the input data to a set of homography matrices satisfying the constraints. The function is invariant to a change of any of the individual scales of the input matrices. The proposed approach is applied to the particular problem of estimating a set of homography matrices induced by multiple planes in the 3D scene between two views. An optimisation algorithm for this problem is developed that operates on natural underlying latent variables, with the use of those variables ensuring that all consistency constraints are satisfied. Experimental results indicate that the algorithm outperforms previous schemes proposed for the same task and is fully comparable in accuracy with the `gold standard' bundle adjustment technique, rendering the whole approach both of practical and theoretical interest. With a view to practical application, it is shown that the proposed algorithm can be incorporated into the familiar random sampling and consensus technique, so that the resulting modified scheme is capable of robust fitting of fully consistent homographies to data with outliers.

Is Hamming distance only way for matching binary image feature descriptors?

Brute force matching of binary image feature descriptors is conventionally performed using the Hamming distance. This paper assesses the use of alternative metrics in order to see whether they can produce feature correspondences that yield more accurate homography matrices. Two statistical tests, namely ANOVA (Analysis of Variance) and McNemar's test were employed for evaluation. Results show that Jackard-Needham and Dice metrics can display better performance for some descriptors. Yet, these performance differences were not found to be statistically significant.

Sampson distance based joint estimation of multiple homographies with uncalibrated cameras

Two images of a scene consisting of multiple flat surfaces are related by a collection of homography matrices. Practitioners typically estimate these homographies separately thereby violating inherent inter-homography constraints that arise naturally out of the rigid geometry of the scene. We demonstrate that through a suitable choice of parametrisation multiple homographies can be jointly estimated in a manner so as to satisfy all inter-homography constraints. Unlike the cost functions used previously for solving this problem, our cost function does not correspond to fitting one set of homography matrices to another set of homography matrices. Instead, we utilise the Sampson distance for homography matrix estimation and operate directly on image data points. By using the Sampson distance and working directly on data points, we expedite the application of a vast amount of knowledge that already exists for Sampson-distance-based single homography or fundamental matrix estimation. The estimation framework reported in this paper establishes a new baseline for joint multiple homography estimation and at the same time raises intriguing new research questions. The work may be of interest to a broad range of researchers who require the estimation of homography matrices with uncalibrated cameras as part of their solution.