Homography Matrix Research Articles

Bilevel progressive homography estimation via correlative region-focused transformer

We propose a novel correlative region-focused transformer for accurate homography estimation by a bilevel progressive architecture. Existing methods typically consider the entire image features to establish correlations for a pair of input images, but irrelevant regions often introduce mismatches and outliers. In contrast, our network effectively mitigates the negative impact of irrelevant regions through a bilevel progressive homography estimation architecture. Specifically, in the outer iteration, we progressively estimate the homography matrix at different feature scales; in the inner iteration, we dynamically extract correlative regions and progressively focus on their corresponding features from both inputs. Moreover, we develop a quadtree attention mechanism based on the transformer to explicitly capture the correspondence between the input images, localizing and cropping the correlative regions for the next iteration. This progressive training strategy enhances feature consistency and enables precise alignment with comparable inference rates. Extensive experiments on qualitative and quantitative comparisons show that the proposed method exhibits competitive alignment results while reducing the mean average corner error (MACE) on the MS-COCO dataset compared to previous methods, without increasing additional parameter cost.

Atomic force microscopy wide-field scanning imaging using homography matrix optimization

During the offline combination of multiple atomic force microscopy (AFM) images, changes in probe displacement can be affected by dynamic noise, leading to dislocation and tearing in the stitching. To overcome this, we designed a homography matrix optimization method to describe the relative positional relationship between images by constructing the original image matrix and applying singular value decomposition to denoise the homography matrix. Additionally, we implemented a scale-invariant feature transform (SIFT) to extract feature points. To verify the effectiveness of this method, the SIFT + RANSAC (R), SIFT + affine transformation (AT), and oriented FAST and rotated BRIEF (ORB) algorithms were compared with the proposed algorithm. The experimental results demonstrate that the proposed method precisely computes the transformation matrix, thereby guaranteeing the geometric consistency of mosaic imaging. The proposed method preserves the intricate details of the original image and enhances the stitching quality of wide-field images.

Strain concentration applied on crack inspection based on marker-based and homography matrix algorithm

ABSTRACT The detection and identification of ship damages have become a significant area of research. This study proposes an image-based structural crack monitoring method that marks the crack locations in the form of strain concentrations by analyzing the overall strain of the structure. Most existing studies focus on using fixed cameras for structural crack monitoring. However, during the actual operation of structures, continuous monitoring around these structures is challenging. To overcome this limitation, this paper implements the transformation from non-fixed camera images to fixed images using a marker-based approach and homography matrix algorithm. In laboratory-scale structural experiments, the algorithm is capable of locating structural crack positions by observing the behavior of strain concentration. Additionally, this study explores the application of markers in structures and verifies the capability of marker attachment methods in crack inspection.

A three-dimensional trajectory measurement system for quasi-one-dimensional high-speed moving objects

Abstract The three-dimensional (3D) trajectory measurement of a quasi–one-dimensional (quasi-1D) high-speed moving object often cannot achieve the optimizations of both the measurement range and the object’s resolution at the same time. In this paper, a 3D trajectory measurement system for quasi-1D high-speed moving objects is proposed. The system was composed of a single camera and a galvo mirror. The posture of the virtual camera is changed by adjusting the angle of the galvo mirror, thereby achieving the tracking and imaging of a quasi-1D high-speed moving object. For a background with complex textures, a measurement method based on the reconstructed homography matrix is proposed. The extrinsic matrix of the virtual camera is estimated based on the feature points on the background, and the equation of the object trajectory’s plane is estimated based on the feature points on the object. Then, by combining the extrinsic matrix and the equation of the plane, a new homography matrix is reconstructed, realizing the measurement of the 3D coordinates of the object. For a background with weak textures, a method based on the deflection angle of the galvo mirror is proposed for the calibration of the extrinsic matrix of the virtual camera. This method can calculate the extrinsic matrix of the virtual camera based on the distance from the camera to the center point of the galvo mirror and the deflection angle of the galvo mirror. The experimental results show that the proposed system is able to increase the field of view by 430%. For complex-texture backgrounds, the relative error of the system is 0.79%, and for weak-texture backgrounds, the relative error is 4.65%.

Estimation of video sequence homography based on a first-order estimation method

As it is a pre-processing task, estimation of video-sequence-based homography requires low computational costs and fast evaluation. However, current algorithms for video sequence tasks are commonly based on image-pair homography and do not consider the inner properties of the video sequences. Therefore they take unnecessary computational resources. In this work, we propose a novel algorithm with a first-order estimation method to fill the gap between estimation of image-pairs and video sequence homography. By considering the continuous movement of the camera, the proposed algorithm adopts a first-order estimation to accelerate the estimation process while maintaining its robustness. Instead of extracting many image features from every frame, we demonstrate that estimating a homography matrix with pixel-based texture patterns is effective and sufficient for video sequences. Experiments show that homography estimation with simple one-dimensional texture vectors, as used in our algorithm, can surpass state-of-the-art feature-based algorithms and deep-learning-based methods. This first-order estimation method was more than 40 times faster and real-time estimation used only the CPU.

Image Registration Algorithm for Stamping Process Monitoring Based on Improved Unsupervised Homography Estimation

Homography estimation is a crucial task in aligning template images with target images in stamping monitoring systems. To enhance the robustness and accuracy of homography estimation against random vibrations and lighting variations in stamping environments, this paper proposes an improved unsupervised homography estimation model. The model takes as input the channel-stacked template and target images and outputs the estimated homography matrix. First, a specialized deformable convolution module and Group Normalization (GN) layer are introduced to expand the receptive field and enhance the model’s ability to learn rotational invariance when processing large, high-resolution images. Next, a multi-scale, multi-stage unsupervised homography estimation network structure is constructed to improve the accuracy of homography estimation by refining the estimation through multiple stages, thereby enhancing the model’s resistance to scale variations. Finally, stamping monitoring image data is incorporated into the training through data fusion, with data augmentation techniques applied to randomly introduce various levels of perturbation, brightness, contrast, and filtering to improve the model’s robustness to complex changes in the stamping environment, making it more suitable for monitoring applications in this specific industrial context. Compared to traditional methods, this approach provides better homography matrix estimation when handling images with low texture, significant lighting variations, or large viewpoint changes. Compared to other deep-learning-based homography estimation methods, it reduces estimation errors and performs better on stamping monitoring images, while also offering broader applicability.

Visual guidance technology of flying cars based on multilevel markers and depth

Split-type flying car will play an important role in the future transportation. This paper adopts a guidance method that couples visual information and depth information, and improves the docking accuracy through the mutual cooperation of the drone and the vehicle. Firstly, a multilevel docking marker is designed to achieve adaptive target matching within different distances during the docking process. The marker has strong robustness and can adapt to complex scenes such as occlusion, strong light, and large angle tilting, providing the redundant corner points required for machine vision detection pose information accurately. Secondly, a three-dimensional pose estimation algorithm is proposed, which can introduce depth information to correct the homography matrix. The algorithm combines the advantages of strong robustness to multilevel marker detection and high accuracy of depth information, and can output millimeter-level precision pose information in different environments, different inclination angles, and different occlusions. Finally, a flying car model experiment was carried out, and the results showed that the guidance technology can obtain millimeter-level precise pose information during the entire process of long distance-near distance-completion of docking, thus realizing precise docking.

Research on PCB tiny displacement estimation method for ORB feature-guided FREAK descriptor enhancement

Abstract Given the hurdles posed by existing techniques for identifying microscopic offsets in miniature printed circuit boards (PCBs), including their low accuracy and high iteration count, this study presents an innovative approach that measures micrometer offsets by utilizing the ORB feature-guided FREAK descriptor enhancement. It improves matching outcomes with a brute-force matcher and RANSAC via ORB feature extraction to guide FREAK descriptors for better feature uniqueness. A PCB small offset detection algorithm is developed with a homography matrix and a theoretical offset model. A novel distance threshold performance evaluation method for this suggested matching algorithm that objectively evaluates matching results is also proposed in this article. The proposed matching method outperforms alternative algorithms in precision and recall, according to its experiments. This research conducts simulation detection of PCB calibration bare boards and instance verification of real PCB boards by constructing a PCB tiny offset detection experimental system. The results show that micron-level detection precision can be achieved without iteration.

CrossHomo: Cross-Modality and Cross-Resolution Homography Estimation.

Multi-modal homography estimation aims to spatially align the images from different modalities, which is quite challenging since both the image content and resolution are variant across modalities. In this paper, we introduce a novel framework namely CrossHomo to tackle this challenging problem. Our framework is motivated by two interesting findings which demonstrate the mutual benefits between image super-resolution and homography estimation. Based on these findings, we design a flexible multi-level homography estimation network to align the multi-modal images in a coarse-to-fine manner. Each level is composed of a multi-modal image super-resolution (MISR) module to shrink the resolution gap between different modalities, followed by a multi-modal homography estimation (MHE) module to predict the homography matrix. To the best of our knowledge, CrossHomo is the first attempt to address the homography estimation problem with both modality and resolution discrepancy. Extensive experimental results show that our CrossHomo can achieve high registration accuracy on various multi-modal datasets with different resolution gaps. In addition, the network has high efficiency in terms of both model complexity and running speed.

Robust pedestrian multi-object tracking in the intelligent bus environment

Abstract Pedestrian multi-object tracking algorithms aim to maintain identity information of pedestrians by comparing the similarity between trajectories and detections, predicting pedestrian motion trajectories. However, within the context of intelligent bus, challenges arise due to factors such as passenger growth and vehicle vibrations, rendering existing pedestrian multi-object tracking algorithms less accurate. Therefore, this paper proposes an intelligent bus terminal robust pedestrian multi-object tracking algorithm named pure motion (PM), which can consistently and stably track pedestrians. The proposed algorithm employs several key strategies. Firstly, it optimizes trajectory prediction by adapting the aspect ratio of the prediction box based on pedestrian movement, automatically adjusting its shape, and selecting velocity weight coefficients according to different tracking targets. Secondly, it decomposes the homography matrix to acquire motion components and correct predicted results under motion conditions. Subsequently, the algorithm leverages the similarity between detection results and trajectories to retain high-confidence detections, eliminating low-confidence ones associated with background, thereby reducing false negatives and enhancing trajectory coherence. Futhermore, the introduction of detection confidence into trajectory updates to enhances the precision of measurement noise. The proposed algorithm underwent testing in intelligent bus driving scenarios, including turns, waiting for traffic lights, emergency braking, and approaching bus stops. The tracking accuracy on the MOT17-13-val dataset reaches 81.8. The results demonstrate that PM significantly improves the robustness of pedestrian multi-object tracking algorithms in the environment of intelligent bus.

Identifying Goalkeeper Movement Timing from Single-Camera Broadcast Footage through Pose Estimation: A Pilot Study

This study explores how open-source pose estimation can be utilized to identify goalkeeper dive initiation during soccer penalty kicks. The purpose of this study is to provide an accessible, low-cost heuristic methodology for identifying goalkeeper dive initiation. This study uses single-camera broadcast footage (1080 p resolution, 50 frames per second) of all 41 penalty shootout kicks attempted during the 2022 FIFA Men’s World Cup. We isolated each penalty kick and recorded the frames of goalkeeper dive initiation and flight. We then identified goalposts to create a homography matrix to account for camera movement and identified the goalkeeper’s skeletal keypoints through pose estimation. From these keypoints, we derived frontal plane kinematics for the torso and legs. We identified local extrema for each kinematic variable and isolated the last observed extrema prior to goalkeeper flight for each variable. Using OLS regression, we found that the last local extremum of the goalkeeper centroid’s y-value was the strongest predictor of labeled commitment to the dive side, with an R2 of 0.998 and a p-value of 0.00. The results of this research are preliminary but demonstrate the promise of pose estimation in identifying sport-specific action timing during live game play using a single camera.

A new visual sensing system for motion state estimation of lateral localization of intelligent vehicles

Accurate lateral localization is the basis of intelligent vehicle decision making and control and one of the core problems of autonomous driving. In the process of vehicle driving, the vehicle body vibration, road bumps, and slope change will affect the motion parameters of the camera, which influences the localization accuracy. Therefore, we investigate a vehicle visual localization method based on the motion state estimation of vehicle camera by Gaussian Bayes sphere. This method uses Gaussian spherical crown sampling and maximum likelihood search to estimate the motion state of the vehicle camera in real time. Then, the real-time homography matrix between the sampling ground plane and the pixel plane is calculated according to the results to convert the coordinate points of the lane lines to the sampling ground plane for lateral localization of vehicles. Last, the localization results are restored to the real scale by the lane line width. The road experimental results show that the average error of the proposed method for lateral localization of vehicles is 5.7 cm under different road conditions on campus roads, and that under different road conditions on urban roads is 5.9 cm. At the same time, the localization accuracy of the proposed method is tested on the slope and under different weather conditions, and good results are obtained.

Vortex-induced vibration dynamic characteristics monitoring with AI-based target object detection for long-span bridges

This research proposes an innovative framework for monitoring bridge Vortex-induced Vibration (VIV) dynamic characteristics utilizing AI-based machine-learning target object detection and tracking techniques with keypoint detection. Two camera calibration methods are implemented for cases with and without intrinsic and extrinsic camera setup information based on homography matrix conversion and distance-based conversion. The framework is verified on a video recording of a real-bridge VIV event and a simulation animation of bridge VIV with a peak/trough-based statistical method for calculating VIV frequency and amplitude. Accurate detection is achieved in both cases with short video durations. The framework demonstrates great potential for real-time bridge VIV monitoring, requiring minimal camera calibration and a straightforward device setup. It offers reliable and accurate results while remaining cost-effective.

Image-Based Bolt-Loosening Detection Using a Checkerboard Perspective Correction Method.

In this paper, a new image-correction method for flange joint bolts is proposed. A checkerboard is arranged on the side of a flange node bolt, and the homography matrix can be estimated using more than four feature points, which include the checkerboard corner points. Then, the perspective distortion of the captured image and the deviation of the camera position angle are corrected using the estimated homography matrix. Due to the use of more feature points, the stability of homography matrix identification is effectively improved. Simultaneously, the influence of the number of feature points, camera lens distance, and light intensities are analyzed. Finally, based on a bolt image taken using an iPhone 12, the prototype structure of the flange joint in the laboratory is verified. The results show that the proposed method can effectively correct image distortion and camera position angle deviation. The use of more than four correction points not only effectively improves the stability of bolt image correction but also improves the stability and accuracy of bolt-loosening detection. The analysis of influencing factors shows that the proposed method is still effective when the number of checkerboard correction points is reduced to nine, and the average error of the bolt-loosening detection result is less than 1.5 degrees. Moreover, the recommended camera shooting distance range is 20 cm to 60 cm, and the method exhibits low sensitivity to light intensity.

A geometric approach for homography-based visual servo control of underactuated UAVs

This paper proposes a new geometric control method for homography-based visual servo control of Underactuated UAVs. In order to solve the application difficulties of geometric control in HBVS and explore a visual servo control technology that can be applied to aerial detection operations, this paper integrates the geometric control into the visual servoing framework and design a new homography-based geometric visual servoing controller. The outer loop is used as feedback information using the virtual homography matrix between the two images. The inner loop controls the orientation of the UAVs through geometric control. The stability of the proposed controller is proved based on Lyapunov’s theory. The proposed method has better transient performance and dynamic performance than the conventional visual servo method. The excellent performance of the controller has been proven by a large number of experiments. In addition, the application of the controller on an unmanned aerial manipulator is demonstrated.

IHUVS: Infinite Homography-Based Uncalibrated Methodology for Robotic Visual Servoing

The construction of task functions in robotic manipulation is of paramount importance for uncalibrated visual servoing. The existing methods generally use image information as control variables and estimate the image Jacobian matrix online, thus possessing issues relating to convergence, and image Jacobian matrix singularities. Therefore, this work proposes a novel methodology dubbed infinite homography-based uncalibrated visual servoing (IHUVS), in which the visual control of the robot end-effector pose is decomposed into its rotational and translational components. The corresponding rotational controller designs the visual servoing task function using the relationship between the infinite homography matrix and rotation matrix, and employs the Kronecker product to derive linear equations for rotational control, as well as to conduct the associated task error analysis. Meanwhile, the translational controller utilizes Kalman filtering for online estimation of the Jacobian matrix that is required by the proportional control scheme. The robot end-effector motion in Cartesian space is generated via the IHUVS method, without knowing the camera's intrinsic parameters and the robot hand-eye relationship. A simulation analysis is carried out to assess the algorithm's numerical performance, while robotic visual servoing experiments are also conducted to verify the accuracy and efficacy of the proposed IHUVS method.

High-precision binocular camera calibration method based on a 3D calibration object.

A high-precision binocular camera calibration method is proposed to address the issues of poor calibration accuracy and large calibration errors in current practical applications. This method uses a triangular stereo sphere as the calibration object and employs steps, such as ellipse fitting, Cholesky decomposition, homography matrix solution, and nonlinear optimization, to compute the intrinsic and extrinsic parameters, distortion parameters, and relative pose of the binocular camera. Moreover, this method simplifies the correspondences between primitives, enabling simultaneous calibration of multiple viewpoint cameras. This method is also suitable for both binocular cameras consisting of two different structured monocular cameras and those composed of two image sensors within the same structure. Experimental results showed that this method outperforms traditional algorithms in terms of binocular camera calibration accuracy, calibration errors between left and right cameras, and robustness, resulting in a significant improvement in overall algorithm performance.

PARSAC: Accelerating Robust Multi-Model Fitting with Parallel Sample Consensus

We present a real-time method for robust estimation of multiple instances of geometric models from noisy data. Geometric models such as vanishing points, planar homographies or fundamental matrices are essential for 3D scene analysis. Previous approaches discover distinct model instances in an iterative manner, thus limiting their potential for speedup via parallel computation. In contrast, our method detects all model instances independently and in parallel. A neural network segments the input data into clusters representing potential model instances by predicting multiple sets of sample and inlier weights. Using the predicted weights, we determine the model parameters for each potential instance separately in a RANSAC-like fashion. We train the neural network via task-specific loss functions, i.e. we do not require a ground-truth segmentation of the input data. As suitable training data for homography and fundamental matrix fitting is scarce, we additionally present two new synthetic datasets. We demonstrate state-of-the-art performance on these as well as multiple established datasets, with inference times as small as five milliseconds per image.

Spacecraft Homography Pose Estimation with Single-Stage Deep Convolutional Neural Network.

Spacecraft pose estimation using computer vision has garnered increasing attention in research areas such as automation system theory, control theory, sensors and instruments, robot technology, and automation software. Confronted with the extreme environment of space, existing spacecraft pose estimation methods are predominantly multi-stage networks with complex operations. In this study, we propose an approach for spacecraft homography pose estimation with a single-stage deep convolutional neural network for the first time. We formulated a homomorphic geometric constraint equation for spacecraft with planar features. Additionally, we employed a single-stage 2D keypoint regression network to obtain homography 2D keypoint coordinates for spacecraft. After decomposition to obtain the rough spacecraft pose based on the homography matrix constructed according to the geometric constraint equation, a loss function based on pixel errors was employed to refine the spacecraft pose. We conducted extensive experiments using widely used spacecraft pose estimation datasets and compared our method with state-of-the-art techniques in the field to demonstrate its effectiveness.

Comprehensive Evaluation of Multispectral Image Registration Strategies in Heterogenous Agriculture Environment.

This article is focused on the comprehensive evaluation of alleyways to scale-invariant feature transform (SIFT) and random sample consensus (RANSAC) based multispectral (MS) image registration. In this paper, the idea is to extensively evaluate three such SIFT- and RANSAC-based registration approaches over a heterogenous mix containing Triticum aestivum crop and Raphanus raphanistrum weed. The first method is based on the application of a homography matrix, derived during the registration of MS images on spatial coordinates of individual annotations to achieve spatial realignment. The second method is based on the registration of binary masks derived from the ground truth of individual spectral channels. The third method is based on the registration of only the masked pixels of interest across the respective spectral channels. It was found that the MS image registration technique based on the registration of binary masks derived from the manually segmented images exhibited the highest accuracy, followed by the technique involving registration of masked pixels, and lastly, registration based on the spatial realignment of annotations. Among automatically segmented images, the technique based on the registration of automatically predicted mask instances exhibited higher accuracy than the technique based on the registration of masked pixels. In the ground truth images, the annotations performed through the near-infrared channel were found to have a higher accuracy, followed by green, blue, and red spectral channels. Among the automatically segmented images, the accuracy of the blue channel was observed to exhibit a higher accuracy, followed by the green, near-infrared, and red channels. At the individual instance level, the registration based on binary masks depicted the highest accuracy in the green channel, followed by the method based on the registration of masked pixels in the red channel, and lastly, the method based on the spatial realignment of annotations in the green channel. The instance detection of wild radish with YOLOv8l-seg was observed at a mAP@0.5 of 92.11% and a segmentation accuracy of 98% towards segmenting its binary mask instances.