EPnP Algorithm Research Articles

High-accuracy pose estimation algorithm with monocular camera based on residual network

Pose estimation is a typical problem encountered in computer vision. Therefore, it is important to improve the accuracy of this method. Aiming at the accuracy of pose estimation, we propose a high-accuracy pose estimation algorithm based on the you only look once network and residual network with a monocular camera as the sensor for visual acquisition. This algorithm uses ArUco markers as a reference for object localization and uses the red, green, and blue (RGB) image as the input. The input image is sampled 16 and 32 times to extract the feature image, and the feature image extracted by 16 times sampling is passed through the pass-through layer and then combined with the feature image extracted by 32 times sampling to accomplish the dimension expansion. The feature image is identified by the convolutional layer. The EPnP algorithm is used to solve the camera poses. The pose information of the target object in the RGB image is used as the output. By comparing the pose estimation accuracy for the LINEMOD dataset with three evaluation metrics—the 2D projection metric, ADD metric, and 5 cm to 5 deg metric—it can be observed that the pose estimation algorithm proposed has advantages in terms of accuracy compared with traditional pose estimation algorithms. When the target is very similar to the background objects, the algorithm also achieves good performance.

An attitude estimation algorithm for the telescopic arm of the boarding bridge based on YOLOv5 and EPnP

Aiming at the attitude estimation of the telescopic arm of the boarding bridge in the process of docking with the offshore wind turbine, an attitude estimation algorithm for the telescopic arm of the boarding bridge based on YOLOv5 and EPnP is proposed in this paper. YOLOv5 algorithm is used to detect four marks in the offshore wind turbine logo image, and then the EPnP algorithm is used to solve attitude angles of the telescopic arm relative to the landing port of the offshore wind turbine according to the 2D pixel coordinates of the center point of each mark. The experimental results show that the attitude estimation error of the proposed algorithm is less than 0.4° and the data update frequency of the algorithm implemented in NVIDIA Jetson AGX Xavier is 25Hz, which meets the real-time and accuracy requirements for the attitude detection of the boarding bridge telescopic arm.

Research on Identification and Location of Charging Ports of Multiple Electric Vehicles Based on SFLDLC-CBAM-YOLOV7-Tinp-CTMA

With the gradual maturity of autonomous driving and automatic parking technology, electric vehicle charging is moving towards automation. The charging port (CP) location is an important basis for realizing automatic charging. Existing CP identification algorithms are only suitable for a single vehicle model with poor universality. Therefore, this paper proposes a set of methods that can identify the CPs of various vehicle types. The recognition process is divided into a rough positioning stage (RPS) and a precise positioning stage (PPS). In this study, the data sets corresponding to four types of vehicle CPs under different environments are established. In the RPS, the characteristic information of the CP is obtained based on the combination of convolutional block attention module (CBAM) and YOLOV7-tinp, and its position information is calculated using the similar projection relationship. For the PPS, this paper proposes a data enhancement method based on similar feature location to determine the label category (SFLDLC). The CBAM-YOLOV7-tinp is used to identify the feature location information, and the cluster template matching algorithm (CTMA) is used to obtain the accurate feature location and tag type, and the EPnP algorithm is used to calculate the location and posture (LP) information. The results of the LP solution are used to provide the position coordinates of the CP relative to the robot base. Finally, the AUBO-i10 robot is used to complete the experimental test. The corresponding results show that the average positioning errors (x, y, z, rx, ry, and rz) of the CP are 0.64 mm, 0.88 mm, 1.24 mm, 1.19 degrees, 1.00 degrees, and 0.57 degrees, respectively, and the integrated insertion success rate is 94.25%. Therefore, the algorithm proposed in this paper can efficiently and accurately identify and locate various types of CP and meet the actual plugging requirements.

A novel method for dense point cloud reconstruction and weld seam detection for tubesheet welding robot

Welding is not yet fully automated, which is mainly limited by the manual drawing of the welding model. A novel method of automatically establishing a dense point cloud model of the tubesheet and detecting the weld seam was proposed to solve the issue in this paper. The multi-sensor system was calibrated with a fast calibration method, and a laser filter algorithm was then applied to fuse the multi-sensor data. The vocabulary tree method was carried out for tubesheet image similarity analysis, followed by the Random Sample Consensus (RANSAC) algorithm used for feature matching, aiming to establish a point cloud model based on epipolar constraint, EPNP and PMVS algorithm. Finally, a weld seam detection algorithm based on voxel point cloud density was proposed to detect the weld seam in the point cloud. After comparison, the reconstruction method has better robustness than the reference. The point cloud measurement results showed that the average row and column length errors of the reconstructed point cloud were both less than 1 %, which can meet the requirements in current welding applications. And the proposed weld seam detection method can reduce the detection error rate from 20.83 % to 9.03 %.

Learning-Based Pose Estimation of Non-Cooperative Spacecrafts with Uncertainty Prediction

Estimation of spacecraft pose is essential for many space missions, such as formation flying, rendezvous, docking, repair, and space debris removal. We propose a learning-based method with uncertainty prediction to estimate the pose of a spacecraft from a monocular image. We first used a spacecraft detection network (SDN) to crop out the rectangular area in the original image where only spacecraft exist. A keypoint detection network (KDN) was then used to detect 11 pre-selected keypoints with obvious features from the cropped image and predict uncertainty. We propose a keypoints selection strategy to automatically select keypoints with higher detection accuracy from all detected keypoints. These selective keypoints were used to estimate the 6D pose of the spacecraft with the EPnP algorithm. We evaluated our method on the SPEED dataset. The experiments showed that our method outperforms heatmap-based and regression-based methods, and our effective uncertainty prediction can increase the final precision of the pose estimation.

Research on Fast Recognition and Localization of an Electric Vehicle Charging Port Based on a Cluster Template Matching Algorithm.

With the gradual maturity of driverless and automatic parking technologies, electric vehicle charging has been gradually developing in the direction of automation. However, the pose calculation of the charging port (CP) is an important part of realizing automatic charging, and it represents a problem that needs to be solved urgently. To address this problem, this paper proposes a set of efficient and accurate methods for determining the pose of an electric vehicle CP, which mainly includes the search and aiming phases. In the search phase, the feature circle algorithm is used to fit the ellipse information to obtain the pixel coordinates of the feature point. In the aiming phase, contour matching and logarithmic evaluation indicators are used in the cluster template matching algorithm (CTMA) proposed in this paper to obtain the matching position. Based on the image deformation rate and zoom rates, a matching template is established to realize the fast and accurate matching of textureless circular features and complex light fields. The EPnP algorithm is employed to obtain the pose information, and an AUBO-i5 robot is used to complete the charging gun insertion. The results show that the average CP positioning errors (x, y, z, Rx, Ry, and Rz) of the proposed algorithm are 0.65 mm, 0.84 mm, 1.24 mm, 1.11 degrees, 0.95 degrees, and 0.55 degrees. Further, the efficiency of the positioning method is improved by 510.4% and the comprehensive plug-in success rate is 95%. Therefore, the proposed CTMA in this paper can efficiently and accurately identify the CP while meeting the actual plug-in requirements.

A UAV Visual Relocalization Method Using Semantic Object Features Based on Internet of Things

Unmanned Air Vehicle (UAV) has the advantages of high autonomy and strong dynamic deployment capabilities. At the same time, with the rapid development of the Internet of Things (IoT) technology, the construction of the IoT based on UAVs can break away from the traditional single-line communication mode of UAVs and control terminals, which makes the UAVs more intelligent and flexible when performing tasks. When using UAVs to perform IoT tasks, it is necessary to track the UAVs’ position and pose at all times. When the position and pose tracking fails, relocalization is required to restore the current position and pose. Therefore, how to perform UAV relocalization accurately by using visual information has attracted much attention. However, the complex changes in light conditions in the real world have brought huge challenges to the visual relocalization of UAV. Traditional visual relocalization algorithms mostly rely on artificially designed low-level geometric features which are sensitive to light conditions. In this paper, oriented to the UAV-based IoT, a UAV visual relocalization method using semantic object features is proposed. Specifically, the method uses YOLOv3 as the object detection framework to extract the semantic information in the picture and uses the semantic information to construct a topological map as a sparse description of the environment. With prior knowledge of the map, the random walk algorithm is used on the association graphs to match the semantic features and the scenes. Finally, the EPnP algorithm is used to solve the position and pose of the UAV which will be returned to the IoT platform. Simulation results show that the method proposed in this paper can achieve robust real-time UAVs relocalization when the scene lighting conditions change dynamically and provide a guarantee for UAVs to perform IoT tasks.

Pose Estimation of Omnidirectional Camera with Improved EPnP Algorithm.

The omnidirectional camera, having the advantage of broadening the field of view, realizes 360° imaging in the horizontal direction. Due to light reflection from the mirror surface, the collinearity relation is altered and the imaged scene has severe nonlinear distortions. This makes it more difficult to estimate the pose of the omnidirectional camera. To solve this problem, we derive the mapping from omnidirectional camera to traditional camera and propose an omnidirectional camera linear imaging model. Based on the linear imaging model, we improve the EPnP algorithm to calculate the omnidirectional camera pose. To validate the proposed solution, we conducted simulations and physical experiments. Results show that the algorithm has a good performance in resisting noise.

Research on the Fast Solution Method of Pilots’ Head Attitude

Aiming at the problem of poor accuracy and poor real-time performance in the process of head pose calculation in pilot cabin simulation, a high accuracy and fast algorithm for head pose calculation is proposed. On the basis of traditional EPNP algorithm was improved, and combining with POSIT algorithm’s speed advantage, select face five key feature points as virtual control points, according to the mapping relationship between space and virtual control point coordinates, establish a system of linear equations, then the internal camera parameters is calculated using Zhengyou Zhang camera calibration method, finally using POSIT algorithm of calculating to get the head rotation matrix and translation vector, by conversion from the head of Euler angle in real time. The experimental results show that compared with DLT, EPNP, POSIT and other traditional methods, the proposed method has higher accuracy, better real-time performance and better robustness in solving pilots’ head attitude in continuous video.

DIRECT CO-REGISTRATION OF TIR IMAGES AND MLS POINT CLOUDS BY CORRESPONDING KEYPOINTS

Abstract. In this work, we discussed how to directly combine thermal infrared image (TIR) and the point cloud without additional assistance from GCPs or 3D models. Specifically, we propose a point-based co-registration process for combining the TIR image and the point cloud for the buildings. The keypoints are extracted from images and point clouds via primitive segmentation and corner detection, then pairs of corresponding points are identified manually. After that, the estimated camera pose can be computed with EPnP algorithm. Finally, the point cloud with thermal information provided by IR images can be generated as a result, which is helpful in the tasks such as energy inspection, leakage detection, and abnormal condition monitoring. This paper provides us more insight about the probability and ideas about the combining TIR image and point cloud.

Multi-feature analysis fatigue driving based on Conditional Local Neural Fields algorithm detection method

In order to effectively strengthen the monitoring technology of fatigue driving and reduce the incidence of traffic accidents, this paper proposes a new method based on conditional local neural fields algorithm for comprehensive evaluation of fatigue driving state. At present, most of the fatigue driving detection algorithms are based on extracting a single characteristic index, which is strict in environmental requirements and not high in detection. In this paper, the HOG feature is combined with the CLNF algorithm to implement face detection and feature point localization. Then, the EPnP algorithm is used to estimate the head anomaly frequency based on the feature point information, and the blink frequency is calculated according to the EAR eye length aspect ratio concept according to the feature points around the eye. Finally the threshold set by the P80 fatigue detection standard in the PERCLOS method is integrated, and the distributed information fusion strategy is used for fatigue evaluation. Experimental results confirm the effectiveness of the method.

基于EPNP算法的单目视觉测量系统研究

The method of pose measurement using computer vision is widely used in modern control, navigation, tracking and other fields. A monocular visual pose measurement method was studied and designd based on P4P rectangular distribution of planar target and EPNP algorithm in this paper. Firstly, a single camera was used to obtain the plane target image. After the image processing, the pixel coordinates of the four feature points were obtained, and the EPNP algorithm was used to perform the attitude calculation. Secondly, the simulation analysis of pose error measurement provides theoretical guidance and basis for improving attitude measurement accuracy. Finally, a coordinate system registration method combined with a high-precision two-dimensional turntable was proposed. The method was used to verify the accuracy of the three-direction attitude angles. The experimental results show that the rotation angle around the x and y axes is[-6, 6], the pose measurement error is less than 0.1 to meet the measurement application requirements.

An on-line calibration method for camera with large FOV based on prior information

In the large dimensional vision measurement, the parameters of camera with large FOV (field-of-view) are easy to change for the influence of working conditions, which directly affects the precision of measurement. Concerning this issue, an on-line calibration method for camera with large FOV based on prior information is proposed. In this method, sufficient prior information is firstly obtained by the lights whose directions and positions in the space are known before measurement, so as to achieve the high accuracy pre-calibration of camera with large FOV, in which the initial values of camera parameters are acquired. During the measurement, on-line calibration is conducted with several fixed points in the FOV by combining EPnP algorithm, a pose estimation algorithm based on lines, and LM (Levenberg–Marquardt) optimization algorithm, which adjusts the parameters of camera in time. The experiments results show that the precision of the on-line calibration method is equal to that of the off-line traditional calibration methods, whose reconstruction error is within 2mm with the FOV of 3000mm×4000mm, while the method is less time consuming. The method in this paper is proved suitable for the on-line calibration of camera with large FOV.

Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation

We propose a novel approach for the estimation of the pose and focal length of a camera from a set of 3D-to-2D point correspondences. Our method compares favorably to competing approaches in that it is both more accurate than existing closed form solutions, as well as faster and also more accurate than iterative ones. Our approach is inspired on the EPnP algorithm, a recent O(n) solution for the calibrated case. Yet we show that considering the focal length as an additional unknown renders the linearization and relinearization techniques of the original approach no longer valid, especially with large amounts of noise. We present new methodologies to circumvent this limitation termed exhaustive linearization and exhaustive relinearization which perform a systematic exploration of the solution space in closed form. The method is evaluated on both real and synthetic data, and our results show that besides producing precise focal length estimation, the retrieved camera pose is almost as accurate as the one computed using the EPnP, which assumes a calibrated camera.

FULLY AUTOMATIC IMAGE-BASED REGISTRATION OF UNORGANIZED TLS DATA

Abstract. The estimation of the transformation parameters between different point clouds is still a crucial task as it is usually followed by scene reconstruction, object detection or object recognition. Therefore, the estimates should be as accurate as possible. Recent developments show that it is feasible to utilize both the measured range information and the reflectance information sampled as image, as 2D imagery provides additional information. In this paper, an image-based registration approach for TLS data is presented which consists of two major steps. In the first step, the order of the scans is calculated by checking the similarity of the respective reflectance images via the total number of SIFT correspondences between them. Subsequently, in the second step, for each SIFT correspondence the respective SIFT features are filtered with respect to their reliability concerning the range information and projected to 3D space. Combining the 3D points with 2D observations on a virtual plane yields 3D-to-2D correspondences from which the coarse transformation parameters can be estimated via a RANSAC-based registration scheme including the EPnP algorithm. After this coarse registration, the 3D points are again checked for consistency by using constraints based on the 3D distance, and, finally, the remaining 3D points are used for an ICP-based fine registration. Thus, the proposed methodology provides a fast, reliable, accurate and fully automatic image-based approach for the registration of unorganized point clouds without the need of a priori information about the order of the scans, the presence of regular surfaces or human interaction.

Fast and automatic image-based registration of TLS data

The fast and automatic registration of laser scanner data is of great interest in photogrammetric research. Recent developments show that for registration purposes characteristic 3D points can be extracted from the measured laser data, where the data are also represented as image. In this paper, radiometric and geometric information derived from TLS data are utilized for estimating the transformation parameters between two unregistered point clouds. After the extraction of characteristic 2D points based on SIFT features, these points are projected into 3D space by using interpolated range information. From these 3D conjugate points and their corresponding 2D projections onto a virtual plane 3D-to-2D correspondences are established. The fast, accurate and robust RANSAC-based registration scheme including the EPnP algorithm provides a framework to estimate the coarse transformation parameters from these 3D-to-2D correspondences. The coarse estimates are further refined by a single step outlier removal to gain a higher accuracy by introducing additional geometric constraints. These new constraints are based on 3D-to-3D correspondences which are much stronger than the 3D-to-2D correspondences alone. It will be shown that the presented approach is successfully applied to a benchmarked data set with millions of points resulting in a fast and accurate estimation of the transformation parameters with a processing speed of several seconds on a standard PC and an accuracy in the low centimeter range.