3D Feature Points Research Articles

Fast Attitude Estimation System for Unmanned Ground Vehicle Based on Vision/Inertial Fusion

The attitude estimation system based on vision/inertial fusion is of vital importance and great urgency for unmanned ground vehicles (UGVs) in GNSS-challenged/denied environments. This paper aims to develop a fast vision/inertial fusion system to estimate attitude; which can provide attitude estimation for UGVs during long endurance. The core idea in this paper is to integrate the attitude estimated by continuous vision with the inertial pre-integration results based on optimization. Considering that the time-consuming nature of the classical methods comes from the optimization and maintenance of 3D feature points in the back-end optimization thread, the continuous vision section calculates the attitude by image matching without reconstructing the environment. To tackle the cumulative error of the continuous vision and inertial pre-integration, the prior attitude information is introduced for correction, which is measured and labeled by an off-line fusion of multi-sensors. Experiments with the open-source datasets and in road environments have been carried out, and the results show that the average attitude errors are 1.11° and 1.96°, respectively. The road test results demonstrate that the processing time per frame is 24 ms, which shows that the proposed system improves the computational efficiency.

IMPROVING RANSAC FEATURE MATCHING BASED ON GEOMETRIC RELATION

Abstract. Feature Matching between images is an essential task for many computer vision and photogrammetry applications, such as Structure from Motion (SFM), Surface Extraction, Visual Simultaneous Localization and Mapping (VSLAM), and vision-based localization and navigation. Among the matched point pairs, there are typically false positive matches. Therefore, outlier detection and rejection are important steps in any vision application. RANSAC has been a well-established approach for outlier detection. The outlier ratio and the number of required correspondences used in RANSAC determine the number of iterations needed, which ultimately, determines the computation time. We propose a simple algorithm (GR_RANSAC) based on the two-dimensional spatial relationships between points in the image domain. The assumption is that the distances and bearing angles between the 2D feature points should be similar in images with small disparity, such as the case for video image sequences. In the proposed approach, the distances and angles are measured from a reference point in the first image and its correspondence in the other image, and the points with any significant differences are considered as outliers. This process can pre-filter the matched points, and thus increase the inliers’ ratio. As a result, GR_RANSAC can converge to the correct hypothesis in fewer trial runs than ordinary RANSAC.

Autonomous vehicle self-localization in urban environments based on 3D curvature feature points – Monte Carlo localization

AbstractThis paper proposes a map-based localization system for autonomous vehicle self-localization in urban environments, which is composed of a pose graph mapping method and 3D curvature feature points – Monte Carlo Localization algorithm (3DCF-MCL). The advantage of 3DCF-MCL is that it combines the high accuracy of the 3D feature points registration and the robustness of particle filter. Experimental results show that 3DCF-MCL can provide an accurate localization for autonomous vehicles with the 3D point cloud map that generated by our mapping method. Compared with other map-based localization algorithms, it demonstrates that 3DCF-MCL outperforms them.

3D Face Modeling Algorithm for Film and Television Animation Based on Lightweight Convolutional Neural Network

Through the analysis of facial feature extraction technology, this paper designs a lightweight convolutional neural network (LW-CNN). The LW-CNN model adopts a separable convolution structure, which can propose more accurate features with fewer parameters and can extract 3D feature points of a human face. In order to enhance the accuracy of feature extraction, a face detection method based on the inverted triangle structure is used to detect the face frame of the images in the training set before the model extracts the features. Aiming at the problem that the feature extraction algorithm based on the difference criterion cannot effectively extract the discriminative information, the Generalized Multiple Maximum Dispersion Difference Criterion (GMMSD) and the corresponding feature extraction algorithm are proposed. The algorithm uses the difference criterion instead of the entropy criterion to avoid the “small sample” problem, and the use of QR decomposition can extract more effective discriminative features for facial recognition, while also reducing the computational complexity of feature extraction. Compared with traditional feature extraction methods, GMMSD avoids the problem of “small samples” and does not require preprocessing steps on the samples; it uses QR decomposition to extract features from the original samples and retains the distribution characteristics of the original samples. According to different change matrices, GMMSD can evolve into different feature extraction algorithms, which shows the generalized characteristics of GMMSD. Experiments show that GMMSD can effectively extract facial identification features and improve the accuracy of facial recognition.

Multi-view 2D–3D alignment with hybrid bundle adjustment for visual metrology

High-precision measurement based on multi-view geometry benefits from aligning a template CAD model to the multi-view observations of a target object. It not only improves the multi-camera calibration but also assists the measurement by serving as a “scaffold.” A straightforward approach is to reconstruct the target object and perform a 3D registration with the CAD model. However, the accuracy of such 3D alignment cannot meet the high-precision requirement. We formulate the problem as a bundle adjustment where we jointly optimize the 6DoF poses of both the template model and the multiple cameras. To accommodate the manufacturing error of products, we propose a simple and robust solution based on a discrete–continuous optimization which interleaves between correspondence selection and pose optimization. In the discrete step, a robust RANSAC-based selection process selects well-matched 2D–3D feature points according to the current model/camera poses. In the continuous step, it jointly optimizes the 6D poses of the CAD model and the multiple cameras. This interleaving optimization constitutes a novel hybrid bundle adjustment (HBA). In HBA, the reprojection error of a feature point is measured either with the 2D–3D correspondence between the observation image and the CAD model or with the cross-view correspondence between multiple images, whichever is more reliable according to the discrete selection step. Through extensive evaluation on real-world data, we demonstrate that HBA achieves high-precision multi-camera calibration, outperforming alternative approaches significantly.

Pose Measurement for Unmanned Aerial Vehicle Based on Rigid Skeleton

Pose measurement is a necessary technology for UAV navigation. Accurate pose measurement is the most important guarantee for a UAV stable flight. UAV pose measurement methods mostly use image matching with aircraft models or 2D points corresponding with 3D points. These methods will lead to pose measurement errors due to inaccurate contour and key feature point extraction. In order to solve these problems, a pose measurement method based on the structural characteristics of aircraft rigid skeleton is proposed in this paper. The depth information is introduced to guide and label the 2D feature points to eliminate the feature mismatch and segment the region. The space points obtained from the marked feature points fit the space linear equation of the rigid skeleton, and the UAV attitude is calculated by combining with the geometric model. This method does not need cooperative identification of the aircraft model, and can stably measure the position and attitude of short-range UAV in various environments. The effectiveness and reliability of the proposed method are verified by experiments on a visual simulation platform. The method proposed can prevent aircraft collision and ensure the safety of UAV navigation in autonomous refueling or formation flight.

Research on Transmission Fault Location Based on Scanning Data

Aiming at the problem that the conventional transmission line fault location method has a delay in positioning and cannot be timely feedback, a method for fault location of transmission lines based on 3D scan data matching is proposed. 2D feature points are used to obtain three-dimensional point pairs. Structured light projectors and 3D scanning technology are applied to match image feature points and generate mapping relationships with reconstructed 3D space points. Then use point cloud computing to solve three-dimensional coordinates and use features. Click the interpolation result to complete the data matching process. The simulation experiment applies the 3D scanning data matching technology to the operation process of transmission line fault location. The experimental results show that compared with other methods, the method of this paper has the highest detection stability and the advantage of high accuracy of intelligent fault diagnosis, and can give feedback on transmission line faults in time.

Research on mathematical model of characteristic curve of surface perception by PVDF array based on Ferguson function

The traditional 3D shape perception and reconstruction requires the use of relatively expensive 3D scanning equipment to obtain the coordinates of discrete feature points of the 3D shape. The reconstruction accuracy is determined by the scanning accuracy of the 3D scanning device, generally, the accuracy is low. In this study, based on the PVDF piezoelectric film sensing array, the 3D topographic feature point coordinate value is perceived and acquired. According to the feature point coordinate value, the feature curve fitting of the 3D shape is obtained, and the slope information of the 3D topography characteristic curve is obtained. The mathematical model of the feature surface of PVDF array surface topography based on Ferguson curve is studied to realise the surface reconstruction of 3D complex topography. Theoretical research shows that three-dimensional complex topographic reconstruction technique based on Ferguson curve for PVDF piezoelectric film array surface topography perception has the advantages of simple principle, easy implementation and high precision of shape reconstruction. It can develop intelligent structure with three-dimensional complex shape perception function.

Highly Accurate Scale Estimation from Multiple Keyframes Using RANSAC Plane Fitting With a Novel Scoring Method

Due to the projective nature of a single camera, monocular visual simultaneous localization and mapping (vSLAM) algorithms suffer from scale drift. Recent studies have addressed the scale drift by estimating the height of the camera from the fitted ground plane. However, the low accuracy of 3D ground point (3DGP) reconstruction and the insufficient number of 3DGPs from the textureless ground region make the fitted plane inaccurate for scale estimation. In this paper, we introduce a novel accuracy measure of the fitted plane. From the previous keyframe, we first obtain the 3D points from the intersection of the fitted plane with a ray passing through the camera center and the 2D feature points on the ground region. We then transform the 3D points and 2D feature points to the current keyframe using their respective planar homography. Finally, we project the transformed 3D points onto the image plane of the current keyframe and measure the average Euclidean distance between the projected points and the transformed feature points. The proposed method achieves the average translation error of 1.03% on the KITTI dataset, which outperforms the state-of-the-art monocular vSLAM methods.

Identification of Building Damage from UAV-Based Photogrammetric Point Clouds Using Supervoxel Segmentation and Latent Dirichlet Allocation Model.

Accurate assessment of building damage is very important for disaster response and rescue. Traditional damage detection techniques using 2D features at a single observing angle cannot objectively and accurately reflect the structural damage conditions. With the development of unmanned aerial vehicle photogrammetric techniques and 3D point processing, automatic and accurate damage detection for building roof and facade has become a research hotspot in recent work. In this paper, we propose a building damage detection framework based on the boundary refined supervoxel segmentation and random forest–latent Dirichlet allocation classification. First, the traditional supervoxel segmentation method is improved to segment the point clouds into good boundary refined supervoxels. Then, non-building points such as ground and vegetation are removed from the generated supervoxels. Next, latent Dirichlet allocation (LDA) model is used to construct the high-level feature representation for each building supervoxel based on the selected 2D image and 3D point features. Finally, LDA model and random forest algorithm are employed to identify the damaged building regions. This method is applied to oblique photogrammetric point clouds collected from the Beichuan Country Earthquake Site. The research achieves the 3D damage assessment for building facade and roof. The result demonstrates that the proposed framework is capable of achieving around 94% accuracy for building point extraction and around 90% accuracy for damage identification. Moreover, both of the precision and recall for building damage detection reached around 89%. Concluded from comparison analysis, the proposed method improved the damage detection accuracy and the highest improvement ratio is over 8%.

Rapid Calibration Method for 3D Laser Scanner

Background: The 3D laser scanner is a non-contact active-sensing system, which has a number of applications. Many patents have been filed on the technologies for calibrating 3D laser scanner. A precise calibration method is important for measuring the accuracy of the 3D laser scanner. The system model contains three categories of parameters to be calibrated which include the camera intrinsic parameters, distortion coefficients and the light plane parameters. Typically, the calibration process is completed in two steps. Based on Zhang’s method, the calibration of the camera intrinsic parameters and distortion coefficients can be performed. Then, 3D feature points on the light plane should precisely be formed and extracted. Finally, the points are used to calculate the light plane parameters. Methods: In this paper, a rapid calibration method is presented. Without any high precision auxiliary device, only one coplanar reference target is used. By using a group of captured images of the coplanar reference target placed in the field of view arbitrarily, calibration can be performed in one step. Based on the constraint from the planes formed by the target in different directions and the camera imaging model, a large amount of 3D points on the light plane can easily be obtained. The light plane equation in the camera coordinates system can be gathered by executing plane fitting to the 3D points. Results: During the experimental process, the developed 3D laser scanner was calibrated by the proposed method. Then, the measuring accuracy of the system was verified with known distance in vertical direction of 1mm with sequential shifting motion generated by precision translation stage. The average value of the measured distances was found to be 1.010mm. The standard deviation was 0.008mm. Conclusion: Experimental results prove that the proposed calibration method is simple and reliable.

The 3D Emotion Recognition Using SVM and HoG Features

Emotion recognition is becoming commercially popular due to the major role of analytics in various aspects of marketing and strategy management. Several papers have been proposed in emotion recognition. They are mainly classified in the past under 2D and 3D emotion recognition, out of which 2D emotion recognition has been more popular. Various aspects like facial posture, light intensity variations and sensor-independent recognition have been studied by different authors in the past. However, in reality, 3D emotion recognition has been found to be more efficient which has a broader area of use. In this paper, a 3D tracking plane with 2D feature points has enabled us to recognize emotions by statistical voting method from all planes having over threshold number of points in their respective contour area. The proposed technique’s results are comparable to existing methods in terms of time, space complexity and accuracy improvement.

Single image-based head pose estimation with spherical parametrization and 3D morphing

Head pose estimation plays a vital role in various applications, e.g., driver-assistance systems, human-computer interaction, virtual reality technology, and so on. We propose a novel geometry-based method for accurately estimating the head pose from a single 2D face image at a very low computational cost. Specifically, the rectangular coordinates of only four non-coplanar feature points from a predefined 3D facial model as well as the corresponding ones automatically/manually extracted from a 2D face image are first normalized to exclude the effect of external factors (i.e., scale factor and translation parameters). Then, the four normalized 3D feature points are represented in spherical coordinates with reference to the uniquely determined sphere by themselves. Due to the spherical parametrization, the coordinates of feature points can then be morphed along all the three directions in the rectangular coordinates effectively. Finally, the rotation matrix indicating the head pose is obtained by minimizing the Euclidean distance between the normalized 2D feature points and the 2D re-projections of the morphed 3D feature points. Comprehensive experimental results over two popular datasets, i.e., Pointing’04 and Biwi Kinect, demonstrate that the proposed method can estimate head poses with higher accuracy and lower run time than state-of-the-art geometry-based methods. Even compared with start-of-the-art learning-based methods or geometry-based methods with additional depth information, our method still produces comparable performance.

SLAM-Based Self-Calibration of a Binocular Stereo Vision Rig in Real-Time.

The calibration problem of binocular stereo vision rig is critical for its practical application. However, most existing calibration methods are based on manual off-line algorithms for specific reference targets or patterns. In this paper, we propose a novel simultaneous localization and mapping (SLAM)-based self-calibration method designed to achieve real-time, automatic and accurate calibration of the binocular stereo vision (BSV) rig’s extrinsic parameters in a short period without auxiliary equipment and special calibration markers, assuming the intrinsic parameters of the left and right cameras are known in advance. The main contribution of this paper is to use the SLAM algorithm as our main tool for the calibration method. The method mainly consists of two parts: SLAM-based construction of 3D scene point map and extrinsic parameter calibration. In the first part, the SLAM mainly constructs a 3D feature point map of the natural environment, which is used as a calibration area map. To improve the efficiency of calibration, a lightweight, real-time visual SLAM is built. In the second part, extrinsic parameters are calibrated through the 3D scene point map created by the SLAM. Ultimately, field experiments are performed to evaluate the feasibility, repeatability, and efficiency of our self-calibration method. The experimental data shows that the average absolute error of the Euler angles and translation vectors obtained by our method relative to the reference values obtained by Zhang’s calibration method does not exceed 0.5˚ and 2 mm, respectively. The distribution range of the most widely spread parameter in Euler angles is less than 0.2˚ while that in translation vectors does not exceed 2.15 mm. Under the general texture scene and the normal driving speed of the mobile robot, the calibration time can be generally maintained within 10 s. The above results prove that our proposed method is reliable and has practical value.

Stereoscopic Image Feature Indexing Based on Hybrid Grid Multiple Suffix Tree and Hierarchical Clustering

In order to achieve content-based binocular stereoscopic image or video retrieval efficiently, a feature indexing algorithm based on hybrid grid multiple suffix tree and hierarchical clustering is proposed. With the RGB-D image model, the shape features of depth map obtained from the matching of binocular stereoscopic left image and right image and the color features of left image are extracted respectively. The features are quantified and hashed, and the optimized underlying features are sorted as leaf nodes of hierarchical indexing. Then the shape and color feature values of leaf nodes are mapped to the two-dimensional coordinates, and the 2D feature points are put into different grid hash areas respectively by clustering and labeled with multiple suffix tree. Furthermore, to construct the global index, a pointer to an array of clustering grid center point is defined according to the computation of grid area feature values. The experimental results show that compared to the double grid suffix tree and typical stereoscopic image feature indexing structure, the proposed algorithm can effectively reduce the indexing construction complexity While maintaining high recall, it can also greatly improve the query efficiency, which can better realize the feature indexing of binocular stereoscopic images or videos.

Digital correlation of computer-generated holograms for 3D face recognition

Three-dimensional (3D) face recognition has been a crucial task in human biometric verification and identification. A digital correlation method of a computer-generated hologram (CGH) for 3D face recognition is proposed, which encodes 3D data into a 2D hologram for recognition. The 3D face models are preprocessed and compressed to into groups of feature points. The CGH templates corresponding to the 3D feature points are generated by point- and layer-oriented algorithms based on three different numerical algorithms to encode depth values into 2D holograms. A 2D digital correlation is performed between the CGH templates. It is demonstrated that the generated CGHs templates could be effectively classified based on the correlation performance metrics of discrimination ratio, peak-to-correlation plane energy, and peak-to-noise ratio. With the essence of the CGH algorithm being the conversion of 3D data to a 2D hologram, the proposed encoding and decoding method has great advantages in reducing computational efforts and potential applications in 3D face recognition, storage, and display.

A Novel to Identify Multiple faces by tracking 2D Face Images over 3D Plane

Nowadays, face recognition is very much effective way of counter security threats in various aspects of human life. Though, other means of defending security attacks exists but they have their own drawbacks and overheads. Human face can be recognized from 2D face images or from 3D geometry of human faces. Although very popular, 2D face recognition algorithms are constrained by various factors like change in illumination, varying facial expressions, make-up on the face and orientation of the head. On the other hand face recognition based on 3D geometry of the faces has been shown to have more correctness than 2D face recognition. The only technological drawback however is that the 3D cameras are much less common than 2D based cameras. Therefore the work propose a novel facial expression recognition in real-time by tracking 2D face over a 3D plane. We use multiple 2D planes projected on each-other such that a 2D facial feature point is projected over all the planes, selecting the closest points over the plane and finally creating a contour enclosing the projected feature point over multiple planes, thereby creating a 3D tracking plane with 2D feature point. We use Cambridge landmark markers [4, 5] for facial tracking with multiple Homomorphic projection [5] for creating the 3D feature points. This technique has been proven better in terms of accuracy improvement.

A fast registration algorithm of rock point cloud based on spherical projection and feature extraction

Point cloud registration is an essential step in the process of 3D reconstruction. In this paper, a fast registration algorithm of rock mass point cloud is proposed based on the improved iterative closest point (ICP) algorithm. In our proposed algorithm, the point cloud data of single station scanner is transformed into digital images by spherical polar coordinates, then image features are extracted and edge points are removed, the features used in this algorithm is scale-invariant feature transform (SIFT). By analyzing the corresponding relationship between digital images and 3D points, the 3D feature points are extracted, from which we can search for the two-way correspondence as candidates. After the false matches are eliminated by the exhaustive search method based on random sampling, the transformation is computed via the Levenberg-Marquardt-Iterative Closest Point (LM-ICP) algorithm. Experiments on real data of rock mass show that the proposed algorithm has the similar accuracy and better registration efficiency compared with the ICP algorithm and other algorithms.

Equivalent Constraints for Two-View Geometry: Pose Solution/Pure Rotation Identification and 3D Reconstruction

Two-view relative pose estimation and structure reconstruction is a classical problem in computer vision. The typical methods usually employ the singular value decomposition of the essential matrix to get multiple solutions of the relative pose, from which the right solution is picked out by reconstructing the three-dimension (3D) feature points and imposing the constraint of positive depth. This paper revisits the two-view geometry problem and discovers that the two-view imaging geometry is equivalently governed by a Pair of new Pose-Only (PPO) constraints: the same-side constraint and the intersection constraint. From the perspective of solving equation, the complete pose solutions of the essential matrix are explicitly derived and we rigorously prove that the orientation part of the pose can still be recovered in the case of pure rotation. The PPO constraints are simplified and formulated in the form of inequalities to directly identify the right pose solution with no need of 3D reconstruction and the 3D reconstruction can be analytically achieved from the identified right pose. Furthermore, the intersection inequality also enables a robust criterion for pure rotation identification. Experiment results validate the correctness of analyses and the robustness of the derived pose solution/pure rotation identification and analytical 3D reconstruction.

An Improved Method for Spatial Measurement in Surveillance Video

Video detection technology plays an increasingly significant role in forensics and criminology. Measurement of real distance from single image frames is a key technology for many applications. There are several methods for the measurement, including proportion measurement, investigative experiment, perspective and projection transformation. Although these methods are applicable to the special scenarios, there are still shortcomings of their own. Therefore, we propose an approach called two perspective transformations to achieve the real size of objects in 3-D world measurement. Firstly, we use close-range photogrammetry method to obtain 3-D structure of the multiple feature points at the crime scene. Then the 3-D structure will be corresponded to 2D feature points in the measured single image frames, and the collinear equations are used to establish the perspective transformation between 2D features in the measured image frame and 3-D structure in 3-D real world. Finally, we use the perspective transformation matrix to reconstruct the 3-D structure of any feature point in the measured image frame. The method we proposed is successfully applied in a corpse-dismantling case for measuring a box’s height.