Ground Plane Constraint Research Articles

LiDAR Vehicle Point Cloud Reconstruction Framework for Axle-Based Classification

As Light Detection and Range (LiDAR) technology rapidly advances, it is becoming increasingly viable as a solution to collect vehicle classification data. The main challenge of LiDAR compared with video in vehicle classification lies in its resolution, which limits the ability of LiDAR-based models to classify vehicles in detail from a single captured frame. This paper proposes a novel framework for reconstructing the vehicle point clouds generated by a roadside LiDAR sensor, by considering ground plane constraints and developing a bootstrap aggregating deep neural network (bagging DNN) model to classify the reconstructed vehicle point clouds, according to the US Federal Highway Administration (FHWA) axle-based vehicle classification scheme. First, a probabilistic-based registration algorithm is used to estimate the transformation matrix between consecutive frames of each vehicle point cloud. Then, a multiway registration is conducted to fine-tune the estimated transformation matrices to rebuild the 3D model of each moving vehicle. Second, key features are extracted from the reconstructed vehicle models and fed into a bagging DNN model to provide classifications based on the FHWA scheme. The classification model with the reconstruction framework outperforms the latest developed LiDAR-based FHWA classification model in terms of both accuracy and robustness. The model has an 83 percent average correct classification rate (CCR) on the test set. Remarkably, the proposed model can accurately distinguish Classes 5 and 8 trucks, which have overlapping axle configurations, with a 97 percent and a 90 percent CCR, respectively.

Visual-Gyroscope-Wheel Odometry With Ground Plane Constraint for Indoor Robots in Dynamic Environment

Localization is an essential prerequisite for autonomous robots to execute tasks. In this letter, a multisensor fused odometry is proposed for the robot applied to dynamic indoor environments. The gyroscope and wheel encoder information is integrated into visual odometry, where the robot pose is parameterized with 3-D position and 3-D Euler angle to relieve the estimation drift during planar motion and conveniently fuse wheel encoder measurements. According to the results of semantic segmentation, the dynamic elements are removed. On this basis, ground plane constraint is exerted on the optimization procedure. As a result, the estimation accuracy is improved. The experimental results on the OpenLORIS-scene dataset demonstrate the effectiveness of the proposed odometry.

Pedestrian- and Vehicle-Detection Algorithm Based on Improved Aggregated Channel Features

In advanced driver-assistance systems (ADAS), the accuracy and real-time performance of pedestrian- and vehicle-detection algorithms based on vision sensors are crucial for safety. Here, a lightweight detection algorithm based on aggregated channel features (ACFs),consisting of a context pixel ACF (CP-ACF) pedestrian detector and a multiview ACF (Mv-ACF) vehicle detector, is proposed to rapidly and precisely understand road scenes. The former fuses local and context information to improve the robustness to pedestrian deformation, while the latter contains a number of subclass detectors to alleviate intraclass differences due to different viewing angles. Compared to the original ACF, the CP-ACF pedestrian detector reduces the average miss rate (AMR) by 6.34%. The Mv-ACF vehicle detector improves the average precision (AP) by 40.26% on average at easy, moderate and hard levels. This remarkable effectiveness is due to the spectrum clustering of multiview samples and the resulting integration of these subclass detectors via confidence score calibration, which reduces the intraclass differences of vehicles. Since feature extraction takes up 68.8% of the total detection time, a mechanism of feature sharing between pedestrian and vehicle detectors is advanced to reduce the time spent in feature extraction. A strategy based on ground-plane constraints (GPCs) is proposed to control false detection of pedestrians and vehicles by incorporating road prior information, which reduces the AMR by 1.07% for CP-ACF pedestrian detectors and improves the AP by 0.27% on average for Mv-ACF vehicle detectors. Thus, the proposed algorithm can effectively control false detection by road prior information.

Ground Plane Polling for 6DoF Pose Estimation of Objects on the Road

This paper introduces an approach to produce accurate 3D detection boxes for objects on the ground using single monocular images. We do so by merging 2D visual cues, 3D object dimensions, and ground plane constraints to produce boxes that are robust against small errors and incorrect predictions. First, we train a single-shot convolutional neural network (CNN) that produces multiple visual and geometric cues of interest: 2D bounding boxes, 2D keypoints of interest, coarse object orientations and object dimensions. Subsets of these cues are then used to poll probable ground planes from a pre-computed database of ground planes, to identify the “best fit” plane with highest consensus. Once identified, the “best fit” plane provides enough constraints to successfully construct the desired 3D detection box, without directly predicting the 6DoF pose of the object. The entire ground plane polling (GPP) procedure is constructed as a non-parametrized layer of the CNN that outputs the desired “best fit” plane and the corresponding 3D keypoints, which together define the final 3D bounding box. Doing so allows us to poll thousands of different ground plane configurations without adding considerable overhead, while also creating a single CNN that directly produces the desired output without the need for post processing. We evaluate our method on the 2D detection and orientation estimation benchmark from the challenging KITTI dataset, and provide additional comparisons for 3D metrics of importance. This single-stage, single-pass CNN results in superior localization and orientation estimation compared to more complex and computationally expensive monocular approaches.

Optimized LOAM Using Ground Plane Constraints and SegMatch-Based Loop Detection

Reducing the cumulative error in the process of simultaneous localization and mapping (SLAM) has always been a hot issue. In this paper, in order to improve the localization and mapping accuracy of ground vehicles, we proposed a novel optimized lidar odometry and mapping method using ground plane constraints and SegMatch-based loop detection. We only used the lidar point cloud to estimate the pose between consecutive frames, without any other sensors, such as Global Positioning System (GPS) and Inertial Measurement Unit (IMU). Firstly, the ground plane constraints were used to reduce matching errors. Then, based on more accurate lidar odometry obtained from lidar odometry and mapping (LOAM), SegMatch completed segmentation matching and loop detection to optimize the global pose. The neighborhood search was also used to accomplish the loop detection task in case of failure. Finally, the proposed method was evaluated and compared with the existing 3D lidar SLAM methods. Experiment results showed that the proposed method could realize low drift localization and dense 3D point cloud map construction.

Mono3D++: Monocular 3D Vehicle Detection with Two-Scale 3D Hypotheses and Task Priors

We present a method to infer 3D pose and shape of vehicles from a single image. To tackle this ill-posed problem, we optimize two-scale projection consistency between the generated 3D hypotheses and their 2D pseudo-measurements. Specifically, we use a morphable wireframe model to generate a fine-scaled representation of vehicle shape and pose. To reduce its sensitivity to 2D landmarks, we jointly model the 3D bounding box as a coarse representation which improves robustness. We also integrate three task priors, including unsupervised monocular depth, a ground plane constraint as well as vehicle shape priors, with forward projection errors into an overall energy function.

Three-Dimensional Deformable-Model-Based Localization and Recognition of Road Vehicles

We address the problem of model-based object recognition. Our aim is to localize and recognize road vehicles from monocular images or videos in calibrated traffic scenes. A 3-D deformable vehicle model with 12 shape parameters is set up as prior information, and its pose is determined by three parameters, which are its position on the ground plane and its orientation about the vertical axis under ground-plane constraints. An efficient local gradient-based method is proposed to evaluate the fitness between the projection of the vehicle model and image data, which is combined into a novel evolutionary computing framework to estimate the 12 shape parameters and three pose parameters by iterative evolution. The recovery of pose parameters achieves vehicle localization, whereas the shape parameters are used for vehicle recognition. Numerous experiments are conducted in this paper to demonstrate the performance of our approach. It is shown that the local gradient-based method can evaluate accurately and efficiently the fitness between the projection of the vehicle model and the image data. The evolutionary computing framework is effective for vehicles of different types and poses is robust to all kinds of occlusion.

3-D model-based vehicle tracking

This paper aims at tracking vehicles from monocular intensity image sequences and presents an efficient and robust approach to three-dimensional (3-D) model-based vehicle tracking. Under the weak perspective assumption and the ground-plane constraint, the movements of model projection in the two-dimensional image plane can be decomposed into two motions: translation and rotation. They are the results of the corresponding movements of 3-D translation on the ground plane (GP) and rotation around the normal of the GP, which can be determined separately. A new metric based on point-to-line segment distance is proposed to evaluate the similarity between an image region and an instantiation of a 3-D vehicle model under a given pose. Based on this, we provide an efficient pose refinement method to refine the vehicle's pose parameters. An improved EKF is also proposed to track and to predict vehicle motion with a precise kinematics model. Experimental results with both indoor and outdoor data show that the algorithm obtains desirable performance even under severe occlusion and clutter.

Efficient image gradient based vehicle localization

This paper reports novel algorithms for the efficient localization and recognition of traffic in traffic scenes. The algorithms eliminate the need for explicit symbolic feature extraction and matching. The pose and class of an object is determined by a form of voting and one-dimensional (1-D) correlations based directly on image gradient data, which can be computed "on the fly." The algorithms are therefore very well suited to real-time implementation. The algorithms make use of two a priori sources of knowledge about the scene and the objects expected: (1) the ground-plane constraint and (2) the fact that the overall shape of road vehicles is strongly rectilinear. Additional efficiency is derived from making the weak perspective assumption. These assumptions are valid in the road traffic application domain. The algorithms are demonstrated and tested using routine outdoor traffic images. Success with a variety of vehicles in several traffic scenes demonstrates the efficiency and robustness of context-based image understanding in road traffic scene analysis. The limitations of the algorithms are also addressed in the paper.

Vehicle-type motion estimation by the fusion of image point and line features

Three-dimensional (3D) motion estimation is a very important topic in machine vision. However, reliability of the estimated 3D motion seems to be the most challenging problem, especially to the linear algorithms developed for solving a general 3D motion problem (six degrees of freedom). In real applications such as the traffic surveillance and auto-vehicle systems, the observed 3D motion has only three degrees of freedom because of the ground plane constraint (GPC). In this paper, a new iterative method is proposed for solving the above problem. Our method has several advantages: (1) It can handle both the point and line features as its input image data. (2) It is very suitable for parallel processing. (3) Its cost function is so well-conditioned that the final 3D motion estimation is robust and insensitive to noise, which is proved by experiments. (4) It can handle the case of missing data to a certain degree. The above benefits make our method suitable for a real application. Experiments including simulated and real-world images show satisfactory results.

Model-Based Localisation and Recognition of Road Vehicles

Objects are often constrained to lie on a known plane. This paper concerns the pose determination and recognition of vehicles in traffic scenes, which under normal conditions stand on the ground-plane. The ground-plane constraint reduces the problem of localisation and recognition from 6 dof to 3 dof. The ground-plane constraint significantly reduces the pose redundancy of 2D image and 3D model line matches. A form of the generalised Hough transform is used in conjuction with explicit probability-based voting models to find consistent matches and to identify the approximate poses. The algorithms are applied to images of several outdoor traffic scenes and successful results are obtained. The work reported in this paper illustrates the efficiency and robustness of context-based vision in a practical application of computer vision. Multiple cameras may be used to overcome the limitations of a single camera. Data fusion in the proposed algorithms is shown to be simple and straightforward.

Human-robot interface by pointing with uncalibrated stereo vision

Here we present the results of an investigation into the use of a pointing-based interface for robot guidance. The system requires no physical contact with the operator, but uses uncalibrated stereo vision with active contours to track the position and pointing direction of a hand in real time. With a ground plane constraint, it is possible to find the indicated position in the robot's workspace, by considering only two-dimensional collineations. Experimental and simulation data show that a resolution of within 1 cm can be achieved in a 40 cm workspace, allowing simple pick-and-place operations to be specified by finger pointing.

Closed-form algorithms for object pose and scale recovery in constrained scenes

This paper concerns the recovery of pose and scale of vehicles in traffic scenes which, under normal conditions, are constrained to be in contact with the ground-plane. Several closed-form algorithms are described for pose and scale recovery using known 2D-to-3D line matches. The algorithms directly exploit the ground-plane constraint and are applicable to an arbitrary number of line matches. The algorithms are tested extensively with both synthetic and real outdoor traffic images. They are found to be robust and perform satisfactorily with real images.

Recognizing objects on the ground-plane

Objects such as vehicles are often constrained to lie on a known plane. The ground-plane constraint reduces the problem of localization and recognition from 6 to 3 DOF. A novel algorithm is presented which makes effective use of the ground-plane constraint to derive pose estimates. A form of the generalized Hough transform is used to group evidence from line features, and to identify approximate poses. The single orientation parameter is decoupled from the two location parameters, and dealt with separately. The method is fast and robust. It copes well with complex outdoor scenes including multiple occluded objects, and image clutter from irrelevant structures.

3D structure and motion estimation from 2D image sequences

Two novel algorithms are presented for depth estimation using point correspondences and the ground plane constraint. One is a direct non-iterative method, and the other a simple, well-behaved iterative technique where the choice of initial value is straightforward. The algorithms are capable of handling any number of points and frames as well as points which become occluded. Once the point depths are determined, motion parameters can be obtained by a linear least squares technique. Extensive test results are included which show that the proposed algorithms are robust to noise, and perform satisfactorily using real outdoor image sequences.