Fast 3D Object Research Articles

ESGN: Efficient Stereo Geometry Network for Fast 3D Object Detection

Fast stereo based 3D object detectors have made great progress recently. However, they suffer from the inferior accuracy. We argue that the main reason is due to the poor geometry-aware feature representation in 3D space. To solve this problem, we propose an efficient stereo geometry network (ESGN). The key in our ESGN is an efficient geometry-aware feature generation (EGFG) module. Our EGFG module first uses a stereo correlation and reprojection module to construct multi-scale stereo volumes in camera frustum space, second employs a multi-scale bird&#x2019;s eye view (BEV) projection and fusion module to generate multiple geometry-aware features. In these two steps, we adopt deep multi-scale information fusion for discriminative geometry-aware feature generation, without any complex aggregation networks. In addition, we introduce a deep geometry-aware feature distillation scheme to guide stereo feature learning with a LiDAR-based detector. The experiments are performed on the classical KITTI dataset. On KITTI test set, our ESGN outperforms the fast state-of-art-art detector YOLOStereo3D by 5.14% on mAP_3<i>d</i> at 62<i>ms</i>. To the best of our knowledge, our ESGN achieves a best trade-off between accuracy and speed. We hope that our efficient stereo geometry network can provide more possible directions for fast 3D object detection.

Fast 3D Object Measurement Based on Point Cloud Modeling

Automated object measurement is becoming increasingly important due to its ability to reduce manual costs, increase production efficiency, and minimize errors in various fields. In this paper, we present a novel approach to three-dimensional (3D) object measurement based on point cloud modeling. Our method introduces a fast point cloud modeling computation framework consisting of five stages: coordinate centralization, rotation and translation, noise filtering, plane projection, and geometric computation. Furthermore, we propose a fast convex hull optimization algorithm to reduce the high complexity problem of traditional convex hull calculation. Our extensive experiments demonstrate that our approach outperforms existing methods in terms of measurement error rate and time savings, with a maximum time saving of 31.03% under certain error conditions.

CenterNet3D: An Anchor Free Object Detector for Point Cloud

Accurate and fast 3D object detection from point clouds is a key task in autonomous driving. Existing one-stage 3D object detection methods can achieve real-time performance, however, they are dominated by anchor-based detectors which are inefficient and require additional post-processing. In this paper, we eliminate anchors and model an object as a single point--the center point of its bounding box. Based on the center point, we propose an anchor-free CenterNet3D network that performs 3D object detection without anchors. Our CenterNet3D uses keypoint estimation to find center points and directly regresses 3D bounding boxes. However, because inherent sparsity of point clouds, 3D object center points are likely to be in empty space which makes it difficult to estimate accurate boundaries. To solve this issue, we propose an extra corner attention module to enforce the CNN backbone to pay more attention to object boundaries. Besides, considering that one-stage detectors suffer from the discordance between the predicted bounding boxes and corresponding classification confidences, we develop an efficient keypoint-sensitive warping operation to align the confidences to the predicted bounding boxes. Our proposed CenterNet3D is non-maximum suppression free which makes it more efficient and simpler. We evaluate CenterNet3D on the widely used KITTI dataset and more challenging nuScenes dataset. Our method outperforms all state-of-the-art anchor-based one-stage methods and has comparable performance to two-stage methods as well. It has an inference speed of 20 FPS and achieves the best speed and accuracy trade-off. Our source code will be released at https://github.com/wangguojun2018/CenterNet3d.

Phase-SLAM: Phase Based Simultaneous Localization and Mapping for Mobile Structured Light Illumination Systems

Structured Light Illumination (SLI) systems have been used for reliable indoor dense 3D scanning via phase triangulation. However, mobile SLI systems for 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> 3D reconstruction demand 3D point cloud registration, involving high computational complexity. In this letter, we propose a phase based Simultaneous Localization and Mapping (Phase-SLAM) framework for fast and accurate SLI sensor pose estimation and 3D object reconstruction. The novelty of this work is threefold: (1) developing a reprojection model from 3D points to 2D phase data towards phase registration with low computational complexity; (2) developing a local optimizer to achieve SLI sensor pose estimation (odometry) using the derived Jacobian matrix for the 6 DoF variables; (3) developing a compressive phase comparison method to achieve high-efficiency loop closure detection. The whole Phase-SLAM pipeline is then exploited using existing global pose graph optimization techniques. We build datasets from both the unreal simulation platform and a robotic arm based SLI system in real-world to verify the proposed approach. The experiment results demonstrate that the proposed Phase-SLAM outperforms other state-of-the-art methods in terms of the efficiency and accuracy of pose estimation and 3D reconstruction.

HCCG: Efficient high compatibility correspondence grouping for 3D object recognition and 6D pose estimation in cluttered scenes

Recognizing and localizing queried objects in point clouds is a critical technique for robotic manipulation and bin picking tasks. Even though it has been steadily studied, it is still a challenging task for scenes with heavy occlusion and clutter. To copy with this intractable problem, this paper presents a fast and robust 3D object recognition framework, especially an efficient high compatibility correspondence grouping (HCCG) technique achieved by the correspondence ranking, clustering and expanding operations. Utilizing the HCCG technique, we first cluster the compatible correspondences into several high compatibility correspondence groups. Then, for each group, a 6DoF pose hypothesis is generated by using a point pair feature constraints (PPFCs)-based outlier removal module and a local reference frame (LRF)-based pose estimation algorithm. Finally, a robust pose verification operator is carried out to reject the false positive pose hypotheses and pick up the correct target object pose. Our approach is not only suitable for the multi-object recognition task, but also inherently yields the capability of detecting multiple instances thanks to the efficient HCCG technique. Extensive experiments on four challenging datasets show that our approach yields efficient and timely solutions and its advantages are further verified by comparing with the latest methods.

Realtime Single-Shot Refinement Neural Network With Adaptive Receptive Field for 3D Object Detection From LiDAR Point Cloud

Object detection plays an important role in autonomous driving systems. LiDAR is widely used in autonomous driving vehicles and robots as a sensor for environmental perception. Recently, the development of computational power and deep learning technology makes it possible to classify and locate objects from LiDAR point cloud in a single end-to-end learnable network. However, objects are sparsely distributed in large point cloud field, and are always been partly scanned by LiDAR, which pose a challenge for accurate and rapid object positioning and classification from the raw point cloud. In this paper, we introduce a new single-shot refinement neural network for fast and accurate 3D object detection from the raw LiDAR point cloud. Firstly, we exploit self-attention mechanism in main object detection branch to enhance object feature representation. Secondly, we apply deformable convolution for learning adaptive receptive fields to fully capture the features of rotating and partially visible objects. Thirdly, an object refinement branch is introduced to produce a finer regression of objects upon the primary estimation from the main detection branch. All proposed modules have been proven to effectively improve the accuracy of object detection. Our method is evaluated on KITTI 3D detection benchmark and achieves state-of-the-art results while maintains real-time efficiency. Furthermore, real-time test in autonomous driving vehicle demonstrates that our method is robust to 16 channels LiDAR and can meet the demands of accuracy, efficiency, and visibility of object detection in various urban scenarios.

Fast 3D texture-less object tracking with geometric contour and local region

The reprojection process is a crucial step in 3D object tracking, which is used to obtain the object’s 3D points to optimize pose. Although multiple reprojections can significantly improve the accuracy, it is also a time-consuming process, especially when reading the depth buffer from video memory. In this paper, we propose a fast 3D object tracking framework to improve the calculation efficiency, which reduces the reprojection process to one time while keeping the optimization accuracy. We first use geometric contour to estimate a coarse pose fastly, in which process we avoid the reprojection process by utilizing the object’s geometric properties. Then we use the local region to refine the pose precisely while reducing the reprojection process to one time. We further propose a faster version, which can achieve 40 FPS on a mobile phone by sacrificing some accuracy. Experiments show that the proposed method can achieve results comparable to the state-of-the-art methods, while the efficiency has been nearly doubled, and our faster version can track stably on mobile phones.

Fast and Accurate 3D Object Detection for Lidar-Camera-Based Autonomous Vehicles Using One Shared Voxel-Based Backbone

Currently, many kinds of LiDAR-camera-based 3D object detectors have been developed with two heavy neural networks to extract view-specific features, while a LiDAR-camera-based 3D detector with only one neural network has not been implemented. To tackle this issue, this paper first presents an early-fusion method to exploit both LiDAR and camera data for fast 3D object detection with only one backbone, achieving a good balance between accuracy and efficiency. We propose a novel point feature fusion module to directly extract point-wise features from raw RGB images and fuse them with their corresponding point cloud with no backbone. In this paradigm, the backbone that extracts RGB image features is abandoned to reduce the large computation cost. Our method first voxelizes a point cloud into a 3D voxel grid and utilizes two strategies to reduce information loss during voxelization. The first strategy is to use a small voxel size (0.05m, 0.05m, 0.1m) in X-axis, Y-axis, and Z-axis, respectively, while the second one is to project the feature (e.g. intensity or height information) of point clouds onto RGB images. Numerous experiments evaluated on the KITTI benchmark suite show that the proposed approach outperforms state-of-the-art LiDAR-camera-based methods on the three classes in 3D performance (Easy, Moderate, Hard): cars (88.04%, 77.60%, 76.23%), pedestrians (66.65%, 60.49%, 54.51%), and cyclists (75.87%, 60.07%, 54.51%). Additionally, the proposed model runs at 17.8 frames per second (FPS), which is almost 2× faster than state-of-the-art fusion methods for LiDAR and camera.

A fast 3D object recognition algorithm using plane-constrained point pair features

The point pair feature (PPF) algorithm is one of the best-performing 3D object recognition algorithms. However, the high dimensionality of its search space is a disadvantage of this algorithm. This high dimensionality means the feature matching process contains a large number of uninformative features, which reduces recognition speed. To solve this problem and improve the object recognition speed, this paper proposes a fast 3D object recognition algorithm based on the plane-constrained point pair features. By utilizing the property of the coplanar point pair features and the characteristics of the object placement plane, the proposed algorithm extracts the object placement plane through convex hull area calculation, eliminates irrelevant point pair features, and then performs object recognition with the reduced point pair feature descriptors for the feature matching. Experimental results demonstrate that the proposed algorithm significantly reduces the number of feature descriptors and accelerates the recognition speed of 3D objects in a complex background. Compared to the original point pair feature algorithm, the proposed method can achieve better performance and efficiency for 3D object recognition.

A Proposal of 3D Feature Based on Occupancy of Point Cloud in Multiscale Shell Region

SUMMARYIn this paper, we propose a novel keypoint detection and feature description method called “SHORT” (Shell Histograms and Occupancy from Radial Transform) for fast 3D object recognition. Conventional keypoint detection and feature description methods such as the SHOT method have been necessary to calculate many normal vectors or other statistical values from the point cloud data in local regions, and therefore its computational costs are expensive. By contrast, the SHORT method consists of a fast keypoint detector that does not calculate statistics and a fast feature descriptor that uses a small number of points in the restricted local regions. The keypoint detector uses the occupancy measure which can be estimated by only counting the number of points in multiple spherical shell regions. Also, the feature descriptor uses a small number of points included in distinctive shell regions of multiple scales. Experimental results in 3D object recognition using real dataset show that the processing speed of the proposed method is approximately nine times faster than that of comparative methods.

DEPTH CAMERAS ON UAVs: A FIRST APPROACH

Abstract. Accurate depth information retrieval of a scene is a field under investigation in the research areas of photogrammetry, computer vision and robotics. Various technologies, active, as well as passive, are used to serve this purpose such as laser scanning, photogrammetry and depth sensors, with the latter being a promising innovative approach for fast and accurate 3D object reconstruction using a broad variety of measuring principles including stereo vision, infrared light or laser beams. In this study we investigate the use of the newly designed Stereolab's ZED depth camera based on passive stereo depth calculation, mounted on an Unmanned Aerial Vehicle with an ad-hoc setup, specially designed for outdoor scene applications. Towards this direction, the results of its depth calculations and scene reconstruction generated by Simultaneous Localization and Mapping (SLAM) algorithms are compared and evaluated based on qualitative and quantitative criteria with respect to the ones derived by a typical Structure from Motion (SfM) and Multiple View Stereo (MVS) pipeline for a challenging cultural heritage application.

Optimal Local Searching for Fast and Robust Textureless 3D Object Tracking in Highly Cluttered Backgrounds

Edge-based tracking is a fast and plausible approach for textureless 3D object tracking, but its robustness is still very challenging in highly cluttered backgrounds due to numerous local minima. To overcome this problem, we propose a novel method for fast and robust textureless 3D object tracking in highly cluttered backgrounds. The proposed method is based on optimal local searching of 3D-2D correspondences between a known 3D object model and 2D scene edges in an image with heavy background clutter. In our searching scheme, searching regions are partitioned into three levels (interior, contour, and exterior) with respect to the previous object region, and confident searching directions are determined by evaluating candidates of correspondences on their region levels; thus, the correspondences are searched among likely candidates in only the confident directions instead of searching through all candidates. To ensure the confident searching direction, we also adopt the region appearance, which is efficiently modeled on a newly defined local space (called a searching bundle). Experimental results and performance evaluations demonstrate that our method fully supports fast and robust textureless 3D object tracking even in highly cluttered backgrounds.

Fast and accurate 3D object recognition directly from digital holograms

Pattern recognition methods can be used in the context of digital holography to perform the task of object detection, classification, and position extraction directly from the hologram rather than from the reconstructed optical field. These approaches may exploit the differences between the holographic signatures of objects coming from distinct object classes and/or different depth positions. Direct matching of diffraction patterns, however, becomes computationally intractable with increasing variability of objects due to the very high dimensionality of the dictionary of all reference diffraction patterns. We show that most of the diffraction pattern variability can be captured in a lower dimensional space. Good performance for object recognition and localization is demonstrated at a reduced computational cost using a low-dimensional dictionary. The principle of the method is illustrated on a digit recognition problem and on a video of experimental holograms of particles.

GPU Based Fast 3D-Object Modeling with Kinect

摘要: 便宜的物体快速三维建模技术是当前计算机视觉领域重要的研究课题.给出了一种基于Kinect传感器的快速物体重建方法,以及基于该方法的一种图形处理器 (Graphic processing unit, GPU)原型系统实现.本文方法主要分为两步: 1)系统的初始标定; 2)全自动的物体重建.对于系统初始标定,提出了一种简单易用的粗标定方法;对于物体重建,提出一种全自动的快速物体重建方法.本文方法鲁棒性高,在出现点云配准错误时仍然能够稳定地得到较理想的重建模型.针对环闭合(Loop-closure)问题,提出了一种全局的点云配准方法.对几类物体的重建实验结果表明,本文方法方便实用,且能得到较好的重建效果.此外,本文还探索了有遮挡物体的重建问题.将本文方法应用于有遮挡物体的重建,也取得了较好的重建效果. 关键词: 三维物体建模 / 图形处理器 / Kinect / 遮挡问题 / 环闭合

High performance bio-image database retrieval using MPI

Fast and accurate 3D object reconstruction and partial 3D component retrieval from 2D image slices represent a difficult and challenging problem. To group related objects on different layers in an image stack, image segmentation and sequential matching of adjacent 2D objects have to be preformed. Object matching involves heavy computing and is time consuming. In this paper, we propose a new approach for parallel implementation of object contour matching and partial 3D component retrieval based on image contour structure. The method has been implemented in MPI on a SGI Origin 2000 machine. The experimental results show a good speedup for sequential object matching and partial 3D component retrieval.

Rethinking Robot Vision – Combining Shape and Appearance

Equipping autonomous robots with vision sensors provides a multitude of advantages by simultaneously bringing up difficulties with regard to different illumination conditions. Furthermore, especially with service robots, the objects to be handled must somehow be learned for a later manipulation. In this paper we summarise work on combining two different vision sensors, namely a laser range scanner and a monocular colour camera, for shape-capturing, detecting and tracking of objects in cluttered scenes without the need of intermediate user interaction. The use of different sensor types provides the advantage of separating the shape and the appearance of the object and therefore overcome the problem with changing illumination conditions. We describe the framework and its components of visual shape-capturing, fast 3D object detection and robust tracking as well as examples that show the feasibility of this approach.

RECOGNITION AND 6DOF LOCALISATION OF PARTS FOR LOTSIZE1 AUTOMATION

The paper gives an overview of a novel innovative vision system embedded in manufacturing applications. The system is able to handle (1) unknown products, (2) in-available data, (3) large disturbances and deviations (shape, position, or even part type) and (4) even very small lot-sizes. The concept foresees a full integration into production technology. Products to be processed are past through a flexible 3D sensor cell. The proposed system adapts its sensor system to arbitrary product geometry and builds a 3D world model of the task space using a fast and robust 3D object recognition system.

INTEGRATION OF GEOMETRIC AND NON-GEOMETRIC ATTRIBUTES FOR FAST OBJECT RECOGNITION

Both man-made and natural objects are described by both their geometric shapes and by their non-geometric attributes such as color. The objective of the proposed research is to create a system which integrates geometric and non-geometric attribute information for fast 3D model-based object recognition. Hashing is employed in a hypothesize and verify approach to the 3D model-based object recognition problem. Viewpoint independent attributes are used in the hypothesis generation stage to eliminate model objects from consideration during hypothesis formation. Utilizing more than one attribute in the proposed hashing scheme helps to ensure a reduction in the actual execution time for object recognition over a larger number of model bases. A nice feature of the system is that new object attributes can be added with relative ease. Issues concerning the ranking of attributes by their distinctiveness with respect to the objects in the model base are discussed. Current work on the use of an information theoretic measure in the design of an optimal multiple attribute hash table is presented.