Edge extraction by merging 3D point cloud and 2D image data

Abstract

Edges provide important visual information by corresponding to discontinuities in the physical, photometrical and geometrical properties of scene objects, such as significant variations in the reflectance, illumination, orientation and depth of scene surfaces. The significance has drawn many people to work on the detection and extraction of edge features. The characteristics of 3D point clouds and 2D digital images are thought to be complementary, so the combined interpretation of objects with point clouds and image data is a promising approach to describe an object in computer vision area. However, the prerequisite for different levels of integrated data interpretation is the geometric referencing between the 3D point cloud and 2D image data, and a precondition for geometric referencing lies in the extraction of the corresponding features. Addressing the wide-ranged applications of edge detection in object recognition, image segmentation and pose identification, this paper presents a novel approach to extract 3D edges. The main idea is combining the edge data from a point cloud of an object and its corresponding digital images. Our approach is aimed to make use of the advantages of both edge processing and analysis of point clouds and image processing to represent the edge characteristics in 3D with increased accuracy. On the 2D image processing part, an edge extraction is applied on the image by using the Canny edge detection algorithm after the raw image data pre-processing. An easily-operating pixel data mapping mechanism is proposed in our work for corresponding 2D image pixels with 3D point cloud pixels. By referring to the correspondence map, 2D edge data are merged into 3D point cloud. On the point cloud part, the border extracting operator is performed on the range image. As a preparation work, the raw point cloud data are used to generate a range image. Edge points in the range image, points with range, are converted to 3D point type with the application of the Point Cloud Library (PCL) to define the edges in the 3D point cloud.