Stitching of depth and color images from multiple RGB-D sensors for extended field of view

Changquan Ding,Hengyu Li,Hang Liu

doi:10.1177/1729881419851665

Changquan Ding, Hengyu Li + Show 1 more

Open Access

https://doi.org/10.1177/1729881419851665

Copy DOI

Abstract

Existing commodity RGB-D sensors have a limited field of view compared with a nearly 360° horizontal field of view of Lidar. This article presents a method to extend the field of view by stitching depth and color images from multiple RGB-D sensors to form a depth and a color panorama. Firstly, a set of 3-D matched key points are constructed based on 2-D matched key points and corresponding depth values to estimate transformation matrices between multiple RGB-D sensors. Then, depth and color images are projected to the 3-D space using estimated transformation matrices to construct textured 3-D points. Finally, projecting textured 3-D points to a compositing surface can create the depth panorama and the color panorama. Experiments validated the accuracy and efficiency of the proposed method.

Highlights

LIDAR is a 3-D scanning device with a 360 field of view (FOV) that measures the distance to a target by illuminating the target with pulsed laser light and measuring the reflected pulses with a sensor
The depth sensor of the RGB-D sensor consists of an infrared (IR) projector combined with an IR camera
We propose a novel method to stitch depth and color images from multiple RGB-D sensors to form depth and color panorama to extend the FOV of a single RGB-D sensor

Summary

Introduction

LIDAR is a 3-D scanning device with a 360 field of view (FOV) that measures the distance to a target by illuminating the target with pulsed laser light and measuring the reflected pulses with a sensor. The IR projector projects a set of IR dots in 3-D space and the IR camera observes each dot and matches it with a dot in the known projector pattern to obtain a depth image. These RGB-D sensors are low cost and can provide a depth image with a video graphics array (VGA) resolution (640 Â 480) at video rate (e.g. 30 Hz) using only the on-board calculation unit.

Methods

Results

Conclusion