A Real-Time Stereo Vision System for Obstacle Detection and Recognition

Abstract

Obstacle detection and recognition are important topics for applications in autonomous navigation and robotics. In this research, we employed a stereo vision system composed of dual cameras, and developed efficient algorithms for obstacle detection and recognition. The dual cameras are mounted on a specially designed mechanism to satisfy the geometric restrictions of ideal stereo vision system with parallel optical axes. With the current design and when the distortion of images due to camera lens is corrected by calibration, the disparity image can be estimated by the correspondence matching method; thus the distance and three-dimensional information of obstacles can be obtained in real-time. To detect and locate the obstacles, the disparity image is projected onto a non-linear top-view map for blob segmentation of possible obstacles. Pre-defined three-dimensional constraints are then imposed to filter noisy blobs and thus to accurately detect obstacles. Following the detection of obstacles, the support vector machine (SVM) method is used to classify the obstacle into categories of small agricultural vehicle, large agricultural vehicle, pedestrian, and unknown object. To enhance the obstacle detection rate and object recognition rate, an algorithm incorporating inter-frame information in a video sequence was also developed. The stereo vision system was tested in various conditions in real outdoor environments. The developed stereo vision system operates at a speed of 10 frames per second processing video frames of 640x480 resolution. The overall obstacle detection rate was above 90%. The error of distance estimation of obstacles in the range between 2.5 ~ 20 m was within 1.8%. The error of height estimation increased with the distance of the obstacle from the dual cameras. For obstacles in the range between 2.5 ~ 20 m, the error of height estimation was within 1.8%.