Robust Elevator Button Recognition in the Presence of Partial Occlusion and Clutter by Specular Reflections

Abstract

This paper deals with vision-based elevator button recognition for a robot arm manipulating elevator buttons. The major difficulties in elevator button recognition are the presence of partial occlusion of the target objects and image clutter caused by specular reflection from mirrorlike walls inside the elevator. As a remedy for the elevator button recognition problem in highly complicated settings, we propose a robust button recognition algorithm, which is modularized into feature extraction, initial recognition, and postrefinement modules. In consideration of the diverse button shapes in the form of convex quadrilaterals with rounded button corners, a set of features is specially designed to describe each button contour subject to perspective distortion. A homography-based image transform is also employed for the template matching to achieve a more reliable matching performance. Like a grammar in linguistics, geometric button alignment, plays a key role in the elimination of false positives and the estimation of the missing buttons. The proposed algorithm is extensively tested in a robotic platform to verify effectiveness, robustness, and real-time performance.