The Finger-Based Interactive Projection using a Monocular Camera

Abstract

Ordinary projection screen is not sensitive to interaction, it cannot meet the demands of teaching, virtual reality, and other applications. Due to the fact that people always use hands to complete a variety of human–computer interaction, the finger-based interactive projection technology is worth being researched. In this paper, an ordinary monocular camera is used to acquire video frame on projection screen, and the touch signal of finger in frame is used as the input of interactive projection system. Because the differences between spatial frequency of common digital camera and the projection screen is small, the frame obtained from camera will contain moire fringe, which needs to be filtered in image frequency domain. Then the difference between current frame edge and previous frame edge is calculated to obtain moving object edge clues. According to these clues, the most possible contour curve is searched in current frame edge, and the curve is fitted by polynomial approximation method. Its curvature integration is used to match with the curvature integration of finger template curve. After that the fingers in the curve are recognized. Because color information is not needed, this method can be used to recognize gloved fingers. Finally, finger shadow is used to judge whether the finger touches projection screen to complete interactive process. The experiments of writing and collaboratively rotating picture on projector screen show that this method can effectively complete interactive operation with the projection screen and can realize the multi-user operation.