This paper presents a real-time video-based pointing method that allows the sketching and writing of English text over air in front of a mobile camera. The proposed method has two main tasks. First, it tracks the colored fingertip in the video frames and then applies English optical character reorganization over the plotted images to recognize the written characters. Moreover, the proposed method provides a natural human–system interaction without requiring a keypad, stylus, pen, glove, or any other device for character inputs. For the experiments, an application was developed using OpenCv with the JAVA language. The proposed method was tested on a Samsung Galaxy3 android mobile phone. The proposed algorithm showed an average accuracy rate of 92.083% when tested for different shaped alphabets. Here, more than 3000 different shaped characters were used. The proposed system is a software-based approach and is relevantly very simple, fast, and easy to use. It does not require sensors or any hardware other than a camera and red tape, and is applicable to all disconnected languages. It has one issue, though it is color-sensitive in that the existence of any red-colored object in the background before the start of and during the character writing can lead to false results.
Read full abstract