Digitization of Hand-Written Notes Using a Wearable Camera

Abstract

In taking hand-written note, the taking process plays important roles for recalling related information in addition of notations (words and figures) in the note. In this paper, we focus on the process through taking hand-written note. First, introduce an image processing method for the deduction of the position of pen head by using a wearable type camera. Next, we propose a method for the transformation of the physical position of the pen head into the logical position based on behaviors by students. Here, the logical position denotes the line and character numbers in the hand-written note. Moreover, we detect the face movement by using the block-matching method. In the progress of digital devices such as tablet type PC, we can easily input information. However, it is important for human to take notes and memorandum in everyday life such as learning, recording and communication. Especially, students have to receive information (characters, speech and images) from teachers and audio-visual devices in the educational opportunity. Moreover, hand-written notes and memorandum play important roles for understanding of contents and recalling related information in lectures. In this case, students have to process information efficiently and write summarized information on their notes. Moreover, Kishi et al. have analyzed the relations between note taking and scores on tests by students and discussed the importance of the process of note taking in lectures (1). However, we have to consider analog type data (hand-written note) to digitize in consideration of the usage in computers and digital devices efficiently. Therefore, various types of digital pens have been developed and many systems for digital pens have been proposed and discussed (2), (3). In this paper, we discuss the extraction of the taking process of hand-written note and discuss the digitization of hand-written notes by a wearable camera. Here, we propose an image processing method for the extraction of the process of taking the note by using a sunglasses type camera. Concretely, the position of pen head can be detected by image processing based on images recorded by a sunglasses type camera. Next, we propose an image processing method for the analysis of hand-written note (4). Authors have been already proposed an image processing method for the detection of the pen head (5). In this method, the position of the note corner is utilized for the compensation of the position of the pen head. However, it is difficult to get the position of the note corner in real case by a wearable camera. Therefore, in this paper, we transform the physical position of the pen head into the logical position by using the detection of the face movement. The face movement can be detected by the block-matching method. Here, the logical position denotes the line and character numbers in the hand-written note. Finally, we show the experimental results for a few hand-written notes.