Natural Handwriting Research Articles

A Virtual Reality Whiteboard System for Remote Collaboration Using Natural Handwriting

The COVID-19 pandemic has increased the significance of remote collaboration. This study proposes a virtual reality (VR) whiteboard system that enables remote collaboration among multiple participants using natural handwriting. In total, three experiments were conducted to investigate, respectively, collaboration efficiency, user experience, and system delay. First, we compared the collaboration efficiency of the traditional whiteboard, the electronic whiteboard, and the proposed virtual reality whiteboard in a series of controlled experiments. It was discovered that the VR whiteboard significantly improves collaboration efficiency in comparison to the mouse-based electronic whiteboard and is comparable to the traditional whiteboard. Second, we assessed the user experience with a survey scale (questionnaires). The subsequent results demonstrate that the VR whiteboard provides a superior user experience in terms of efficiency, usability, etc., compared to the traditional whiteboard. We also measured an end-to-end latency of approximately 115 milliseconds, which is sufficient for remote collaboration.

On developing complete character set Meitei Mayek handwritten character database

This paper introduces a large-scale Meitei Mayek handwritten character database. It consists of the complete character set of the script. There are a total of 85,124 character images of 55 character classes with 72,330 and 12,794 images in training and test sets, respectively. The present work focuses on collecting the natural handwriting of individuals by carrying out sample collection in two phases: (a) unconstrained handwriting in the form of answer sheets and classroom notes and (b) tabular forms. A total of nearly 500 individuals have contributed in the development of the database. Recognition of the character images in the database is carried out using different feature descriptors with four popular classifiers, namely KNN, Linear Support Vector Classifier, Random Forest and Support Vector Machine. The paper also proposes a convolutional neural network (CNN) model by enhancing a base CNN architecture by optimally tuning the hyperparameters. Experimental results show that the CNN model can be benchmarked against the concerned database with a test accuracy of 95.56%.

Infrastructure for Long-term Preservation and OCR Analysis of Herbarium Images

Herbaria hold large numbers of specimens: approximately 22 million herbarium specimens exist as botanical reference objects in Germany, 20 million in France and about 500 million worldwide. High resolution digital images of these specimens take up substantial bandwidth and disk space. New methods of extracting information from the specimen labels have been developed using OCR (Optical character recognition) techniques, but the exploitation of this technology for biological specimens is particularly complex due to the presence of biological material in the image with the text, the non-standard vocabularies, alongside the variation and age of the fonts. Much of the information is handwritten and natural handwriting pattern recognition is a less mature technology than OCR. Today, our system (eTDR-European Trusted digital Repository) provides the OCR technology (using Tesseract software) adapted to the requirements of herbarium specimen images and requires minimal installation in each institution. This is what we propose to make available to botanists with our portal. The goal for a museum is to be able to submit a large number of scanned images easily to a long-term archiving system in order to automatically obtain OCR texts and retrieve them by a full text search on an open data portal. Most of the images are provided for reuse through CC-BY licenses. In each case, the rights of reuse associated with the data are specified in associated metadata. This pilot was an opportunity to test the long-term storage service eTDR provided by CINES. The services (B2SAFE, B2Handle) developed by EUDAT were used to facilitate the transfer of data to the storage repository and to provide indexing services for access to that repository. This workflow that has been tested for the european project ICEDIG is presented as a poster: See the document (Suppl. material 1).

Distinguishing Characteristics of Robotic Writing.

Advances in robotic handwriting technology create new challenges for forensic document examiners. In the past, devices such as the autopen were used to replicate signatures of government officials and corporate companies. In today's technology, companies such as Bond utilize robots to create written documents, which mimic natural patterns of handwriting. They generate customizable written samples by simulating pen movements and letter formations. Four forensic document examiners were given various reproduced questioned documents and utilized a modified ACE-V (analysis, comparison, evaluation, verification) methodology to determine their genuineness. Examiners were able to make a distinction between the human writing samples and the skilled robotic equivalents. Several distinct features that are not seen in natural handwriting, such as even pen pressure and the superimposition of letterforms were observed in the robotic samples. Careful examination of identifying features of the Bond produced documents resulted in an opinion of nongenuineness.

ELECTRONIC ONLINE HANDWRITING CHARACTER RECOGNITION SYSTEM USING ARDUINO PLATFORM

This paper proposes an online prototype approach of handwriting character recognition system using a microcontroller. The main contribution of this work is the simple method of translating a handwriting input from analog to a digital font using XOR bit-wise operation. This system recognizes characters and numbers in natural handwriting with a stylus. This electronic system is used to improve the technology from the use of a bulky keyboard system to a portable and convenient way which is more suitable for smaller electronic devices nowadays such as smart phone, tablet and etc. In order to construct the system, a Arduino Mega is used as the microcontroller along with the Arduino TFT Touch Shield and an LCD screen as the input and output respectively as the hardware components. For the software part, this system uses C language and Arduino software (IDE). The recognition result achieved is 80.0%.

Handwritten Urdu character recognition using one-dimensional BLSTM classifier

The recognition of cursive script is regarded as a subtle task in optical character recognition due to its varied representation. Every cursive script has different nature and associated challenges. As Urdu is one of cursive language that is derived from Arabic script, that is why it nearly shares the similar challenges and complexities but with more intensity. We can categorize Urdu and Arabic language on basis of its script they use. Urdu is mostly written in Nasta’liq style, whereas Arabic follows Naskh style of writing. This paper presents new and comprehensive Urdu handwritten offline database name Urdu-Nasta’liq handwritten dataset (UNHD). Currently, there is no standard and comprehensive Urdu handwritten dataset available publicly for researchers. The acquired dataset covers commonly used ligatures that were written by 500 writers with their natural handwriting on A4 size paper. UNHD is publically available and can be download form https://sites.google.com/site/researchonurdulanguage1/databases . We performed experiments using recurrent neural networks and reported a significant accuracy for handwritten Urdu character recognition.

Individual English Handwriting Synthesis

Handwriting is considered as an input method through pen-based or touch-based mean. Consequently, it is an unique feature preserving users’ individuality. Since, it is becoming more lively aspect of user interaction, it is a very facile and more theoretical measure to reproduce an individual’s cursive and noncursive English handwriting from ASCII transcription. Special input arrangement is designed to collect user’s natural handwriting. Then, the system depicts the individuality features and characteristics of anyone’s handwriting that machine learns afterwards. And at last, at a given set of instructions, for any set of ASCII value, user natural handwriting is synthesized hierarchically.

My Text in Your Handwriting

There are many scenarios where we wish to imitate a specific author’s pen-on-paper handwriting style. Rendering new text in someone’s handwriting is difficult because natural handwriting is highly variable, yet follows both intentional and involuntary structure that makes a person’s style self-consistent. The variability means that naive example-based texture synthesis can be conspicuously repetitive. We propose an algorithm that renders a desired input string in an author’s handwriting. An annotated sample of the author’s handwriting is required; the system is flexible enough that historical documents can usually be used with only a little extra effort. Experiments show that our glyph-centric approach, with learned parameters for spacing, line thickness, and pressure, produces novel images of handwriting that look hand-made to casual observers, even when printed on paper.

Character recognition based on non-linear multi-projection profiles measure

In this paper, we study a method for isolated handwritten or hand-printed character recognition using dynamic programming for matching the non-linear multi-projection profiles that are produced from the Radon transform. The idea is to use dynamic time warping (DTW) algorithm to match corresponding pairs of the Radon features for all possible projections. By using DTW, we can avoid compressing feature matrix into a single vector which may miss information. It can handle character images in different shapes and sizes that are usually happened in natural handwriting in addition to difficulties such as multi-class similarities, deformations and possible defects. Besides, a comprehensive study is made by taking a major set of state-of-the-art shape descriptors over several character and numeral datasets from different scripts such as Roman, Devanagari, Oriya, Bangla and Japanese-Katakana including symbol. For all scripts, the method shows a generic behaviour by providing optimal recognition rates but, with high computational cost.

Estimating Writing Neatness from Online Handwritten Data

Handwriting is the most fundamental expressive activity in learning. To utilize the intuitiveness and the nature of handwriting, digital pen technology has emerged to capture and transfer notes. We developed AirTransNote, a student note-sharing system that facilitates collaborative and interactive learning in conventional classrooms. A teacher can use the AirTransNote system to share student notes with the class on a projected screen immediately to enhance the group learning experience. However, to improve the effectiveness of sharing notes, the teacher must be able to select an effective note for sharing. This can be difficult and time consuming during a lecture. Moreover, students should be encouraged to improve the presentation of their handwritten notes. Well-written notes are more accessible for other students and reduce irrelevant and careless mistakes. To facilitate learning improvements based on note sharing, we require a method to estimate the neatness of a note automatically. If a method is established, the teacher can easily select effective notes. Furthermore, this method can help provide feedback to the student to improve their writing. We examined 14 basic features from handwritten notes by considering correlation coefficients and found that the variance of pen speed, angular point average, and pen speed average were the significant features for evaluating the neatness of handwritten notes.

Handwriting synthesis: classifications and techniques

Handwriting synthesis is the automatic generation of data that resemble natural handwriting. Although handwriting synthesis has recently gained increasing interest, the area still lacks a stand-alone review. This paper provides classifications for the different aspects of handwriting synthesis. It presents the applications, techniques, and evaluation methods for handwriting synthesis based on the several aspects that we identify. Then, it discusses various synthesis techniques. To the best of our knowledge, this paper is the only stand-alone survey on this topic, and we believe it can serve as a useful reference for the researchers in the field of handwriting synthesis.

ISWC 2012 best papers

This special issue of Pervasive and Ubiquitous Computing includes three invited papers that are extended versions of best paper-nominated submissions to the sixteenth International Symposium on Wearable Computers (ISWC), held from June 18–22, 2012, in Newcastle, UK. The first paper, entitled Airwriting: A Wearable Handwriting Recognition System by Christoph Amma, Marcus Georgi, and Tanja Schultz describes a system for the recognition of handwriting gestures in real time that allows users to write in free space in an equivalent manner to writing on a white board or wall. It includes a description of a wireless gloveworn device to sense writing movement along with a description and evaluation of the algorithms and language model used to predict handwriting. It will be of interest to researchers wanting to exploit gesturing in user interfaces by extending them to natural handwriting. The second paper, entitled The Distant N-Gram Topic Model: A Probabilistic Approach to Mining Sequences by Katayoun Farrahi and Daniel Gatica-Perez, describes a method and mathematical model for extracting or discovering data patterns from large data sets, for example those resulting from collecting sensor data over an extended period of time. This work will be of interest to those wishing to model activity or other behavioral patterns that can occur over long-measurement intervals under relatively unstructured conditions. The third paper, entitled Activity Recognition for Creatures of Habit: Energy-Efficient Embedded Classification using Prediction by Dawud Gordon, Jurgen Czerny, and Michael Beigl, was the best paper winner and describes a novel algorithm for classifying human activities reported by wearable sensors followed by a prediction of future actions. This algorithm can be used by applications to assist hardware power management, for example by activating sensors that are the most likely to be needed, while shutting off ones that are less likely to be used resulting in significant energy savings. ISWC is overseen by the IEEE Technical Committee on Wearable Information Systems (TC WIS). ISWC has been the leading forum for wearable computing since the conference was first held in 1997. For the last 2 years, the conference has been co-located with Pervasive, uniting the growing ubiquitous computing community. Through ISWC, related conferences such as ISMAR, and special issues of journals such as this issue of PUC, TC WIS has worked to bring together the diverse range of people who are interested in wearable computing, from computing researchers to apparel designers. TC WIS concentrates on all issues related to wearing information technology, such as user interfaces, low power hardware, and design for wearability. The TC steering committee invites anyone interested in wearable information systems to join the TC. The TC website (http://www.iswc.net) has instructions on joining the TC, as well as information on upcoming events and links to previous years of ISWC.

SIGNATURE SEGMENTATION FROM MACHINE PRINTED DOCUMENTS USING CONTEXTUAL INFORMATION

Automatic signature segmentation from a printed document is a challenging task due to the nature of handwriting of the signatory, overlapping/touching of signature strokes with printed text, graphics, noise, etc. In this paper, we propose a two-stage approach for signature segmentation from a document page. In the first stage, a document is segmented into blocks and then blocks are classified into two classes: signature block and printed word block. Gradient-based features are used for block feature extraction and support vector machine classifier is used for block-wise classification. In the second stage, printed characters that may be present in isolated form or overlapped/touched with signature part are removed from signature blocks. From each of the detected signature blocks, the isolated printed characters (if exist) are removed using context information. To detect overlapping/touching printed stroke in a signature block, at first some hypothetical zones are detected where possible overlapping/touching may occur. Bounding box information of neighboring printed word block and local linearity of character strings near the signature blocks are used to detect hypothetical zones. Next, to detect the overlapping/touching printed strokes in hypothetical zones of a signature block, the corner points of contours obtained by Douglas and Peucker polygonal approximation algorithm and skeleton junction points are used. Finally, the touching strokes of signature are separated from text characters using the contour smoothness information near skeleton junction points. The experiment is performed in "Tobacco-800" dataset [The legacy tobacco document library (ltdl), available at http://legacy.library.ucsf.edu/ , University of California, San Francisco, 2007.] and the results obtained from the experiment are promising.

Pen-Based Interaction Forms for Smarter Product Customization

AbstractThis paper proposes a novel digital system for ordering customized products in a convenient pen and paper setting. In particular we integrate pen-based interaction forms which automatically recognize natural handwriting. The integration of these forms in a factory environment describes a novel way of addressing orders to a producing facility beside usual ways like direct access or web-interaction forms.

Handwriting Evidence Evaluation Based on the Shape of Characters: Application of Multivariate Likelihood Ratios*,†

A novel Bayesian methodology has been developed to quantitatively assess handwriting evidence by means of a likelihood ratio (LR) designed for multivariate data. This methodology is presented and its applicability is shown through a simulated case of a threatening anonymous text where a suspect is apprehended. The shape of handwritten characters a, d, o, and q of the threatening text was compared with characters of the true writer, and then with two other writers, one with similar and one with dissimilar characters shape compared to the true writer. In each of these three situations, 100 draws of characters were made and the resulting distributions of LR were established to consider the natural handwriting variation. LR values supported the correct hypothesis in every case. This original Bayesian methodology provides a coherent and rigorous tool for the assessment of handwriting evidence, contributing undoubtedly to integrate the field of handwriting examination into science.

Towards robust writer verification by correcting unnatural slant

Slant is a salient feature of Western handwriting and it is considered to be an important writer-specific feature. In disguised handwriting however, slant is often modified. It was tested whether slant is indeed an important factor and it was tested whether the distorting effect of deliberate slant change can be countered by a simple shear transform. This was done in two off-line writer verification experiments in image processing conditions of slant elimination and slant correction. The experiments were performed using three features based on statistical pattern recognition, including the state-of-the-art features Fraglets and Hinge. A new public dataset was created and used, containing natural and slanted handwriting by 47 writers. A striking result is that the average natural slant value is much less important for biometric systems than is usually assumed: eliminating slant yields just a 1–5% performance loss. A second result is that the effects of deliberate slant change cannot be fully countered by a simple shear transform: it raises performance on the distorted handwriting from 53–68% to 64–90%, but this is still lower than normal operation on natural handwriting: 97–100%.

Supervised chromosome clustering and image classification

In this paper we propose handwritten signature classification using a supervised chromosome clustering technique. Due to the time variant nature of handwriting of human being, a set of hundred sample handwritten signatures were first collected from the user or individual in form of the same sized grayscale images. These grayscale handwritten signature images will be used as the training set in our classification algorithm. Our proposed algorithm will then decide whether the future incoming handwritten signature of an individual can be a member of the training set or not. In this paper, the distance and similarities play an important role, where the greater the dissimilarity measure or distance of genes, the more dissimilar are the two chromosomes.

Basic Test Framework for the Evaluation of Text Line Segmentation and Text Parameter Extraction

Text line segmentation is an essential stage in off-line optical character recognition (OCR) systems. It is a key because inaccurately segmented text lines will lead to OCR failure. Text line segmentation of handwritten documents is a complex and diverse problem, complicated by the nature of handwriting. Hence, text line segmentation is a leading challenge in handwritten document image processing. Due to inconsistencies in measurement and evaluation of text segmentation algorithm quality, some basic set of measurement methods is required. Currently, there is no commonly accepted one and all algorithm evaluation is custom oriented. In this paper, a basic test framework for the evaluation of text feature extraction algorithms is proposed. This test framework consists of a few experiments primarily linked to text line segmentation, skew rate and reference text line evaluation. Although they are mutually independent, the results obtained are strongly cross linked. In the end, its suitability for different types of letters and languages as well as its adaptability are its main advantages. Thus, the paper presents an efficient evaluation method for text analysis algorithms.

Persian/arabic handwritten word recognition using M-band packet wavelet transform

The extraction of rotation and scale invariant features is an essential problem in document image analysis. This paper proposes an effective rotation and scale invariant holistic handwritten word recognition scheme. This approach utilizes M-band packet wavelet transform to extract feature vector of Farsi word image. The global and local features extracted are exploited in recognition of limited-size lexicon of handwritten words. The rotation and scale invariant feature of a word image involves applying a polar transform to eliminate rotation and scale effects, but this produces M-row shifted polar image, which is passed to a row shift invariant M-band wavelet packet transform to eliminate the row shift effects. The output wavelet coefficients are rotation and scale invariant. For each subband of these wavelet coefficients a set of local energy features are computed and we extract feature vectors from the subbands of wavelet coefficients. The proposed polar M-band wavelet features have been tested by employing Mahalanobis algorithm to classify a set of distinct natural handwriting Farsi words. We compared the proposed scheme with two well-known rotation invariant methods; Fourier-wavelet and Zernike moments. The experimental results show that the proposed algorithm improves the recognition rate about 12 percents.

Implementation of an Intuitive Writing Interface and a Laparoscopic Robot for Gynaecological Laser Assisted Surgery

The research reported in this paper aims at applying the human handwriting skill to improve and facilitate the control of laser-assisted laparoscopic surgery operations performed by gynaecological surgeons. For the purpose, a laparoscopic robot was interfaced with a digitizing tablet. This interface, further called the intuitive writing interface (IWI), directly converts the hand trajectory, handwritten on the tablet, into an input signal to the robot. It replaces the traditional complex manipulations performed by the surgeon during manual laparoscopic surgery by natural handwriting. It provides the surgeon with an intuitive 'what-you-draw-is-what-you-cut' control facility by employing his/her familiar handwriting skills to control the laser ablation process accurately. The system was successfully built and tested in vitro. Performance tests on the robot resulted in tracking errors in the order of 1 mm in the target plane at an ablation speed of 20 mm/s. The high accuracy of the system was successfully demonstrated by cutting characters 4 mm high on an apple. These results indicate that laser ablation performance is upgraded by the IWI to the accuracy levels of human handwriting, which is much higher than can be obtained with manual laser laparoscopy. Safety features include the use of pen contact with the tablet as a safety switch, and back drivability in the robot joints for easy manual positioning and evacuation in case of emergency.