Neural networks provide robust character recognition for Newton PDAs

Abstract

While online handwriting recognition is an area of long standing and ongoing research, the recent emergence of portable, pen based computers (personal digital assistants, or PDAs) has focused urgent attention on usable, practical solutions. Pen based PDAs depend wholly on fast and accurate handwriting recognition, because the pen serves as the primary means for inputting data to the devices. To meet this need, we have combined an artificial neural network (ANN) character classifier with context driven search over character segmentation, word segmentation, and word recognition hypotheses to provide robust recognition of hand printed English text in new models of Apple Computer's Newton MessagePad. Earlier attempts at handwriting recognition used strong, limited language models to maximize accuracy. However, this approach failed in real world applications, generating disturbing and seemingly random word substitutions known within Apple as "The Doonesbury Effect". We have taken an alternative approach, using bottom up classification techniques based on trainable ANNs, in combination with comprehensive but weakly applied language models. By simultaneously providing accurate character level recognition, via the ANN, with dictionaries exhibiting very wide coverage of the language, plus the ability to write entirely outside those dictionaries, we have produced a hand print recognizer that some have called the first usable handwriting recognition system.