Larry Yaeger Handwriting Recognition Technology in the Newton's Second Generation 'Print Recognizer' (The One That Worked)

While on-line handwriting recognition is an area of long-standing and ongoing research, the emergence of PDAs and pen-based computers focused urgent attention on usable, practical solutions. Despite its widely heralded early difficulties, the second generation of Apple Computer's Newton PDA is widely acclaimed for having delivered the world's first truly functional handwriting recognition system.

In a broad and deep presentation of the overall recognition framework, I will discuss the combination of artificial neural network (ANN) character classifier, large-coverage, loosely-applied language model, and context-driven search over character segmentation, word segmentation, and word recognition hypotheses that make the Newton's "Print Recognizer" (and Mac OS X's "Inkwell") actually work. I will also discuss issues related to recognizer training, generalization, segmentation, language models, geometric context, probabilistic formalisms, etc., that had to be addressed and resolved in order to obtain sufficient recognition accuracy to be genuinely usable. Additionally, I will present some unique innovations in the application of ANNs to the problem of character classification for word recognition, including multiple representations, normalized output error, negative training, stroke warping, frequency balancing, and error emphasis, all of which are interpretable as methods for reducing prior biases in the training data, and all of which significantly improve the performance of the network classifier when embedded in such a system. Other practical issues, such as quantized (one-byte) weights for the neural network and dictionary stemming will also be addressed.

Some small insight will be offered into why the Newton's first generation recognition failed in the ways it did, producing the so-called "Doonesbury" or "egg freckles" effect, and contrasted to second and subsequent generation recognition efforts.

Slides

Article: Combining Neural Networks and Context-Driven Search for On-Line, Printed Handwriting Recognition in the Newton, Yaeger, L. S., Webb, B. J., Lyon, R. F., AI Magazine, AAAI, 19:1 (Spring 1998) p73-89

Larry Yaeger is the father of Rosetta recognizer (now code-named Inkwell), which he developed while working within Apple's Advanced Technology Group, the world's first truly usable handwriting recognizer.

Larry Yaeger is an invited speaker of the Worldwide Newton Conference and a honorary member of the Worldwide Newton Association.