This article describes a real-time document image processing system. Its objective is to recognize names and addresses from scanned address block images extracted from various tax forms of the United States Internal Revenue Service. The Name and Address Block Reader (NABR) system accepts both machine- and hand-printed address block images as input. Salient aspects of the system are presented, including document analysis (connected component analysis, address block extraction, label detection, hand-print/machine-print discrimination) and document recognition. Document recognition is performed in two nonidentical streams for machine-and hand-print; key steps are address parsing, character recognition, word recognition, and postal data base lookup (ZIP+4 and city-state-ZIP files). System output is a packet containing the results of recognition together with data base access status file. Real-time throughput (8500 forms/h) is achieved by employing a loosely coupled multiprocessing architecture where successive input images are distributed to available address recognition processors. The functional architecture, software design, system architecture, and the hardware implementation are described. Performance evaluation on machine- and hand-written addresses are presented. © 1996 John Wiley & Sons, Inc.