A framework for object recognition in a visually complex environment and its application to locating address blocks on mail pieces

Abstract

A computational framework for recognizing an object of interest in a complex visual environment is developed. Arising from the problem of finding the destination address block on a mail piece, a general framework for coordinating a collection of specialized image analysis tools is described. The framework is based on the blackboard model of problem-solving with enhancements to account for object recognition in a visually complex environment. Those enhancements include (i) a systematic derivation of knowledge sources that describes a methodology for developing knowledge sources in an application domain, (ii) a dependency graph to specify the interdependency between knowledge sources, (iii) a control mechanism that detects the varying degrees of complexity in the input, and adjust the behavior of the system so that the effort expended in gathering evidence is in accordance with the complexity of inputs, (iv) a formalism for estimating the utility of a tool so that the effective knowledge sources get high priorities to be invoked, (v) the organization of a knowledge source which allows the image analysis related knowledge sources to be reapplied on unsatisfactory initial results, and (vi) an evidence combination scheme to combine confidence values associated with pieces of evidence generated by various knowledge sources. An address block location system (ABLS) is developed and implemented in accordance with this framework and shown to be capable of dealing with a wide range of environments: from those having a high degree of global spatial structure (e.g., letter mail envelopes which conform to specifications) to those with no structure (e.g., magazines with randomly pasted address labels). Experimental results with a database of difficult cases demonstrating the promise of the computational framework are presented.