On the application of Bayes networks to semantic understanding of consumer photographs

Abstract

Belief networks, such as Bayes nets, have emerged as an effective knowledge representation and inference engine in artificial intelligence and expert systems research. Their effectiveness is due to the ability to explicitly integrate domain knowledge in the network structure and to reduce a joint probability distribution to conditionally independence relationships. Current research in content-based image processing and analysis is largely limited to low-level feature extraction and classification. The ability to extract both low-level and semantic features and perform knowledge integration of different types of features would be very useful. We present a general knowledge integration framework that incorporates Bayes networks and has been used in two applications involving semantic understanding of consumer photographs. The first application aims at detecting main photographic subjects in an image and the second aims at selecting the most appealing image in an event. With these diverse examples, we demonstrate that effective inference engines can be built according to specific domain knowledge and available training data to solve inherently uncertain vision problems.