What, Where and Who? Telling the Story of an Image by Activity Classification, Scene Recognition and Object Categorization

Abstract

AbstractWe live in a richly visual world. More than one third of the entire human brain is involved in visual processing and understanding. Psychologists have shown that the human visual system is particularly efficient and effective in perceiving high-level meanings in cluttered real-world scenes, such as objects, scene classes, activities and the stories in the images. In this chapter, we discuss a generativemodel approach for classifying complex human activities (such as croquet game, snowboarding, etc.) given a single static image.We observe that object recognition in the scene as well as scene environment classification of the image facilitate each other in the overall activity recognition task. We formulate this observation in a graphical model representation where activity classification is achieved by combining information from both the object recognition and the scene classification pathways. For evaluating the robustness of our algorithm, we have assembled a challenging dataset consisting real-world images of eight different sport events, most of them collected from the Internet. Experimental results show that our hierarchical model performs better than existing methods.KeywordsObject RecognitionObject ClassEvent CategoryForeground ObjectVisual WorldThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.