A SYNERGISTIC MODEL FOR REPRESENTING AND INTERPRETING HUMAN ACTIVITIES AND EVENTS FROM VIDEO

Abstract

This paper describes a new and innovative and approach for representing, recognizing and interpreting human activity from video, contributing to an automated system capable of recognition of complex human behaviors. This technology is directly applicable for monitoring public safety and law enforcement, and capturing of activities is crucial for supporting virtual collaborations between citizens, and between citizens and government. Digital video stores of terabytes are now common, and will continue to increase until they dominate stored data. The government of the future will have to manage, organize, recall, and interpret information from this resource, This paper addresses one important facet of this. The approach presented here is a model based on the hierarchical synergy of three other models: the Local/Global (L-G) graph, the Stochastic Petri Net (SPN) graph and a neural network (NN) model. The application focus is the description of activity of actors in a video (or multi-sensor) scene, from the snapshot state description through higher levels of organization into events. The concept of importance is the distinction and interaction between structural knowledge, or knowledge about physical state, and functional knowledge, knowledge about change and events. The L-G graph provides a powerful description of the structural image features presented in an event, and the SPN model offers a description of the functional behavior. The NN (or other adaptive) model provides the capability of leaning behavioral patterns for classification of posture and activity, and forecasting possible events in a free environment.