Temporal Consistency Among Five-Sensory Events: Application to Multimedia Technology

Abstract

While transmitting multi or five- sensory information after transduction, the concept of temporal consistency becomes an important issue at the receiving end from the point of view of context. Any violation of temporal consistency among the different decoded sensory information can lead to a failure of context, thereby making the multisensory system ineffective in creating unified percept and bringing in reality to any scene. Therefore, when a multisensory information is captured, transduced and the resulting convergent (fused or mashed) information is transmitted as a multimedia information, and again reverse transduced and interpreted at the receiving end, the temporal constraints among the different sensory responses need to be consistent and preserved. Since a multimedia message containing fused five-sensory information describes patterns of occurrence of events in the real world, the invocation of a partially ordered sequence of such messages in a given context should take us from one consistent state to another consistent state and checking for context violations. In this paper we describe a cooperating-agents based algorithm to detect temporal consistency among the five sensory events and for multimedia five-sensory processing. The theoretical formulation requires elementary principles of temporal logic. The cooperative aspects of the agent-based algorithm is illustrated using consistent and inconsistent examples and the UML activity diagram. This algorithm will be very useful for applications in Web 2.0 location–based context awareness in multimedia technology.