Abstract
The multimodal fusion for natural human-computer interaction involves complex intelligent architectures which are subject to the unexpected errors and mistakes of users. These architectures should react to events occurring simultaneously, and possibly redundantly, from different input media. In this paper, intelligent agent-based generic architectures for multimedia multimodal dialog protocols are proposed. Global agents are decomposed into their relevant components. Each element is modeled separately. The elementary models are then linked together to obtain the full architecture. The generic components of the application are then monitored by an agent-based expert system which can then perform dynamic changes in reconfiguration, adaptation, and evolution at the architectural level. For validation purposes, the proposed multiagent architectures and their dynamic reconfiguration are applied to practical examples, including a W3C application.
Highlights
With the growth in technology, many applications supporting more transparent and flexible human-computer interactions have emerged
If the time proximity between a word event and a click event is below the variable ProxyTime and if these two events verify the grammatical and semantic conditions, these two events are fused into one command
New agent-based architectural paradigms for multimedia multimodal fusion purposes are proposed. These paradigms lead to new generic structures unifying applications based on multimedia multimodal dialog
Summary
With the growth in technology, many applications supporting more transparent and flexible human-computer interactions have emerged. Each application is based on a dialog architecture combining modalities to match and elaborate on the relevant multimodal information. The architecture obtained is refined either by adding details or by decomposing components or connectors (recursively, through modeling, choice of a style, instantiation, and refinement) This process should result in an architecture which is defined, abstract, and reusable. This interface uses the fusion engine modeled with the CPN scheme
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
