Open issues in model transformations for multimodal applications

Abstract

Multimodal human–computer interaction refers to the interaction with the virtual and physical environment through natural modes of communication. Multimodal input user interfaces have significant role in different domains, for example industrial plants or hospitals, also they have implications for accessibility. To develop multimodal applications in formally rigorous settings, software developer teams may use tools or a software development kit to increase the efficiency and the quality of the resulted software artifacts. Such a technique is performing the design with software modeling and applying model transformations to generate well-defined components of the software. Furthermore, representation-bridging communication is a discipline of cognitive infocommunications, where the sensory information transferred to the receiver entity is filtered and/or converted. Whenever such approaches are used, the challenges associated with the modeling of information requirements, user capabilities and cross-model interactions are compounded and further increase the need for formal design and verification tools. Applying model transformations is a way to support this activity. Communication-intensive solutions often require complex methods, i.e. significant model transformation efforts between the different representations. Important semantic information should be preserved and not misinterpreted in a complex model transformations. Therefore, methods are required to verify that the semantics used during the application generation and analysis are indeed preserved across the transformation. As a case in point, such a model transformation could yield embedded code for a given type of electronic driver assistant system based on a high-level characterizations of the information to be transferred and the driver’s cognitive capabilities. Later, a multimodal interactions expert could easily modify those characterizations on demand, and regenerate a modified version of the software without having to know about the low-level details of the embedded platform. This paper provides a strong motivation regarding the necessity of methods to support verification and validation of model transformations supporting multimodal application development and cognitive infocommunications. As the main result of the paper, we compile a list of open issues in the field of verification/validation of model transformations, and link those issues to the development of multimodal interfaces. Through its discussions, the paper makes the point that the design practices behind multimodal interfaces could strongly benefit from the use of formal modeling techniques in general, and model transformation approaches in particular.