Towards Multimodal Information Systems for Assisting Humans in Production and Logistics Processes

Abstract

Information support systems for coping with and mastering increasing process complexity are becoming a success factor for many companies. The information systems available today are predominantly implemented as mobile applications and, as the range of functions grows, place increasingly high demands on the user interface. To reduce the cognitive load, these are tailored to the user and are limited in functionality or require special training, which is costly and impractical for some user groups. Current voice interaction interfaces offer advantages in this context, but the communication of data to the user can be done more meaningfully and clearly in graphical or multimodal form. Furthermore, both interaction modes neglect essential sources of information in industrial and logistics use cases, such as available information about documents and physical properties of objects. Methodologically, this contribution proceeds as follows: First, a literature review was conducted on Cognitive Load Theory, Human-Computer Interaction, Social Response Theory, and the cooperative principle of conversation with a focus on human-machine interaction. Subsequently, expert interviews on relevant application areas in the field of production and logistics were conducted in order to identify use cases and derive system requirements. Based on the findings, a concept for an stationary adaptive multimodal information system was developed, which, to our knowledge, does not exist yet due to the combination of a 2D/3D scanner and conversational intelligence and can be used to improve human-machine interaction in the context of Industry 5.0.