A human factors’ approach for multimodal collaboration with Cognitive Computing to create a Human Intelligent Machine Team: a Review

Abstract

In the Horizon 2020 funded Clean Sky 2 programme, the Cognitive Collaboration for Teaming project aims at positioning a roadmap towards Single Pilot Operations (SPO) and human intelligent machine teaming. Built on top of the states-of-the art of human factors, a careful examination of the factors and parameters to be taken into account to form a Human intelligent Machine Team (HiMT) was carried out. Each parameter can influence positively or negatively teaming. A good HiMT is a HiMT that communicates, shares knowledge, information, collaborates and trusts each other to ensure flight safety at the highest level. This review shows the added value of multimodality for bidirectional communication in the HiMT. Multimodality will allow both verbal and non-verbal communication to be maintained in both directions. A review was conducted on the benefits of each of the modalities and means of interaction under different conditions and how each modality can complement each other for better bidirectional communication that is natural, efficient, and reliable. The objective is to transmit information in a clear, accurate and concise manner, but also to ensure that it is well received by the receiver (i.e. the CCTeammate and the pilot) and well understood. Each modality will allow the CCTeammate to present and/or represent the information in the best possible way to the pilot depending on the context and the task at hand.