Accountable system design architecture for embodied AI: a focus on physical human support robots

Abstract

Although the development of robot-based support systems for elderly people has become more popular, it is difficult for humans to understand the actions, plans, and behavior of autonomous robots and the reasons behind them, particularly when the robots include learning algorithms. Learning-based autonomous systems which are called AI are treated as an inherently untrustworthy ‘black box,’ because machine learning or deep learning algorithms are difficult for humans to understand. Robot systems such as assistive robots, which work closely with humans, however, should be trusted. Systems should therefore achieve accountability for all stakeholders. However, most research in this field has focused on particular systems and situations, and no general design architecture exists. In this study, we propose a new design method, focused on accountability and transparency, for learning-based robot systems. Describing the entire system is a necessary first step, and transcribing the described system for each stakeholder based on several principles is effective for achieving accountability. The method improves transparency for systems, including learning algorithms. A standing assistive robot is used as an example of the entire system to clarify which system parts require greater transparency. This study adopted the Systems Modeling Language (SysML) to describe the system and the described system is used for the information representation. Information should be represented considering the relationships between stakeholders, information, and the system interface. Because of their complexity, it is difficult for humans to understand the complete set of information available in robot systems. Systems should therefore present only the information required, depending on the situation. The stakeholder–interface relationship is also important because it is more beneficial for professionals to view information relevant to their specialized field, which would be difficult for others to understand. By contrast, the interface should be intuitive for general users. Visualization and sound are very useful means of transmitting information, with advantages and disadvantages for different circumstances. These relationships are important for achieving accountability. Finally, we show an example of implementation with a developed support system. It is confirmed that accountable systems can be designed based on the proposed design architecture.