Augmented Reality Approach for a User Interface in a Robotic Production System

Abstract

In a Cyber-physical system, the information flow from the cyber part to the physical part plays a crucial role. This paper presents the work of development and initial testing of an augmented reality approach to provide a user interface for operators that could be a part of a robotic production system. The solution is distributed and includes a communication hub that allows the exchange of data and information between multiple clients e.g. robot controllers, an optimization platform, and visualization devices. The main contributions of the presented work are visualization of optimization results and visualization of information obtained from the robot controller and the integrated communication framework. The paper also presents challenges faced during the development work and opportunities related to the presented approach. The implemented interface uses HoloLens 2 mixed reality device to visualize in real-time information obtained from a robot controller as well as from simulation. Information regarding the placement of work objects and targets or currently executed lines of code can be useful for robotic cell programmers and commissioning teams to validate robot programs and to select more optimal solutions toward sustainable manufacturing. The operator can simulate the execution of the robot program and visualize it by overlying the robot cell with the 3D model of the simulated robot. Moreover, visualization of future robot motion could support human-robot collaboration. Furthermore, the interface allows providing the user with details from multi-objective optimization performed on a digital twin of the robotic cell with the aim to reduce cycle time and energy consumption. It allows visualizing selected scenarios to support decision-making by allowing comparison of proposed solutions and the initial one. The visualization includes cell layout, robot path, cycle time, robot energy consumption. The presented approach is demonstrated in industry-inspired cases and with the use of an industrial ABB robot.