Human-in-the-loop control of dynamics and robotics using augmented reality

Abstract

Human-machine interaction (HMI) and human-robot interaction (HRI) offer innovative approaches for the analysis of structural dynamics both in laboratory and real-world scenarios. In vibratory experimentation, an external force is generated to evaluate dynamic responses of structures. One mode of generation is to use electrodynamic shakers. Sensors are attached to the structure to measure dynamic responses and can be deployed by robots. In post-disaster inspections robotic deployment ensures the safety of inspectors and requires control. If the interface between operators and the controls was augmented, then operators can visualize experiments, exciter levels, and define robot input while maintaining awareness of the area of interest. Robots can provide better aid to humans if control of the robot is: (1) quantified and presented to the human; and (2) conducted in real-time for human feedback. This research proposes using Augmented Reality (AR) to provide operators with sensor feedback and control for analysis of dynamic structures with robots. This method improves cognition by allowing the operator to maintain awareness of structures while adjusting conditions accordingly with the assistance of the new real-time interface. One interface application is developed to plot sensor data in addition to voltage, frequency, and duration controls for vibration simulations. Two more applications are developed following the same framework, one to control the position of a mediating robot and one to control the frequency of the robot's movement. The applications serve as a first-step approach investigating potential robotic sensor deployment in disaster assessment of structures. This paper presents the proposed model for the new vibratory control loop and then compares the new approach with a traditional method by measuring time delay in control input. The study also demonstrates that AR enhances time efficiency and accuracy in controlling robotic arm positions, matching actuator frequencies, and deploying sensors with minimal time delay.