Robotic Feedback Research Articles

Robot Feedback Design for Response Delay

We investigated an effective human-robot interaction design to alleviate the user’s negative experience caused by the response delay of a robot. The feedback regarding a robot’s delayed response was designed using human-like and machine-like approaches. The first experiment was conducted in task-based interaction to explore the effects of the feedback type (human-like or machine-like feedback versus a baseline robot) and task type (a high-cognitive-demand task versus a low-cognitive-demand task) on the understandability, perceived waiting time, and service evaluation of the robot (N = 36). The robot with machine-like feedback was rated the most positive in terms of understandability, perceived waiting time, and service evaluation. The perceived waiting time mediated the effect of feedback type on service evaluation. There were significant effects between the feedback types and the task types on understandability, perceived waiting time, and service evaluation. In a low-cognitive-demand task, machine-like feedback was evaluated more positively than human-like feedback, while in a high-cognitive-demand task, there was no significant difference between human-like feedback and machine-like feedback. In addition, the second experiment was conducted in social interaction between humans and robots to explore the effect of the two feedback types on the understandability, perceived waiting time, sociability, and service evaluation of the robot (N = 36). The robots providing feedback were evaluated more positively than the baseline robot, which did not provide any feedback, and sociability mediated the effect of feedback type on service evaluation.

A Multi‐Module Sensing and Bi‐Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

AbstractRapid advances in robotics have placed urgent demands on more intelligent human‐machine interaction technologies. Specifically, the way of establishing dual‐way intuitive communication with a consistent sensory system can greatly enhance efficiency and reliability. Here, a bi‐directional human‐machine interface (HMI) is designed by applying starch‐based hydrogel sensors. The whole system consists of a multi‐modal wearable sensory exoskeleton with a haptic feedback module and sensory robotic hand. The sensory exoskeleton with strain‐sensing glove and rotation‐sensing arm can capture and project the motion of the entire upper limb. The system offers object recognition functions by utilizing a sensing array on the robotic hands and machine learning algorithms, which can identify the shape and hardness information. The recognized results can be delivered back to the operator via vision and vibrational haptic feedback, respectively. This dual‐way intelligent sensory system shows potential application in many key fields such as the Internet of Things, teleoperation, and medical robotics.

Feasibility of a Wearable Soft Robot Triggered by Brain-Computer Interface for Stroke Upper Extremity Rehabilitation

Abstract Conventional robot designs have been applied as feedback stages in brain-computer interfaces (BCI) for stroke upper limb neurorehabilitation, showing promising results. Soft -robotic devices can be simpler and less expensive to manufacture, and provide ergonomic advantages over traditional designs, allowing to increase the efficacy and availableness of BCI systems for stroke neurorehabilitation. However, patients' degrees of control, neurophysiological activity, and system's usability with a BCI, using a soft robotic device as feedback, have not been assessed in stroke. For these reasons, a BCI system with a soft robotic feedback device was assessed in stroke patients. Fifty trials were acquired to setup the system, and another fifty trials were performed for evaluating patients' BCI control and cortical activity during movement intention (MI) and robotic feedback. User experience with the BCI was also assessed. Classification accuracy was in the range of 71.3% to 97.5%. Significant decrease in alpha power was observed during both motor intention and robotic feedback, but significant decrease in beta power was only observed during motor intention. BCI performance was high and in the range of reported BCI stroke interventions that used traditional robotics as feedback. Power decrease observed predominantly in alpha during soft robotic feedback was likely due to the eliciting of motor-related mechanisms. Quantification of user experience with the BCI implied that the system complexity is adequate for stroke patients. Therefore, a BCI system aimed at stroke neurorehabilitation can incorporate a soft robotic design as feedback and has potential for upper extremity interventions.

Differences in Social Expectations About Robot Signals and Human Signals.

In our daily lives, we are continually involved in decision-making situations, many of which take place in the context of social interaction. Despite the ubiquity of such situations, there remains a gap in our understanding of how decision-making unfolds in social contexts, and how communicative signals, such as social cues and feedback, impact the choices we make. Interestingly, there is a new social context to which humans are recently increasingly more frequently exposed-social interaction with not only other humans but also artificial agents, such as robots or avatars. Given these new technological developments, it is of great interest to address the question of whether-and in what way-social signals exhibited by non-human agents influence decision-making. The present study aimed to examine whether robot non-verbal communicative behavior has an effect on human decision-making. To this end, we implemented a two-alternative-choice task where participants were to guess which of two presented cups was covering a ball. This game was an adaptation of a "Shell Game." A robot avatar acted as a game partner producing social cues and feedback. We manipulated robot's cues (pointing toward one of the cups) before the participant's decision and the robot's feedback ("thumb up" or no feedback) after the decision. We found that participants were slower (compared to other conditions) when cues were mostly invalid and the robot reacted positively to wins. We argue that this was due to the incongruence of the signals (cue vs. feedback), and thus violation of expectations. In sum, our findings show that incongruence in pre- and post-decision social signals from a robot significantly influences task performance, highlighting the importance of understanding expectations toward social robots for effective human-robot interactions.

Live semantic data from building digital twins for robot navigation: Overview of data transfer methods

Increasing reliance on automation and robotization presents great opportunities to improve the management of construction sites as well as existing buildings. Crucial in the use of robots in a built environment is their capacity to locate themselves and navigate as autonomously as possible. Robots often rely on planar and 3D laser scanners for that purpose, and building information models (BIM) are seldom used, for a number of reasons, namely their unreliability, unavailability, and mismatch with localization algorithms used in robots. However, while BIM models are becoming increasingly reliable and more commonly available in more standard data formats (JSON, XML, RDF), they become more promising and reliable resources for localization and indoor navigation, in particular in the more static types of existing infrastructure (existing buildings). In this article, we specifically investigate to what extent and how such building data can be used for such robot navigation. Data flows are built from BIM model to local repository and further to the robot, making use of graph data models (RDF) and JSON data formats. The local repository can hereby be considered to be a digital twin of the real-world building. Navigation on the basis of a BIM model is tested in a real world environment (university building) using a standard robot navigation technology stack. We conclude that it is possible to rely on BIM data and we outline different data flows from BIM model to digital twin and to robot. Future work can focus on (1) making building data models more reliable and standard (modelling guidelines and robot world model), (2) improving the ways in which building features in the digital building model can be recognized in 3D point clouds observed by the robots, and (3) investigating possibilities to update the BIM model based on robot feedback.

Investigating the impact of visual perspective in a motor imagery-based brain-robot interaction: A pilot study with healthy participants.

Motor Imagery (MI)-based Brain Computer Interfaces (BCI) have raised gained attention for their use in rehabilitation therapies since they allow controlling an external device by using brain activity, in this way promoting brain plasticity mechanisms that could lead to motor recovery. Specifically, rehabilitation robotics can provide precision and consistency for movement exercises, while embodied robotics could provide sensory feedback that can help patients improve their motor skills and coordination. However, it is still not clear whether different types of visual feedback may affect the elicited brain response and hence the effectiveness of MI-BCI for rehabilitation. In this paper, we compare two visual feedback strategies based on controlling the movement of robotic arms through a MI-BCI system: 1) first-person perspective, with visual information that the user receives when they view the robot arms from their own perspective; and 2) third-person perspective, whereby the subjects observe the robot from an external perspective. We studied 10 healthy subjects over three consecutive sessions. The electroencephalographic (EEG) signals were recorded and evaluated in terms of the power of the sensorimotor rhythms, as well as their lateralization, and spatial distribution. Our results show that both feedback perspectives can elicit motor-related brain responses, but without any significant differences between them. Moreover, the evoked responses remained consistent across all sessions, showing no significant differences between the first and the last session. Overall, these results suggest that the type of perspective may not influence the brain responses during a MI- BCI task based on a robotic feedback, although, due to the limited sample size, more evidence is required. Finally, this study resulted into the production of 180 labeled MI EEG datasets, publicly available for research purposes.

Talk like me: Exploring the feedback speech rate regulation strategy of the voice user interface for elderly people.

Voice user interface (VUI) is widely used in developing intelligent products due to its low learning cost. However, most of such products do not consider the cognitive and language ability of elderly people, which leads to low interaction efficiency, poor user experience, and unfriendliness to them. Firstly, the paper analyzes the factors which influence the voice interaction behavior of elderly people: speech rate of elderly people, dialog task type, and feedback word count. And then, the voice interaction simulation experiment was designed based on the wizard of Oz testing method. Thirty subjects (M = 61.86 years old, SD = 7.16; 15 males and 15 females) were invited to interact with the prototype of a voice robot through three kinds of dialog tasks and six configurations of the feedback speech rate. Elderly people's speech rates at which they speak to a person and to a voice robot, the feedback speech rates they expected for three dialog tasks were collected. The correlation between subjects' speech rate and the expected feedback speech rate, the influence of dialog task type, and feedback word count on elderly people's expected feedback speech rate were analyzed. The results show that elderly people speak to a voice robot with a lower speech rate than they speak to a person, and they expected the robot feedback speech rate to be lower than the rate they speak to the robot. There is a positive correlation between subjects' speech rate and the expected speech rate, which implies that elderly people with faster speech rates expected a faster feedback speech rate. There is no significant difference between the elderly people's expected speech rate for non-goal-oriented and goal-oriented dialog tasks. Meanwhile, a negative correlation between the feedback word count and the expected feedback speech rate is found. This study extends the knowledge boundaries of VUI design by investigating the influencing factors of voice interaction between elderly people and VUI. These results also provide practical implications for developing suitable VUI for elderly people, especially for regulating the feedback speech rate of VUI.

Continuous versus discrete robotic feedback for brain-computer interfaces aimed for neurorehabilitation.

Brain-Computer Interfaces (BCI) can allow control of external devices using motor imagery (MI) decoded from electroencephalography (EEG). Although BCI have a wide range of applications including neurorehabilitation, the low spatial resolution of EEG, coupled to the variability of cortical activations during MI, make control of BCI based on EEG a challenging task. An assessment of BCI control with different feedback timing strategies was performed. Two different feedback timing strategies were compared, comprised by passive hand movement provided by a robotic hand orthosis. One of the timing strategies, the continuous, involved the partial movement of the robot immediately after the recognition of each time segment in which hand MI was performed. The other feedback, the discrete, was comprised by the entire movement of the robot after the processing of the complete MI period. Eighteen healthy participants performed two sessions of BCI training and testing, one with each feedback. Significantly higher BCI performance (65.4 ± 17.9% with the continuous and 62.1 ± 18.6% with the discrete feedback) and pronounced bilateral alpha and ipsilateral beta cortical activations were observed with the continuous feedback. It was hypothesized that these effects, although heterogenous across participants, were caused by the enhancement of attentional and closed-loop somatosensory processes. This is important, since a continuous feedback timing could increase the number of BCI users that can control a MI-based system or enhance cortical activations associated with neuroplasticity, important for neurorehabilitation applications.

Curvature-based force estimation for an elastic tube

AbstractContact force is one of the most significant feedback for robots to achieve accurate control and safe interaction with environment. For continuum robots, it is possible to estimate the contact force based on the feedback of robot shapes, which can address the difficulty of mounting dedicated force sensors on the continuum robot body with strict dimension constraints. In this paper, we use local curvatures to estimate the magnitude and location of single or multiple contact forces based on Cosserat rod theory. We validate the proposed method in a thin elastic tube and calculate the curvatures via Fiber Bragg Grating (FBG) sensors or image feedback. For the curvature feedback obtained from multicore FBG sensors, the overall force magnitude estimation error is$0.062 \pm 0.068$N and the overall location estimation error is$3.51 \pm 2.60$mm. For the curvature feedback obtained from image, the overall force magnitude estimation error is$0.049 \pm 0.048$N and the overall location estimation error is$2.75 \pm 1.71$mm. The results demonstrate that the curvature-based force estimation method is able to accurately estimate the contact force.

Game Theory-based Framework for Bi-manual Rehabilitation

This paper presents a shared control scheme for bi-manual robot-assisted rehabilitation exercises. Using game theory-based techniques, we design control schemes for the haptic devices to provide assistance for the upper limb to complete the rehabilitation task. Suitable cost functions are selected for human and robots to achieve shared assistance during the rehabilitation task. Further, the Nash equilibrium solution to the bi-manual rehabilitation task is presented considering a cooperative game between human and robot. An adaptive input observer is used to estimate the applied human inputs and to solve for the robot's shared control. An iterative algorithm that simultaneously estimate the human control gains and solve for the robot's feedback control is presented. Convergence of the estimated human control gains and calculated robot control gains verify that human-robot collaboration is achieved.

Development and assessment of a contactless 3D joystick approach to industrial manipulator gesture control

This paper explores a novel design of ergonomic gesture control with visual feedback for the UR3 collaborative robot that aims to allow users with little to no familiarity with robots to complete basic tasks and programming. The principle behind the design mirrors that of a 3D joystick but utilises the Leapmotion device to track the user's hands and prevents any need for a physical joystick or buttons. The Rapid Upper Limb Assessment (RULA) ergonomic tool was used to inform the design and ensure the system was safe for long-term use. The developed system was assessed using the RULA tool for an ergonomic score and through an experiment requiring 19 voluntary participants to complete a basic task with both the gesture system and the UR3's RTP (Robot Teach Pendant), then filling out SUS (System Usability Scale) questionnaires to compare the usability of both systems. The task involved controlling the robot to pick up a pipe and then insert it into a series of slots of decreasing diameter, allowing for both the speed and accuracy of each system to be compared. The experiment found that even those with no previous robot experience were able to complete the tasks after only a brief description of how the gesture system works. Despite beating the RTP's ergonomic score, the system narrowly lost on average usability scores. However, as a contactless gesture system it has other advantages over the RTP and through this experiment many potential improvements were identified, paving the way for future work into assessing the significance of including the visual feedback and comparing this system against other gesture-based systems. Relevance to IndustryIn industrial environments where the robots may need to be frequently reprogrammed through complex RTPs, companies must call on those with specialist training even to make minor adjustments. The presented system takes advantage of gesture control to allow for easy interaction with industrial robots, even for untrained operators.

A warm hug from a robot: A dual-mode e-skin with programming compliance.

Recent achievements in the field of electronic skin (e-skin) have provided promising technology for service robots. However, the development of a bionic perception system that exhibits superior performance in terms of safety and interaction quality remains a challenge. Here, we demonstrate a biomimetic soft e-skin that is composed of an array of capacitors and air pouches. It is a single platform that shows dual-mode sensing capabilities of tactile sensing and proximity perception. We optimized the shape and area of the electrode via simulation of the approach of a robot to an object. Moreover, the compliance and temperature of the e-skin can be actively adjusted by tuning the pressure and heat of the air inside the pouches. The e-skin provided dual-mode sensing feedback and soft touch for humanoid service robots, for example, when a robot hugged a man, which illustrated the potential of this e-skin for applications in human-robot interactions.

Integrated sliding mode control with input restriction, output feedback and repetitive learning for space robot with flexible-base, flexible-link and flexible-joint

AbstractIn the control of space robots, flexible vibrations exist in the base, links and joints. When building a motion control scheme, the following three aspects should be considered: (1) the complexity in dynamic modeling; (2) the low accuracy of motion control and (3) the simultaneous suppression of multiple flexible vibrations. In this paper, we propose a motion vibration integrated saturation control scheme. First, the dynamic model of space robot with flexible-base, flexible-link and flexible-joint is established according to the assumed modes method and Lagrange equation. Second, singular perturbation theory is used to decompose the model into two subsystems: a slow subsystem containing the rigid motions of base and joints as well as the vibration of links, and a fast subsystem containing vibrations of base and joints. Third, an integrated sliding mode control with input restriction, output feedback and repetitive learning (ISMC-IOR) is designed, which can track the desired trajectories of base and joints with −3 orders of magnitude accuracy, while suppressing the multiple flexible vibrations of base, links and joints 50%–80% and 37% performance improvement over ISMC-IOR-NV were achieved. Finally, the algorithm is verified by simulations.

A triboelectric multifunctional sensor based on the controlled buckling structure for motion monitoring and bionic tactile of soft robots

Soft robots have unlimited degrees of freedom, which limits the application of traditional sensors in the field of soft robots. Therefore, continuous motion monitoring and tactile feedback for soft robots have been challenging. To address this challenge, this paper proposes a multifunctional sensor by combining a controlled buckling structure with triboelectric nanogenerator for the first time to detect and control grasping, realize two-dimensional (2D) planar motion, and achieve adaptive obstacle avoidance in soft robots. A 2D kirigami electrode is compressed and buckled into a stretchable three-dimensional (3D) structure by pre-stretching the substrate. The proposed soft grasping system sensor can detect the bending angle of a soft gripper to identify the size of a grasped object, achieving a detection accuracy of 91.67 %. By alternately arranging the buckling electrodes to form a 2D spatial decoupling structure, the 2D motion sensor (2DMS) can detect the movement direction and real-time position of a robot with an accuracy rate of 83.33 %. Using the compressible properties of the buckling 3D structure, the inchworm-like soft robot (I-LSR) tactile sensor can sense the surrounding environment, thereby realizing dynamic obstacle avoidance. The proposed novel multifunctional triboelectric sensor by combining the properties of controlled buckling structure provides a theoretical and experimental basis for high-precision motion detection and intelligent obstacle avoidance for soft robots.

Neural Adaptive Robust Motion-Tracking Control for Robotic Manipulator Systems

This paper deals with the motion trajectory tracking control problem based on output feedback and artificial neural networks for anthropomorphic manipulator robots under disturbed operating scenarios. This class of manipulator robots constitutes nonlinear dynamic systems subjected to disturbance torques induced mainly by work payload. Parametric uncertainty and possible dynamic modeling errors stand for other kind of disturbances that can deteriorate the efficiency and robustness of the tracking of controlled nonlinear robotic system trajectories. In fact, the presence of unknown dynamic disturbances is unavoidable in industrial robotic engineering systems. Therefore, for high-precision applications, such as laser cutting, marking, or welding, effective control schemes should be designed to guarantee adequate motion profile tracking planned on this class of disturbed nonlinear robotic system. In this context, a new adaptive robust motion trajectory tracking control scheme based on output feedback and artificial neural networks of anthropomorphic manipulator robots is presented. Three-layer B-spline artificial neural networks and time-series modeling are properly exploited in the design of novel adaptive robust motion tracking controllers for robotic applications of laser manufacturing. In this way, dependency on detailed nonlinear mathematical modeling of robotic systems is considerably reduced, and real-time estimation of uncertain dynamic disturbances is not required. Furthermore, several cases studies to demonstrate the motion planning tracking control robustness for a class of MIMO nonlinear robotic systems are described. blue Insights for the extension of the introduced output-feedback adaptive neural control design approach for other architecture of nonlinear robotic systems are depicted.

Haptic Feedback and Force-Based Teleoperation in Surgical Robotics

This article presents an overview of the current state of research and application of haptic (primarily kinesthetic) feedback and force-based teleoperation in the context of surgical robotics. Telerobotic surgery provides an approach for transferring the sensorimotor skills of a surgeon through a robotic platform to perform surgical intervention inside a patient’s body. Integration of advanced sensing and haptic technologies in telerobotic surgery can help to enhance the sensory awareness and motor accuracy of the surgeon, thereby leading to improved surgical procedures and outcomes for patients. The primary mode of sensory feedback has been through 3-D visual observation using stereo endoscopes. However, until recently, the sense of touch, i.e., haptics, has been missing in the commercial telesurgery robots approved for use in the operating room despite over two decades of research and development in the field of haptics for teleoperated systems (“telehaptics”). Research has shown that high-fidelity force feedback can enhance the performance of telesurgery and potential outcomes by enabling the surgeon to have a more natural feel of interaction between surgical tools and tissue as normally experienced during open surgery. Interaction forces, such as those generated during palpation of tissue, insertion of a needle, unintentional (and potentially unsafe) exertion of force by a tool, suture breakage, needle slippage, or tool interaction, are replaced by indirect (virtual) sensations, termed visual haptics, which provides an alternative to sensory compensation. Although there is a significant amount of literature supporting this benefit, there are still several important technical challenges in introducing haptics in telesurgery, including instrumentation, fidelity (transparency), stability, and modalities for force reflection, e.g., direct or indirect. This article examines these challenges and discusses recent work on haptics-based teleoperated surgical robotic systems.

Effects of Human and Robot Feedback on Shaping Human Movement Behaviors during Reaching Tasks

This was the first of a series of experiments to investigate whether adults respond to a robot’s feedback as effectively as to a human’s during a virtual-reality game. Twenty adults volunteered to pop bubbles in 7 virtual-reality games, including 2 in baseline, 3 in acquisition, and 2 in extinction. Participants were randomized to receive feedback from a robot or a human during acquisition. The 80% of the average movement time at baseline was used as the target time to determine the feedback content. Feedback was the same except for the agent (robot or human). Results indicated adults responding to a robot’s feedback to shape their reaching during acquisition and extinction had faster reaching than baseline; however, their patterns were jerkier, less straight, and slower than participants’ receiving a human’s feedback. This study shows the potential to use robots as the feedback provider, though participants performed slightly better with human feedback.

Continuum Robot Proprioception: The Ionic Liquid Approach

Slender continuum robots are uniquely placed to address inspection and intervention tasks in difficult-to-access environments. Tasks such as minimally invasive surgery and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in situ</i> maintenance of complex machinery require dexterous robots with accurate motion control, and benefit from instruments with small diameters and large internal passages for services. Intrinsic proprioception, or built-in position sensory feedback, has been described in prior work on continuum and soft robots but seldom employed for motion closed-loop control. In this research article, we present a novel tubular resistive stretch sensor with ionic liquid conductors to enable closed-loop position control of low-cost, 3-D printed continuum robots. We describe a methodology for sensor characterization and calibration drawing from a model of their behavior under strain, allowing us to apply four stretch sensors as proprioceptive feedback for motion control of a four-axis slender continuum manipulator. We demonstrate a substantial (approximately 65%) improvement in position control accuracy and repeatability over open-loop methods and show a promising step toward low-cost, scalable position feedback for slender continuum robots.

Balance Therapy With Hands-Free Mobile Robotic Feedback for At-Home Training Across the Lifespan.

Providing aging adults with engaging, at-home balance therapy is essential to promote long-term adherence to unsupervised training and to foster independence. We developed a portable interactive balance training system that provides real-world visual cues on balance performance using wobble board tilt angles to control the speed of a robotic car platform in a three-dimensional environment. The goal of this study was to validate this mobile balance therapy system for home use across the lifespan. Twenty younger (18-39 years) and nineteen older (58-74 years) healthy adults performed balance training with and without visual feedback while standing on a wobble board instrumented with a consumer-grade inertial measurement unit (IMU) and optical motion tracking markers. Participants performed feedback trials based on either the robotic car's movements or a commercially-available virtual game. Wobble board tilt measurements were highly correlated between IMU and optical measurement systems ( [Formula: see text]), with high agreement in outcome metrics ( [Formula: see text]) and small bias ( [Formula: see text]). Both measurement systems identified similar aging, feedback, and stance type effects including (1) altered movement control when older adults performed tilting trials with either robotic or virtual feedback compared to without feedback, (2) two-fold greater wobble board oscillations in older vs. younger adults during steady standing, (3) no difference in board oscillations during steady standing in narrow vs. wide double support, and (4) greater wobble board oscillations for single compared to double support. These findings demonstrate the feasibility of implementing goal-directed robotic balance training with mobile tracking of balance performance in home environments.

Upper-body haptic system for snake robot teleoperation in pipelines

Snake robots have shown a great potential for operations in confined workplaces that are less accessible or dangerous to human workers, such as the in-pipe inspection. However, the snake robot teleoperation remains a nontrivial task due to the unique locomotion mechanism (e.g., helical motion) and the constraints of the workplaces including the low visibility and indistinguishable features. Most snake robot feedback systems are based on the live camera view only. It is hard for the human operator to develop a correct spatial understanding of the remote workplace, leading to problems such as disorientation and motion sickness in snake robot teleoperation. This study designs and evaluates an innovative haptic assistant system for snake robot teleoperation in the in-pipe inspection. An upper-body haptic suit with 40 vibrators on both the front and back sides of the human operator was developed to generate haptic feedback corresponding to the bottom and up sides of the snake robot, transferring the egocentric sensation of the snake robot to the human operator. A human-subject experiment (n = 31) was performed to evaluate the efficacy of the developed system. The results indicate that the proposed haptic assistant system outperformed other feedback systems in terms of both task performance and subjective workload and motion sickness evaluations. It inspires new control and feedback designs for the future snake robot in industrial operations.