Human-swarm Interaction Research Articles

Swarm Robotic Interactions in an Open and Cluttered Environment: A Survey

Recent population migrations have led to numerous accidents and deaths. Little research has been done to help migrants in their journey. For this reason, a literature review of the latest research conducted in previous years is required to identify new research trends in human-swarm interaction. This article presents a review of techniques that can be used in a robots swarm to find, locate, protect and help migrants in hazardous environment such as militarized zone. The paper presents a swarm interaction taxonomy including a detailed study on the control of swarm with and without interaction. As the interaction mainly occurs in cluttered or crowded environment (with obstacles) the paper discussed the algorithms related to navigation that can be included with an interaction strategy. It focused on comparing algorithms and their advantages and disadvantages.

Autonomous recommender system for reconnaissance tasks using a swarm of UAVs and asynchronous shepherding

Several artificial intelligence-based systems have been proposed to help human operators in safety-critical domains such as air-traffic control systems, health and farming. In scenarios where multiple autonomous systems are utilised, the cognitive capacity of humans hits its limits quickly, and sometimes the success of these systems is obstructed. Designing systems to sustain human performance over longer periods through balancing the load between human operators and autonomous systems can reduce errors and improve overall performance. Future human-autonomy systems need to be designed in a way that makes them scalable in terms of capabilities and safety, yet flexible enough to adapt to the cognitive workload of human operators. In this paper, we propose an adaptive human swarm interaction system for maintaining human operator performance in supervisory-control aerial-surveillance tasks. A bio-inspired guidance and control shepherding agent is proposed to support the human operator to guide multiple assets simultaneously using a single point of control. The proposed system employs an adjustable autonomous recommender system to provide the human operator with timely recommendations in real-time sittings. The proposed system is designed in a flexible way that allows it to adapt its rate of providing these recommendations in response to the needs of human operators. Simulation-based experiments are conducted to validate and evaluate the proposed recommender system. Experimental results demonstrate the effectiveness and efficiency of the proposed recommender system in minimising control errors while maintaining low computational costs that allow it to be used in a real-time applications.

The reliability and transparency bases of trust in human-swarm interaction: principles and implications

Automation reliability and transparency are key factors for trust calibration and as such can have distinct effects on human reliance behaviour and mission performance. One question that remains unexplored is: what are the implications of reliability and transparency on trust calibration for human-swarm interaction? We investigate this research question in the context of human-swarm interaction, as swarm systems are becoming more popular for their robustness and versatility. Thirty-two participants performed swarm-based tasks under different reliability and transparency conditions. The results indicate that trust, whether it is reliability- or transparency-based, indicates high reliance rates and shorter response times. Reliability-based trust is negatively correlated with correct rejection rates while transparency-based trust is positively correlated with these rates. We conclude that reliability and transparency have distinct effects on trust calibration. Practitioner Summary: Reliability and transparency have distinct effects on trust calibration. Findings from our human experiments suggest that transparency is a necessary design requirement if and when humans need to be involved in the decision-loop of human-swarm systems, especially when swarm reliability is high. Abbreviations: HRI: human-robot interaction; IOS: inter-organisational systems; LMM: liner mixed models; MANOVA: multivariate analysis of variance; UxV: heterogeneous unmanned vehicles; UAV: unmanned aerial vehicle

Human-swarm Interactions for Formation Control Using Interpreters

In this work, we develop a novel Human-Swarm Interaction (HSI) framework for formation control using the notion of an interpreter, enabling the user to control a robotic swarm's shape and formation using abstraction. The user conveys their intended commands by drawing shapes through arm gestures and motions which are recorded by an off-the-shelf wearable device. We propose a novel interpreter system, which acts as an intermediary between the user and the swarm to simplify the roles of both. The interpreter takes in high level input in the form of shapes drawn by the user, and translates it into swarm control commands by planning in the shape space using novel shape morphing dynamics (SMD), which is also used for user feedback. The proposed interpreter employs machine learning, estimation and optimal control techniques to translate the users intention into swarm control parameters. The dynamics of the swarm are realized by means of a novel decentralized formation controller based on distributed linear iterations and dynamic average consensus. Theoretical guarantees of convergence along with convergence rate of the proposed swarm controller are given. The resulting shape morphing dynamics are illustrated and discussed through simulations. The entire framework is demonstrated theoretically as well as experimentally in a 2D environment, with a human controlling a swarm of simulated robots in real time with the help of a Graphical User Environemnt (GUI).

Assisted Teleoperation for a Human-Swarm Interaction System

Abstract The trend in human-controlled robotic systems is to move away from the current many-to-one paradigm, where multiple operators are forced to control a single system to a one-to-many mode of operation in which a single operator can control a large number of autonomous vehicles. For this transition to be successful, it is essential that we clearly understand how humans should interact with a large number of robots, this is precisely the problem of a growing field of research called Human-Swarm Interaction (HSI). In this article, the training of a recurrent neural network (LSTM) is proposed to model the behavior of the human operator in the task of rendezvous performed by swarms, in order to develop an assisted teleoperation system. The system is tested in the presence of humans and in the presence of faults in the communications.

A high-performance millirobot for swarm-behaviour studies: Swarm-topology estimation

In this article, we present a novel high-performance millirobot ( milli- robot- Toronto), designed to allow for the testing of complex swarm-behaviours, including human–swarm interaction. milli- robot- Toronto, built only with off-the-shelf components, has locomotion, processing and sensing capabilities that significantly improve upon existing designs, while maintaining one of the smallest footprints among current millirobots. As complementary software to this hardware development, herein, we also present a new global swarm-topology estimation algorithm. The method is novel in that it uniquely fuses incomplete location data collected by the individual robots in a distributed manner to optimally estimate the topology of the overall swarm using a centralized computer. It is a generalized technique usable by any swarm comprising robots capable of collecting location estimates of neighbouring robots. Numerous experiments, evaluating the performance of milli- robot- Toronto and the proposed optimal swarm-topology estimation algorithm, are also included.

Trust Mediating Reliability-Reliance Relationship in Supervisory Control of Human-Swarm Interactions.

This work aims to further test the theory that trust mediates the interdependency between automation reliability and the rate of human reliance on automation. Human trust in automation has been the focus of many research studies. Theoretically, trust has been proposed to impact human reliance on automation by mediating the relationship between automation reliability and the rate of human reliance. Experimentally, however, the results are contradicting as some confirm the mediating role of trust, whereas others deny it. Hence, it is important to experimentally reinvestigate this role of trust and understand how the results should be interpreted in the light of existing theory. Thirty-two subjects supervised a swarm of unmanned aerial vehicles (UAVs) in foraging missions in which the swarm provided recommendations on whether or not to collect potential targets, based on the information sensed by the UAVs. By manipulating the reliability of the recommendations, we observed changes in participants' trust and their behavioral responses. A within-subject mediation analysis revealed a significant mediation role of trust in the relationship between swarm reliability and reliance rate. High swarm reliability increased the rate of correct acceptances, but decreased the rate of correct rejections. No significant effect of reliability was found on response time. Trust is not a mere by-product of the interaction; it possesses a predictive power to estimate the level of reliance on automation. The mediation role of trust confirms the significance of trust calibration in determining the appropriate level of reliance on swarm automation.

SwarmTouch: Guiding a Swarm of Micro-Quadrotors With Impedance Control Using a Wearable Tactile Interface.

To achieve a smooth and safe guiding of a drone formation by a human operator, we propose a novel interaction strategy for a human-swarm communication, which combines impedance control and vibrotactile feedback. The presented approach takes into account the human hand velocity and changes the formation shape and dynamics accordingly using impedance interlinks simulated between quadrotors, which helps to achieve a natural swarm behavior. Several tactile patterns representing static and dynamic parameters of the swarm are proposed. The user feels the state of the swarm at the fingertips and receives valuable information to improve the controllability of the complex formation. A user study revealed the patterns with high recognition rates. A flight experiment demonstrated the possibility to accurately navigate the formation in a cluttered environment using only tactile feedback. Subjects stated that tactile sensation allows guiding the drone formation through obstacles and makes the human-swarm communication more interactive. The proposed technology can potentially have a strong impact on the human-swarm interaction, providing a higher level of awareness during the swarm navigation.

Gesture based Human-Swarm Interactions for Formation Control using interpreters

Abstract We propose a novel Human-Swarm Interaction (HSI) framework which enables the user to control a swarm’s shape and formation. The user commands the swarm utilizing just arm gestures and motions which are recorded by an off-the-shelf wearable armband. We propose a novel interpreter system, which acts as an intermediary between the user and the swarm to simplify the user’s role in the interaction. The interpreter takes high level input drawn using gestures by the user, and translates it into low level swarm control commands. This interpreter employs machine learning, Kalman filtering and optimal control techniques to translate the user input into swarm control parameters. A notion of Human Interpretable dynamics is introduced, which is used by the interpreter for planning as well as to provide feedback to the user. The dynamics of the swarm are controlled using a novel decentralized formation controller based on distributed linear iterations and dynamic average consensus. The framework is demonstrated theoretically as well as experimentally in a 2D environment, with a human controlling a swarm of simulated robots in real time.

A Passivity-Based System Design of Semi-Autonomous Cooperative Robotic Swarm

This article presents a novel semiautonomous cooperative robotic system. Information flows in the architecture are designed based on the passivity property. The human passivity needed to guarantee the control goal is demonstrated using a human-in-the-loop simulator and system identification techniques. As a solution to stable coordination via bilateral human–swarm interactions, the research group built the system consisting of multiple planar robots, an operator and a computer whose role is to implement the distributed motion synchronization law. Experimental results on remote control of the robots are finally demonstrated. A typical approach to stabilization of an interconnected system that includes a highly uncertain component like a human operator is to assume passivity of the component block. Specifically, modeling of the operator has been addressed in this manner for several studies in bilateral teleoperation. It is observed from the figures provided in the article that the robots form and maintain the specified formation while they are maneuvered stably toward the desired references by the operator.

Investigating the effect of the reality gap on the human psychophysiological state in the context of human-swarm interaction

The reality gap is the discrepancy between simulation and reality—the same behavioural algorithm results in different robot swarm behaviours in simulation and in reality (with real robots). In this paper, we study the effect of the reality gap on the psychophysiological reactions of humans interacting with a robot swarm. We compare the psychophysiological reactions of 28 participants interacting with a simulated robot swarm and with a real (non-simulated) robot swarm. Our results show that a real robot swarm provokes stronger reactions in our participants than a simulated robot swarm. We also investigate how to mitigate the effect of the reality gap (i.e., how to diminish the difference in the psychophysiological reactions between reality and simulation) by comparing psychophysiological reactions in simulation displayed on a computer screen and psychophysiological reactions in simulation displayed in virtual reality. Our results show that our participants tend to have stronger psychophysiological reactions in simulation displayed in virtual reality (suggesting a potential way of diminishing the effect of the reality gap).

Investigating the effect of increasing robot group sizes on the human psychophysiological state in the context of human–swarm interaction

We study the psychophysiological state of humans when exposed to robot groups of varying sizes. In our experiments, 24 participants are exposed sequentially to groups of robots made up of 1, 3 and 24 robots. We measure both objective physiological metrics (skin conductance level and heart rate), and subjective self-reported metrics (from a psychological questionnaire). These measures allow us to analyse the psychophysiological state (stress, anxiety, happiness) of our participants. Our results show that the number of robots to which a human is exposed has a significant impact on the psychophysiological state of the human and that higher numbers of robots provoke a stronger response.

Human Interaction With Robot Swarms: A Survey

Recent advances in technology are delivering robots of reduced size and cost. A natural outgrowth of these advances are systems comprised of large numbers of robots that collaborate autonomously in diverse applications. Research on effective autonomous control of such systems, commonly called swarms, has increased dramatically in recent years and received attention from many domains, such as bioinspired robotics and control theory. These kinds of distributed systems present novel challenges for the effective integration of human supervisors, operators, and teammates that are only beginning to be addressed. This paper is the first survey of human–swarm interaction (HSI) and identifies the core concepts needed to design a human–swarm system. We first present the basics of swarm robotics. Then, we introduce HSI from the perspective of a human operator by discussing the cognitive complexity of solving tasks with swarm systems. Next, we introduce the interface between swarm and operator and identify challenges and solutions relating to human–swarm communication, state estimation and visualization, and human control of swarms. For the latter, we develop a taxonomy of control methods that enable operators to control swarms effectively. Finally, we synthesize the results to highlight remaining challenges, unanswered questions, and open problems for HSI, as well as how to address them in future works.

Two Invariants of Human Swarm Interaction

The search for invariants is a fundamental aim of scientific endeavors. These invariants, such as Newton's laws of motion, allow us to model and predict the behavior of systems across many different problems. In the nascent field of Human-Swarm Interaction (HSI), a systematic identification of fundamental invariants is still lacking. Discovering and formalizing these invariants will provide a foundation for developing, and better understanding, effective methods for HSI. We propose two invariants underlying HSI for geometric-based swarms: (1) collective state is the fundamental percept associated with a bio-inspired swarm, and (2) a human's ability to influence and understand the collective state of a swarm is determined by the balance between the span and persistence. We provide evidence of these invariants by synthesizing much of our previous work in the area of HSI with several new results, including a novel user study where users manage multiple swarms simultaneously. We also discuss how these invariants can be applied to enable more efficient and successful teaming between humans and bio-inspired collectives and identify several promising directions for future research into the invariants of HSI.

Swarm Robotics: A Technological Advancement for Human-Swarm Interaction in Recent Era from Swarm-Intelligence Concept

Swarm Robotics: A Technological Advancement for Human-Swarm Interaction in Recent Era from Swarm-Intelligence Concept

Haptic Interactions With Multi-Robot Swarms Using Manipulability

This paper investigates how haptic interactions can be defined for enabling a single operator to control and interact with a team of mobile robots. Since there is no unique or canonical mapping from the swarm configuration to the forces experienced by the operator, a suitable mapping must be developed. To this end, multi-agent manipulability is proposed as a potentially useful mapping, whereby the forces experienced by the operator relate to how inputs, injected at precise locations in the team, translate to swarm-level motions. Small forces correspond to directions in which it is easy to move the swarm, while larger forces correspond to more costly directions. Initial experimental results support the viability of the proposed, haptic, human-swarm interaction mapping, through a user study where operators are tasked with driving a collection of robots through a series of way points.

Analyzing human-swarm interactions using control Lyapunov functions and optimal control

A number of different interaction modalities have been proposed for human engagement with networked systems. In this paper, we establish formal guarantees for whether or not a given human-swarm interaction (HSI) strategy is appropriate for achieving particular multi-robot tasks, such as guiding a swarm of robots into a particular geometric configuration. In doing so, we define what it means to impose an HSI control structure on a multi-robot system. Control Lyapunov functions are used to establish feasibility for a user to achieve a particular geometric configuration with a multi-robot system under some selected HSI control structure. Several examples of multi-robot systems with unique HSI control structures are provided to illustrated the use of CLFs to establish feasibility. Additionally, we also uses these examples to illustrate how to use optimal control tools to compute three metrics for evaluating an HSI control structure: attention, effort, and scalability.

Biologically-Inspired Human-Swarm Interaction Metrics

Human-swarm interaction is an emerging field encompassing questions related to biology, robotics, computer science, human-computer interaction, and psychology. Swarms are large groups of individual entities that enact group behaviors; biological examples include fish, birds and insects. Swarms overwhelm humans’ abilities to monitor and interact with each entity. Human-robot and human-computer interaction metrics are inappropriate to describe human-swarm interactions alone due to the interaction challenges posed by swarms. It is unknown precisely how humans respond to interacting with swarms. The theory is that biological swarm metrics may be appropriate for analyzing human-swarm interaction. Nine human-swarm interaction metric categories derived from the biological and robotic swarm literature are presented, including example metrics from each category. This paper opens the discussion regarding what types of existing swarm metrics may be applicable and what categories of metrics will be important for human-swarm interaction assessment.

Using Haptic Feedback in Human Robotic Swarms Interaction

Robotic swarms display emergent behaviors that are robust to failure of individual robots, although they can not necessarily accomplish complex tasks with these behaviors. The research objective is to make use of their robust behaviors to accomplish complex tasks in many types of environments. For now, it is difficult to affect swarm “goals”, and therefore difficult them to direct to perform complex tasks. The extant literature on Human Swarm Interaction (HSI) focuses on demonstrating the usefulness of human operator inputs for swarms to accomplish complex tasks. The human typically gets visual feedback of the state of the swarm and influences the robots through a computer interface. This paper presents a user study investigating the effectiveness of haptic feedback in improving HSI. We use methods developed in studies using haptics in multi-robot systems (where the communication and structure is very rigid) and potential field algorithms developed for fully-autonomous swarms to determine the benefits of haptic feedback from the semi-autonomous control algorithm. In some environments, haptic feedback proved beneficial whereas in other environments haptic feedback did not improve performance over visual feedback alone. However, presence of haptic feedback did not degrade the performance under any of the experimental conditions. This supports our working hypothesis that haptic feedback is potentially useful in HSI.

Levels of Automation for Human Influence of Robot Swarms

Autonomous swarm algorithms and human-robot interaction (HRI) have both attracted increasing attention from researchers in recent years. However, HRI has rarely extended beyond single robots or small multi-robot teams. While one of the benefits of robot swarms is their robust capabilities and the ability of their distributed algorithms to deal autonomously with the complex interactions amongst swarm members, there is undoubtedly a need for humans to influence such swarms in some circumstances—especially when these swarms are operating in unknown or hostile environments. In this paper, we approach the problem of human-swarm interaction (HSI) using previous research in levels of automation (LOAs) in HRI. We create a target searching task whereby the swarm can operate at two different levels of autonomy: an autonomous dispersion algorithm, or user-defined goto points. We investigate what environmental conditions are conducive to different amounts of human influence, and at what point further human intervention has a detrimental effect on the swarm’s performance. The results show that for complex environments containing numerous obstacles and small passageways, there is indeed a need for some human influence; however, after a certain point, further influence causes performance degradation.