Human-human Collaboration Research Articles

Robust and Context-Aware Real-Time Collaborative Robot Handling via Dynamic Gesture Commands

This paper studies real-time collaborative robot (cobot) handling, where the cobot maneuvers an object under human dynamic gesture commands. Enabling dynamic gesture commands is useful when the human needs to avoid direct contact with the robot or the object handled by the robot. However, the key challenge lies in the heterogeneity in human behaviors and the stochasticity in the perception of dynamic gestures, which requires the robot handling policy to be adaptable and robust. To address these challenges, we introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conditional Collaborative Handling Process</i> (CCHP) to encode a context-aware cobot handling policy and a procedure to learn such policy from human-human collaboration. We thoroughly evaluate the adaptability and robustness of CCHP and apply our approach to a real-time cobot assembly task with Kinova Gen3 robot arm. Results show that our method leads to significantly less human effort and smoother human-robot collaboration than state-of-the-art rule-based approach even with first-time users.

Coevolution of internal representations in physical human-robot orchestration – models of the surgeon and the robot in robotic surgery

In teleoperated Robot-Assisted Minimally-Invasive Surgery (RAMIS), a surgeon controls the movements of instruments inside the patient’s body via a pair of robotic joysticks. RAMIS has transformed many surgical disciplines, but its full potential is still to be realized. In this chapter we propose a pathway towards overcoming several bottlenecks that are related to transparency and stability of the teleoperation channels that mediate RAMIS. We describe the traditional system centered and the more recent human-centred approaches to teleoperation, and the special considerations for RAMIS as an application of teleoperation. However, the human-centered approach is still one sided view focusing on the surgeon but neglecting the learning capabilities of robotic systems. Hence, we consider a more general idea of physical human-robot orchestration with coevolution of mutual internal representations – of the human and the robot, and discuss it in comparison to human-human collaboration over teleoperated channels.

An electromyography signals-based human-robot collaboration method for human skill learning and imitation

Human beings have strong perception and decision-making ability. In human-human collaboration, the follower can adjust the motion speed by perceiving the change of external force information, and cooperate with the leader to complete the collaboration task. Transferring human collaboration skills to robots will help improve the flexibility and compliance of human-robot collaboration. For this purpose, an Electromyography (EMG) signals-based human-robot collaboration method is proposed. In the collaboration method, the human arm force, obtained by EMG signals and joint angles, is taken as the interface of human-robot interaction, and the robot is controlled to complete the collaboration task by learning the speed adjustment skills of human tutors. To reduce the inaccuracy and instability of human-robot interaction caused by the fluctuation of EMG signals, the adaptive data correction unit based on tremor information and the input-output control unit based on Naive Bayes are added to the Long Short-Term Memory (LSTM) neural network, and the parallel network structure is used to estimate the three-dimensional arm force. Aiming at the problem that the traditional collaborative control model cannot balance the accuracy and rapidity, a multi-model Gaussian process regression algorithm is used to capture the collaboration skills from inaccurate human-human demonstrations in the way of probability estimation. Finally, taking peg-in-hole assembly as an example, the proposed human-robot collaboration method is verified. Compared with the traditional adaptive admittance control, the proposed collaboration method shows better interaction performance, and its assembly time and assembly success rate are improved.

Artificial Intelligence and a Microperspective of Organizational Corruption

Building on previous research on the collaborative root of corruption, we propose and conclude that people cheat more when they collaborate with another person in comparison with an artificial intelligence (AI) robot. We suggest that people infer that a human collaborator would cheat more than an AI collaborator when working together. Having drawn this ethical inference about a collaborator’s behavior, people are more likely to cheat subsequently in human-human collaboration than in human-AI collaboration. Therefore, the ethical inference about a collaborator’s behavior is the reason why people are more inclined to cheat in human-human collaboration than human-AI collaboration. We tested our theory in two lab experiments. Additionally, in two online experiments, we showed our findings were counterintuitive with regard to the common belief of laypeople. Managers are unaware of the positive effect of human-AI collaboration on reducing cheating behavior in organizations. Our research contributes to the microperspective of organizational corruption and has implications for anticorruption and compliance policies in organizations and society as a whole.

Investigating a Mixed-Initiative Workflow for Digital Mind-Mapping

AbstractIn this paper, we report on our investigation of human-AI collaboration for mind-mapping. We specifically focus on problem exploration in pre-conceptualization stages of early design. Our approach leverages the notion of query expansion—the process of refining a given search query for improving information retrieval. Assuming a mind-map as a network of nodes, we reformulate its construction process as a sequential interaction workflow wherein a human user and an intelligent agent take turns to add one node to the network at a time. Our contribution is the design, implementation, and evaluation of algorithm that powers the intelligent agent (IA). This paper is an extension of our prior work (Chen et al., 2019, “Mini-Map: Mixed-Initiative Mind-Mapping Via Contextual Query Expansion,” AIAA Scitech 2020 Forum, p. 2347) wherein we developed this algorithm, dubbed Mini-Map, and implemented a web-based workflow enabled by ConceptNet (a large graph-based representation of “commonsense” knowledge). In this paper, we extend our prior work through a comprehensive comparison between human-AI collaboration and human-human collaboration for mind-mapping. We specifically extend our prior work by: (a) expanding on our previous quantitative analysis using established metrics and semantic studies, (b) presenting a new detailed video protocol analysis of the mind-mapping process, and (c) providing design implications for digital mind-mapping tools.

Unsupervised early prediction of human reaching for human–robot collaboration in shared workspaces

This paper focuses on human–robot collaboration in industrial manipulation tasks that take place in a shared workspace. In this setting we wish to predict, as quickly as possible, the human’s reaching motion so that the robot can avoid interference while performing a complimentary task. Given an observed part of a human’s reaching motion, we thus wish to predict the remainder of the trajectory, and demonstrate that this is effective as a real-time input to the robot for human–robot collaboration tasks. We propose a two-layer framework of Gaussian Mixture Models and an unsupervised online learning algorithm that updates these models with newly-observed trajectories. Unlike previous work in this area which relies on supervised learning methods to build models of human motion, our approach requires no offline training or manual labeling. The main advantage of this unsupervised approach is that it can build models on-the-fly and adapt to new people and new motion styles as they emerge. We test our method on motion capture data from a human-human collaboration experiment to show the early prediction performance. We also present two human–robot workspace sharing experiments of varying difficulty where the robot predicts the human’s motion every 0.1 s. The experimental results suggest that our framework can use human motion predictions to decide on robot motions that avoid the human in real-time applications with high reliability.

Sustainable Design and Postindustrial Society: Our Ethical and Aesthetic Crossroads

Mid-20th century transitions from industrial product society to postindustrial information society have marked profound but now familiar conversions to service economy, knowledge workers, and cybernetic reasoning. Second order, but equally important consequences of this change involve the transformation from predominantly human-machine heroics to human-human collaboration. Collectively, these events have revolutionized the bases of production and value across the developed world. Less appreciated however, are the more subtle shifts of postindustrialism and their ultimate epochal transformations of contemporary life. The short list of these more elusive transitions includes local scale isolation to macro and global scale interaction, mechanistic routine to systemic reasoning, static to dynamic assumptions, short-termism to scenario planning, profit to value motives, hero to team attribution, intuitive to cybernetic decisions, and a move away from rote procedural expertise in favor of reasoned principle, wisdom, and theory. Our historical perspective thus argues for the relevance of postindustrial society in the emergence of a sustainable future, with particular reference to the built environment and to the complex, collaborative, evidence based and cybernetic processes it involves. The difficulty here is that without a vivid and operational understanding of the aesthetic connections and ethical mandates inherent in these more sublime postindustrial events; it is entirely possible that all the best scientific, technical, and political efforts toward sustainability are hampered by old habits of piecemeal procedures, mechanistic approaches, individual expertise, quick profit, and simplistic short-termism. Postindustrial ethics and aesthetics, on the other hand, offer a new and different apparatus by embracing complexity and dynamic interaction. Within that new aesthetic lies a set of principles and sensitivities towards postindustrial and sustainable era ethics. As such, this present argument attempts to form a cohesive framework contextualizing sustainability, societal trends and nascent evolutions within an aspirational agenda. The underlying theory of this framework describes, explains, and predicts the co-evolution of sustainability and postindustrial events. Finally, the aesthetic basis of the theory is functionally aligned with human cognition. Just as humankind did not quit building with masonry at the end of the Stone Age however, the argument presented here does not demonize the progress of the industrial era that has doubled life expectancy in the last hundred years; nor does it belittle the advent of antibiotics, space travel, telecommunications, rapid transit and the like. It is now necessary however to acknowledge that the pioneer era nature-as-antagonist and industrial era of nature-as-resource have given way to an era of nature-as-model-and-host relation.

Ontology Modeling for Intelligent Computer-Aided Design of Apparel Products

This paper presents a study developing a model of the ontology of an apparel product and the design thereof. Here, “ontology” means the concepts and relationships between them for apparel and apparel design; ontology models are also called conceptual models. To the best of our knowledge, such a model has not yet been made in the literature of apparel and apparel design. A conceptual model is a foundation for (1) effective human-human communication and collaboration, (2) facilitating the communication among different computer programs for computer-aided design (CAD) of apparel products, and (3) facilitating human-computer interactions necessary to design an apparel product. The conceptual model developed completely captures the semantics of apparel and design, including the 2D pattern and 3D apparel product. An example is presented, providing an impression of the usefulness of this model.

Affective Computing as Complex Systems Science

Pioneered in the early ‘90s by Rosalind Picard, a professor and IEEE Fellow of the MIT Media Lab, Affective Computing – rooted originally in artificial intelligence – now branches into wearable computing, big data, psychology, neuroscience, and modeling in order to advance the knowledge, understanding, and development of systems for sensing, recognizing, categorizing, and reacting to human emotion. Yet, the challenges of sensing multiple modalities simultaneously, disambiguating complex emotional states non-linearly, and modeling multiple individuals’ emotional states dynamically have continued to ring true, despite dramatic advances in affective computing. This paper seeks to serve two objectives. The first objective is to discuss how these three challenges are related to the three characteristics of complex systems – namely multiple components, non-linearity, and emergent behaviors. The second objective is to identify opportunities from the complex systems domain to address these challenges in novel and comprehensive ways. Recent advances in the utilization of Dynamical Systems Theory (an applied complexity science methodology) have shown that complex human interaction can be rigorously studied and modeled. Coupling the technological advances that cloud-based affective computing have brought with the emerging complex systems science-perspective may well catalyze a new era of human-machine and human-human collaboration.

Tacit knowledge

This workshop provides a forum for discussing experiences and issues related to tacit knowledge [Polanyi 1967] in collaborative systems. Beginning with early CSCW systems, tacit knowledge of work practice, in terms of unspoken assumptions and exceptions, has posed difficult problems for system designers. Analyses by Ehn [1988], Grudin [t994] and others [Bullen and Bennett 1990] show that tacit knowledge continues to play a disturbingly large role in the problems most CSCW systems struggle with. Humans make excellent use of tacit knowledge. Anaphora, ellipses, unstated shared understanding are all used in the service of our collaborative relationships. But when human-human collaboration becomes human-computer-human collaboration, tacit knowledge becomes a problem.