Programming By Demonstration Research Articles

Can Human-Inspired Learning Behaviour Facilitate Human–Robot Interaction?

The evolution of production systems for smart factories foresees a tight relation between human operators and robots. Specifically, when robot task reconfiguration is needed, the operator must be provided with an easy and intuitive way to do it. A useful tool for robot task reconfiguration is Programming by Demonstration (PbD). PbD allows human operators to teach a robot new tasks by showing it a number of examples. The article presents two studies investigating the role of the robot in PbD. A preliminary study compares standard PbD with human–human teaching and suggests that a collaborative robot should actively participate in the teaching process as human practitioners typically do. The main study uses a wizard of oz approach to determine the effects of having a robot actively participating in the teaching process, specifically by controlling the end-effector. The results suggest that active behaviour inspired by humans can lead to a more intuitive PbD.

Trained Behavior Trees: Programming by Demonstration to Support AI Game Designers

Programming by demonstration (PbD) has a straightforward application in the development of the artificial intelligence (AI) for nonplayer characters (NPCs) in a video game: a game designer controls the NPC during a training session in the game, and thus demonstrates the expected behavior for that character in different situations. Afterwards, applying some machine learning technique on the traces recorded during the demonstration, an AI for the NPC can be generated. Nevertheless, with this approach, it is very hard for the game designer to fully control the resulting behavior, which is a key requirement for game designers, who are responsible for putting together a fun experience for the player. In this paper, we present trained behavior trees (TBTs). TBTs are behavior trees (BTs) generated from traces obtained in a game through PbD. BTs are a technique widely used for AI game programming that are created and modified through special purpose visual editors. By inducing a BT from a PbD game session, we combine the ease of use of PbD with the ability to fine-tune the learned behavior of BTs. Furthermore, TBTs facilitate the use of BTs by game designers and promote their authoring control on game AI.

Design and Optimization of a BCI-Driven Telepresence Robot Through Programming by Demonstration

Improving the life quality of people with severe motor paralysis has a significant impact on restoring their functional independence to perform activities of daily living (ADL). Telepresence is a subfield of the robotic-assisted route, where human plays the role of an operator, sending high-level instructions to an assistive robot while receiving sensory feedback. However, for severely motor-impaired people, conventional interaction modalities may not be suitable due to their complete paralysis. Thus, designing alternative ways of interaction such as Brain-Computer Interfaces (BCI) is essential for a telepresence capability. We propose a novel framework that integrates a BCI system and a humanoid robot to develop a brain-controlled telepresence system with multimodal control features. In particular, the low-level control is executed by Programming by Demonstration (PbD) models, and the higher-level cognitive commands are produced by a BCI system to perform vital ADLs. The presented system is based on real-time decoding of attention-modulated neural responses elicited in the brain electroencephalographic signals and generating multiple control commands. As a result, the system allows a user to interact with a humanoid robot while receiving auditory and visual feedback from the robot’s sensors. We validated our system across ten subjects in a realistic scenario. The experimental results show the feasibility of the approach in the design of a telepresence robot with high BCI decoding performances.

Instrumented Tool based Robot Programming - Parameterization of Screwing Process Macros

Programming by Demonstration (PbD), is a powerful mechanism for reducing the complexity of search space for learning. Current approaches for skill representation can be broadly divided between two trends. Low-level representations of skills, taking into account the form of a non-linear mapping between sensory and motor information, which is referred to as “trajectories encoding”. On the other hand high-level representations of skills that decompose them in a sequence of action-perception units which is referred to as “symbolic encoding” [1]. While for symbolic encoding visual perception of states and actions is one preferred approach trajectories usually are acquired from direct demonstration by hand guidance of robot arms carrying robotic tools by skilled workers. This approach requires both the process knowledge of the experienced worker and the possibility of intuitive hand guidance of the robotic tool which is negatively affected by the robots limited working space, friction or the reachability of the robot. Therefore and because it is also highly unintuitive to parameterize macro based robot programs with numerical values considering process parameters (like required forces or position parameters) we propose to let the worker use ordinary power tools during demonstration which are instrumented with additional sensors in order to be able to measure relevant parameters, so called “instrumented tools”. The paper presents the concept for a robotic workstation to be programmed by demonstration with an instrumented tool, the components of the instrumented tool as well as relevant calibration steps and results from application of the technology to an automotive preassembly process..

MASD: A Multimodal Assembly Skill Decoding System for Robot Programming by Demonstration

Programming by demonstration (PBD) transforms the robot programming from the code level to automated interface between robot and human, promoting the flexibility of robotized automation. In this paper, we focus on programming the industrial robot for assembly tasks by parsing the human demonstration into a series of assembly skills and compiling the skill to the robot executables. To achieve this goal, an identification system using multimodal information to recognize the assembly skill, called MASD, is proposed including: 1) an initial learning stage using a hierarchical model to recognize the action by considering the features from action-object effect, gesture, and trajectory and 2) a retrospective thinking stage using a segmentation method to cut the continuous demonstrations into multiple assembly skills optimally. Using MASD, the demonstration of assembly tasks can be explained with high accuracy in real time, driving a hypothesis that a PBD system on the top of MASD can be extended to more realistic assembly tasks beyond pure positional moving and picking. In experiments, the skill identification module is used to recognize the five kinds of assembly skills in demonstrations of both single and multiple assembly skills, and outperforms the comparative action identification methods. Besides integrated with the MASD, the PBD system can generate the program based on the demonstration and successfully enable an ABB industrial robotic arm simulator to assemble a flashlight and a switch, verifying the initial hypothesis. Note to Practitioners —In the conventional robotized automation, the key role of the robot mainly owes to its capacity for repeating a wide variety of tasks with high speed and accuracy in long term, with a cost of days to months of programming for deployment. On the other hand, the new trend of customization brings the new characteristics: production in short cycle and small volume. This irreversible momentum urges the robot to switch from task to task efficiently. The biggest bottleneck here is the tedious programming, which also has high prerequisites for most practitioners in manufacturing. This situation motivates the development of a PBD system that can understand the assembly skills performed by the human experts in the demonstration and accordingly generate the program for robot’s execution of the taught task. In this paper, we present a skill decoding system to parse the observational raw demonstration into symbolic sequences, which is the crucial bridge to enable the automatic programming. The system achieves high performance in recognition and is tailored for the PBD in assembly tasks by considering both advantages and disadvantages in the background of assembly, such as controllable environment and limited computational resources. It is particularly useful for assembly tasks with modularized actions based on a set of standard parts. At the perspective of industrial application, the PBD upon the proposed system is a promising solution to improve the flexibility of manufacture, which is expected to be true in midterm but an important step toward this goal.

Perception of Demonstration for Automatic Programing of Robotic Assembly: Framework, Algorithm, and Validation

In this paper, we focus on developing an intelligent perception system that can generate the task program for the robot to realize programming by demonstration (PBD) in industrial assembly tasks. The core problem of the system is to understand the semantics of the parts and skills in the visually observed demonstration. We present a probabilistic framework to state this problem as the joint inference of actions in the skill, as well as the parts poses and relations. To estimate the former, a multimodal information fusion action recognition algorithm is developed that makes use of the prior knowledge in the assembly task, so that real-time performance is achieved. While for the latter, an expectation maximization based algorithm is designed to generate solutions meeting the high accuracy required by the assembly. The perception system is validated on comprehensive simulation data and the real assembly tasks by committing the generated program to an ABB industrial robot. The satisfactory results indicate the effectiveness and feasibility of the system, finally verifying our hypothesis that PBD can be applied in assembly tasks to achieve the “teacher” and “student” scenario with the proposed perception system.

Programming a Pneumatic Processes Sequence based on PLC by Demonstration

Abstract
 Pneumatic processes sequence (PPS) is used widely in industrial applications. It is common to do a predetermined PPS to achieve a specific larger task within the industrial application like the PPS achieved by the pick and place industrial robot arm. This sequence may require change depending on changing the required task and usually this requires the programmer intervention to change the sequence’ sprogram, which is costly and may take long time. In this research a PLC-based PPS control system is designed and implemented, in which the PPS is programmed by demonstration. The PPS could be changed by demonstrating the new required sequence via the user by following simple series of manual steps without having to change the PLC’s original program, which leads to decreasing the cost and time.
 Keywords: FIFO function block, Programming by Demonstration (PbD), System Frame.

Programming by Demonstration in DLProlog

Programming by Demonstration in DLProlog

WirelessSyncroVision: Wireless synchronization for industrial stereoscopic systems

The research proposes a novel technological solution for marker-based human motion capture called WirelessSyncroVision (WSV). The WSV is formed by two main modules: the visual node (WSV-V) which is based on a stereoscopic vision system and the marker node (WSV-M) that is constituted by a 6-DOF active marker. The solution synchronizes the acquisition of images in remote muti-cameras with the ON period of the active marker. This increases the robustness of the stereoscopic system to illumination changes, which is extremely relevant for programming industrial robotic-arms using a human demonstrator programming by demonstration (PbD). In addition, the research presents a robust method named Adaptive and Robust Synchronization (ARS), that is designed for temporal alignment of remote devices using a wireless network. The algorithm models the phase difference as a function of time, measuring the parameters that must be known to predict the synchronization instant between the active marker and the remote cameras. Results demonstrate that the ARS creates a balance between the real-time capability and the performance estimation of the phase difference. Therefore, this research proposes an elegant solution to synchronize image acquisition systems in real-time that is easy to implement with low operational costs; however, the major advantage of the WSV is related to its high level of flexibility since it can be extended toward to other devices besides the PbD, for instance, motion capture, motion analysis, and remote sensoring systems.

Robot Programming by Demonstration with Crowdsourced Action Fixes

Programming by Demonstration (PbD) can allow end-users to teach robots new actions simply by demonstrating them. However, learning generalizable actions requires a large number of demonstrations that is unreasonable to expect from end-users. In this paper, we explore the idea of using crowdsourcing to collect action demonstrations from the crowd. We propose a PbD framework in which the end-user provides an initial seed demonstration, and then the robot searches for scenarios in which the action will not work and requests the crowd to fix the action for these scenarios. We use instance-based learning with a simple yet powerful action representation that allows an intuitive visualization of the action. Crowd workers directly interact with these visualizations to fix them. We demonstrate the utility of our approach with a user study involving local crowd workers (N=31) and analyze the collected data and the impact of alternative design parameters so as to inform a real-world deployment of our system.

Soft Sensor for Multiphase and Multimode Processes Based on Gaussian Mixture Regression

For complex industrial plants with multiphase and multimode characteristic, traditional multivariate statistical soft sensor methods are not applicable as Gaussian distribution assumption of data is not met. Thus Gaussian mixture model (GMM) is used to approximate data distribution. In the previous GMM-based soft sensor modeling researches, GMM is only used to identify operating mode, then other regression algorithms like PLS are used for quality prediction in different localized modes. In this article, an existing method—Gaussian mixture regression (GMR) is introduced for soft sensor modeling, which has been already successfully applied in robot programming by demonstration (PbD). Different from past GMM-based soft sensors, GMR is directly used for regression. In GMR, data mode identification and regression are incorporated into one model, thus there is no need to switch prediction model when data mode has changed. Feasibility and efficiency of GMR based soft sensor are validated in the fermentation process and TE process.

Ensuring kinetostatic consistency in observation of human manipulation

The aim of this paper is to present a method to guarantee the kinetostatic consistency in observation of human manipulation, i.e. the consistency between the observed hand posture and the tactile information on the contact between the fingertips and the objects. The core idea of the proposed algorithm is to compare the fingertip contact information, obtained by tactile sensors, with the contact information computed in a virtual environment, that reproduces the real environment where the observation is carried out. In case the estimation of the joint angles and the relative pose between the hand and the object are not consistent, a correction of the hand posture is computed. For some tasks, collisions might occur between parts of the hand (e.g. palm) and the grasped object. To handle this problem, the corrected hand posture is computed by adopting a closed loop inverse kinematic (CLIK) approach that exploits the redundant Degrees of Freedom (DoFs) of the hand. The algorithm has been designed to work on-line. This feature is particularly important for Programming by Demonstration (PbD) applications, since it allows the trainer to actively adapt the demonstration to measurement noise and model errors. The effectiveness of the proposed method has been tested in five different tasks: grasping a cup, unscrewing a bottle, grasping a plate, grasping a ketchup bottle, and grasping a measuring cup.

A Method for Learning Macro-Actions for Virtual Characters Using Programming by Demonstration and Reinforcement Learning

The decision-making by agents in games is commonly based on reinforcement learning. To improve the quality of agents, it is necessary to solve the problems of the time and state space that are required for learning. Such problems can be solved by Macro-Actions, which are defined and executed by a sequence of primitive actions. In this line of research, the learning time is reduced by cutting down the number of policy decisions by agents. Macro-Actions were originally defined as combinations of the same primitive actions. Based on studies that showed the generation of Macro-Actions by learning, Macro-Actions are now thought to consist of diverse kinds of primitive actions. However an enormous amount of learning time and state space are required to generate Macro-Actions. To resolve these issues, we can apply insights from studies on the learning of tasks through Programming by Demonstration (PbD) to generate Macro- Actions that reduce the learning time and state space. In this paper, we propose a method to define and execute Macro-Actions. Macro-Actions are learned from a human subject via PbD and a policy is learned by reinforcement learning. In an experiment, the proposed method was applied to a car simulation to verify the scalability of the proposed method. Data was collected from the driving control of a human subject, and then the Macro- Actions that are required for running a car were generated. Furthermore, the policy that is necessary for driving on a track was learned. The acquisition of Macro-Actions by PbD reduced the driving time by about 16% compared to the case in which Macro-Actions were directly defined by a human subject. In addition, the learning time was also reduced by a faster convergence of the optimum policies.

A Sequential Action Generation Framework for Intelligent Agents in Cyberspace

Intelligent agents learn their behaviors in games or crowd simulations for diverse behaviors. Given that iterative learning is required to generate behaviors by learning, programming by demonstration (PbD) can be applied to such intelligent agents for reducing learning time. One line of research has focused on how to define portions of sequential actions executed by a predecessor as building blocks of sequential actions to be executed by an intelligent agent. These studies face the following problems: first, the approach to divide collected actions using the variance of actions in consecutive order cannot derive consecutive actions in various ways; and second, the approach deriving consecutive actions in all cases increases the amount of calculation depending on the number of actions. This paper proposes a behavior generation framework that includes functions to reduce the amount of computation to generate a set of behaviors for an intelligent agent. In an experiment applying the framework to a driving simulation, the number of movements was reduced to 24% by filtering movements. In addition, the generated behaviors were reduced to 23% by quantization and filtering. Thus, the amount of calculation required to generate the behaviors was reduced.

Coupled dynamical system based arm–hand grasping model for learning fast adaptation strategies

Performing manipulation tasks interactively in real environments requires a high degree of accuracy and stability. At the same time, when one cannot assume a fully deterministic and static environment, one must endow the robot with the ability to react rapidly to sudden changes in the environment. These considerations make the task of reach and grasp difficult to deal with. We follow a Programming by Demonstration (PbD) approach to the problem and take inspiration from the way humans adapt their reach and grasp motions when perturbed. This is in sharp contrast to previous work in PbD that uses unperturbed motions for training the system and then applies perturbation solely during the testing phase. In this work, we record the kinematics of arm and fingers of human subjects during unperturbed and perturbed reach and grasp motions. In the perturbed demonstrations, the target’s location is changed suddenly after the onset of the motion. Data show a strong coupling between the hand transport and finger motions. We hypothesize that this coupling enables the subject to seamlessly and rapidly adapt the finger motion in coordination with the hand posture. To endow our robot with this competence, we develop a coupled dynamical system based controller, whereby two dynamical systems driving the hand and finger motions are coupled. This offers a compact encoding for reach-to-grasp motions that ensures fast adaptation with zero latency for re-planning. We show in simulation and on the real iCub robot that this coupling ensures smooth and “human-like” motions. We demonstrate the performance of our model under spatial, temporal and grasp type perturbations which show that reaching the target with coordinated hand–arm motion is necessary for the success of the task.

Visual programming by demonstration of grasping skills in the context of a mobile service robot using 1D-topology based self-organizing-maps

An approach to the task of Programming by demonstration (PbD) of grasping skills is introduced, where a mobile service robot is taught by a human instructor how to grasp a specific object. In contrast to other approaches the instructor demonstrates the grasping action several times to the robot to increase reconstruction performance. Only the robot’s stereoscopic vision system is used to track the instructor’s hand. The developed tracking algorithm is designed to not need artificial markers, data gloves or being restricted to fixed or difficult to calibrate sensor installations while at the same time being real-time capable on a mobile service robot with limited resources. Due to the instructor’s repeated demonstrations and his low repeating accuracy, every time a grasp is demonstrated the instructor performs it differently. To compensate for these variations and also to compensate for tracking errors, the use of a Self-Organizing-Map (SOM) with a one-dimensional topology is proposed. This SOM is used to generalize over differently demonstrated grasping actions and to reconstruct the intended approach trajectory of the instructor’s hand while grasping an object. The approach is implemented and evaluated on the service robot TASER using synthetically generated data as well as real world data.

Haptic and Visual Training of System Behavior – a case study for Robotic Programming-by-Demonstration

Programming-by-demonstration (PBD) is a new paradigm for programming industrial robots enabled by the development of the DLR/KUKA light-weight robot. Although the PBD approach facilitates and simplifies the generation of robot programs, the technician still needs to have skills and knowledge about the robotic system in order to produce efficient trajectories and to exploit the abilities of the robot in an optimal way. Within the EU-SKILLS project a robotic training system and protocol was developed to enable enactive learning of robotic skills and abilities. The paper presents the evaluation of the skill transfer for robotic PBD based on enactive learning.

Grasp programming by demonstration in virtual reality with automatic environment reconstruction

A virtual reality system enabling high-level programming of robot grasps is described. The system is designed to support programming by demonstration (PbD), an approach aimed at simplifying robot programming and empowering even unexperienced users with the ability to easily transfer knowledge to a robotic system. Programming robot grasps from human demonstrations requires an analysis phase, comprising learning and classification of human grasps, as well as a synthesis phase, where an appropriate human-demonstrated grasp is imitated and adapted to a specific robotic device and object to be grasped. The virtual reality system described in this paper supports both phases, thereby enabling end-to-end imitation-based programming of robot grasps. Moreover, as in the PbD approach robot environment interactions are no longer explicitly programmed, the system includes a method for automatic environment reconstruction that relieves the designer from manually editing the pose of the objects in the scene and enables intelligent manipulation. A workspace modeling technique based on monocular vision and computation of edge-face graphs is proposed. The modeling algorithm works in real time and supports registration of multiple views. Object recognition and workspace reconstruction features, along with grasp analysis and synthesis, have been tested in simulated tasks involving 3D user interaction and programming of assembly operations. Experiments reported in the paper assess the capabilities of the three main components of the system: the grasp recognizer, the vision-based environment modeling system, and the grasp synthesizer.

Advances in Robot Programming by Demonstration

Robot Programming by Demonstration (PbD) has been dealt with in the literature as a promising way to teach robots new skills in an intuitive way. In this paper we describe our current work in the field toward the implementation of PbD system which allows robots to learn continuously from human observation, build generalized representations of human demonstration and apply such representations to new situations.

Sheepdog, parallel collaborative programming-by-demonstration

We introduce parallel collaborative programming-by-demonstration (PBD) as a principled approach to capturing knowledge on how to perform computer-based procedures by independently recording multiple experts executing these tasks and combining the recordings via a learning algorithm. Traditional PBD has focused on end-user programming for a single user, and does not support parallel collaborative procedure model construction from examples provided by multiple experts. In this paper we discuss how to extend the main aspects of PBD (instrumentation, abstraction, learning, and execution), and we describe the implementation of these extensions in a system called Sheepdog.