Robot Execution Research Articles

Detection of Subtle Context-Dependent Model Inaccuracies in High-Dimensional Robot Domains.

Autonomous robots often rely on models of their sensing and actions for intelligent decision making. However, when operating in unconstrained environments, the complexity of the world makes it infeasible to create models that are accurate in every situation. This article addresses the problem of using potentially large and high-dimensional sets of robot execution data to detect situations in which a robot model is inaccurate-that is, detecting context-dependent model inaccuracies in a high-dimensional context space. To find inaccuracies tractably, the robot conducts an informed search through low-dimensional projections of execution data to find parametric Regions of Inaccurate Modeling (RIMs). Empirical evidence from two robot domains shows that this approach significantly enhances the detection power of existing RIM-detection algorithms in high-dimensional spaces.

English

English

Needle Grasp and Entry Port Selection for Automatic Execution of Suturing Tasks in Robotic Minimally Invasive Surgery.

This paper presents algorithms for selection of needle grasp and for selection of entry ports of robotic instruments, for autonomous robotic execution of the minimally invasive surgical suturing task. A critical issue for automatic execution of surgical tasks, such as suturing, is the choice of needle grasp for the robotic system. Inappropriate needle grasp increases operating time requiring multiple regrasps to complete the desired task. In robotic minimally invasive surgery, the entry port that the surgical robot goes through into the patient's body has a significant role on the performance of the robot. Improper entry port affects the robot's dexterity, manipulability and reachability. The proposed methods use manipulability, dexterity and torque metrics for needle grasp selection, and employ needle grasp robustness and target location robustness metrics for port selection. The results of a case study simulation in thoracoscopic surgery is also presented to demonstrate the proposed methods. Note to Practitioners-This paper is motivated by the problem of automating low-level surgical tasks in robotic surgery, such as, suturing, retraction, dissection, and providing exposure. Specifically, this paper focuses on needle grasp and entry port selection for automating robotic surgical suturing. Selection of an appropriate way of grasping a needle is critical for successfully and robustly completing autonomous suturing. To the best authors' knowledge, there are no earlier studies in the literature which focus on the needle grasp selection problem. The proposed approach determines how to grasp the needle by optimizing the surgical system's manipulation performance. The existing approaches in the literature for selecting entry ports for the robotic surgical tools only consider the teleoperated robotic minimally invasive surgery, in which the surgeons directly control the robotic instruments. However, automated performance of suturing introduces additional challenges due to uncertainties in needle localization and grasping. This paper proposes two new performance metrics on selecting port locations from the perspective of autonomously performing surgical suturing, without direct involvement of the human user. The paper also presents preliminary experiments which demonstrate the effectiveness of the proposed methods.

A probabilistic approach to workspace sharing for human–robot cooperation in assembly tasks

Abstract Recently, human–robot cooperation (HRC) research activities have focused on the development of new methodologies for the generation of safe robot trajectories. However, the applicability of such methodologies in a real context is limited due to the inherent uncertainty of robot trajectory execution time (i.e. the robot can avoid the worker by modifying its velocity along the path). This paper proposes an approach to estimate a confidence interval on robot trajectory execution time for scenarios in which human–robot space sharing is required. First, human arm movements are studied for a given set of assembly collaborative tasks: worker occupancy volumes and occupancy volume probabilities are derived. Then, a finite number of alternative robot trajectories, crossing human occupancy volumes with different occupancy probabilities, are generated. It is therefore possible to estimate a probability for the robot to reduce its velocity, and a confidence interval on the robot execution time. The application on a real assembly case is discussed.

An automatic system for humanoid dance creation

Abstract The paper describes a novel approach to allow a robot to dance following musical rhythm. The proposed system generates a dance for a humanoid robot through the combination of basic movements synchronized with the music. The system made up of three parts: the extraction of features from audio file, estimation of movements through the Hidden Markov Models and, finally, the generation of dance. Starting from a set of given movements, the robot choices sequence of movements a suitable Hidden Markov Model, and synchronize them processing musical input. The proposed approach has the advantage that movement execution probabilities could be changed according evaluation of the dance execution in order to have an artificial creative system. In the same way, a choreographer could give more importance to some movements and/or exclude others, using the system as a co-creation tool. The approach has been tested on Aldebaran NAO humanoid using different genres of music, and experimentations was conduct at presence of real human dancers to have feedback of the goodness of the robot execution. Three professional judges expressed their evaluations about the following points: appropriateness of movements for a given musical genre; the precision to track the rhythm; the aesthetic impact of the whole sequence of movements; and the overall judgment of the robot performance. All the evaluations are very satisfying, and confirm that robot dance is realistic and aesthetically acceptable. The robustness and flexibility of the system allow us to embed the system in artificial creative system in future work. In the discussion we introduce some issues to pursuit this aim, using a previous proposed cognitive architecture based on needs and motivations.

A Study on Bipedal and Mobile Robot Behavior Through Modeling and Simulation

The purpose of this work is to study and analyze mobile robot behavior. In performing this, a framework is adopted and developed for mobile and bipedal robot. The robots are design, build, and run as proceed from the development of mechanical structure, electronics and control integration, and control software application. The behavior of those robots are difficult to be observed and analyzed qualitatively. To evaluate the design and behavior quality, modeling and simulation of robot structure and its task capability is performed. The stepwise procedure to robot behavior study is explained. Behavior cases study are experimented to bipedal robots, transporter robot and Autonomous Guided Vehicle (AGV) developed at our institution. The experimentation are conducted on those robots by adjusting their dynamic properties and/or surrounding environment. Validation is performed by comparing the simulation result and the real robot execution. The simulation gives a more idealistic behavior execution rather than realistic one. Adjustments are performed to fine tuning simulation's parameters to provide a more realistic performance.

A Study on Bipedal And Mobile Robot Behavior through Modeling And Simulation

The purpose of this work is to study and analyze mobile robot behavior. In performing this, a framework is adopted and developed for mobile and bipedal robot. The robots are design, build, and run as proceed from the development of mechanical structure, electronics and control integration, and control software application. The behavior of those robots are difficult to be observed and analyzed qualitatively. To evaluate the design and behavior quality, modeling and simulation of robot structure and its task capability is performed. The stepwise procedure to robot behavior study is explained. Behavior cases study are experimented to bipedal robots, transporter robot and Autonomous Guided Vehicle (AGV) developed at our institution. The experimentation are conducted on those robots by adjusting their dynamic properties and/or surrounding environment. Validation is performed by comparing the simulation result and the real robot execution. The simulation gives a more idealistic behavior execution rather than realistic one. Adjustments are performed to fine tuning simulation's parameters to provide a more realistic performance.

Bioanalytical laboratory automation development: why should we and how could we collaborate?

Bioanalytical laboratory automation development: why should we and how could we collaborate?

Development of a Survivable Cloud Multi-Robot Framework for Heterogeneous Environments

Cloud robotics is a paradigm that allows for robots to offload computationally intensive and data storage requirements into the cloud by providing a secure and customizable environment. The challenge for cloud robotics is the inherent problem of cloud disconnection. A major assumption made in the development of the current cloud robotics frameworks is that the connection between the cloud and the robot is always available. However, for multi-robots working in heterogeneous environments, the connection between the cloud and the robots cannot always be guaranteed. This work serves to assist with the challenge of disconnection in cloud robotics by proposing a survivable cloud multi-robotics (SCMR) framework for heterogeneous environments. The SCMR framework leverages the combination of a virtual ad hoc network formed by robot-to-robot communication and a physical cloud infrastructure formed by robot-to-cloud communications. The quality of service (QoS) on the SCMR framework was tested and validated by determining the optimal energy utilization and time of response (ToR) on drivability analysis with and without cloud connection. The design trade-off, including the result, is between the computation energy for the robot execution and the offloading energy for the cloud execution.

A new benchmark for pose estimation with ground truth from virtual reality

The development of programming paradigms for industrial assembly currently gets fresh impetus from approaches in human demonstration and programming-by-demonstration. Major low- and mid-level prerequisites for machine vision and learning in these intelligent robotic applications are pose estimation, stereo reconstruction and action recognition. As a basis for the machine vision and learning involved, pose estimation is used for deriving object positions and orientations and thus target frames for robot execution. Our contribution introduces and applies a novel benchmark for typical multi-sensor setups and algorithms in the field of demonstration-based automated assembly. The benchmark platform is equipped with a multi-sensor setup consisting of stereo cameras and depth scanning devices (see Fig. 1). The dimensions and abilities of the platform have been chosen in order to reflect typical manual assembly tasks. Following the eRobotics methodology, a simulatable 3D representation of this platform was modelled in virtual reality. Based on a detailed camera and sensor simulation, we generated a set of benchmark images and point clouds with controlled levels of noise as well as ground truth data such as object positions and time stamps. We demonstrate the application of the benchmark to evaluate our latest developments in pose estimation, stereo reconstruction and action recognition and publish the benchmark data for objective comparison of sensor setups and algorithms in industry.

Exploring the acquisition and production of grammatical constructions through human-robot interaction with echo state networks.

One of the principal functions of human language is to allow people to coordinate joint action. This includes the description of events, requests for action, and their organization in time. A crucial component of language acquisition is learning the grammatical structures that allow the expression of such complex meaning related to physical events. The current research investigates the learning of grammatical constructions and their temporal organization in the context of human-robot physical interaction with the embodied sensorimotor humanoid platform, the iCub. We demonstrate three noteworthy phenomena. First, a recurrent network model is used in conjunction with this robotic platform to learn the mappings between grammatical forms and predicate-argument representations of meanings related to events, and the robot's execution of these events in time. Second, this learning mechanism functions in the inverse sense, i.e., in a language production mode, where rather than executing commanded actions, the robot will describe the results of human generated actions. Finally, we collect data from naïve subjects who interact with the robot via spoken language, and demonstrate significant learning and generalization results. This allows us to conclude that such a neural language learning system not only helps to characterize and understand some aspects of human language acquisition, but also that it can be useful in adaptive human-robot interaction.

Technical Activities Execution with a TiltRotor UAS employing Explicit Model Predictive Control

An innovative application field for Unmanned Aerial Systems (UASs) is the subject of this paper. The aerial robotic execution of technical activities based on autonomous aerial platforms poses important challenges, as realistic industrial activities are physically demanding. Typical environment-modifying tasks, such as surface grinding, require the exertion of significant forces in order to be successfully executed. For such purposes, the exploitation of thrust-vectoring actuation is proposed, and a methodology for achieving longitudinal force exertion while retaining safe operation is developed, relying on the platform's exceptional actuation features, a piecewise-affine representation of the system modes, and an explicit model predictive control scheme. The experimental demonstration of the proposed strategy is conducted utilizing a compound UAS, consisting of a high-end tilt-rotor vehicle, mounted with an end-effector which carries a motorized tool customized for surface grinding tasks.

An improved kernel based extreme learning machine for robot execution failures.

Robot execution failures prediction (classification) in the robot tasks is a difficult learning problem due to partially corrupted or incomplete measurements of data and unsuitable prediction techniques for this prediction problem with little learning samples. Therefore, how to predict the robot execution failures problem with little (incomplete) or erroneous data deserves more attention in the robot field. For improving the prediction accuracy of robot execution failures, this paper proposes a novel KELM learning algorithm using the particle swarm optimization approach to optimize the parameters of kernel functions of neural networks, which is called the AKELM learning algorithm. The simulation results with the robot execution failures datasets show that, by optimizing the kernel parameters, the proposed algorithm has good generalization performance and outperforms KELM and the other approaches in terms of classification accuracy. Other benchmark problems simulation results also show the efficiency and effectiveness of the proposed algorithm.

Neural networks for prediction of robot failures

It is known that the supervision and learning of robotic executions is not a trivial problem. Nowadays, robots must be able to tolerate and predict internal failures in order to successfully continue performing their tasks. This study presents a novel approach for prediction of robot execution failures based on neural networks. Real data consisting of robot forces and torques recorded immediately after the system failure are used for the neural network training. The multilayer feedforward neural networks are employed in order to find optimal solution for the failure prediction problem. In total, 7 learning algorithms and 24 neural architectures are implemented in two environments – Matlab and specially designed software titled BPnet. The results show that the neural networks can successfully be applied for the problem in hand with prediction rate of 95.4545%, despite having the erroneous or otherwise incomplete sensor measurements invoked in the dataset. Additionally, the real-world experiments are conducted on a mobile robot for obstacle detection and trajectory tracking problems in order to prove the robustness of the proposed prediction approach. In over 96% for the detection problem and 99% for the tracking experiments, neural network successfully predicted the failed information, which evidences the usefulness and the applicability of the developed intelligent method.

A robot learning from demonstration framework to perform force-based manipulation tasks

This paper proposes an end-to-end learning from demonstration framework for teaching force-based manipulation tasks to robots. The strengths of this work are manyfold. First, we deal with the problem of learning through force perceptions exclusively. Second, we propose to exploit haptic feedback both as a means for improving teacher demonstrations and as a human---robot interaction tool, establishing a bidirectional communication channel between the teacher and the robot, in contrast to the works using kinesthetic teaching. Third, we address the well-known what to imitate? problem from a different point of view, based on the mutual information between perceptions and actions. Lastly, the teacher's demonstrations are encoded using a Hidden Markov Model, and the robot execution phase is developed by implementing a modified version of Gaussian Mixture Regression that uses implicit temporal information from the probabilistic model, needed when tackling tasks with ambiguous perceptions. Experimental results show that the robot is able to learn and reproduce two different manipulation tasks, with a performance comparable to the teacher's one.

Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals

Previous studies have demonstrated the possibility of using functional MRI to control a robot arm through a brain–machine interface by directly coupling haemodynamic activity in the sensory–motor cortex to the position of two axes. Here, we extend this work by implementing interaction at a more abstract level, whereby imagined actions deliver structured commands to a robot arm guided by a machine vision system. Rather than extracting signals from a small number of pre-selected regions, the proposed system adaptively determines at individual level how to map representative brain areas to the input nodes of a classifier network. In this initial study, a median action recognition accuracy of 90% was attained on five volunteers performing a game consisting of collecting randomly positioned coloured pawns and placing them into cups. The “pawn” and “cup” instructions were imparted through four mental imaginery tasks, linked to robot arm actions by a state machine. With the current implementation in MatLab language the median action recognition time was 24.3s and the robot execution time was 17.7s. We demonstrate the notion of combining haemodynamic brain–machine interfacing with computer vision to implement interaction at the level of high-level commands rather than individual movements, which may find application in future fMRI approaches relevant to brain-lesioned patients, and provide source code supporting further work on larger command sets and real-time processing.

Framework and service allocation for network robot platform and execution of interdependent services

This research introduces a framework for network robot platform (NWR-PF) and a service allocation method for heterogeneous distributed robots; they cooperate to perform services that include interdependent tasks. The proposed framework is composed of three layers: connection units, the area management gateway, and the robot-user interaction database. The area management gateway and robot-user interaction database treat the information of the users, robots, services, and service history in a uniform manner, because the connection units hide the differences, such as format and protocol, among the heterogeneous robots. The 4W1H matching method is also proposed for service allocation. The method selects the most suitable robots by comparing the elements of user, robot, and service information. Moreover, it generates robot commands in a common format, by combining the service scenario, and service history. A feature of the method is that both robotic functions to execute service and functions that robots possess are described as independent parameters. After the service trigger is received, both the robot and scenario needed to execute the service are decided by matching these parameters. Experiments show that robots controlled by NWR-PF can perform interdependent services by referring to the service history.

From Abstract Task Knowledge to Executable Robot Programs

Robots that are capable of learning new tasks from humans need the ability to transform gathered abstract task knowledge into their own representation and dimensionality. New task knowledge that has been collected e.g. with Programming by Demonstration approaches by observing a human does not a-priori contain any robot-specific knowledge and actions, and is defined in the workspace of the human demonstrator. This article presents a new approach for mapping abstract human-centered task knowledge to a robot execution system based on the target system properties. Therefore the required background knowledge about the target system is examined and defined explicitly.

Robotic execution of everyday tasks by means of external vision/force control

In this article, we present an integrated manipulation framework for a service robot, that allows to interact with articulated objects at home environments through the coupling of vision and force modalities. We consider a robot which is observing simultaneously his hand and the object to manipulate, by using an external camera (i.e. robot head). Task-oriented grasping algorithms (Proc of IEEE Int Conf on robotics and automation, pp 1794–1799, 2007) are used in order to plan a suitable grasp on the object according to the task to perform. A new vision/force coupling approach (Int Conf on advanced robotics, 2007), based on external control, is used in order to, first, guide the robot hand towards the grasp position and, second, perform the task taking into account external forces. The coupling between these two complementary sensor modalities provides the robot with robustness against uncertainties in models and positioning. A position-based visual servoing control law has been designed in order to continuously align the robot hand with respect to the object that is being manipulated, independently of camera position. This allows to freely move the camera while the task is being executed and makes this approach amenable to be integrated in current humanoid robots without the need of hand-eye calibration. Experimental results on a real robot interacting with different kind of doors are presented.

다변량 스트림 데이터 축소 기법 평가

센서 네트워크는 애플리케이션 분야에 따라 데이터 특성과 사용자의 요구사항이 다양함에도 불구하고, 현존하는 스트림 데이터 축소 연구는 데이터의 본질적인 특징보다 특정 축소 기법의 성능 향상 측면에 중점을 두고 있다. 이 논문은 계층/분산형 센서 네트워크 구조와 데이터 모델을 소개하고, 선택적으로 축소 기법을 적용하기 위해 데이터 특성과 사용자의 요구에 적합한 다변량 데이터 축소 기법을 비교 평가한다. 다변량 데이터 축소 기법의 성능을 비교 분석하기 위해, 우리는 웨이블릿, HCL(Hierarchical Clustering), SVD(Singular Value Decomposition), 샘플링과 같은 표준화 된 다변량 축소 기법을 이용한다. 실험 데이터는 다차원 시계열 데이터와 로봇 센서 데이터를 사용한다. 실험 결과 SVD와 샘플링 기법이 상대 에러 비율과 수행 성능 측면에서 웨이블릿과 HCL기법에 비해 우수하였다. 특히 각 데이터 축소 기법의 상대 에러 비율은 입력 데이터 특성에 따라 다르기 때문에 선택적으로 데이터 축소 기법을 적용하는 것이 좋은 성능을 보였다. 이 논문은 다차원 센서 데이터가 수집되는 센서 네트워크를 디자인하고 구축하는 응용 분야에 유용하게 활용될 것이다. Even though sensor networks are different in user requests and data characteristics depending on each application area, the existing researches on stream data transmission problem focus on the performance improvement of their methods rather than considering the original characteristic of stream data. In this paper, we introduce a hierarchical or distributed sensor network architecture and data model, and then evaluate the multivariate data reduction methods suitable for user requirements and data features so as to apply reduction methods alternatively. To assess the relative performance of the proposed multivariate data reduction methods, we used the conventional techniques, such as Wavelet, HCL(Hierarchical Clustering), Sampling and SVD (Singular Value Decomposition) as well as the experimental data sets, such as multivariate time series, synthetic data and robot execution failure data. The experimental results shows that SVD and Sampling method are superior to Wavelet and HCL ia respect to the relative error ratio and execution time. Especially, since relative error ratio of each data reduction method is different according to data characteristic, it shows a good performance using the selective data reduction method for the experimental data set. The findings reported in this paper can serve as a useful guideline for sensor network application design and construction including multivariate stream data.