DARPA Robotics Challenge Research Articles

Context-aware system synthesis, task assignment, and routing

The design and organization of complex robotic systems traditionally requires laborious trial-and-error processes to ensure both hardware and software components are correctly connected with the resources necessary for computation. This paper presents a novel generalization of the quadratic assignment and routing problem, introducing formalisms for selecting components and interconnections to synthesize a complete system capable of providing some user-defined functionality. By introducing mission context, functional requirements, and modularity directly into the assignment problem, we derive a solution where components are automatically selected and placed into an optimal organization of hardware and software, all while respecting restrictions on component viability and required functionality. The ability to generate complete functional systems directly from individual components reduces manual design effort by allowing for a guided exploration of the design space. Additionally, our formulation increases resiliency by quantifying resource margins and enabling adaptation of system structure in response to changing environments, hardware or software failure. The proposed formulation is cast as an integer linear program which is provably \({\mathcal {N}}{\mathcal {P}}\)-hard. Two case studies are analyzed to highlight the expressiveness and complexity of problems that are addressable by this approach: the first explores the iterative development of a ground-based search-and-rescue robot in varying mission contexts, while the second compares an automated humanoid design against a manual design that was finalist in the DARPA Robotics Challenge. Numerical simulations quantify real-world performance and demonstrate tractable time complexity for the scale of problems encountered in many modern robotic systems.

How Robots Can Help US Act and Feel Younger: Toyota's Gill Pratt on Enhancing Independence in Old Age

The number of people over the age of 80 will triple, approaching half a billion. Supporting an aging population is a worldwide concern, but this demographic shift is especially pronounced in Japan, where more than a third of Japanese will be 65 or older by midcentury.• Toyota Research Institute (TRI), which was established by Toyota Motor Corp. in 2015 to explore autonomous cars, robotics, and “human amplification technologies,” has also been focusing a significant portion of its research on ways to help older people maintain their health, happiness, and independence as long as possible. While it's an important goal in itself, improving self-sufficiency for the elderly also reduces the amount of support they need from society more broadly. And without technological help, sustaining this population in an effective and dignified manner will grow increasingly difficult-first in Japan, but globally soon after.• Gill Pratt, Toyota's chief scientist and the CEO of TRI, believes that robots have a significant role to play in assisting older people by solving physical problems as well as providing mental and emotional support. With a background in robotics research and five years as a program manager at the Defense Advanced Research Projects Agency, during which time he oversaw the DARPA Robotics Challenge in 2015, Pratt understands how difficult it can be to bring robots into the real world in a useful, responsible, and respectful way. In an interview earlier this year in Washington, D.C., with IEEE Spectrum's Evan Ackerman, he said that the best approach to this problem is a human-centric one: “It's not about the robot, it's about people.”

Carved Turn Control with Gate Vision Recognition of a Humanoid Robot for Giant Slalom Skiing on Ski Slopes.

The performance of humanoid robots is improving, owing in part to their participation in robot games such as the DARPA Robotics Challenge. Along with the 2018 Winter Olympics in Pyeongchang, a Skiing Robot Competition was held in which humanoid robots participated autonomously in a giant slalom alpine skiing competition. The robots were required to transit through many red or blue gates on the ski slope to reach the finish line. The course was relatively short at 100 m long and had an intermediate-level rating. A 1.23 m tall humanoid ski robot, ‘DIANA’, was developed for this skiing competition. As a humanoid robot that mimics humans, the goal was to descend the slope as fast as possible, so the robot was developed to perform a carved turn motion. The carved turn was difficult to balance compared to other turn methods. Therefore, ZMP control, which could secure the posture stability of the biped robot, was applied. Since skiing takes place outdoors, it was necessary to ensure recognition of the flags in various weather conditions. This was ensured using deep learning-based vision recognition. Thus, the performance of the humanoid robot DIANA was established using the carved turn in an experiment on an actual ski slope. The ultimate vision for humanoid robots is for them to naturally blend into human society and provide necessary services to people. Previously, there was no way for a full-sized humanoid robot to move on a snowy mountain. In this study, a humanoid robot that transcends this limitation was realized.

A Shared Control Method for Collaborative Human-Robot Plug Task

Humans are superior to robots in performing complex assembly tasks primarily due to their dexterity and sensorimotor abilities. When humans and robots collaborate to complete an assembly task, it is critical to provide the robot with sufficiently precise manipulation capabilities and formulate a shared control method that coordinates human and robot actions. In this study, we are investigating human-robot collaboration in completing an assembly task, namely the plug task. The plug task, motivated by the 2015 DARPA Robotics Challenge Finals, involves inserting a plug connected with a cable into a paired socket. The human holds the socket while the robot is grasping the cable and manages to insert the plug. We estimate an initial socket pose based on human subjects' data. With the statistically calculated initial guess, the robot could act instantly from the beginning to move the plug towards an initial target. The actual socket pose is updated using the Kalman filter in real-time. Besides, the system continuously updates the cable model and tracks the plug to enable visual servoing to complete the plug task. The result of an extensive experimental study demonstrates that the proposed approach demonstrates fast and accurate performance as well as flexibility when compared to a state-of-the-art method based on interactive probabilistic movement primitives.

Model-Based Manipulation of Linear Flexible Objects: Task Automation in Simulation and Real World

Manipulation of deformable objects is a desired skill in making robots ubiquitous in manufacturing, service, healthcare, and security. Common deformable objects (e.g., wires, clothes, bed sheets, etc.) are significantly more difficult to model than rigid objects. In this research, we contribute to the model-based manipulation of linear flexible objects such as cables. We propose a 3D geometric model of the linear flexible object that is subject to gravity and a physical model with multiple links connected by revolute joints and identified model parameters. These models enable task automation in manipulating linear flexible objects both in simulation and real world. To bridge the gap between simulation and real world and build a close-to-reality simulation of flexible objects, we propose a new strategy called Simulation-to-Real-to-Simulation (Sim2Real2Sim). We demonstrate the feasibility of our approach by completing the Plug Task used in the 2015 DARPA Robotics Challenge Finals both in simulation and real world, which involves unplugging a power cable from one socket and plugging it into another. Numerical experiments are implemented to validate our approach.

Ground Vehicle Driving by Full Sized Humanoid

This paper presents a technical overview of the Team DRC-Hubo@UNLV’s approach to the driving task in DARPA Robotics Challenge Finals (DRC-Finals). First, hardware updates in the main body and the perception head of the contestant robot, DRC-Hubo+, were presented by comparison with the previous platform (DRC-Hubo in DRC-Trials). Then, the control system which enabled the full-sized humanoid to drive the off-the-shelf utility vehicle in the competition was provided. For this, the sensor data fusion process and the advanced driving assistant techniques were emphasized. Next, the driving pattern analysis of other type operators (such as human-drivers or pure tele-operation systems) was provided to demonstrate the performance of the developed system. Lastly, test and evaluation of the built robot and control system was provided via experimentation which DRC-Hubo+ drove the vehicle in a real-world setting. The presented approach was also verified in DRC-Finals as the author’s team was placed in the top record at the driving task in the competition.

Developing Humanoid Robots for Applications in Real-World Scenarios [From the Guest Editors

Five years after the DARPA Robotics Challenge (DRC) and close to the 50th anniversary of the first fullscale anthropomorphic robot developed in the world, it is our great pleasure to present the first IEEE Robotics and Automation Magazine special issue on humanoid robot applications in real-world scenarios. The aim of this special issue is to collect in a single volume the experience of current experts in the field, represented by practical applications of humanoid robotics on real-world problems.

The Seednoid Robot Platform: Designing a Multipurpose Compact Robot From Continuous Evaluation and Lessons From Competitions

Robots are designed to tackle specific problems or are designed as a multipurpose platform that achieves a concept. In this letter, we approach the hardware design problem from practice and evaluation through numerous competitions including the Darpa Robotics Challenge, the Amazon Picking Challenge, and other competitions that were held in Japan. The paper aims to answer the question what hardware design would we achieve if we continuously refine designs through competitions? We show that competitions lead to a more compact but capable platform design that is durable and has more end effector solutions. We also point out the limitations and benefits of using competitions for designing robot platforms. Our platform began from a human-inspired design for teleoperation and has evolved to mix both human-like and industrial robot structure.

An Efficient Acyclic Contact Planner for Multiped Robots

We present a contact planner for complex legged locomotion tasks: standing up, climbing stairs using a handrail, crossing rubble, and getting out of a car. The need for such a planner was shown at the DARPA Robotics Challenge, where such behaviors could not be demonstrated (except for egress). Current planners suffer from their prohibitive algorithmic complexity because they deploy a tree of robot configurations projected in contact with the environment. We tackle this issue by introducing a reduction property: the reachability condition. This condition defines a geometric approximation of the contact manifold, which is of low dimension, presents a Cartesian topology, and can be efficiently sampled and explored. The hard contact planning problem can then be decomposed into two subproblems: first, we plan a path for the root without considering the whole-body configuration, using a sampling-based algorithm; then, we generate a discrete sequence of whole-body configurations in static equilibrium along this path, using a deterministic contact-selection algorithm. The reduction breaks the algorithm complexity encountered in previous works, resulting in the first interactive implementation of a contact planner (open source). While no contact planner has yet been proposed with theoretical completeness, we empirically show the interest of our framework: in a few seconds, with high success rates, we generate complex contact plans for various scenarios and two robots: HRP-2 and HyQ. These plans are validated in dynamic simulations or on the real HRP-2 robot.

WALK‐MAN: A High‐Performance Humanoid Platform for Realistic Environments

In this work, we present WALK‐MAN, a humanoid platform that has been developed to operate in realistic unstructured environment, and demonstrate new skills including powerful manipulation, robust balanced locomotion, high‐strength capabilities, and physical sturdiness. To enable these capabilities, WALK‐MAN design and actuation are based on the most recent advancements of series elastic actuator drives with unique performance features that differentiate the robot from previous state‐of‐the‐art compliant actuated robots. Physical interaction performance is benefited by both active and passive adaptation, thanks to WALK‐MAN actuation that combines customized high‐performance modules with tuned torque/velocity curves and transmission elasticity for high‐speed adaptation response and motion reactions to disturbances. WALK‐MAN design also includes innovative design optimization features that consider the selection of kinematic structure and the placement of the actuators with the body structure to maximize the robot performance. Physical robustness is ensured with the integration of elastic transmission, proprioceptive sensing, and control. The WALK‐MAN hardware was designed and built in 11 months, and the prototype of the robot was ready four months before DARPA Robotics Challenge (DRC) Finals. The motion generation of WALK‐MAN is based on the unified motion‐generation framework of whole‐body locomotion and manipulation (termed loco‐manipulation). WALK‐MAN is able to execute simple loco‐manipulation behaviors synthesized by combining different primitives defining the behavior of the center of gravity, the motion of the hands, legs, and head, the body attitude and posture, and the constrained body parts such as joint limits and contacts. The motion‐generation framework including the specific motion modules and software architecture is discussed in detail. A rich perception system allows the robot to perceive and generate 3D representations of the environment as well as detect contacts and sense physical interaction force and moments. The operator station that pilots use to control the robot provides a rich pilot interface with different control modes and a number of teleoperated or semiautonomous command features. The capability of the robot and the performance of the individual motion control and perception modules were validated during the DRC in which the robot was able to demonstrate exceptional physical resilience and execute some of the tasks during the competition.

Autonomous car driving by a humanoid robot

AbstractEnabling a humanoid robot to drive a car requires the development of a set of basic primitive actions. These include walking to the vehicle, manually controlling its commands (e.g., ignition, gas pedal, and steering) and moving with the whole body to ingress/egress the car. We present a sensor‐based reactive framework for realizing the central part of the complete task, consisting of driving the car along unknown roads. The proposed framework provides three driving strategies by which a human supervisor can teleoperate the car or give the robot full or partial control of the car. A visual servoing scheme uses features of the road image to provide the reference angle for the steering wheel to drive the car at the center of the road. Simultaneously, a Kalman filter merges optical flow and accelerometer measurements to estimate the car linear velocity and correspondingly compute the gas pedal command for driving at a desired speed. The steering wheel and gas pedal reference are sent to the robot control to achieve the driving task with the humanoid. We present results from a driving experience with a real car and the humanoid robot HRP‐2Kai. Part of the framework has been used to perform the driving task at the DARPA Robotics Challenge.

Team VALOR's ESCHER: A Novel Electromechanical Biped for the DARPA Robotics Challenge

The Electric Series Compliant Humanoid for Emergency Response (ESCHER) platform represents the culmination of four years of development at Virginia Tech to produce a full‐sized force‐controlled humanoid robot capable of operating in unstructured environments. ESCHER's locomotion capability was demonstrated at the DARPA Robotics Challenge (DRC) Finals when it successfully navigated the 61 m loose dirt course. Team VALOR, a Track A team, developed ESCHER leveraging and improving upon bipedal humanoid technologies implemented in previous research efforts, specifically for traversing uneven terrain and sustained untethered operation. This paper presents the hardware platform, software, and control systems developed to field ESCHER at the DRC Finals. ESCHER's unique features include custom linear series elastic actuators in both single and dual actuator configurations and a whole‐body control framework supporting compliant locomotion across variable and shifting terrain. A high‐level software system designed using the robot operating system integrated various open‐source packages and interfaced with the existing whole‐body motion controller. The paper discusses a detailed analysis of challenges encountered during the competition, along with lessons learned that are critical for transitioning research contributions to a fielded robot. Empirical data collected before, during, and after the DRC Finals validate ESCHER's performance in fielded environments.

Technical Overview of Team DRC‐Hubo@UNLV's Approach to the 2015 DARPA Robotics Challenge Finals

This paper presents a technical overview of Team DRC‐Hubo@UNLV's approach to the 2015 DARPA Robotics Challenge Finals (DRC‐Finals). The Finals required a robotic platform that was robust and reliable in both hardware and software to complete tasks in 60 min under degraded communication. With this point of view, Team DRC‐Hubo@UNLV integrated methods and algorithms previously verified, validated, and widely used in the robotics community. For the communication aspect, a common shared memory approach that the team adopted to enable efficient data communication under the DARPA controlled network is described. A new perception head design (optimized for the tasks of the Finals) and its data processing are then presented. In the motion planning and control aspect, various techniques, such as wheel‐driven navigation, zero‐moment‐point (ZMP) ‐based locomotion, and position‐based manipulation and controls, are described in this paper. By introducing strategically critical elements and key lessons learned from DRC‐Trials 2013 and the testbed of Charleston, we also illustrate how DRC‐Hubo has evolved successfully toward the DRC‐Finals.

Developing a Robust Disaster Response Robot: CHIMP and the Robotics Challenge

CHIMP, the CMU Highly Intelligent Mobile Platform, is a humanoid robot capable of executing complex tasks in dangerous, degraded, human‐engineered environments, such as those found in disaster response scenarios. CHIMP is uniquely designed for mobile manipulation in challenging environments, as the robot performs manipulation tasks using an upright posture, yet it uses more stable prostrate postures for mobility through difficult terrain. In this paper, we report on the improvements made to CHIMP—both in its mechanical design and its software systems—in preparation for the DARPA Robotics Challenge Finals in June 2015. These include details on CHIMP's novel mechanical design, actuation systems, robust construction, all‐terrain mobility, supervised autonomy approach, and unique user interfaces utilized for the challenge. Additionally, we provide an overview of CHIMP's performance, and we detail the various lessons learned over the course of the challenge. CHIMP was one of the winners of the DARPA Robotics Challenge, completing all tasks and finishing in 3rd place out of 23 teams. Notably, CHIMP was the only robot to stand back up after accidentally falling over, a testament to the robustness engineered into the robot and a remote operator's ability to execute complex tasks using a highly capable robot. We present CHIMP as a concrete engineering example of a successful disaster response robot.

An Architecture for Human‐Guided Autonomy: Team TROOPER at the DARPA Robotics Challenge Finals

Recent robotics efforts have automated simple, repetitive tasks to increase execution speed and lessen an operator's cognitive load, allowing them to focus on higher‐level objectives. However, an autonomous system will eventually encounter something unexpected, and if this exceeds the tolerance of automated solutions, there must be a way to fall back to teleoperation. Our solution is a largely autonomous system with the ability to determine when it is necessary to ask a human operator for guidance. We call this approach human‐guided autonomy. Our design emphasizes human‐on‐the‐loop control where an operator expresses a desired high‐level goal for which the reasoning component assembles an appropriate chain of subtasks. We introduce our work in the context of the DARPA Robotics Challenge (DRC) Finals. We describe the software architecture Team TROOPER developed and used to control an Atlas humanoid robot. We employ perception, planning, and control automation for execution of subtasks. If subtasks fail, or if changing environmental conditions invalidate the planned subtasks, the system automatically generates a new task chain. The operator is able to intervene at any stage of execution, to provide input and adjustment to any control layer, enabling operator involvement to increase as confidence in automation decreases. We present our performance at the DRC Finals and a discussion about lessons learned.

Team WPI‐CMU: Achieving Reliable Humanoid Behavior in the DARPA Robotics Challenge

In the DARPA Robotics Challenge (DRC), participating human‐robot teams were required to integrate mobility, manipulation, perception, and operator interfaces to complete a simulated disaster mission. We describe our approach using the humanoid robot Atlas Unplugged developed by Boston Dynamics. We focus on our approach, results, and lessons learned from the DRC Finals to demonstrate our strategy, including extensive operator practice, explicit monitoring for robot errors, adding additional sensing, and enabling the operator to control and monitor the robot at varying degrees of abstraction. Our safety‐first strategy worked: we avoided falling, and remote operators could safely recover from difficult situations. We were the only team in the DRC Finals that attempted all tasks, scored points (14/16), did not require physical human intervention (a reset), and did not fall in the two missions during the two days of tests. We also had the most consistent pair of runs.

仮想マリオネットシステムによる脚型ロボットの遠隔操作

Generally, a target position of a hand or a foot is input in a virtual space by using a mouse as a user interface of a legged robot. This type of a legged robot interface is used by teams participating in Darpa Robotics Challenge. However, the operation time is too long. Also it is difficult for the operators to recognize the spatial relationship between a robot and an environment. To solve these problems, we have proposed an interface that mixes and presents both visual and haptic sense. In the previous research, we developed a visual-haptic fusion system for teleoperation of a crawler robot. In this study, we adapt the system to the operation of a legged robot. We verify the effectiveness of the proposed system by comparison with the conventional method.

Director: A User Interface Designed for Robot Operation with Shared Autonomy

Operating a high degree of freedom mobile manipulator, such as a humanoid, in a field scenario requires constant situational awareness, capable perception modules, and effective mechanisms for interactive motion planning and control. A well‐designed operator interface presents the operator with enough context to quickly carry out a mission and the flexibility to handle unforeseen operating scenarios robustly. By contrast, an unintuitive user interface can increase the risk of catastrophic operator error by overwhelming the user with unnecessary information. With these principles in mind, we present the philosophy and design decisions behindDirector—the open‐source user interface developed by Team MIT to pilot the Atlas robot in the DARPA Robotics Challenge (DRC). At the heart of Director is an integrated task execution system that specifies sequences of actions needed to achieve a substantive task, such as drilling a wall or climbing a staircase. These task sequences, developeda priori, make online queries to automated perception and planning algorithms with outputs that can be reviewed by the operator and executed by our whole‐body controller. Our use of Director at the DRC resulted in efficient high‐level task operation while being fully competitive with approaches focusing on teleoperation by highly trained operators. We discuss the primary interface elements that comprise Director, and we provide an analysis of its successful use at the DRC.

Team THOR's Entry in the DARPA Robotics Challenge Finals 2015

This paper describes Team THOR's approach to human-in-the-loop disaster response robotics for the 2015 DARPA Robotics Challenge (DRC) Finals. Under the duress of unpredictable networking and terrain, fluid operator interactions and dynamic disturbance rejection become major concerns for effective teleoperation. We present a humanoid robot designed to effectively traverse a disaster environment while allowing for a wide range of manipulation abilities. To complement the robot hardware, a hierarchical software foundation implements network strategies that provide real-time feedback to an operator under restricted bandwidth using layered user interfaces. Our strategy for humanoid locomotion includes a backward-facing knee configuration paired with specialized toe and heel lifting strategies that allow the robot to traverse difficult surfaces while rejecting external perturbations. With an upper body planner that encodes operator preferences, predictable motion plans are executed in unforeseen circumstances. These plans are critical for manipulation in unknown environments. Our approach was validated during the DRC Finals competition, where Team THOR scored three points in 18 min of operation time, and the results are presented along with an analysis of each task.

Team SNU's Control Strategies for Enhancing a Robot's Capability: Lessons from the 2015 DARPA Robotics Challenge Finals

This paper presents the technical approaches used and experimental results obtained by Team SNU (Seoul National University) at the 2015 DARPA Robotics Challenge (DRC) Finals. Team SNU is one of the newly qualified teams, unlike 12 teams who previously participated in the December 2013 DRC Trials. The hardware platform THORMANG, which we used, has been developed by ROBOTIS. THORMANG is one of the smallest robots at the DRC Finals. Based on this platform, we focused on developing software architecture and controllers in order to perform complex tasks in disaster response situations and modifying hardware modules to maximize manipulability. Ensuring stability and modularization are two main keywords in the technical approaches of the architecture. We designed our interface and controllers to achieve a higher robustness level against disaster situations. Moreover, we concentrated on developing our software architecture by integrating a number of modules to eliminate software system complexity and programming errors. With these efforts on the hardware and software, we successfully finished the competition without falling, and we ranked 12th out of 23 teams. This paper is concluded with a number of lessons learned by analyzing the 2015 DRC Finals.