Model-based manipulation of linear flexible objects

Abstract

Manipulation of deformable objects plays an important role in various scenarios such as manufacturing, service, healthcare, and security. Linear flexible objects are common in these scenarios, e.g., cables, wires, ropes. However, the high dimensionality of the linear flexible objects brings challenges to the modeling and planning in manipulation tasks, and automatic manipulation of these objects is computationally expensive due to their infinite degrees of freedom in the free spaces. In this dissertation, we investigate model-based manipulation of linear flexible objects such as cables. We contribute to different models including geometrical and physical models to represent the linear flexible objects. With these models, we then develop manipulation plans and strategies to achieve the automation of the linear flexible object manipulation tasks in both simulation and real-world. Besides, we also investigate human-robot collaboration to complete a sample assembly task involving linear flexible object manipulation. To develop a general representation of the linear flexible objects those are subject to gravity, we propose a geometric modeling method that is based on visual feedback. This model characterizes the shape of the object by combining the learned curves on two projection planes. In this approach, we achieve tracking of the pose of a cable-like object, the pose of its tip, and the pose of the selected grasp point on the object, which enables closed-loop automatic manipulation of the object. We demonstrate the feasibility of our approach by automatically completing the Plug Task used in the 2015 DARPA Robotics Challenge Finals (DRC Plug Task), which involves unplugging a power cable from one socket and plugging it into another. Due to the lack of a high-fidelity simulation for the linear flexible objects, we propose a physical multi-link model. This model consists of multiple links connected by revolute joints and identified model parameters. A new identification approach for building the model of the linear flexible objects is derived from real-world to simulation. To bridge the gap between simulation and real world and automate the DLO manipulation tasks (e.g. the DRC Plug Task), we introduce a Simulation-to-Real-to-Simulation (Sim2Real2Sim) strategy which involves three steps: (1) using the rough environment with the estimated models to develop the methods to complete the manipulation task in the simulation; (2) applying the methods from simulation to real world and comparing their performance; (3) updating the models and methods in simulation based on the differences between the real world and the simulation. The automation of the DRC plug task in both simulation and real-world proves the success of the Sim2Real2Sim strategy. Based on the proposed models of linear flexible objects, we then explore the manipulation and task planning for manipulating the linear flexible objects, especially in the constrained spaces. Manipulating linear flexible objects in a constrained space raises more challenges than in a free space due to restricted robot motions. In this research study, we propose a method for automating the manipulation of linear flexible objects in the constrained spaces. The method contains a geometric model of cable-like objects based on multimodal features including point cloud, color, and shape. With the estimated 3-D cable model and environment information, we propose a visual servoing based planning methodology for manipulating the cable in the constrained spaces. We demonstrate the efficiency and robustness of our method by automating a cable threading task with real robot experiments using a benchmarking task board designed by the National Institute of Standards and Technology. Humans are superior to robots in manipulating linear flexible objects primarily due to their dexterity and sensorimotor abilities. When humans and robots collaborate to complete the assembly tasks involving the manipulation of linear flexible objects, it is important to enable the robot's capability in precise manipulation of deformable objects and coordinate human and robot actions. To complete the plug task we mentioned above with human-robot collaboration, we propose a shared control method. In our setup, the human holds the socket while the robot is holding the cable and manages to insert the plug into the socket. An initial socket pose estimation is provided based on multiple human subjects' data. With the statistically calculated initial guess, the robot could act instantly from the beginning to move the plug towards an initial target. The actual socket pose is constantly updated using the Kalman filter in real-time. Moreover, the system is continuously modeling the cable and tracking the cable-tip to enable a visual servoing control approach for the plug task. The approaches and algorithms introduced in this thesis were demonstrated on the Kinova Jaco v2 and Kinova Gen3 robotic arms with numerous experiments.--Author's abstract