Dual-arm Manipulation Tasks Research Articles

Sensor placement determination for a wearable device in dual-arm manipulation tasks

To enhance the quality of life for individuals in various sectors, such as industry, healthcare, and everyday activities, the development of a highly efficient and cost-effective wearable device is essential. This paper presents a feasibility study for a general estimation methodology for human activity assessment using a single sensor placed on the back. The methodology considers the minimal variability in human body movements when carrying different weights. Our approach involves optimizing the sensor's position through a simple pre-processing technique. This technique utilizes virtual sensor position and orientation data derived from a virtual mannequin based on real inertial measurement unit (IMU) measurements obtained during two-arm manipulation tasks in a warehouse setting. The primary contribution is the optimization of a single, non-invasive sensor placement on the back to accurately assess activities involving two-hand manipulation with low variability in movement. We identified 54 different activity classes with an accuracy of 91.77% using a hybrid two-dimensional Convolutional Neural Network (CNN) combined with a Bidirectional Long Short-Term Memory (BiLSTM) network (2D CNN-BiLSTM) model. This model does not require additional sensors that measure other physical quantities, such as electromyography activity. Identifying repetitive tasks involving significant bending, stretching, and twisting, which pose risks of musculoskeletal disorders and back pain, offers a solution for designing wearable devices. The portability and autonomy of these devices are crucial, and current sensor technology meets these needs with low size, cost, and consumption. Wearable medical devices can thus be effectively used for self-monitoring health, preventive medicine, and rehabilitation.

Global Model Learning for Large Deformation Control of Elastic Deformable Linear Objects: An Efficient and Adaptive Approach

Robotic manipulation of deformable linear objects (DLOs) has broad application prospects in many fields. However, a key issue is to obtain the exact deformation models (i.e., how robot motion affects DLO deformation), which are hard to theoretically calculate and vary among different DLOs. Thus, shape control of DLOs is challenging, especially for large deformation control which requires global and more accurate models. In this paper, we propose a coupled offline and online data-driven method for efficiently learning a global deformation model, allowing for both accurate modeling through offline learning and further updating for new DLOs via online adaptation. Specifically, the model approximated by a neural network is first trained offline on random data, then seamlessly migrated to the online phase, and further updated online during actual manipulation. Several strategies are introduced to improve the model's efficiency and generalization ability. We propose a convex-optimization-based controller, and analyze the system's stability using the Lyapunov method. Detailed simulations and real-world experiments demonstrate that our method can efficiently and precisely estimate the deformation model, and achieve large deformation control of untrained DLOs in 2D and 3D dual-arm manipulation tasks better than the existing methods. It accomplishes all 24 tasks with different desired shapes on different DLOs in the real world, using only simulation data for the offline learning.

Miniature Tangible Cube: Concept and Design of Target-Object-Oriented User Interface for Dual-Arm Telemanipulation

In recent years, there has been a great deal of research on teleoperation of robots, and many end-effector-oriented control systems have been proposed, but these systems have difficulties in performing manipulation tasks with physical contacts between the target object, the robot, and the environment, such as dual-arm manipulation, because these systems can only express the end-effectors' pose and force commands. In this study, we propose Target-object-oriented User Interface (TOOUI) that focuses on the target object rather than the end effectors of the robot, and develop a tangible device called Miniature Tangible Cube as a TOOUI. Through this device, the pilot can express the target orientation and contact state of the target object, and the robot can be teleoperated to do dual-arm manipulation tasks with geometric and physical constraints. For dual-arm manipulation tasks, we also implement a dual-arm telemanipulation system with the Miniature Tangible Cube using the motion planner based on state machine. In the experiments, we evaluated our system with the Miniature Tangible Cube by conducting dual-arm manipulation tasks with the dual-arm robot PR2 in the real world.

Learning context-adaptive task constraints for robotic manipulation

Constraint-based control approaches offer a flexible way to specify robotic manipulation tasks and execute them on robots with many degrees of freedom. However, the specification of task constraints and their associated priorities usually requires a human-expert and often leads to tailor-made solutions for specific situations. This paper presents our recent efforts to automatically derive task constraints for a constraint-based robot controller from data and adapt them with respect to previously unseen situations (contexts). We use a programming-by-demonstration approach to generate training data in multiple variations (context changes) of a given task. From this data we learn a probabilistic model that maps context variables to task constraints and their respective soft task priorities. We evaluate our approach with 3 different dual-arm manipulation tasks on an industrial robot and show that it performs better than comparable approaches with respect to reproduction accuracy in previously unseen contexts.

Constrained Manipulability for Humanoid Robots Using Velocity Polytopes

Robot performance measures are important tools for quantifying the ability to carry out manipulation tasks. Generally, these measures examine the system’s kinematic transformations from configuration to task space. This means that environmental constraints are neglected in spite of the significant effects they may have on the robot’s admissible motions. In this paper, we propose a new measure called the constrained manipulability polytope (CMP) that considers the system’s kinematic structure, including closed chains or composite sub-mechanisms, joint limits and the presence of obstacles. For an illustrative planar case, we demonstrate how the CMP can evaluate a robot’s performance in a cluttered scene and how this evaluation can be extrapolated to obtain a workspace visualization. Additionally, we show the advantages and limitations of the CMP compared to the state of the art. Furthermore, the method is demonstrated both in simulation and experimentally for NASA’s Valkyrie robot. We show how the CMP provides a measure for single-arm and dual-arm manipulation tasks, analyze the workspace and be used to optimize the robot’s posture.

Human to humanoid motion conversion for dual-arm manipulation tasks

SUMMARYA conversion process for the imitation of human dual-arm motion by a humanoid robot is presented. The conversion process consists of an imitation algorithm and an algorithm for generating human-like motion of the humanoid. The desired motions in Cartesian and joint spaces, obtained from the imitation algorithm, are used to generate the human-like motion of the humanoid. The proposed conversion process improves existing techniques and is developed with the aim to enable imitating of human motion with a humanoid robot, to perform a task with and/or without contact between hands and equipment. A comparative analysis shows that our algorithm, which takes into account the situation of marker frames and the position of joint frames, ensures more precise imitation than previously proposed methods. The results of our conversion algorithm are tested on the robot ROMEO through a complex “open/close drawer” task.

Toward optimal mapping of human dual-arm motion to humanoid motion for tasks involving contact with the environment

In this article, we explore human motion skills in the dual-arm manipulation tasks that include contact with equipment with the final aim to generate human-like humanoid motion. Human motion is analyzed using the optimization approaches starting with the assumption that human motion is optimal. A combination of commonly used optimization criteria in the joint space with the weight coefficients is considered: minimization of kinetic energy, minimization of joint velocities, minimization of the distance between the current and ergonomic positions, and maximization of manipulability. The contribution of each criterion for seven different dual-arm manipulation tasks to provide the most accurate imitation of the human motion is given via suggested inverse optimization approach calculating values of weight coefficients. The effects on actors’ body characteristics and the characteristics of the environment (involved equipment) on the choice of criterion functions are additionally analyzed. The optimal combination of weight coefficients calculated by the inverse optimization approach is used in our inverse kinematics algorithm to transfer human motion skills to the motion of the humanoid robots. The results show that the optimal combination of weight coefficients is able to generate human-like humanoid motions rather than individual one of the considered criterion functions. The recorded human motion and the motion of the humanoid robot ROMEO, obtained with the strategy used by human and defined by our inverse optimal control approach, for the tasks “opening/closing a drawer” are assessed visually and quantitatively.

Virtual mechanism approach for dual-arm manipulation

SUMMARYWe propose a novel control approach for cooperative dual-arm manipulation tasks. Our scheme has three typical features: (1) the task performed by two robots is represented as a motion of a virtual mechanism and the task execution is accomplished by controlling the virtual mechanism; (2) the two arms and the task form a joined kinematic chain; (3) the scheme allows a cooperative dual-arm system to perform the task also when robot base is moving. The calculation of the Jacobian matrix of a chained two-arm mechanisms is based on a methodology which is using the Jacobian matrices of particular robot mechanisms and their end-effector positions and orientations. The proposed algorithm for dual-arm manipulation is verified by simulations of two cooperating planar robots and by experiments on a dual-arm robot consisting of two KUKA LWR arms.

Two-arm manipulation tasks with friction-assisted grasping

—The main objective of this paper is to study human dual-arm manipulation tasks and to develop a computational model that predicts the trajectories and force distribution for the coordination of two arms moving an object between two given positions and orientations in a horizontal plane. Our ultimate goal is to understand the dynamics of human dual-arm coordination in order to develop better robot control algorithms. We propose a computational model based on the hypothesis proposed by Uno et al. that suggests that human movements minimize the integral of the norm of the rate of change of actuator torques. We compare the experimental trajectories and force distributions with those obtained from the computational model. The observed trajectories show a significant degree of repeatability across trials and across subjects. We show that the computational model predicts the trajectories and the distribution of forces (torques) for a certain class of trajectories. However, the trajectories in the sagittal and frontal plane are characterized by asymmetric features that are hard to model using any integral cost function. Finally, we show that the computational model can be used to generate smooth trajectories and actuator forces for cooperating robots and discuss the advantages of such an approach to motion planning.