Robot Manipulator Research Articles

Multi-objective optimal trajectory planning for manipulators based on CMOSPBO

Feasible, smooth, and time-jerk optimal trajectory is essential for manipulators utilized in manufacturing process. A novel technique to generate trajectories in the joint space for robotic manipulators based on quintic B-spline and constrained multi-objective student psychology based optimization (CMOSPBO) is proposed in this paper. In order to obtain the optimal trajectories, two objective functions including the total travelling time and the integral of the squared jerk along the whole trajectories are considered. The whole trajectories are interpolated by quintic B-spline and then optimized by CMOSPBO, while taking into account kinematic constraints of velocity, acceleration, and jerk. CMOSPBO mainly includes improved student psychology based optimization, archive management, and an adaptive ε-constraint handling method. Lévy flights and differential mutation are adopted to enhance the global exploration capacity of the improved SPBO. The ε value is varied with iterations and feasible solutions to prevent the premature convergence of CMOSPBO. Solution density estimation corresponding to the solution distribution in decision space and objective space is proposed to increase the diversity of solutions. The experimental results show that CMOSPBO outperforms than SQP, and NSGA-II in terms of the motion efficiency and jerk. The comparison results demonstrate the effectiveness of the proposed method to generate time-jerk optimal and jerk-continuous trajectories for manipulators.

Assessment of first-touch skills in robotic surgical training using hi-Sim and the hinotori surgical robot system among surgeons and novices

PurposeSurgeons’ adaptability to robotic manipulation remains underexplored. This study evaluated the participants’ first-touch robotic training skills using the hinotori surgical robot system and its simulator (hi-Sim) to assess adaptability.MethodsWe enrolled 11 robotic surgeons (RS), 13 laparoscopic surgeons (LS), and 15 novices (N). After tutorial and training, participants performed pegboard tasks, camera and clutch operations, energizing operations, and suture sponge tasks on hi-Sim. They also completed a suture ligation task using the hinotori surgical robot system on a suture simulator. Median scores and task completion times were compared.ResultsPegboard task scores were 95.0%, 92.0%, and 91.5% for the RS, LS, and N groups, respectively, with differences between the RS group and LS and N groups. Camera and clutch operation scores were 93.1%, 49.7%, and 89.1%, respectively, showing differences between the RS group and LS and N groups. Energizing operation scores were 90.9%, 85.2%, and 95.0%, respectively, with a significant difference between the LS and N groups. Suture sponge task scores were 90.6%, 43.1%, and 46.2%, respectively, with differences between the RS group and LS and N groups. For the suture ligation task, completion times were 368 s, 666 s, and 1095 s, respectively, indicating differences among groups. Suture scores were 12, 10, and 7 points, respectively, with differences between the RS and N groups.ConclusionFirst-touch simulator-based robotic skills were partially influenced by prior robotic surgical experience, while suturing skills were affected by overall surgical experience. Thus, robotic training programs should be tailored to individual adaptability.

Leveraging Environmental Contact and Sensor Feedback for Precision in Robotic Manipulation

Leveraging Environmental Contact and Sensor Feedback for Precision in Robotic Manipulation

Smart automation in luxury leather shoe polishing: a human centric robotic approach

ABSTRACT The polishing of luxury leather shoes is a delicate, labor-intensive process traditionally performed by skilled craftsmen. Footwear companies aim to automate parts of this process to enhance quality, productivity and operator well-being, but the unique nature of luxury shoe production presents challenges. This paper introduces a solution involving a collaborative robotic cell to assist in shoe polishing. A collaborative robotic manipulator, equipped with a specialized tool and governed by force control, executes the polishing tasks. Key factors such as trajectory design, applied force, polishing speed and polish amount were analyzed. Polishing trajectories are designed using CAM software and transferred to the robot’s control system. Human operators design the process, supervise the robot and perform final finishing, ensuring their expertise is integral to achieving quality. Extensive testing on various shoe models showed significant improvements in quality and reliability, leading to successful implementation on an industrial production line.

Rearranging Deformable Linear Objects for Implicit Goals with Self‐Supervised Planning and Control

The robotic manipulation of deformable linear objects is a frontier problem with many potential applications in diverse industries. However, most existing research in this area focuses on shape control for a provided explicit goal and does not consider physical constraints, which limits its applicability in many real‐world scenarios. In this study, a self‐supervised planning and control approach are proposed to address the challenge of rearranging deformable linear objects for implicit goals. Specifically, the context of making both ends of the object reachable (inside the robotic access range) and graspable (outside potential collision regions) by dual‐arm robots is considered. Firstly, the object is described with sequential keypoints and the correspondence‐based action is parameterized. Secondly, a generator capable of producing multiple explicit targets is developed, which adhere to implicit conditions. Thirdly, value models are learnt to assign the most promising explicit target as guidance and determine the goal‐conditioned action. All models within the policy are trained in a self‐supervised manner based on data collected from simulations. Importantly, the learned policy can be directly applied to real‐world settings since we do not rely on accurate dynamic models. The performance of the new method is validated with simulations and real‐world experiments.

A Multi-Posture Grasping Manipulator Actuated by Shape Memory Alloy with Different Functional Modules

Currently, multi-posture robots have complex grasping robotic manipulators with low power density, making it difficult to miniaturize and integrate. In this paper, a multi-posture grasping manipulator actuated by shape memory alloy with different functional modules is presented. It is composed of deflection, translation, rotation and grasping modules. Based on a D-H parameter method, the end motion trajectory model is established and the end motion space is drawn. Finally, the grasping experiment of a light circular object is carried out to verify the validity of the multi-posture grasping function of the multi-module combination manipulator, which provides a choice for future intelligent robot manipulators.

Instrumented assessment of lower and upper motor neuron signs in amyotrophic lateral sclerosis using robotic manipulation: an explorative study

BackgroundAmyotrophic lateral sclerosis (ALS) is a lethal progressive neurodegenerative disease characterized by upper motor neuron (UMN) and lower motor neuron (LMN) involvement. Their varying degree of involvement results in a clinical heterogenous picture, making clinical assessments of UMN signs in patients with ALS often challenging. We therefore explored whether instrumented assessment using robotic manipulation could potentially be a valuable tool to study signs of UMN involvement.MethodsWe examined the dynamics of the wrist joint of 15 patients with ALS and 15 healthy controls using a Wristalyzer single-axis robotic manipulator and electromyography (EMG) recordings in the flexor and extensor muscles in the forearm. Multi-sinusoidal torque perturbations were applied, during which participants were asked to either relax, comply or resist. A neuromuscular model was used to study muscle viscoelasticity, e.g. stiffness (k) and viscosity (b), and reflexive properties, such as velocity, position and force feedback gains (kv, kp and kf, respectively) that dominated the responses. We further obtained clinical signs of LMN (muscle strength) and UMN (e.g. reflexes, spasticity) dysfunction, and evaluated their relation with the estimated neuromuscular model parameters.ResultsOnly force feedback gains (kf) were elevated in patients (p = 0.033) compared to controls. Higher kf, as well as the resulting reflexive torque (Tref), were both associated with more severe UMN dysfunction in the examined arm (p = 0.040 and p < 0.001). Patients with UMN symptoms in the examined arm had increased kf and Tref compared to controls (both p = 0.037). Neither of these measures was related to muscle strength, but muscle stiffness (k) was lower in weaker patients (p = 0.012). All these findings were obtained from the relaxed test. No differences were observed during the instructions comply and resist.ConclusionsThis findings are proof-of-concept that instrumented assessment using robotic manipulation is a feasible technique in ALS, which may provide quantitative, operator-independent measures relating to UMN symptoms. Elevated force feedback gains, driving larger reflexive muscle torques, appear to be particularly indicative of clinically established levels of UMN dysfunction in the examined arm.

Retraction Note: Deep belief network-based probabilistic generative model for detection of robotic manipulator failure execution

Retraction Note: Deep belief network-based probabilistic generative model for detection of robotic manipulator failure execution

Muscle‐Inspired Robust Anisotropic Cellulose Conductive Hydrogel for Multidirectional Strain Sensors and Implantable Bioelectronics

AbstractIntegrating superior mechanical performance, anisotropic conductivity, and biocompatibility into conductive hydrogels as all‐in‐one human‐machine interaction device remains challenging. Herein, by mimicking the anisotropic structures of human muscles, a robust anisotropic conductive hydrogel is developed by initially aligning polyvinyl alcohol with polypyrrole decorated cellulose nanofibrils to form an anisotropically oriented polymer networks, followed by post‐crosslinking with tannic acid (TA). Introducing TA into hydrogel network permanently secures its hierarchically anisotropic structure through multiple hydrogen bonds, thus endowing the hydrogel with exceptional mechanical properties (tensile strength of 11.41 MPa, toughness of 12.44 MJ m−3), anisotropic adhesive property, and direction‐dependent conductivity. With these attributes, a hydrogel strain sensor with excellent multidirectional sensitivity is developed, enabling stable monitoring of multi‐degrees of freedom joint movements in the human body and facilitating the control of a multiaxial virtual robot manipulator. Moreover, the in vitro/vivo tests demonstrate exceptional biocompatibility and anti‐biofouling properties of the as‐prepared hydrogel sensor, maintaining stable electronic response signals for over 14 days after successful implantation into the Achilles tendon of mice. Overall, this study presents a promising approach for designing conductive hydrogels with superior mechanical properties and anisotropic functionality for emerging applications in both in vitro and in vivo human‐machine interface materials.

A Force Control Method Integrating Human Skills for Complex Surface Finishing

Force control is one of the core modules for surface finishing such as grinding, polishing and sanding. However, the current force control methods based on human skills lack in-depth analysis of data patterns or are only applicable to flat surfaces. In addition, surface finishing is mainly performed by hand, resulting in low processing efficiency and poor product consistency. Therefore, this paper proposes a force control method that incorporates human skills to achieve relatively accurate force skill transfer and complex surface finishing. Firstly, human skills consisting of the force skill and the motion skill are learned. The force skill is used to generate the desired force. Then, a series of discrete poses are obtained based on human demonstration and combined with the motion skill to generate the desired trajectory. Finally, a computed-torque impedance control method is proposed to achieve relatively accurate force skill transfer and complex surface finishing by incorporating the desired trajectory and the desired force. The experiments are conducted on a platform composed of a 7-DOF collaborative robot manipulator from Franka Emika and a complex violin surface. The results demonstrate that the proposed force control method can achieve relatively accurate force skill transfer and improve the surface quality of the workpiece.

Development of 6DOF Hardware-in-the-Loop Ground Testbed for Autonomous Robotic Space Debris Removal

This paper presents the development of a hardware-in-the-loop ground testbed featuring active gravity compensation via software-in-the-loop integration, specially designed to support research in autonomous robotic removal of space debris. The testbed is designed to replicate six degrees of freedom (6DOF) motion maneuvering to accurately simulate the dynamic behaviors of free-floating robotic manipulators and free-tumbling space debris under microgravity conditions. The testbed incorporates two industrial 6DOF robotic manipulators, a three-finger robotic gripper, and a suite of sensors, including cameras, force/torque sensors, and tactile tensors. Such a setup provides a robust platform for testing and validating technologies related to autonomous tracking, capture, and post-capture stabilization within the context of active space debris removal missions. Preliminary experimental results have demonstrated advancements in motion control, computer vision, and sensor fusion. This facility is positioned to become an essential resource for the development and validation of robotic manipulators in space, offering substantial improvements to the effectiveness and reliability of autonomous capture operations in space missions.

Robotic manipulation of cardiomyocytes to identify gap junction modifiers for arrhythmogenic cardiomyopathy.

Arrhythmogenic cardiomyopathy (ACM) is a leading cause of sudden cardiac death among young adults. Aberrant gap junction remodeling has been linked to disease-causative mutations in plakophilin-2 (PKP2). Although gap junctions are a key therapeutic target, measurement of gap junction function in preclinical disease models is technically challenging. To quantify gap junction function with high precision and high consistency, we developed a robotic cell manipulation system with visual feedback from digital holographic microscopy for three-dimensional and label-free imaging of human induced pluripotent stem cell-derived cardiomyocytes (iPSC-CMs). The robotic system can accurately determine the dynamic height changes in the cells' contraction and resting phases, microinject drug-treated healthy and diseased iPSC-CMs in their resting phase with constant injection depth across all cells, and deposit a membrane-impermeable dye that solely diffuses between cells through gap junctions for measuring the gap junction diffusion function. The robotic system was applied toward a targeted drug screen to identify gap junction modulators and potential therapeutics for ACM. Five compounds were found to dose-dependently enhance gap junction permeability in cardiomyocytes with PKP2 knockdown. In addition, PCO 400 (pinacidil) reduced beating irregularity in a mouse model of ACM expressing mutant PKP2 (R735X). These results highlight the utility of the robotic cell manipulation system to efficiently assess gap junction function in a relevant preclinical disease model, thus providing a technique to advance drug discovery for ACM and other gap junction-mediated diseases.

Kinematic Reliability of Manipulators Based on an Interval Approach

Robotic manipulators inevitably experience the impact of uncertainties and errors, such as dimensional tolerances and joint clearances. Therefore, this paper proposes a novel method based on an interval approach to evaluate the kinematic reliability of manipulators. Kinematic reliability quantifies the probability of positioning errors that fall within allowable boundaries. As a result, reliability evaluates the probability that the interval end-effector error produced by dimensional tolerances exceeds an acceptable rate. The proposed reliability index is based on the interval error that conveys an alternative approach to the kinematic reliability methods based on probabilistic frameworks reported in the literature based on probabilistic approaches. The obtained numerical results demonstrate the viability of the proposed methodology by evaluating the reliability of a serial manipulator subjected to joint clearances and a parallel manipulator with dimensional tolerances.

Robust tube-based MPC with smooth computation for dexterous robot manipulation

Robust tube-based MPC with smooth computation for dexterous robot manipulation

A Human-Centered View of Continual Learning: Understanding Interactions, Teaching Patterns, and Perceptions of Human Users Toward a Continual Learning Robot in Repeated Interactions

Continual learning (CL) has emerged as an important avenue of research in recent years, at the intersection of Machine Learning (ML) and Human–Robot Interaction (HRI), to allow robots to continually learn in their environments over long-term interactions with humans. Most research in CL, however, has been robot-centered to develop CL algorithms that can quickly learn new information on systematically collected static datasets. In this article, we take a human-centered approach to CL, to understand how humans interact with, teach, and perceive CL robots over the long term, and if there are variations in their teaching styles. We developed a socially guided CL system that integrates CL models for object recognition with a mobile manipulator robot and allows humans to directly teach and test the robot in real time over multiple sessions. We conducted an in-person study with 60 participants who interacted with the CL robot in 300 sessions with 5 sessions per participant. In this between-participant study, we used three different CL models deployed on a mobile manipulator robot. An extensive qualitative and quantitative analysis of the data collected in the study shows that there is significant variation among the teaching styles of individual users indicating the need for personalized adaptation to their distinct teaching styles. Our analysis shows that the constrained experimental setups that have been widely used to test most CL models are not adequate, as real users interact with and teach CL robots in a variety of ways. Finally, our analysis shows that although users have concerns about CL robots being deployed in our daily lives, they mention that with further improvements CL robots could assist older adults and people with disabilities in their homes.

Enhancing Grapevine Node Detection to Support Pruning Automation: Leveraging State-of-the-Art YOLO Detection Models for 2D Image Analysis

Automating pruning tasks entails overcoming several challenges, encompassing not only robotic manipulation but also environment perception and detection. To achieve efficient pruning, robotic systems must accurately identify the correct cutting points. A possible method to define these points is to choose the cutting location based on the number of nodes present on the targeted cane. For this purpose, in grapevine pruning, it is required to correctly identify the nodes present on the primary canes of the grapevines. In this paper, a novel method of node detection in grapevines is proposed with four distinct state-of-the-art versions of the YOLO detection model: YOLOv7, YOLOv8, YOLOv9 and YOLOv10. These models were trained on a public dataset with images containing artificial backgrounds and afterwards validated on different cultivars of grapevines from two distinct Portuguese viticulture regions with cluttered backgrounds. This allowed us to evaluate the robustness of the algorithms on the detection of nodes in diverse environments, compare the performance of the YOLO models used, as well as create a publicly available dataset of grapevines obtained in Portuguese vineyards for node detection. Overall, all used models were capable of achieving correct node detection in images of grapevines from the three distinct datasets. Considering the trade-off between accuracy and inference speed, the YOLOv7 model demonstrated to be the most robust in detecting nodes in 2D images of grapevines, achieving F1-Score values between 70% and 86.5% with inference times of around 89 ms for an input size of 1280 × 1280 px. Considering these results, this work contributes with an efficient approach for real-time node detection for further implementation on an autonomous robotic pruning system.

Towards full autonomy in mobile robot navigation and manipulation

AbstractRobot autonomy is a key for automatizing industrial production plants, facilitating responsive search and rescue (SAR), and for interacting in urban settings. This work presents mobile systems capable of carrying out these tasks and summarises experience from practical deployments in three different domains for navigation. For industry, a mobile manipulator is presented, which is used for dynamic precise docking and object handling. For SAR, a semi-autonomous system is presented which is used for sampling hazardous materials, manipulating tools, and for localizing nuclear radiation sources. To improve urban logistics, a mobile robot autonomously navigates complex outdoor environments using semantic mapping techniques. Future work will explore further advancements that strive toward increased flexibility, reasoning and universal applicability of mobile robots. The goal is to reduce training, onboarding and tool changing times to reduce manual interfacing times.

Hierarchically‐Interlocked, Three‐Axis soft Iontronic Sensor for Omnidirectional Shear and Normal Forces

AbstractArtificial tactile sensing capable of measuring shear and normal forces is crucial for the diverse human‐machine interactions and dexterous robotic manipulations. However, existing soft multi‐axis force sensors usually suffer from limited detectable directions and complex coupling mechanisms, limiting their applications in realistic wearable and robotic systems. Here, a hierarchically‐interlocked, three‐axis soft iontronic sensor is presented with an asymmetrical electrode pattern that leverages the mortise‐and‐tenon structure and ultracapacitive principle to detect omnidirectional shear and normal forces. The designed sensor facilitates the decoupling process and achieves enhanced sensing performances including high accuracy, fast response ability, and mechanical robustness. Prototypical integration and application of the sensor are demonstrated to perform wearable telecontrol of virtual platforms and assist robot gripper through closed‐loop force feedback. A sensing array is further developed to construct a touch panel and identify handwriting based on the continuously measured directions of applied forces. This work may offer a potentially promising solution for the next‐generation intelligent electronic skins requiring multimodal tactile sensing information.

Distributed leader-following bipartite consensus for one-sided Lipschitz multi-agent systems via dual-terminal event-triggered mechanism

This article analyses leader-following bipartite consensus for one-sided Lipschitz multi-agent systems by dual-terminal event-triggered output feedback control approach. A distributed observer is designed to estimate unknown system states by employing relative output information at triggering time instants, and then an event-triggered output feedback controller is proposed. Dual-terminal dynamic event-triggered mechanisms are proposed in sensor–observer channel and controller–actuator channel, which can save communication resources to a great extent, and the Zeno behavior is ruled out. A new generalized one-sided Lipschitz condition is proposed to handle the nonlinear term and achieve bipartite consensus. Some stability conditions are presented to guarantee leader-following bipartite consensus. Finally, one-link robot manipulator systems are introduced to demonstrate the availability of the designed scheme. The results demonstrate that the agents of the robot manipulators can track the reference trajectories bi-directionally, and effectively reduce communication resources by 61.22% and 68.04% at the sensor–observer and controller–actuator channels, respectively.

Model predictive control for space robot manipulator modeled by switched systems: A data‐driven approach

AbstractThis article investigates the data‐driven based model predictive control (MPC) problem for a class of space robot manipulator (SRM). First, the SRM system is modeled as a class of discrete‐time switched system to reflect more system characteristics. Second, only the input‐state data are used for end‐to‐end controller design, and the system identification process is avoided. In order to meet the real‐time requirements of the manipulator arm angles, a kind of constrained predictive control method is designed to prevent the overheating or damage of SRM motor, optimizing the cost function for each decision cycle under the safe control input constraints. Then, based on the average dwell time method, the asymptotic stability is assured for the presented SRM system. The design conditions for data‐driven based MPC are provided in the form of linear matrix inequalities. Finally, the effectiveness and advantage for the developed data‐driven based MPC strategy are validated via a case study of SRM system.