Robot Manipulator Research Articles

Towards full autonomy in mobile robot navigation and manipulation

AbstractRobot autonomy is a key for automatizing industrial production plants, facilitating responsive search and rescue (SAR), and for interacting in urban settings. This work presents mobile systems capable of carrying out these tasks and summarises experience from practical deployments in three different domains for navigation. For industry, a mobile manipulator is presented, which is used for dynamic precise docking and object handling. For SAR, a semi-autonomous system is presented which is used for sampling hazardous materials, manipulating tools, and for localizing nuclear radiation sources. To improve urban logistics, a mobile robot autonomously navigates complex outdoor environments using semantic mapping techniques. Future work will explore further advancements that strive toward increased flexibility, reasoning and universal applicability of mobile robots. The goal is to reduce training, onboarding and tool changing times to reduce manual interfacing times.

Hierarchically‐Interlocked, Three‐Axis soft Iontronic Sensor for Omnidirectional Shear and Normal Forces

AbstractArtificial tactile sensing capable of measuring shear and normal forces is crucial for the diverse human‐machine interactions and dexterous robotic manipulations. However, existing soft multi‐axis force sensors usually suffer from limited detectable directions and complex coupling mechanisms, limiting their applications in realistic wearable and robotic systems. Here, a hierarchically‐interlocked, three‐axis soft iontronic sensor is presented with an asymmetrical electrode pattern that leverages the mortise‐and‐tenon structure and ultracapacitive principle to detect omnidirectional shear and normal forces. The designed sensor facilitates the decoupling process and achieves enhanced sensing performances including high accuracy, fast response ability, and mechanical robustness. Prototypical integration and application of the sensor are demonstrated to perform wearable telecontrol of virtual platforms and assist robot gripper through closed‐loop force feedback. A sensing array is further developed to construct a touch panel and identify handwriting based on the continuously measured directions of applied forces. This work may offer a potentially promising solution for the next‐generation intelligent electronic skins requiring multimodal tactile sensing information.

Distributed leader-following bipartite consensus for one-sided Lipschitz multi-agent systems via dual-terminal event-triggered mechanism

This article analyses leader-following bipartite consensus for one-sided Lipschitz multi-agent systems by dual-terminal event-triggered output feedback control approach. A distributed observer is designed to estimate unknown system states by employing relative output information at triggering time instants, and then an event-triggered output feedback controller is proposed. Dual-terminal dynamic event-triggered mechanisms are proposed in sensor–observer channel and controller–actuator channel, which can save communication resources to a great extent, and the Zeno behavior is ruled out. A new generalized one-sided Lipschitz condition is proposed to handle the nonlinear term and achieve bipartite consensus. Some stability conditions are presented to guarantee leader-following bipartite consensus. Finally, one-link robot manipulator systems are introduced to demonstrate the availability of the designed scheme. The results demonstrate that the agents of the robot manipulators can track the reference trajectories bi-directionally, and effectively reduce communication resources by 61.22% and 68.04% at the sensor–observer and controller–actuator channels, respectively.

Model predictive control for space robot manipulator modeled by switched systems: A data‐driven approach

AbstractThis article investigates the data‐driven based model predictive control (MPC) problem for a class of space robot manipulator (SRM). First, the SRM system is modeled as a class of discrete‐time switched system to reflect more system characteristics. Second, only the input‐state data are used for end‐to‐end controller design, and the system identification process is avoided. In order to meet the real‐time requirements of the manipulator arm angles, a kind of constrained predictive control method is designed to prevent the overheating or damage of SRM motor, optimizing the cost function for each decision cycle under the safe control input constraints. Then, based on the average dwell time method, the asymptotic stability is assured for the presented SRM system. The design conditions for data‐driven based MPC are provided in the form of linear matrix inequalities. Finally, the effectiveness and advantage for the developed data‐driven based MPC strategy are validated via a case study of SRM system.

Automatic differential kinematics of serial manipulator robots through dual numbers

Dual Numbers are an extension of real numbers known for its capability of performing exact automatic differentiation of one-valued functions theoretically without error approximation. Also, Differential Kinematics of robots involves the computation of the Jacobian, which is a matrix of partial derivatives of the Forward Kinematic equations with respect to the robot’s joints. Thus, to perform the automatic calculation of the Jacobian matrix, this paper presents an extension of dual numbers composed of a scalar real part and a vector dual part, where the real part represents the angular value of the robot joint, and the dual part represents the direction of the corresponding partial derivative for each joint. The presented method was implemented in Matlab through Object Orientes Programming (OOP), and the results for calculating the Jacobian of the KUKA KR 500 robot model for 1000 random postures were subsequently compared in terms of execution time and Mean Squared Error (MSE) with other conventional methods: the geometric method, the symbolic method, and the finite difference method. The results showed a significant improvement in the computing time for calculating the Jacobian of the robotic model compared to the other methods, as well as a minimum MSE having as reference the numerical value of the symbolic calculations.

A Depth Awareness and Learnable Feature Fusion Network for Enhanced Geometric Perception in Semantic Correspondence.

Deep learning is becoming the most widely used technology for multi-sensor data fusion. Semantic correspondence has recently emerged as a foundational task, enabling a range of downstream applications, such as style or appearance transfer, robot manipulation, and pose estimation, through its ability to provide robust correspondence in RGB images with semantic information. However, current representations generated by self-supervised learning and generative models are often limited in their ability to capture and understand the geometric structure of objects, which is significant for matching the correct details in applications of semantic correspondence. Furthermore, efficiently fusing these two types of features presents an interesting challenge. Achieving harmonious integration of these features is crucial for improving the expressive power of models in various tasks. To tackle these issues, our key idea is to integrate depth information from depth estimation or depth sensors into feature maps and leverage learnable weights for feature fusion. First, depth information is used to model pixel-wise depth distributions, assigning relative depth weights to feature maps for perceiving an object's structural information. Then, based on a contrastive learning optimization objective, a series of weights are optimized to leverage feature maps from self-supervised learning and generative models. Depth features are naturally embedded into feature maps, guiding the network to learn geometric structure information about objects and alleviating depth ambiguity issues. Experiments on the SPair-71K and AP-10K datasets show that the proposed method achieves scores of 81.8 and 83.3 on the percentage of correct keypoints (PCK) at the 0.1 level, respectively. Our approach not only demonstrates significant advantages in experimental results but also introduces the depth awareness module and a learnable feature fusion module, which enhances the understanding of object structures through depth information and fully utilizes features from various pre-trained models, offering new possibilities for the application of deep learning in RGB and depth data fusion technologies. We will also continue to focus on accelerating model inference and optimizing model lightweighting, enabling our model to operate at a faster speed.

Towards Autonomous Retail Stocking and Picking: Methods Enabling Robust Vacuum-Based Robotic Manipulation in Densely Packed Environments.

With the advent of robotics and artificial intelligence, the potential for automating tasks within human-centric environments has increased significantly. This is particularly relevant in the retail sector where the demand for efficient operations and the shortage of labor drive the need for rapid advancements in robot-based technologies. Densely packed retail shelves pose unique challenges for robotic manipulation and detection due to limited space and diverse object shapes. Vacuum-based grasping technologies offer a promising solution but face challenges with object shape adaptability. The study proposes a framework for robotic grasping in retail environments, an adaptive vacuum-based grasping solution, and a new evaluation metric-termed grasp shear force resilience-for measuring the effectiveness and stability of the grasp during manipulation. The metric provides insights into how retail objects behave under different manipulation scenarios, allowing for better assessment and optimization of robotic grasping performance. The study's findings demonstrate the adaptive suction cups' ability to successfully handle a wide range of object shapes and sizes, which, in some cases, overcome commercially available solutions, particularly in adaptability. Additionally, the grasp shear force resilience metric highlights the effects of the manipulation process, such as in shear force and shake, on the manipulated object. This offers insights into its interaction with different vacuum cup grasping solutions in retail picking and restocking scenarios.

Nonzero-sum game-based decentralized approximate optimal control of modular robot manipulators with coordinate operation tasks using value iteration

Nonzero-sum game-based decentralized approximate optimal control of modular robot manipulators with coordinate operation tasks using value iteration

A Cooperative and Multifunctional Magnetic Continuum Robot for Noninteractive Access, Dexterous Navigation, and Versatile Manipulation

AbstractSoft continuum robots can navigate through narrow and tortuous lumina. However, its interactions with the lumina during steering pose a risk of luminal injuries, especially if it integrates multifunctional units, which increase both the robot's size and local stiffness. Therefore, a continuum robot comprising coaxially assembled Guider and Follower components is designed that utilize phase change materials for noninteractive navigation and versatile manipulation. Navigation occurs through alternating softening‐hardening cycles: First, the Guider softens and advances under magnetic steering while the rigid Follower provides support. Then, the Guider solidifies to form a new backbone as the Follower softens to replicate its path. This process maintains structural integrity while enabling navigation along desired trajectories, thereby greatly reducing interactions with the lumen. To further accomplish complex tasks in large and open spaces, a continuum robot system with a triple‐individual Guider is developed in which the multi‐degree‐of‐freedom movement of the individuals is precisely and sequentially controlled through segmented stiffness regulation. This configuration achieves versatile manipulation, including targeted delivery, precision coating, grasping, and obstacle clearing in complex environments. Combining cooperative movement and diverse functionalities, this continuum robot design offers a new approach to dexterous navigation and intervention in minimally invasive surgical procedures.

Improved Bald Eagle Search Optimization Algorithm for the Inverse Kinematics of Robotic Manipulators.

The inverse kinematics of robotic manipulators involves determining an appropriate joint configuration to achieve a specified end-effector position. This problem is challenging because the inverse kinematics of manipulators are highly nonlinear and complexly coupled. To address this challenge, the bald eagle search optimization algorithm is introduced. This algorithm combines the advantages of evolutionary and swarm techniques, making it more effective at solving nonlinear problems and improving search efficiency. Due to the tendency of the algorithm to fall into local optima, the Lévy flight strategy is introduced to enhance its performance. This strategy adopts a heavy-tailed distribution to generate long-distance jumps, thereby preventing the algorithm from becoming trapped in local optima and enhancing its global search efficiency. The experiments first evaluated the accuracy and robustness of the proposed algorithm based on the inverse kinematics problem of manipulators, achieving a solution accuracy of up to 10-18 m. Subsequently, the proposed algorithm was compared with other algorithms using the CEC2017 test functions. The results showed that the improved algorithm significantly outperformed the original in accuracy, convergence speed, and stability. Specifically, it achieved over 70% improvement in both standard deviation and mean for several test functions, demonstrating the effectiveness of the Lévy flight strategy in enhancing global search capabilities. Furthermore, the practicality of the proposed algorithm was verified through two real engineering optimization problems.

Inverse Kinematics of Robotic Manipulators Based on Hybrid Differential Evolution and Jacobian Pseudoinverse Approach

Robot manipulators play a critical role in several industrial applications by providing high precision and accuracy. To perform these tasks, manipulator robots require the effective computation of inverse kinematics. Conventional methods to solve IK often encounter significant challenges, such as singularities, non-linear equations, and poor generalization across different robotic configurations. In this work, we propose a novel approach to solve the inverse kinematics (IK) problem in robotic manipulators using a metaheuristic algorithm enhanced with a Jacobian step. Our method overcomes those limitations by selectively applying the Jacobian step to the differential evolution (DE) algorithm. The effectiveness and versatility of the proposed approach are demonstrated through simulations and real-world experimentation on a 5 DOF KUKA robotic arm.

Diffusion policy: Visuomotor policy learning via action diffusion

This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot’s visuomotor policy as a conditional denoising diffusion process. We benchmark Diffusion Policy across 15 different tasks from 4 different robot manipulation benchmarks and find that it consistently outperforms existing state-of-the-art robot learning methods with an average improvement of 46.9%. Diffusion Policy learns the gradient of the action-distribution score function and iteratively optimizes with respect to this gradient field during inference via a series of stochastic Langevin dynamics steps. We find that the diffusion formulation yields powerful advantages when used for robot policies, including gracefully handling multimodal action distributions, being suitable for high-dimensional action spaces, and exhibiting impressive training stability. To fully unlock the potential of diffusion models for visuomotor policy learning on physical robots, this paper presents a set of key technical contributions including the incorporation of receding horizon control, visual conditioning, and the time-series diffusion transformer. We hope this work will help motivate a new generation of policy learning techniques that are able to leverage the powerful generative modeling capabilities of diffusion models. Code, data, and training details are available (diffusion-policy.cs.columbia.edu).

Exploring artificial neural networks for the forward kinematics of a SCARA robotic manipulator using varied datasets and training optimizers

Abstract Although analytical methods are traditionally employed, the solution to the Forward Kinematics (FK) problem for Selective Compliance Assembly Robot Arm (SCARA) manipulator robots can prove intricate and computationally demanding. Recognizing this challenge, this study endeavors to introduce an intelligent approach by leveraging Artificial Neural Networks (ANNs) to address the FK problem specifically tailored for a four-degree-of-freedom (4-DoF) SCARA robot. To train the ANNs, we employ three distinct datasets, one with a fixed step size, one with a random step size, and one based on a sinusoidal signal. Moreover, the objective is to scrutinize the ANNs performance under the influence of three distinct training algorithms: Levenberg-Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG). Through a systematic comparison of various ANN models, diverse training algorithms, and the three chosen datasets, the investigation reveals that optimal Mean Squared Error (MSE) results are achieved with random step size datasets for models with two hidden layers using the LM algorithm (MSE = 8.6099e-05). For the BR algorithm, the best MSE (8.0535e-05) was obtained with sinusoidal datasets and three hidden layers. For the SCG algorithm, the optimal MSE (1.1144e-04) was achieved with fixed step size datasets and one hidden layer. The accuracy of the ANN model is significantly influenced by the dataset, the choice of training optimizer, and the configuration of hidden layers. Consequently, further research could explore hybrid approaches that integrate evolutionary algorithms to leverage their respective strengths and improve overall ANN model performance.

A comprehensive survey of space robotic manipulators for on-orbit servicing.

On-Orbit Servicing (OOS) robots are transforming space exploration by enabling vital maintenance and repair of spacecraft directly in space. However, achieving precise and safe manipulation in microgravity necessitates overcoming significant challenges. This survey delves into four crucial areas essential for successful OOS manipulation: object state estimation, motion planning, and feedback control. Techniques from traditional vision to advanced X-ray and neural network methods are explored for object state estimation. Strategies for fuel-optimized trajectories, docking maneuvers, and collision avoidance are examined in motion planning. The survey also explores control methods for various scenarios, including cooperative manipulation and handling uncertainties, in feedback control. Additionally, this survey examines how Machine learning techniques can further propel OOS robots towards more complex and delicate tasks in space.

Pose Estimation of a Cobot Implemented on a Small AI-Powered Computing System and a Stereo Camera for Precision Evaluation.

The precision of robotic manipulators in the industrial or medical field is very important, especially when it comes to repetitive or exhaustive tasks. Geometric deformations are the most common in this field. For this reason, new robotic vision techniques have been proposed, including 3D methods that made it possible to determine the geometric distances between the parts of a robotic manipulator. The aim of this work is to measure the angular position of a robotic arm with six degrees of freedom. For this purpose, a stereo camera and a convolutional neural network algorithm are used to reduce the degradation of precision caused by geometric errors. This method is not intended to replace encoders, but to enhance accuracy by compensating for degradation through an intelligent visual measurement system. The camera is tested and the accuracy is about one millimeter. The implementation of this method leads to better results than traditional and simple neural network methods.

Disturbance observer based adaptive predefined‐time sliding mode control for robot manipulators with uncertainties and disturbances

AbstractThis article develops a predefined‐time sliding mode control approach for systems with external disturbances and uncertainties through a nonlinear disturbance observer (DO). For addressing predefined‐time stabilization problem of robotic manipulator system, a predefined‐time sliding mode surface is proposed, ensuring system states converge to origin within a predefined‐time once sliding mode surface is attained. Compared to conventional fixed‐time and finite‐time control strategies, a distinctive advantage of this scheme is that system settling time can be explicitly chosen in advance and independent of system states. To achieve predefined‐time performance, a disturbance observer is introduced to generate the disturbance estimate, which can be incorporated into controller to counteract disturbance. To address the systems uncertainty, an adaptive law is employed to estimate the unknown upper boundary of system uncertainties. Finally, the effectiveness and performance of the proposed scheme are illustrated by simulation and experiment.

Flatness-based control in successive loops for dual-arm robotic manipulators

Dual-arm robotic manipulators are used in industry and for assisting humans since they enable dexterous handling of objects and more agile and secure execution of pick-and-place, grasping and lifting, or assembling tasks. In this article, a multi-loop flatness-based controller is proposed for the dynamic model of a dual-arm robot. In the considered application, the dual-arm robotic system has to lift and transfer an object under synchronized motion of its two end-effectors while it has also to compensate for the grasping forces between the two end-effectors and the object. The dynamic model of this robotic system is formulated while it is proven that the state-space description of the robot’s dynamics is differentially flat. The control problem for this robotic system is solved with the use of a flatness-based control approach which is implemented in successive loops. To apply the multi-loop flatness-based control method, the state-space model of the dual-arm robotic manipulator is separated into subsystems, which are connected in cascading loops. Each one of these subsystems can be viewed independently as a differentially flat system and control about it can be performed with inversion of its dynamics as in the case of input–output linearized flat systems. The state variables of the preceding i-th subsystem become virtual control inputs for the subsequent ( i + 1)-th subsystem. In turn, real control inputs are applied to the last subsystem. The whole control method is implemented in successive loops and its global stability properties are also proven through Lyapunov stability analysis. The multi-loop flatness-based control approach is an exact nonlinear control scheme which (i) does not use approximate linearization of the dynamic model of the controlled system and (ii) does not need changes of state variables and complicated state-space model transformations.

Adaptive visual servoing control for uncalibrated robot manipulator with uncertain dead-zone constraint

This paper addresses the adaptive visual tracking control problem of an uncalibrated camera robot system with unknown dead-zone inputs. The uncertainties include camera extrinsic and intrinsic parameters, robot dynamic parameters, and feature depth parameters. The control achieves a better performance by establishing a smooth inverse model of the dead-zone input, which eliminates the nonlinear effect of the actuator constraint. A novel adaptive control scheme is presented which does not require a priori knowledge of the parameter intervals of dead-zone model, to update the parameter values online. Furthermore, adaptive laws are provided to estimate the uncertainties that exist both in robot and in camera models. Meanwhile, in order to avoid the singularity of the image Jacobian matrix, a projection algorithm is also introduced. Subsequently, a novel robust adaptive controller together with dead-zone compensation is established so that the tracking error image space converges to the origin. Simulations and experimental studies are conducted to verify the effectiveness of the proposed method.

Machine learning model‐based optimal tracking control of nonlinear affine systems with safety constraints

AbstractThis work focuses on the development of a machine learning (ML) model‐based framework for safe optimal tracking control of a class of nonlinear control‐affine systems to ensure simultaneous closed‐loop stability and safety. Specifically, a novel multilayer feedforward neural network (FNN) with a control‐affine architecture is designed to model nonlinear dynamic systems. Subsequently, a model‐based reinforcement learning (RL) framework is presented, utilizing a novel cost function with Control Lyapunov‐Barrier Function (CLBF) properties, to learn both the control policy and the optimal value function for an infinite‐horizon optimal tracking control problem for nonlinear systems with safety constraints. The efficacy of the proposed methodology is demonstrated through simulations of a one‐link robot manipulator and a chemical process example.

An Adaptive Sliding Mode Control Using a Novel Adaptive Law Based on Quasi-Convex Functions and Average Sliding Variables for Robot Manipulators

This paper proposes a novel adaptive law that uses a quasi-convex function and a novel sliding variable in an adaptive sliding mode control (ASMC) scheme for robot manipulators. Since the dynamic equations of robot manipulators inevitably include model uncertainties and disturbances, time-delay estimation (TDE) errors occur when using the time-delay control (TDC) approach. Further, the ASMC method used to compensate for TDE errors naturally causes a chattering phenomenon. To improve tracking performance while reducing or maintaining chattering, this paper proposes an adaptive law based on a quasi-convex function that is convex at the origin and concave at the gain switching point, respectively. We also adopt a novel sliding variable that uses previously sampled tracking errors and their time derivatives. Further, this paper proves that the sliding variable of the robot manipulator controlled by the proposed ASMC satisfies uniformly ultimately bounded stability. The simulation and experimental results illustrate the effectiveness of the proposed methods in terms of tracking performance.