End-effector Of Manipulator Research Articles

Image Dynamics-Based Visual Servo Control for Unmanned Aerial Manipulatorl With a Virtual Camera

This article proposes a visual servo control method based on image dynamics for an unmanned aerial manipulator (UAM) combining an unmanned aerial vehicle (UAV) with a multidegree-of-freedom onboard manipulator. First, taking into account the dynamic characteristics of the coupled UAM, the dynamic model of the coupled system is derived. Then, based on the perspective projection model, the appropriate image features are defined on the virtual image plane to obtain decoupled image feature dynamics. Combining image features dynamics and UAM dynamics, the position and attitude controller of the UAM flying platform with the center of gravity compensation is derived. Furthermore, the image feature motion is further applied to the positioning control of the end-effector of the onboard manipulator. In addition, the UAM motion distribution strategy is designed to coordinate the movement between the UAM flying platform and the onboard manipulator. The simulation results illustrate the performance of the proposed visual servo control based on image dynamics. As a result, the trajectories of the feature points are smooth in the real image plane, which means the UAM movement is stable. Finally, the experimental results in an outdoor environment verify the practicability and effectiveness of the proposed visual servo control method for autonomous operations. The proposed control method manages the UAM from an unknown initial position to the desired position, which can be applied to various types of aerial operations missions such as object transmission.

A Design of a 2-DoF Planar Parallel Manipulator with an Electro-Pneumatic Servo-Drive

This paper presents the design of a planar parallel manipulator with a pneumatic drive. Such manipulators are used in production lines for sorting, selecting, packing, and palletizing workpieces. This paper presents simulation studies of the designed manipulator in Matlab/Simulink software and using the SimMechanics library. A simple kinematics problem and an inverse kinematics problem were solved in order to carry out simulation studies of the designed manipulator. Simulation studies were also carried out on the dynamics of the manipulator using a mathematical model describing the physical phenomena occurring during the operation of the manipulator’s electro-pneumatic servo-drives. The main objective of the simulation study was to determine the manipulator working space and the possibility of positional control of the manipulator end-effector using a fuzzy logic controller.

Tomato harvesting robotic system based on Deep-ToMaToS: Deep learning network using transformation loss for 6D pose estimation of maturity classified tomatoes with side-stem

This paper presents the development of an autonomous harvesting robot system for tomato, a representative crop cultivated in the facility horticulture smart farm. The automated harvesting work using a robotic system is very challenging because of the appearance, environmental features, such as the atypical directions of the peduncles or their growing form in a bunch. Also, the robot system should enable the harvesting of the target fruit only without damaging other fruits, stems, and branches. Hence, this paper presents a deep learning network pipeline, Deep-ToMaToS, capable of three-level maturity classification and 6D pose (3D translation + 3D rotation) estimation of the target fruit simultaneously. Due to the difficulties encountered in building a large-scale dataset to train and test the deep learning model for the 6D pose estimation in the real world, we presented an automatic data collection scheme based on a photo-realistic 3D simulator environment. The robotic harvesting system includes a harvesting motion control algorithm based on the result of the 6D pose estimation. The overall process of the motion control phase is described along with the decision way of the appropriate final posture of the harvesting module mounted at the end-effector of the robot manipulator via removal of invalid motions getting out of the valid workspace or redundant motions. We conducted experiments on the 6D pose estimation based on the Deep-ToMaToS and the harvesting motion control in virtual and real smart farm environments. The experimental results showed a 6D pose estimation accuracy of 96 % based on the ADD_S metric, and the proposed harvesting motion control algorithm achieves the harvesting success rate of 84.5 % on average. The experimental results reveal that the harvesting robot system has significant potential to extend to harvesting works for other fruits and crops.

Dual-Mode Synchronization Predictive Control of Robotic Manipulator

Abstract To reduce the contour error of the end-effector of a robotic manipulator during trajectory tracking, a dual-mode synchronization predictive control is proposed. First, the dynamic model of n-degree-of-freedom (n-DOF) robotic manipulator is discretized by using the Taylor expansion method, and the mapping relationship between the joint error in joint space and the contour error in task space is constructed. Second, the synchronization error and the tracking error in joint space are defined, and the coupling error of joints is derived through the coupling coefficient λ. Third, a dual-mode synchronization predictive control is proposed, and the stability of the proposed control system is guaranteed using constraint set conditions. Finally, numerical simulation and experimental results are shown to prove the effectiveness of the proposed control strategy.

End Effector Control System of 6R Manipulator Based on Fuzzy PID

The front three axes of a traditional 6R manipulator move and the back three axes rotate. Given that it is directly connected with the end effector to work and there are some factors such as motor weight, inertia, and material, the end of the manipulator is designed as a system with springs and dampers, so as to improve the stability of the manipulator, reduce errors, and improve production efficiency. Conventional PID control often determines the proportional, integral, and differential effects through simulation, which can not well control the 6R manipulator system in a nonlinear and time-variant state. In order to control the system better, in addition to designing a fuzzy PID control system by making relevant membership functions and fuzzy rules, this paper simulates the end effector of the 6R manipulator by Simulink. The simulation results show that compared with conventional PID, fuzzy PID has better indexes in rapidity, stability, accuracy, and adaptability. The end effector control system of the 6R manipulator based on fuzzy PID is more stable and accurate than the ordinary 6R manipulator.

A gecko-inspired adhesive robotic end effector for critical-contact manipulation

A gecko-inspired adhesive robotic end effector for critical-contact manipulation

Meta-Heuristics Based Inverse Kinematics of Robot Manipulator’s Path Tracking Capability Under Joint Limits

In robot-assisted manufacturing or assembly, following a predefined path became a critical aspect. In general, inverse kinematics offers the solution to control the movement of manipulator while following the trajectory. The main problem with the inverse kinematics approach is that inverse kinematics are computationally complex. For a redundant manipulator, this complexity is further increased. Instead of employing inverse kinematics, the complexity can be reduced by using a heuristic algorithm. Therefore, a heuristic-based approach can be used to solve the inverse kinematics of the robot manipulator end effector, guaranteeing that the desired paths are accurately followed. This paper compares the performance of four such heuristic-based approaches to solving the inverse kinematics problem. They are Bat Algorithm (BAT), Gravitational Search Algorithm (GSA), Particle Swarm Optimization (PSO), and Whale Optimization Algorithm (WOA). The performance of these algorithms is evaluated based on their ability to accurately follow a predefined trajectory. Extensive simulations show that BAT and GSA outperform PSO and WOA in all aspects considered in this work related to inverse kinematic problems.

Model-based recurrent neural network for redundancy resolution of manipulator with remote centre of motion constraints

Redundancy resolution is a critical issue to achieve accurate kinematic control for manipulators. End-effectors of manipulators can track desired paths well with suitable resolved joint variables. In some manipulation applications such as selecting insertion paths to thrill through a set of points, it requires the distal link of a manipulator to translate along such fixed point and then perform manipulation tasks. The point is known as remote centre of motion (RCM) to constrain motion planning and kinematic control of manipulators. Together with its end-effector finishing path tracking tasks, the redundancy resolution of a manipulators has to maintain RCM to produce reliable resolved joint angles. However, current existing redundancy resolution schemes on manipulators based on recurrent neural networks (RNNs) mainly are focusing on unrestricted motion without RCM constraints considered. In this paper, an RNN-based approach is proposed to solve the redundancy resolution issue with RCM constraints, developing a new general dynamic optimisation formulation containing the RCM constraints. Theoretical analysis shows the theoretical derivation and convergence of the proposed RNN for redundancy resolution of manipulators with RCM constraints. Simulation results further demonstrate the efficiency of the proposed method in end-effector path tracking control under RCM constraints based on an industrial redundant manipulator system.

Adaptive NN motion control and predictive coordinate planning for aerial manipulators

The aerial manipulator has attracted great interest since it can interact with the environment through its end-effector. This paper divides the end-effector position tracking control task for the aerial manipulator into two subtasks. The first is motion control, and the second is coordinate planning so that the end-effector of the aerial manipulator can precisely track the given trajectory. The motion control of the aerial manipulator is solved by a partially coupled approach and divided into a multirotor controller and a manipulator controller. The motion controller is designed by the adaptive neural network control. By resorting to radial basis function neural networks with adaptive weight estimation laws, the dynamic couplings between the multirotor and the manipulator can be compensated in real time. With the support of Lyapunov stability criteria, it is proved that the desired trajectories can be boundedly tracked by the multirotor under the proposed controller. Then, a novel predictive coordinate planning method is proposed based on the move blocking method. This method ensures that the solution satisfies physical limits of the aerial manipulator and can be executed in real time. Simulations and comparisons demonstrate effectiveness and performance of the proposed methods.

Hybrid regression model via multivariate adaptive regression spline and online sequential extreme learning machine and its application in vision servo system

To solve the problems of slow convergence speed, poor robustness, and complex calculation of image Jacobian matrix in image-based visual servo system, a hybrid regression model based on multiple adaptive regression spline and online sequential extreme learning machine is proposed to predict the product of pseudo inverse of image Jacobian matrix and image feature error and online sequential extreme learning machine is proposed to predict the product of pseudo inverse of image Jacobian matrix and image feature error. In MOS-ELM, MARS is used to evaluate the importance of input features and select specific features as the input features of online sequential extreme learning machine, so as to obtain better generalization performance and increase the stability of regression model. Finally, the method is applied to the speed predictive control of the manipulator end effector controlled by image-based visual servo and the prediction of machine learning data sets. Experimental results show that the algorithm has high prediction accuracy on machine learning data sets and good control performance in image-based visual servo.

End-Effector Position Estimation and Control of a Flexible Interconnected Industrial Manipulator Using Machine Learning

The control of flexible robot manipulators is a challenging task, especially when one considers parallel and interconnected manipulators under flexibility considerations. This paper proposes a method to estimate the position of the end-effector of a flexible interconnected manipulator based on a virtual sensor principle and function approximation schemes. By using SolidWorks/MSC ADAMS software, we developed a virtual prototype of a flexible interconnected manipulator, and rigorously evaluated the feasibility of using function approximation schemes such as Neural Networks (NN), Support Vector Machines (SVM), and Gaussian Process (GP) in estimating the deflection error arising due to the flexibility of the robot structure. Our rigorous computational experiments have shown that: (1) the NN, SVM, and GP models were are able to attain the promising and reasonable prediction accuracy, (2) a feedforward NN with 535 neurons and an Ascending distribution of its nodes achieves the best prediction and generalization to unseen environments (the upper bound of the error was <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.15\times 10^{-3}$ </tex-math></inline-formula> m); implying the robust estimation of the position of the end-effector under flexibility considerations, and (3) the control based on the inverse Jacobian and a NN-based estimator was able to follow a sinusoidal trajectory with reasonable tracking and error performance in MSC ADAMS & MATLAB/Simulink co-simulation. Our results show the feasibility and effectiveness of the nonlinear relationships learned by NN, SVM, and GP in aiding estimation and control of the position of the end-effector of the flexible manipulator with a promising/desirable capability.

Flatness-based Heavy Rope Control: Experimental results with a 6R Industrial Manipulator

Flatness-based control schemes for a heavy rope with freely movable suspension point are compared in simulation and experiment. These schemes are based on a nonlinear distributed parameter model (DPM) of the chain and two different approximations, firstly a rigid body approximation as a physical pendulum, secondly a tangent linearization of the original model. The flat output is given by the free end of the rope for the DPMs and by the center of oscillation for the rigid body approximation. The design is achieved by prescribing a desired path for the flat output and computing the corresponding path of the suspension point. Numerical simulations are performed by means of a finite-difference approximation of the original non-linear DPM. For the experimental studies the chain is suspended from the end effector of a 6R industrial manipulator. The required joint angles of the manipulator are computed by selecting one of the eight solutions of the inverse kinematics of the robot.

Positional error analysis of 3-RPR planar parallel manipulator under the influence of tolerances

The positional error analysis of the 3-RPR (R-Revolute, P-Prismatic) planar parallel manipulator under the effect of link tolerances is presented. In this research, a new mathematical model is derived to analyse the effect of tolerances on the position of the end-effector of a 3-RPR planar parallel manipulator. The obtained results are verified by CAD model approach. The proposed approach is applied for different positions of the end effector in the linear actuator limit. Different combinations of tolerances are considered, and an effort is made to estimate the maximum error that occurred by varying the input displacement.

Data-Driven Human-Robot Interaction Without Velocity Measurement Using Off-Policy Reinforcement Learning

In this paper, we present a novel data-driven design method for the human-robot interaction (HRI) system, where a given task is achieved by cooperation between the human and the robot. The presented HRI controller design is a two-level control design approach consisting of a task-oriented performance optimization design and a plant-oriented impedance controller design. The task-oriented design minimizes the human effort and guarantees the perfect task tracking in the outer-loop, while the plant-oriented achieves the desired impedance from the human to the robot manipulator end-effector in the inner-loop. Data-driven reinforcement learning techniques are used for performance optimization in the outer-loop to assign the optimal impedance parameters. In the inner-loop, a velocity-free filter is designed to avoid the requirement of end-effector velocity measurement. On this basis, an adaptive controller is designed to achieve the desired impedance of the robot manipulator in the task space. The simulation and experiment of a robot manipulator are conducted to verify the efficacy of the presented HRI design framework.

YOLO 를 이용한 CNN 기반의 물체 인식과 파지 중심점 위치오차 최소화 알고리즘 연구 및 파지 성능 평가

In the last few years, the use of robot manipulators has attracted increasing attention in various industries. Accordingly. Researchers have proposed unique ideas for co-robot control using vision sensors. In this study, you only look once (YOLO) based on convolutional neural network (CNN), and grasping center point position error minimization algorithms were proposed to reduce object misrecognition and increase the performance for grasping an object. In addition, a gripping algorithm was designed for six degree of freedom (DOF) robot manipulators. In addition, machine vision algorithms, including a Grayscale, Gaussian filter, Canny edge, and Contouring, were implemented to detect objects features, such as centroids and orientation. Furthermore, the coordinate system of the vision sensor was converted into a coordinate system of the robot manipulator using a transformation matrix to accurately move the end effector of the robot arm to the center point of the object. The logic implemented in this study not only detected the trained object on the workstation, but also minimized the positional error of the transformation matrix. Additionally, experiments were performed on the 6-DOF robot manipulators. The results revealed that the end effector of the 6-DOF manipulators successfully moved to the center of the detected object, and each of the eight objects was normally gripped.

Direct visual servoing and interaction control for a two-arms on-orbit servicing spacecraft

A direct visual-servoing algorithm for control of a space-based two-arm manipulator is proposed in this paper. The algorithm can be utilized in a two-arm manipulators configuration, where one of the arms performs the manipulation and the second arm is dedicated to the observation of the target zone of manipulation. The algorithm utilizes both visual features extracted from onboard cameras and force and torque measured at the manipulator's end-effector to control the movements of the manipulator during on-orbit servicing operations. The algorithm takes into account the relative dynamics of the bodies involved, it relies on images taken independently from de-localized cameras, e.g. at the end-effector of a second manipulator, and it integrates an impedance control for the compensation of eventual contact reactions when the end effector touches and operates the target body. The analytical derivations demonstrate the stability of the algorithm and incorporate an impedance compliance strategy into an optimal framework formulation. Simulations results in two different scenarios have been presented to show the adequate behavior of the presented approach in on-orbit-servicing operations.

Optimal Path Search for Robot Manipulator using Deep Reinforcement Learning

Much research using deep reinforcement learning has been conducted to make robot manipulators perform certain tasks with little prior knowledge of the work environment. However, due to the high-dimension continuous action-state spaces that a robot manipulator must work in, it is difficult for deep reinforcement learning agents to find or even learn optimal policies. In this paper, we present a method for generating optimal paths for the end effector of a robot manipulator to move through. This is achieved by using images of the workspace that allow us to avoid the problems of high dimensionality while the system learns pick-and-place tasks using deep reinforcement learning. The robot simulator Webots was used to implement the robot model in a virtual work environment. Simulations confirmed that the robot manipulator is able to successfully move target objects from their starting points to their destinations while avoiding moving obstacles in real time, even if the size, position, and number of obstacles are varied.

Dynamics Simulation of Grasping Process of Underwater Vehicle-Manipulator System

Underwater vehicle-manipulator system (UVMS) can be applied to fulfill different complex underwater tasks such as grasping, drilling, sampling, etc. It is widely used in the field of oceanographic research, marine exploration, military, and commercial applications. In this paper, the dynamic simulation of UVMS is presented in the process of grasping an object. First, the dynamic model of UVMS, which considers the change of the load of manipulator when the end effector of manipulator grasps the object, is developed. To compare different conditions, numerical simulation of grasping processes without/with vehicle attitude control are carried out. The simulation results show that the coupling dynamics between the vehicle and the manipulator in the grasping process are clearly illustrated. It deteriorates the positioning accuracy of the end effector of the manipulator and is harmful to underwater precision operations. The tracking position error of end effector without vehicle control is large and UVMS cannot complete the grasping task under this condition. Vehicle control can compensate the motion of the vehicle due to the coupling effect caused by the motion of the manipulator. This study will contribute to underwater operation mission for UVMS with floating base.

Closed-Loop Pose Control and Automated Suturing of Continuum Surgical Manipulators With Customized Wrist Markers Under Stereo Vision

The use of continuum manipulators in surgical applications can be beneficial because of their inherent safety from their structural compliance. However, the tip pose (position and orientation) accuracy of a continuum manipulator can be low when an external load or disturbance is applied. Closed-loop pose control is preferred to improve the pose accuracy. Considering the requirements stemmed from sterilization and medical device electromagnetic compatibility, the approach of fully utilizing the vision feedback from a stereo endoscopic camera for such a closed-loop pose controller is adopted. Wrist markers are attached to the end-effectors of the surgical continuum manipulators for pose information detection. And these wrist markers are designed in a well-considered way to generate enough variants. The closed-loop pose controller is then designed based on a stereo vision tracking algorithm that detects the position and orientation of the wrist marker. A switching feature is also incorporated into the controller to handle the situation when the closed-loop control is disrupted due to the visual occlusion or the operation transition in dual-arm manipulation. Experimental verification showed that the average errors of the pose feedback are 0.65 mm for the position and 1.05° for the orientation. The experiments of trajectory following under a 120-gram external load and automated dual-arm suturing were also conducted to verify the effectiveness of the proposed controller.

Manipulability and Kinematics Analysis of Three-Linked Robotic Arm Using Mathematical Program

The Kinematics study deals with the motion of the bodies and the mechanism without consideration of force. Geometry is applied by robot kinematics to the research movement of multi-degree freedom kinematic links that shape the robot manipulator structure. The most critical problem is locating the end effector and angle of the robot manipulator. Used in automation industries to perform various tasks. To overcome this problem, Singular Value Decomposition (SVD) of the Jacobin matrix has been derived using length and angle for robotic manipulator links. In this work, a mathematical program for solving kinematics of the three linked robotic arms has been developed. End effector position x, y and joint angles θ1, θ2, θ3 for the robotic arm have been simulated and values are obtained by using MATLAB software. From the developed mathematical program, the required progression positions of the robotic arm over time have been achieved.