Robotic Manipulator Arm Research Articles

Robot arm grasping based on YOLOv5 in the perspective of automated production

Abstract The rapid development of industrial intelligence has gradually expanded the application of automated production. As a typical automated production equipment, the robotic arm still faces the problems of low grasping efficiency and high control costs when facing highly integrated and miniaturized components. Given this, to improve the grasping level of the robotic arm in complex production environments, this study first constructs a kinematic mathematical model of the robotic arm. Secondly, based on the algorithm that You Only Look Once, improvements are made to its convolution operation and feature extraction modules, ultimately proposing a new type of robotic arm grasping control model. The results showed that the loss test value of the new model was the lowest at 2.75, the average detection error of the captured object was the lowest at 0.003, and the average detection time was the shortest at 1.28 seconds. The highest success rate for grasping six types of industrial parts was 94%, and the lowest average energy consumption was 35.67 joules. Therefore, research models can significantly improve the grasping performance of robotic arms under various complex conditions, thereby achieving efficient manipulation of robotic arms in industrial automation.

Practical Probabilistic Model-Based Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory Sampling.

This article addresses the prediction stability, prediction accuracy, and control capability of the current probabilistic model-based reinforcement learning (MBRL) built on neural networks. A novel approach to dropout-based probabilistic ensembles with trajectory sampling (DPETS) is proposed, where the system uncertainty is stably predicted by combining the Monte Carlo dropout (MC Dropout) and trajectory sampling in one framework. Its loss function is designed to correct the fitting error of neural networks for more accurate prediction of probabilistic models. The state propagation in its policy is extended to filter the aleatoric uncertainty for superior control capability. Evaluated by several Mujoco benchmark control tasks under additional disturbances and one practical robot arm manipulation task, DPETS outperforms related MBRL approaches in both average return and convergence velocity while achieving superior performance than well-known model-free baselines with significant sample efficiency. The open-source code of DPETS is available at https://github.com/mrjun123/DPETS.

Exploring artificial neural networks for the forward kinematics of a SCARA robotic manipulator using varied datasets and training optimizers

Abstract Although analytical methods are traditionally employed, the solution to the Forward Kinematics (FK) problem for Selective Compliance Assembly Robot Arm (SCARA) manipulator robots can prove intricate and computationally demanding. Recognizing this challenge, this study endeavors to introduce an intelligent approach by leveraging Artificial Neural Networks (ANNs) to address the FK problem specifically tailored for a four-degree-of-freedom (4-DoF) SCARA robot. To train the ANNs, we employ three distinct datasets, one with a fixed step size, one with a random step size, and one based on a sinusoidal signal. Moreover, the objective is to scrutinize the ANNs performance under the influence of three distinct training algorithms: Levenberg-Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG). Through a systematic comparison of various ANN models, diverse training algorithms, and the three chosen datasets, the investigation reveals that optimal Mean Squared Error (MSE) results are achieved with random step size datasets for models with two hidden layers using the LM algorithm (MSE = 8.6099e-05). For the BR algorithm, the best MSE (8.0535e-05) was obtained with sinusoidal datasets and three hidden layers. For the SCG algorithm, the optimal MSE (1.1144e-04) was achieved with fixed step size datasets and one hidden layer. The accuracy of the ANN model is significantly influenced by the dataset, the choice of training optimizer, and the configuration of hidden layers. Consequently, further research could explore hybrid approaches that integrate evolutionary algorithms to leverage their respective strengths and improve overall ANN model performance.

Robust Adaptive Sliding Mode Control Using Stochastic Gradient Descent for Robot Arm Manipulator Trajectory Tracking

This paper presents an innovative control strategy for robot arm manipulators, utilizing an adaptive sliding mode control with stochastic gradient descent (ASMCSGD). The ASMCSGD controller significant improvements in robustness, chattering elimination, and fast, precise trajectory tracking. Its performance is systematically compared with super twisting algorithm (STA) and conventional sliding mode control (SMC) controllers, all optimized using the grey wolf optimizer (GWO). Simulation results show that the ASMCSGD controller achieves root mean squared errors (RMSE) of 0.12758 for θ1 and 0.13387 for θ2. In comparison, the STA controller yields RMSE values of 0.1953 for θ1 and 0.1953 for θ2, while the SMC controller results in RMSE values of 0.24505 for θ1 and 0.29112 for θ2. Additionally, the ASMCSGD simplifies implementation, eliminates unwanted oscillations, and achieves superior tracking performance. These findings underscore the ASMCSGD’s effectiveness in enhancing trajectory tracking and reducing chattering, making it a promising approach for robust control in practical applications of robot arm manipulators.

Developing novel non-assistant help operation in dual-portal robotic-assisted thoracic surgery (neoDRATS)

ObjectiveTo introduce and evaluate the non-assistant help operation in dual-portal robotic-assisted thoracic surgery (neoDRATS), a novel technique designed to eliminate the need for skilled assistants by using all 4 robotic arms independently during anatomical lung surgery. MethodsPatients were placed in the lateral decubitus position under general anesthesia with single-lung ventilation. The da Vinci Xi Surgical System was used, with specific configurations for right- and left-side operations. The neoDRATS technique used a 4-cm working port and a 1.8-cm secondary port, with detailed guidelines for optimal setup and robotic arm manipulation. ResultsThe neoDRATS approach demonstrated successful surgical outcomes without the need for a skilled assistant. The use of a 0° camera and careful placement of instruments minimized interference within the thoracic cavity. The technique provided smooth operability and minimized postoperative discomfort. Video demonstrations of right and left upper lobectomies are provided to illustrate the approach. ConclusionsNeoDRATS offers a practical, safe, and minimally invasive alternative to conventional multiportal and uniportal robotic-assisted thoracic surgeries. This technique simplifies the surgical process, particularly in settings with limited availability of skilled assistants, and represents a significant advancement in robotic thoracic surgery. Further refinement and clinical integration of neoDRATS are anticipated as robotic innovations continue to evolve.

Comparative Analysis: Fractional PID vs. PID Controllers for Robotic Arm Using Genetic Algorithm Optimization

This paper presents a comparative analysis of a fractional-order proportional–integral–derivative (FO-PID) controller against the standard proportional–integral–derivative (PID) controller, applied to a nonlinear robotic arm manipulator systems. The genetic algorithm (GA) optimization method was implemented to tune the gain parameters of the FO-PID and PID controllers. The performance of the FO-PID and PID controllers were evaluated though different cost functions, including integral of squared error (ISE), integral of absolute error (IAE), integral of time-weighted absolute error (ITAE), and integral of time-weighted squared error (ITSE). The performance of these controllers was examined via extensive simulations by using MATLAB/SIMULINK for different operating scenarios of the robotic arm manipulator system. Based on the obtained results, a comparative performance matrix is proposed, wherein cost functions ISE, IAE, ITAE, and ITSE are represented as columns while characteristic parameters (overshoot, rising time, and settling time) are represented as rows. The proposed performance matrix facilitates the selection between the PID and FO-PID controllers.

Computed Torque-NN-GWO Dynamic Hybrid Control of Manipulator Robotic Arm

Robotics involves designing, simulating, and controlling robots for various industries and daily tasks. Manipulator arm robots are precisionoriented industrial robots designed for one-plane movement. Planar robotic manipulators are used in cargo loading, assembly lines, handling radioactive materials, and military applications. In this study theoretical analysis of manipulator robot kinematics using Denavit-Hartenberg notation and the Euler-Lagrange method for a three-degree of freedom (3-DOF) system, requiring an advanced controller for accurate control. A new hybrid controller, Computed Torque Control-Neural-Network -Grey Wolf Optimization (CTC-NNGWO), is proposed for path planning of 3-DOF MR. It uses kinematics and dynamic models, computes torque magnitudes of stepper motors, and denormalizes output values from NN. MATLAB programming is used for the simulation of the theoretical result. A comparison study of CTC-NN-GWO and CTC-PSO found that CTC-NN-GWO outperforms CTC-PSO in minimizing path planning errors, the maximal value of the error of joint angular position 𝑒𝜃1 is (0.29 rad), whereas 𝑒𝜃2 and 𝑒𝜃3 are approximately (0.15 rad) and (0.07 rad), respectively, also, reducing torque magnitudes, improving the performance of 3-DOF MR. Index Terms— Manipulator robotic, Grey wolf, Neural network, Path planning, Computed torque.

Observer-Based Fault-Tolerant Control for Uncertain Robot Manipulators without Velocity Measurements

In recent years, robot manipulator arms have become increasingly prevalent and are playing pivotal roles across various industries. Their ability to replace human labor in arduous and hazardous tasks has positioned them as indispensable assets. Consequently, there has been a surge in research efforts aimed at enhancing their operational performance. The imperative to improve their efficiency and effectiveness has garnered significant attention within the research community. In this study, a novel fault-tolerant control (FTC) scheme for robot manipulators to handle the effects of the unknown input is proposed to aid robots in achieving good tracking performance. In the first step, an extended state observer (ESO) is constructed to approximate both velocities and the unknown input in the robot system. The observer offers estimation information with good accuracy and quick convergence. The estimated signals are then combined with computed torque control (CTC), which is a useful control technique for trajectory tracking of robot manipulator systems, to construct an active FTC to decrease the influences of the unknown input. The proposed algorithm does not require velocity measurement in the design process. In addition, with a novel design approach, the combination of controller and observer provides a novel control signal that delivers higher tracking performance compared to the traditional design approach. The global and asymptotic stability of the suggested technique is proved through the Lyapunov theory. Finally, simulations are implemented on a 2-degree-of-freedom (DOF) robot manipulator to validate the efficiency of the proposed controller–observer method.

Shape Control of Elastic Deformable Linear Objects for Robotic Cable Assembly

Currently, cables are manually installed in aircraft manufacturing scenarios. The utilization of robots in cable assemblies can enhance efficiency and ensure quality, presenting great promise for the future. However, the assembly process must control the shape of the cables, which is a significant challenge for robots. On the one hand, cables have high degrees of freedom in space, making accurate modeling of cable dynamics for robotic arm manipulation difficult; on the other hand, cable deformation has uncertainty, and the applied forces cause simultaneous deformation and motion, which is difficult to control. To address these problems, this study proposes a cable‐shape control method based on graph neural networks and online visual shape‐servoing. The method first approximates cable dynamics with a graph neural network. Then, in practice, the learnt model is used alongside visual‐based shape‐servoing to generate optimal robotic arm movements, controlling the cable to attain the desired shape. In the experiments, precise control of three different cable types is realized, and an example of a completed cable assembly is shown.

Composing MPC With LQR and Neural Network for Amortized Efficiency and Stable Control

Model predictive control (MPC) is a powerful control method that handles dynamical systems with constraints. However, solving MPC iteratively in real time, i.e., implicit MPC, remains a computational challenge. To address this, common solutions include explicit MPC and function approximation. Both methods, whenever applicable, may improve the computational efficiency of the implicit MPC by several orders of magnitude. Nevertheless, explicit MPC often requires expensive pre-computation and does not easily apply to higher-dimensional problems. Meanwhile, function approximation, although scales better with dimension, still requires pre-training on a large dataset and generally cannot guarantee to find an accurate surrogate policy, the failure of which often leads to closed-loop instability. To address these issues, we propose a triple-mode hybrid control scheme, named Memory-Augmented MPC, by combining a linear quadratic regulator, a neural network, and an MPC. From its standard form, we derive two variants of such hybrid control scheme: one customized for chaotic systems and the other for slow systems. The proposed scheme does not require pre-computation and is capable of improving the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">amortized</i> running time of the composed MPC with a well-trained neural network. In addition, the scheme maintains closed-loop stability with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">any</i> neural networks of proper input and output dimensions, alleviating the need for certifying optimality of the neural network in safety-critical applications. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article was motivated by the need to reduce the amortized cost of MPC in repetitive industrial robotic applications, where long-term operational cost is important and safety is critical. Examples of such applications include factory robotic arm manipulation and fixed-route quadcopter payload transport. Unlike explicit MPC or function approximation, our approach does not require any pre-computation or pre-training. Rather, it attains task proficiency over time by learning a surrogate neural network on the spot and by gradually replacing the costly MPC with the more efficient surrogate model so long as safety permits. Consequently, the proposed scheme incurs a learning cost during the initial phase of the deployment but usually becomes more adept on the task afterwards, leading to amortized efficiency.

Unveiling the Significance of Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning

Toddlers evolve from free exploration with sparse feedback to exploiting prior experiences for goal-directed learning with denser rewards. Drawing inspiration from this Toddler-Inspired Reward Transition, we set out to explore the implications of varying reward transitions when incorporated into Reinforcement Learning (RL) tasks. Central to our inquiry is the transition from sparse to potential-based dense rewards, which share optimal strategies regardless of reward changes. Through various experiments, including those in egocentric navigation and robotic arm manipulation tasks, we found that proper reward transitions significantly influence sample efficiency and success rates. Of particular note is the efficacy of the toddler-inspired Sparse-to-Dense (S2D) transition. Beyond these performance metrics, using Cross-Density Visualizer technique, we observed that transitions, especially the S2D, smooth the policy loss landscape, promoting wide minima that enhance generalization in RL models.

Machine learning meets advanced robotic manipulation

Automated industries lead to high quality production, lower manufacturing cost and better utilization of human resources. Robotic manipulator arms have major role in the automation process. However, for complex manipulation tasks, hard coding efficient and safe trajectories is challenging and time consuming. Machine learning methods have the potential to learn such controllers based on expert demonstrations. Despite promising advances, better approaches must be developed to improve safety, reliability, and efficiency of ML methods in both training and deployment phases. This survey aims to review cutting edge technologies and recent trends on ML methods applied to real-world manipulation tasks. After reviewing the related background on ML, the rest of the paper is devoted to ML applications in different domains such as industry, healthcare, agriculture, space, military, and search and rescue. The paper is closed with important research directions for future works.

Automatic Fruit Plucking Machine Using Deep Learning

Abstract: The Automatic Fruit Plucking Machine project aims to develop an innovative automated system to revolutionize fruit harvesting. By integrating computer vision, robotic arm manipulation, and YOLO deep learning algorithms, the machine can accurately identify and pluck ripe fruits. It reduces labor costs, enhances harvest quality, and improves productivity. The machine's core is a trained machine learning algorithm that combines image processing and deep learning models for real-time fruit detection. Through iterative testing and refinement, the machine's performance continually improves. This project offers a sustainable and efficient solution for fruit producers to meet market demands and reduce manual labor reliance

Collision Avoidance for a Selective Compliance Assembly Robot Arm Manipulator Using Topological Path Planning

Industrial applications with robotic manipulators have grown and made production systems increasingly efficient. However, there are still some limitations that can delay production, causing losses. Several factors, such as accidents and collisions of manipulator robots with operators and other machines, can cause unforeseen stops. Thus, this work aims to develop a trajectory planning method to avoid collisions applied to a selective compliance assembly robot arm (SCARA) robotic manipulator in the context of collaborative robotics. The main contribution of this paper is a path planning method based on mathematical morphology, named topological path planning (TPP). Through some evaluation metrics such as the number of path points, computing time, distance, standard deviation of the joint acceleration, and maximum acceleration rate along the path, we show that TPP is a collision-free, deterministic, and predictable route planning. In our experiments, our proposal presented better results for applications in industrial robotic manipulators when compared to the probabilistic roadmap method (PRM) and TPP*, a particular case of TPP that is similar to the generalized Voronoi diagram (GVD).

Simulation Power vs. Immersive Capabilities: Enhanced Understanding and Interaction with Digital Twin of a Mechatronic System

Automation Studio, a specialised simulation software, offers virtual commissioning capabilities and robust tools for modelling the behaviour and performance of a pneumatic robot controlled by a PLC. Conversely, Unity is a versatile platform primarily used for creating high-quality 3D games and interactive simulations, providing immersive experiences with DTs through mixed-reality environments. This paper provides a study that compares and contrasts the simulation power of Automation Studio and the immersive capabilities of Unity in the context of developing digital twins for a mechatronic system. This research explores how these complementary approaches enhance the development, validation, understanding, and interaction with digital twins. By examining both platforms in this context, the article provides valuable insights for engineers, developers, and researchers looking to create digital twins for mechatronic systems, but not only. This study demonstrates the potential of leveraging the combined power of simulation and immersive capabilities to improve the interaction between the real robotic arm manipulator cylindrical type and its digital twin in different scenarios, using an OPC approach for mirroring. The combination of Automation Studio and Unity provides a powerful platform for applied science education in the field of the digital twin of mechatronic systems.

Modeling and Simulation for a Five DOF Robotic Arm Manipulator

Nowadays, service robots are used in a large number of applications in the service industry. With the booming service industry in third world countries, service robots are expected to be used in a wider range of applications, so it is of great importance to conduct research on service robots. This paper develops the model of a kinematic models for a 5 DOF robotic arm that used in service industry in SolidWorks. The robotic arm’s mathematical model is predicated on Denavit-Hartemberg (DH) method, which determine the robot joints angle vector. The procedure aims at describing the forward and inverse kinematic analysis, and finally determine the end effector’s position and orientation through the corresponding joints angle related to the coordinate system (CS), in order to control the robotic arm to reach the designated location in space. A simulation model of the robotic arm was built using the Matlab Robotics Toolbox to verify the correctness of forward and inverse kinematics, which also can simplify the calculation and analysis.

Trajectory planning for a 6-axis robotic arm with particle swarm optimization algorithm

Robotic arms, which are favored for usage in both large- and small-scale industrial regions, run into issues with numerous limits between the starting and ending locations in the working space when attempting to complete a particular task. With the solution to one of these problems, trajectory planning, the robotic arm manipulator can move from the starting point to the target point vibration-free, without hitting obstacles, and by choosing the shortest way. In this study, a robotic arm with 6 degrees of freedom, which is in the Mechatronics Engineering laboratory of Isparta University of Applied Sciences and whose prototype was realized by Acrome company, was used. In the study, the trajectory planning of the robotic arm was carried out using the MATLAB program and particle swarm optimization (PSO). Trajectory planning is developed using the PSO algorithm to determine the position of the robot at each point as it moves from its starting point to its target. Thus, time optimization was achieved by choosing the shortest path between the two points. Trajectory planning in joint space is aimed to ensure that the position, speed, and acceleration between the starting and ending points are continuous by using the fifth-order polynomial. The instant values of the joint variables used to determine the points followed by the manipulator were obtained by forward kinematics through the MATLAB program. Using forward kinematics, the position information of the manipulator was obtained by providing a transition from joint space to Cartesian space.

Adaptive Learning-Based Distributed Control of Cooperative Robot Arm Manipulation for Unknown Objects

This article proposes a distributed cooperative manipulation control scheme for multirobot systems to track reference trajectories with unknown payload dynamics, grasp positions, and external disturbances. An online learning module is established to estimate the payload dynamics. Then a wrench-synthetic trajectory tracking control protocol is thereby developed to manipulate an object under unknown external disturbances no matter where the grasping points are. Moreover, sufficient conditions are derived to guarantee the uniform boundedness of the tracking errors of the closed-loop cooperative manipulation system. Finally, numerical simulations are conducted to substantiate the effectiveness of the proposed cooperative manipulation control scheme.

Minimizing the Effect of Specular Reflection on Object Detection and Pose Estimation of Bin Picking Systems Using Deep Learning

The rapid evolution towards industrial automation has widened the usage of industrial applications, such as robot arm manipulation and bin picking. The performance of these applications relies on object detection and pose estimation through visual data. In fact, the clarity of those data significantly influences the accuracy of object detection and pose estimation. However, a majority of visual data corresponding to metal or glossy surfaces tend to have specular reflections that reduce the accuracy. Hence, this work aims to improve the performance of industrial bin-picking tasks by reducing the effects of specular reflections. This work proposes a deep learning (DL)-based neural network model named SpecToPoseNet to improve object detection and pose estimation accuracy by intelligently removing specular reflections. The proposed work implements a synthetic data generator to train and test the SpecToPoseNet. The conceptual breakthrough of this work is its ability to remove specular reflections from scenarios with multiple objects. With the use of the proposed method, we could reduce the fail rate of object detection to 7%, which is much less compared to specular images (27%), U-Net (20%), and the basic SpecToPoseNet model (11%). Thus, it is claimable that the performance improvements gained are positive influences of the proposed DL-based contexts such as bin-picking.

Dual Arm Impact-Aware Grasping through Time-Invariant Reference Spreading Control

With the goal of increasing the speed and efficiency in robotic dual arm manipulation, a novel control approach is presented that utilizes intentional simultaneous impacts to rapidly grasp objects. This approach uses the time-invariant reference spreading framework, in which partly-overlapping ante and post-impact reference vector fields are used. These vector fields are coupled via the impact dynamics in proximity of the expected impact area, minimizing the otherwise large velocity errors after the impact and the corresponding large control efforts. A purely spatial task is introduced to strongly encourage the synchronization of impact times of the two arms. An interim-impact control phase provides robustness in the execution against the inevitable lack of exact impact simultaneity and the corresponding unreliable velocity error. In this interim phase, a position feedback signal is derived from the ante-impact velocity reference, which is used to enforce sustained contact in all contact points without using velocity error feedback. With an eye towards real-life implementation, the approach is formulated using a QP control framework, and is validated using numerical simulations on a realistic robot model with flexible joints and low-level torque control.