Robot Arm Research Articles

Tracking cloth deformation: A novel dataset for closing the sim-to-real gap for robotic cloth manipulation learning

Robotic learning for deformable object manipulation—such as textiles—is often done in simulation due to the current limitation of perception methods to understand cloth’s deformation. For this reason, the robotics community is always on the search for more realistic simulators to reduce as much as possible the sim-to-real gap, which is still quite large especially when dynamic motions are applied. We present a cloth dataset consisting of 120 high-quality recordings of several textiles during dynamic motions. Using a Motion Capture System, we record the location of key-points on the cloth surface of four types of fabrics (cotton, denim, wool and polyester) of two sizes and at different speeds. The scenarios considered are all dynamic and involve rapid shaking and twisting of the textiles, collisions with frictional objects, strong hits with a long and thin rigid object and even self-collisions. We explain in detail the scenarios considered, the collected data and how to read it and use it. In addition, we propose a metric to use the dataset as a benchmark to quantify the sim-to-real gap of any cloth simulator. Finally, we show that the recorded trajectories can be directly executed by a robotic arm, enabling learning by demonstration and other imitation learning techniques. Dataset: https://doi.org/10.5281/zenodo.14644526 Video: https://fcoltraro.github.io/projects/dataset/

The Design and Analysis of a Lightweight Robotic Arm Based on a Load-Adaptive Hoisting Mechanism

This paper presents the design and control of a lightweight three degrees of freedom robotic arm based on a load-adaptive hoisting mechanism. The proposed design integrates a spring-loaded rope and a variable radius reel into the gripper, enabling efficient load adaptability with minimal structural complexity. By leveraging this mechanism, the robotic arm achieves significant weight reduction while maintaining robust performance under variable payloads. The study includes a comprehensive analysis of the system’s kinematics and dynamics, focusing on the interaction between the adaptive gripper and the arm structure. A prototype of the robotic arm was developed and experimentally tested to validate its functionality.

Robot-Based Procedure for 3D Reconstruction of Abdominal Organs Using the Iterative Closest Point and Pose Graph Algorithms

Image-based 3D reconstruction enables robot-assisted interventions and image-guided navigation, which are emerging technologies in laparoscopy. When a robotic arm guides a laparoscope for image acquisition, hand–eye calibration is required to know the transformation between the camera and the robot flange. The calibration procedure is complex and must be conducted after each intervention (when the laparoscope is dismounted for cleaning). In the field, the surgeons and their assistants cannot be expected to do so. Thus, our approach is a procedure for a robot-based multi-view 3D reconstruction without hand–eye calibration, but with pose optimization algorithms instead. In this work, a robotic arm and a stereo laparoscope build the experimental setup. The procedure includes the stereo matching algorithm Semi Global Matching from OpenCV for depth measurement and the multiscale color iterative closest point algorithm from Open3D (v0.19), along with the multiway registration algorithm using a pose graph from Open3D (v0.19) for pose optimization. The procedure is evaluated quantitatively and qualitatively on ex vivo organs. The results are a low root mean squared error (1.1–3.37 mm) and dense point clouds. The proposed procedure leads to a plausible 3D model, and there is no need for complex hand–eye calibration, as this step can be compensated for by pose optimization algorithms.

Improving 3D Reconstruction Through RGB-D Sensor Noise Modeling

High-resolution RGB-D sensors are widely used in computer vision, manufacturing, and robotics. The depth maps from these sensors have inherently high measurement uncertainty that includes both systematic and non-systematic noise. These noisy depth estimates degrade the quality of scans, resulting in less accurate 3D reconstruction, making them unsuitable for some high-precision applications. In this paper, we focus on quantifying the uncertainty in the depth maps of high-resolution RGB-D sensors for the purpose of improving 3D reconstruction accuracy. To this end, we estimate the noise model for a recent high-precision RGB-D structured light sensor called Zivid when mounted on a robot arm. Our proposed noise model takes into account the measurement distance and angle between the sensor and the measured surface. We additionally analyze the effect of background light, exposure time, and the number of captures on the quality of the depth maps obtained. Our noise model seamlessly integrates with well-known classical and modern neural rendering-based algorithms, from KinectFusion to Point-SLAM methods using bilinear interpolation as well as 3D analytical functions. We collect a high-resolution RGB-D dataset and apply our noise model to improve tracking and produce higher-resolution 3D models.

Enhanced crayfish optimization algorithm: Orthogonal refracted opposition-based learning for robotic arm trajectory planning.

In high-dimensional scenarios, trajectory planning is a challenging and computationally complex optimization task that requires finding the optimal trajectory within a complex domain. Metaheuristic (MH) algorithms provide a practical approach to solving this problem. The Crayfish Optimization Algorithm (COA) is an MH algorithm inspired by the biological behavior of crayfish. However, COA has limitations, including insufficient global search capability and a tendency to converge to local optima. To address these challenges, an Enhanced Crayfish Optimization Algorithm (ECOA) is proposed for robotic arm trajectory planning. The proposed ECOA incorporates multiple novel strategies, including using a tent chaotic map for population initialization to enhance diversity and replacing the traditional step size adjustment with a nonlinear perturbation factor to improve global search capability. Furthermore, an orthogonal refracted opposition-based learning strategy enhances solution quality and search efficiency by leveraging the dominant dimensional information. Additionally, performance comparisons with eight advanced algorithms on the CEC2017 test set (30-dimensional, 50-dimensional, 100-dimensional) are conducted, and the ECOA's effectiveness is validated through Wilcoxon rank-sum and Friedman mean rank tests. In practical robotic arm trajectory planning experiments, ECOA demonstrated superior performance, reducing costs by 15% compared to the best competing algorithm and 10% over the original COA, with significantly lower variability. This demonstrates improved solution quality, robustness, and convergence stability. The study successfully introduces novel population initialization and search strategies for improvement, as well as practical verification in solving the robotic arm path problem. The results confirm the potential of ECOA to address optimization challenges in various engineering applications.

Autonomous robotic ultrasound scanning system: a key to enhancing image analysis reproducibility and observer consistency in ultrasound imaging

PurposeThis study aims to develop an autonomous robotic ultrasound scanning system (auto-RUSS) pipeline, comparing its reproducibility and observer consistency in image analysis with physicians of varying levels of expertise.Design/methodology/approachAn auto-RUSS was engineered using a 7-degree-of-freedom robotic arm, with real-time regulation based on force control and ultrasound visual servoing. Two phantoms were employed for the human-machine comparative experiment, involving three groups: auto-RUSS, non-expert (4 junior physicians), and expert (4 senior physicians). This setup enabled comprehensive assessment of reproducibility in contact force, image acquisition, image measurement and AI-assisted classification. Radiological feature variability was measured using the coefficient of variation (COV), while performance and reproducibility assessments utilized mean and standard deviation (SD).FindingsThe auto-RUSS had the potential to reduce operator-dependent variability in ultrasound examinations, offering enhanced repeatability and consistency across multiple dimensions including probe contact force, images acquisition, image measurement, and diagnostic model performance.Originality/valueIn this paper, an autonomous robotic ultrasound scanning system (auto-RUSS) pipeline was proposed. Through comprehensive human-machine comparison experiments, the auto-RUSS was shown to effectively improve the reproducibility of ultrasound images and minimize human-induced variability.

Universal slip detection of robotic hand with tactile sensing

Slip detection is to recognize whether an object remains stable during grasping, which can significantly enhance manipulation dexterity. In this study, we explore slip detection for five-finger robotic hands being capable of performing various grasp types, and detect slippage across all five fingers as a whole rather than concentrating on individual fingertips. First, we constructed a dataset collected during the grasping of common objects from daily life across six grasp types, comprising more than 200 k data points. Second, according to the principle of deep double descent, we designed a lightweight universal slip detection convolutional network for different grasp types (USDConvNet-DG) to classify grasp states (no-touch, slipping, and stable grasp). By combining frequency with time domain features, the network achieves a computation time of only 1.26 ms and an average accuracy of over 97% on both the validation and test datasets, demonstrating strong generalization capabilities. Furthermore, we validated the proposed USDConvNet-DG in real-time grasp force adjustment in real-world scenarios, showing that it can effectively improve the stability and reliability of robotic manipulation.

DTL-GNN: Digital Twin Lightweight Method Based on Graph Neural Network

In the digital twin system of mechatronics engineering, the scale and accuracy of models are continually improving. Nevertheless, this growth can hinder real-time interaction and decision-making accuracy within digital twins. The resulting delay impacts the entire system’s reliability by reducing its response speed and real-time decision-making. Consequently, there is an imperative demand for a lightweight approach to tackle the challenges arising from the escalating scale of digital twin virtual entity models. This paper presents a digital twin methodology that is lightweight. The procedure comprises three primary phases: graph data modeling, graph neural network analysis, and hierarchical simplification of virtual entities. Specifically, the graph neural network method proposed in this article is used to classify the graph data of virtual entities. Then, the model is hierarchically simplified based on the classification. Finally, experiments were conducted on factory and robotic arm datasets to evaluate the proposed method. The experimental results indicate that the DTL-GNN method can reduce system redundancy while preserving the essential features of virtual entities.

Nonlinear compensation of stretchable strain sensors with application to proprioceptive sensing of soft robotic arm

Abstract With advances in materials and manufacturing techniques, recent years have seen a number of conductive composite materials that exhibit pronounced strain-dependent electrical resistivity, allowing them to be used for embedded, cost-effective strain sensing in various applications. The strain-resistivity relationship of these materials, however, is often highly nonlinear and dynamic, posing challenges for effective use of such strain sensors. In this paper, a computationally efficient scheme is proposed for compensating the nonlinear, dynamic strain-resistance behavior of a soft conductive rubber using a Time Delay Neural Network (TDNN). The accuracy and feasibility of the technique is evaluated with a soft robotic arm incorporating three strain sensors for proprioception. Experimental results show that the sensing scheme is able to predict both the tip position and the shape of the robotic manipulator, achieving an average tip positional error of less than 4% relative to the total length of the manipulator.

Artificial Intelligence in Robotic Manipulators: Exploring Object Detection and Grasping Innovations

The importance of deep learning has heralded transforming changes across different technological domains, not least in the enhancement of robotic arm functionalities of object detection’s and grasping. This paper is aimed to review recent and past studies to give a comprehensive insight to focus in exploring cutting-edge deep learning methodologies to surmount the persistent challenges of object detection and precise manipulation by robotic arms. By integrating the iterations of the You Only Look Once (YOLO) algorithm with deep learning models, our study not only advances the innovations in robotic perception but also significantly improves the accuracy of robotic grasping in dynamic environments. Through a comprehensive exploration of various deep learning techniques, we introduce many approaches that enable robotic arms to identify and grasp objects with unprecedented precision, thereby bridging a critical gap in robotic automation. Our findings demonstrate a marked enhancement in the robotic arm’s ability to adapt to and interact with its surroundings, opening new avenues for automation in industrial, medical, and domestic applications. The impact of this research extends lays the groundwork for future developments in robotic autonomy, offering insights into the integration of deep learning algorithms with robotic systems. This also serves as a beacon for future research aimed at fully unleashing the potential of robots as autonomous agents in complex, real-world settings.

Design, Development and Optimization of a Robotic Weed Control System for Greenhouses

This study presents the design and development of a robotic prototype specifically engineered for weed elimination within cucumber plants growing in greenhouse conditions. Given the limited research in automated weed control tailored to cucumber cultivation, this project fills a crucial gap in agricultural robotics. The cucumber plants are arranged in a single row, maintaining a spacing of approximately 40 cm between each plant, while the rows themselves are positioned about 1 meter apart. The greenhouse environment features sandy loam soil, which informs the robot's design and operational parameters. The robot employs three arrays of ultrasonic sensors strategically positioned to enhance weed detection capabilities. A PIC18 F4550-E/P microcontroller serves as the processing unit, enabling real-time feedback from the sensors that triggers the actuation of a robotic arm. This arm is engineered to maneuver between plant rows, utilizing rotating blades for efficient weed cutting. A comprehensive experimental procedure was conducted, comprising 54 trials within the greenhouse setting, to optimize the performance of the robotic arm. Key variables investigated included arm motor (AM) speed, blade rotation (BR) speed, and blade design. The research tested three BR speeds—3500, 2500, and 1500 RPM—determined based on prior literature concerning rotary movers. Furthermore, two AM speeds (10 and 30 RPM) paired with three distinct blade designs—S-shaped, triangular, and circular—were evaluated primarily for their cost-efficiency and performance based on initial testing outcomes. Results revealed a significant correlation between motor speed and weed cutting efficiency, showing that higher motor speeds correlated with a decreased percentage of cut weeds. Consequently, the 10 RPM motor speed was identified as the optimal setting for arm movement, balancing effective weed removal with operational efficiency.

Softer, Safer, Smarter: Robotic Hands Inspired by Nature

Robot arms can be found in many factories around the world. From the tiniest chip in a phone to the body of a car, robots are responsible for moving things around thousands of times a day. However, robots find themselves a bit stuck when dealing with more natural objects, such as fruits or vegetables. Their rigid bodies feel out of place amongst all the squishiness and different shapes and sizes of nature. One way to help solve that is by making robots softer. Soft grippers allow robots to delicately touch the world around them, grabbing fragile objects almost as easily as humans do. Many robots made of soft materials, called soft robots, are inspired by nature to help them interact with their environments. With hands made of softer materials, these robots are ready to help us not just in factories, but also in our daily lives. How are soft grippers created? What do they look like? In this article, we will explore the captivating realm of soft grippers!

Adaptive Grasp Pose Optimization for Robotic Arms Using Low-Cost Depth Sensors in Complex Environments

This paper presents an efficient grasp pose estimation algorithm for robotic arm systems with a two-finger parallel gripper and a consumer-grade depth camera. Unlike traditional deep learning methods, which suffer from high data dependency and inefficiency with low-precision point clouds, the proposed approach uses ellipsoidal modeling to overcome these issues. The algorithm segments the target and then applies a three-stage optimization to refine the grasping path. Initial estimation fits an ellipsoid to determine principal axes, followed by nonlinear optimization for a six-degree-of-freedom grasp pose. Validation through simulations and experiments showed a target grasp success rate (TGSR) of over 83% under low noise, with only a 4.9% drop under high noise—representing a 68.0% and a 42.4% improvement over GPD and PointNetGPD, respectively. In real-world tests, success rates ranged from 95 to 100%, and the computational efficiency was improved by 56.3% compared to deep learning methods, proving its practicality for real-time applications. These results demonstrate stable and reliable grasping performance, even in noisy environments and with low-cost sensors.

Predictive Maintenance Algorithm of a 6DOF Robotic Arm using Gradient Boosting Regression

The objective of this study is to develop a predictive maintenance algorithm for the ABB IRB 4600, a 6-axis robotic arm, using digital simulations. A variety of tests were conducted using SolidWorks, including calculations pertaining to stress, strain, fatigue, and heat. The simulations included an analysis of the materials used in the construction of the robotic arm, which are gray cast iron and aluminum alloy. The robotic arm was tested in three positions—picking, raised, and placing—with loads of 100 kg, 200 kg, and 300 kg, respectively. The findings indicated that elevated stress, strain, and displacement levels diminish the robot's operational lifespan and accelerate its deterioration over time. The placing position was found to experience the greatest stress, displacement, and strain. The fatigue test also demonstrated that after 10 million cycles, the arm had accumulated damage. The gradient boosting regression algorithm was selected as the Machine Learning (ML) algorithm for the study following a comparison of the performance of various ML regression models. This finding underscores the significance of predictive maintenance in preventing breakdowns and extending the robot's lifespan.

Artificial intelligent based control strategy for reach and grasp of multi-objects using brain-controlled robotic arm system

ABSTRACT Brain-controlled robotic arm systems are designed to provide a method of communication and control for individuals with limited mobility or communication abilities. These systems can be beneficial for people who have suffered from a spinal cord injury, stroke, or neurological disease that affects their motor abilities. The ability of a person to control a robotic arm to reach and grasp multiple objects using their brain signals. This technology involves the use of an electroencephalogram (EEG) cap that captures the electrical activity in the user’s brain, which is then processed by an artificial intelligent to translate it into commands that control the movements of the robotic arm. With this technology, individuals who are unable to move their limbs due to paralysis or other conditions can still perform daily activities such as feeding themselves, drinking from a glass, or grasping objects. In this paper, we propose an artificial intelligent–based control strategy for reach and grasp of multi-objects using brain-controlled robotic arm system. The proposed control strategy consists of threefold process: feature extraction, feature optimization, and control strategy classification. Initially, we design an improved ResNet pre-trained architecture for deep feature extraction from the given EEG signal.

AxiWorm: a new tool using YOLOv5 to test antiparasitic drugs against Trichinella spiralis

Background-ObjectiveTrichinella spiralis drug development and control need an objective high throughput system to assess first stage larvae (L1) viability. YOLOv5 is an image recognition tool easily trained to count muscular first stage larvae (L1) and recognize morphological differences. Here we developed a semi-automated system based on YOLOv5 to capture photographs of 96 well microplates and use them for L1 count and morphological damage evaluation after experimental drug treatments.Material and methodsMorphological properties were used to distinguish L1 from debris after pepsin muscle digestion and distinguish healthy (serpentine) or damaged (coiled) L1s after 72 h untreated or treated with albendazole or mebendazole cultures. An AxiDraw robotic arm with a smartphone was used to scan 96 well microplates and store photographs. Images of L1 were manually annotated, and augmented based on exposure, bounding, blur, noise, and mosaicism.ResultsA total of 1309 photographs were obtained that after L1 labeling and data augmentation gave 27478 images. The final dataset of 12571 healthy and 14907 affected L1s was used for training, testing, and validating in a ratio of 70/20/10 respectively. A correlation of 92% was found in a blinded comparison with bare-eye assessment by experienced technicians.ConclusionYOLOv5 is capable of accurately counting and distinguishing between healthy and affected L1s, thus improving the performance of the assessment of meat inspection and potential new drugs.Graphical

Haptic Glove Guided Robotic Arm

The integration of haptic glove technology with robotic arms provides a leap in the field of teleoperation, offering users a precise interaction with robotic systems. This technology combines feedback mechanisms with a wearable glove interface, allowing users to manipulate objects in remote or hazardous environments through a robotic arm. The haptic glove is embedded with advanced sensors and actuators that capture finger movements, which are then processed in real-time to control the corresponding actions of the robotic arm. Simultaneously, haptic feedback provides a sense of pressure, enabling operators to perform tasks with increased control. The advancements in technology have expanded the applications of haptic glove-controlled robotic arms across various fields.

Feasible Techniques Named "Pure" Robotic Simple Hysterectomy With 4 Robotic Arms "4+0" Mode for Hysterectomy in da Vinci Xi.

The three-arm approach is mainly selected, despite the multiple robotic arms in da Vinci Xi. This type of surgical setup may provide less autonomy to the console surgeon and result in greater dependence on the bedside surgical assistant. Therefore, the 4th arm is used instead of the assist port, which is why we developed "pure" robot simple hysterectomy (PRSH) as a novel surgical technique, in which all ports are operated by robotic arms. After pneumoperitoneum was established, trocars were inserted under visual control: three 8 mm robotic ports on the same horizontal line spaced 8 cm apart at the level of the endoscope port. The 2nd arm was used to insert the endoscope, and the fenestrated bipolar forceps in the 1st arm and Maryland bipolar forceps in the 3rd arm were operated using the double bipolar method. In this technique, the uterine manipulator is not used because the Cadiere forceps in the 4th arm manipulate the uterus. For suturing, the 3rd arm was equipped with a SutureCut needle driver from Maryland bipolar forceps, which enabled suturing and thread cutting. Suction and intra-abdominal transport of the needle was introduced into the abdominal cavity by pulling out the instrument in the 3rd arm. Hence, since all robotic arms are used for all ports, we named this technique "pure" robot simple hysterectomy. The routine use of a fourth robotic arm "4+0" mode during PRSH provides the operating surgeon with greater independence during critical phases of the procedure without requiring a uterine manipulator and assistant. 5043-03.

Dynamic Navigation-Guided Robotic Placement of Zygomatic Implants

Objectives: To assess the feasibility and accuracy of a new prototype robotic implant system for the placement of zygomatic implants in edentulous maxillary models. Methods: The study was carried out on eight plastic models. Cone beam computed tomographs were captured for each model to plan the positions of zygomatic implants. The hand-eye calibration technique was used to register the dynamic navigation system to the robotic spaces. A total of 16 zygomatic implants were placed, equally distributed between the anterior and the posterior parts of the zygoma. The placement of the implants (ZYGAN®, Southern Implants) was carried out using an active six-jointed robotic arm (UR3e, Universal Robots) guided by the dynamic navigation coordinate transformation matrix. The accuracy of the implant placement was assessed using EvaluNav and GeoMagicDesignX® software based on pre- and post-operative CBCT superimposition. Descriptive statistics for the implant deviations and Pearson's correlation analysis of these deviations to force feedback recorded by the robotic arm were conducted. Results: The 3D deviations at the entry and exit points were 1.80 ± 0.96 mm and 2.80 ± 0.95 mm, respectively. The angular deviation was 1.74 ± 0.92°. The overall registration time was 23.8 ± 7.0 minutes for each side of the model. Operative time excluding registration was 66.8 ± 8.8 minutes for each trajectory.The exit point and angular deviations of the implants were positively correlated with the drilling force perpendicular to the long axis of the handpiece and negatively correlated with the drilling force parallel to the long axis of the handpiece. Conclusion: The errors of the dynamic navigation-guided robotic placement of zygomatic implants were within the clinically acceptable limits. Further refinements are required to facilitate the clinical application of the tested integrated robotic-dynamic navigation system. Clinical Significance: Robotic placement of zygomatic implants has the potential to produce a highly predictable outcome irrespective of the operator's surgical experience or fatigue. The presented study paves the way for clinical applications.

DRL-enhanced 3D detection of occluded stems for robotic grape harvesting

Detecting grape stems is essential for the autonomous operation of grape-picking robots. In natural orchards, the complex and irregular positioning of grape bunches, along with frequent occlusions caused by obstacles and leaves, create a dynamic and unpredictable environment that substantially affects the robot’s perception quality and harvesting efficiency. Inspired by human active observation mechanisms, this study introduces an end-to-end active visual perception framework using deep reinforcement learning (DRL) to enhance the detection of grape stems under complex occlusion. An instance segmentation network is trained to obtain 2D masks, which are integrated into an octree volumetric grid to produce detailed volumetric observations for DRL training. Different occupancy weights are assigned to both explored and newly discovered regions, contributing to a novel reward function based on information gain, which subsequently drives the network to optimize camera movements, guiding the robotic arm towards the best viewpoint for efficient grape stem detection. The superior performance of the proposed method has been validated in both laboratory and outdoor orchards settings. The primary contribution of this work lies in presenting a novel fully end-to-end detection framework for occluded grape stems. Compared to existing methods, it enables the system to directly learn optimal viewpoint strategies through interactions between the robotic arm and the environment, effectively handling feature extraction and environmental modeling without the need for manually designed information gain metrics. This advancement provides foundational support for the next generation of highly autonomous picking robots, capable of operating adaptively in complex, unstructured environments.