Pruning Robot Research Articles

Enhancing Grapevine Node Detection to Support Pruning Automation: Leveraging State-of-the-Art YOLO Detection Models for 2D Image Analysis

Automating pruning tasks entails overcoming several challenges, encompassing not only robotic manipulation but also environment perception and detection. To achieve efficient pruning, robotic systems must accurately identify the correct cutting points. A possible method to define these points is to choose the cutting location based on the number of nodes present on the targeted cane. For this purpose, in grapevine pruning, it is required to correctly identify the nodes present on the primary canes of the grapevines. In this paper, a novel method of node detection in grapevines is proposed with four distinct state-of-the-art versions of the YOLO detection model: YOLOv7, YOLOv8, YOLOv9 and YOLOv10. These models were trained on a public dataset with images containing artificial backgrounds and afterwards validated on different cultivars of grapevines from two distinct Portuguese viticulture regions with cluttered backgrounds. This allowed us to evaluate the robustness of the algorithms on the detection of nodes in diverse environments, compare the performance of the YOLO models used, as well as create a publicly available dataset of grapevines obtained in Portuguese vineyards for node detection. Overall, all used models were capable of achieving correct node detection in images of grapevines from the three distinct datasets. Considering the trade-off between accuracy and inference speed, the YOLOv7 model demonstrated to be the most robust in detecting nodes in 2D images of grapevines, achieving F1-Score values between 70% and 86.5% with inference times of around 89 ms for an input size of 1280 × 1280 px. Considering these results, this work contributes with an efficient approach for real-time node detection for further implementation on an autonomous robotic pruning system.

Optimized Design of Robotic Arm for Tomato Branch Pruning in Greenhouses

Aiming at the robotic pruning of tomatoes in greenhouses, a new PRRPR configuration robotic arm consisting of two prismatic (P) joints and three revolute (R) joints was designed to locate the end effector to handle randomly growing branches with an appropriate posture. In view of the various spatial posture of the branches, drawing on the skill of manual pruning operation, we propose a description method of the optimal operation posture of the pruning end effector, proposing a method of solving the inverse kinematics of the pruning arm based on the multi-objective optimization algorithm. According to the spatial distribution characteristics of the tomato branches along the main stem, the robotic arm structure is compact and the reachable space is maximized as the objective function, and a method of optimizing the key geometric parameters of the robotic arm is proposed. The optimal maximum length of the arm’s horizontal slide joint was determined to be 953.149 mm and the extension maximum length of its telescopic joint was 632.320 mm. The verification test of the optimal structural parameter showed that the optimized robotic arm could reach more than 89.94% of the branches in the pruning target area with a posture that meets the pruning requirements. This study is supposed to provide technical support for the development of a tomato pruning robot.

Tree-PointNet: A Novel Neural Network for Dormant Apple Trees Using Semi-Circle-Based Point Cloud for Robotic Pruning

HighlightsTree-PointNet, a novel neural network, integrates geometric down-sampling for efficient pruning.A Semi-Circle-Based point cloud dataset of apple trees was collected to aid in automated pruning.Enhanced algorithm performance was investigated, showing significant improvements in model accuracy. Abstract. Pruning during the dormant season is one of the most costly and labor-intensive operations in the production of specialty crops. In winter, many trained seasonal workers have to carefully remove branches from hundreds of trees according to pre-defined rules. The aim of automated pruning is to reduce this reliance on the large amount of labor currently required for the operation. The first step in the automatic application of pruning rules is the acquisition of point cloud data from trees. To match the optical vision system of an ideal automatic pruning robot, a brand new and challenging apple tree dataset was collected in which the point clouds were all based on semicircular shapes, rather than the carefully crafted point clouds of complete apple trees. 1000 Semi-Circle-Based point clouds of apple trees were used. For the brand-new structure of point cloud part segmentation, a neural network model (namely Tree-PointNet) was proposed. A Semi-Circle-Based point cloud of apple tree was segmented by the Tree-PointNet with the segmentation result consisting of a trunk and primary branches. With a weight size of only 7.9% of PointStack, the best OA of 95.6% was obtained by the Tree-PointNet network proposed in this paper. The GDS was also used, which could obtain more key points in the Branch-to-Trunk connection region as input to the proposed network. On the other hand, the OA and mIoU of the model improved by 10.2% and 7.98%, respectively, compared to the PointNet++. Keywords: Dormant apple trees, Part segmentation, PointNet++, Semi-circle-based point cloud.

Development of an automatic control system for a hydraulic pruning robot

Automatic pruning is capable of trimming trees into complex shapes efficiently and continuously. However, this technology faces more challenges under unstructured agricultural environments. In the present paper, we describe the development of a hydraulic disc-saw mounted pruning robot with a relatively low cost for automated geometric pruning of fruit trees based on a new automatic pruning scenario. A pruning control strategy based on rolling observations and continuous prediction algorithm was designed to precisely control the motion of hydraulic cylinders. A forward kinematic model was described using modified Denavit–Hartenberg (D-H) notation, and the accessible workspace for the end-effector was analyzed in the simulation. The inverse kinematics model based on geometry was established, which provided the foundation for the development of a control system together with the forward kinematics model. Performance tests showed a sufficient control accuracy within ± 1.8° of all cylinders. The results of the waypoint tracking tests suggest that an increase in the step-size from 20 mm to 60 mm not only increases the moving speed and improves tracking accuracy, but also increases the fluctuation of the end-effector in the Z-axis direction. The movement of the chassis can introduce new errors and results in a larger average absolute positional error along the X-axis. The average startup and stop lag of the control system was 0.55 s and 0.15 s, respectively. This study provides guidance for pruning robot path planning and lays the foundation for future iterations of autonomous pruning robots.

Recent work on robotic pruning of upright fruiting offshoot cherry systems

Recent work on robotic pruning of upright fruiting offshoot cherry systems

Bush spherical center detection algorithm based on depth camera 3D point cloud

Automated pruning is an inevitable trend in the improvement of modern gardens. In order to provide necessary information for automatic garden robots and satisfy the requirement of target detection and positioning during pruning, this paper proposed a bush spherical center detection algorithm based on a 3D depth camera point cloud. Firstly, the depth camera collected the bush image, and the results were aligned to the depth image to obtain the 3D point cloud of bush. Then the ROI was extracted by preprocessing, and the 3D point clouds of bush was obtained after filtering and coordinate transformation. Finally, the spherical center coordinates of the bush were extracted by the minimum bounding box method. Four groups of tests on the bush spherical coordinates detection were carried out outdoors. The maximum location error and the minimum location error of the spherical bush center were 10.23mm and 8.65 mm, respectively, and the average location error was 9.51mm. The bush spherical center detection algorithm based on depth camera 3D point clouds proposed in this paper provides a technical reference for the information acquisition of automatic pruning robot.

Design and test of a tree-climbing and pruning robot

In view of the problems of manual pruning, such as time-consuming, labor-intensive, low efficiency and low degree of automation, the structure and working principle of the robot for tree-climbing and pruning were studied. Finding the problems in the process of product design, analyzing and defining them, transforming them into the standard mode of solving problems, and using TRIZ theory according to the essence of contradictions to solve the contradictions one by one. The virtual prototype was created to carry out the interference analysis and kinematics simulation analysis of the whole machine. The robot for tree-climbing and pruning was designed by modular design method, which can automatically climb to a certain height to realize the function of pruning and cutting the top of the tree trunk. The duty cycle of the motor output is gradually adjusted by PID algorithm to make multiple motors work synchronously and keep the robot stable in the up-tree pruning process and down-tree process. The robot was made and tested. The experimental results show that the diameter of climbing trunk is 100 mm∼300 mm, and the climbing speed is 1 m/min. It can be used for tree-climbing, pruning lateral branches and cutting main trunks of high-straight trunk trees. It can be self-adaptively held, fall-resistant, obstacle-crossing, and manual operate to improve efficiency.

Branch Identification and Junction Points Location for Apple Trees Based on Deep Learning

Branch identification is key to the robotic pruning system for apple trees. High identification accuracy and the positioning of junction points between branch and trunk are important prerequisites for pruning with a robotic arm. Recently, with the development of deep learning, Transformer has been gradually applied to the field of computer vision and achieved good results. However, the effect of branch identification based on Transformer has not been verified so far. Taking Swin-T and Resnet50 as a backbone, this study detected and segmented the trunk, primary branch and support of apple trees on the basis of Mask R-CNN and Cascade Mask R-CNN. The results show that, when Intersection over Union (IoU) is 0.5, the bbox mAP and segm mAP of Cascade Mask R-CNN Swin-T are the highest, which are 0.943 and 0.940; as for the each category identification, Cascade Mask R-CNN Swin-T shows no significant difference with the other three algorithms in trunk and primary branch; when the identified object is a support, the bbox AP and segm AP of Cascade Mask R-CNN Swin-T is significantly higher than that of other algorithms, which are 0.879 and 0.893. Next, Cascade Mask R-CNN SW-T is combined with Zhang & Suen to obtain the junction point. Compared with the direct application of Zhang & Suen algorithm, the skeleton obtained by this method is advantaged by trunk diameter information, and its shape and junction points position are closer to the actual apple trees. This model and method can be applied to follow-up research and offer a new solution to the robotic pruning system for apple trees.

Kinematic-Based Multi-Objective Design Optimization of a Grapevine Pruning Robotic Manipulator

Annual cane pruning of grape vineyards is a time-consuming and labor-intensive job, but no mechanized or automatic way has been developed to do it yet. Robotic pruning can be a perfect alternative to human labor. This article proposes a systematic seven-stage procedure to design a kinematically optimized manipulator, named ‘Prubot’, to manage vineyards’ cane pruning. The manipulator structure was chosen, resulting in a 7R (Revolute) manipulator with a spherical shoulder and wrist. To obtain the design constraints, the manipulator task space was modeled. The robot’s second and third link lengths were determined by optimizing the global translational version of the measure of manipulability and the measure of isotropy of the manipulator arm section. Finally, simulations confirmed the appropriateness of the manipulator workspace. Furthermore, sampling-based path planning simulations were carried out to evaluate the manipulator’s kinematic performance. Results illustrated the impressive kinematic performance of the robot in terms of path planning success rate (≅100%). The simulations also suggest that among the eight single-query sampling-based path planning algorithms used in the simulations, Lazy RRT and KPIECE are the best (≤5 s & ~100%) and worst ≥5 s &≤25% path planning algorithms for such a robot in terms of computation time and success rate, respectively. The procedure proposed in this paper offers a foundation for the kinematic and task-based design of a cane pruning manipulator. It could be promisingly used for designing similar agricultural manipulators.

Design of Control System for a new Intelligent Tree-climbing and Pruning Robot

This paper designed the control system of a new intelligent tree-climbing and pruning robot which included the control of remote control mobile terminal and intelligent control terminal. Remote control module, power circuit, motor drive, automatic holding detection, height detection, side branch detection and other modules were designed, Then the main program based on STM32 embedded MCU, interrupt program and communication program were designed. The control system could enable the tree climbing robot to climb automatically, detect the side branches in the climbing process, trim the side branches automatically, and saw the top branch after climbing to a certain height. The robot also had the function of image monitoring. Through image monitoring, the working process of the robot could be adjusted remotely. Finally, the prototype was tested. The test results showed that the selection of controller and sensors were appropriate, the design of remote control circuit, motor driving circuit and detection circuit was reasonable, and the structure and algorithm of the program were reasonable. The development of the prototype laid a foundation for the popularization of fast-growing forest pruning robot.

Semantics-guided skeletonization of upright fruiting offshoot trees for robotic pruning

Dormant pruning for fresh market fruit trees is a relatively unexplored application of agricultural robotics for which few end-to-end systems exist. One of the biggest challenges in creating an autonomous pruning system is the need to reconstruct a model of a tree which is accurate and informative enough to be useful for deciding where to cut. One useful structure for modeling a tree is a skeleton: a 1D, lightweight representation of the geometry and the topology of a tree. This skeletonization problem is an important one within the field of computer graphics, and a number of algorithms have been specifically developed for the task of modeling trees. These skeletonization algorithms have largely addressed the problem as a geometric one. In agricultural contexts, however, the parts of the tree have distinct labels, such as the trunk, supporting branches, etc. This labeled structure is important for understanding where to prune. We introduce an algorithm which uses topological and geometric priors about Upright Fruiting Offshoot (UFO) trees to produce such a labeled skeleton. We test our skeletonization algorithm on point clouds from 29 UFO trees and, using an accuracy metric based on the number of correctly reconstructed and labeled skeleton segments, demonstrate a median construction accuracy of 70% with respect to a human-evaluated gold standard. We also make point cloud scans of 82 UFO trees open-source to other researchers. Our work represents a significant first step towards a robust tree modeling framework which can be used in an autonomous pruning system.

Automatic branch detection of jujube trees based on 3D reconstruction for dormant pruning using the deep learning-based method

Pruning is a time-consuming and labor-intensive practice for managing of dormant jujube orchards, in which dormant pruning is still mainly dependent on manual operation. Automated pruning using a robotic platform could be a better solution to overcome the skilled labor shortage and increased labor costs. With the development of dwarf and dense planting of jujube trees, the tree architectures are suitable for robotic pruning. This study concentrated on the detection and identification of pruning branches, which was a critical step for automating pruning operations. The method in this study mainly consisted of three steps: first, present a vision system based on two synchronous consumer-level RGB-D cameras to obtain the high-quality 3D point cloud in filed; second, propose the reconstruction pipeline for the desired three-dimensional model (basically a complete 3D model) using only two perspectives; and third, automatically segment the trunks and branches based on deep learning method (SPGNet), and then apply the DBSCAN clustering algorithm for estimating branch counts of jujube trees. For the reconstruction of jujube tree, registration errors II (0°-180°) were greater than errors I (0°-55°), and registration errors I (0°-55°) were less than 1 mm under different light conditions. Experimental results demonstrated that trunks and branches were segmented successfully with class accuracies of 0.93 and 0.84, and another metric intersection-over-union (IoU) was 0.85 and 0.76, respectively; the coefficient of determination (R2) values analyzed between the ground-truth and cluster results were 0.83, 0.88, and 0.89 in sunny, cloudy, and night conditions, respectively. These results showed that the proposed method for detecting branches could be utilized to generate substantial information, such as the diameter length and diameter of the branch, which was critical for the automated dormant pruning of jujube trees in the future orchard field.

Kinematics Analysis and Trajectory Planning of Dual-arm Pruning Robot

In order to overcome the disadvantages of manual pruning of tree branches, the dual-arm pruning robot is proposed in this paper. The structure of the dual-arm pruning robot is designed, and the three-dimensional model is established. Taking the dual-arm of the pruning robot as the study object, the D-H method is used to build the kinematics model so as to better research the process of the two arms cooperative pruning. The kinematics simulation is done by Matlab to verify the rationality of the D-H parameters. The workspace of the dual-arm is analyzed, as well as the trajectory planning in joint space. The torques of each joint are solved to determine the power of the driving motor. The research results provide the theoretical basis for the final prototype manufacturing.

Investigation of Branch Accessibility with a Robotic Pruner for Pruning Apple Trees

HighlightsA branch accessibility simulation was performed for robotic pruning of apple trees.A virtual tree environment was established using a kinematic manipulator model and an obstacle model.Rapidly-exploring random tree (RRT) was combined with smoothing and optimization for improved path planning.Effects on RRT path planning of the approach angle of the end-effector and cutter orientation at the target were studied.Abstract. Robotic pruning is a potential solution to reduce orchard labor and associated costs. Collision-free path planning of the manipulator is essential for successful robotic pruning. This simulation study investigated the collision-free branch accessibility of a six rotational (6R) degrees of freedom (DoF) robotic manipulator with a shear cutter end-effector. A virtual environment with a simplified tall spindle tree canopy was established in MATLAB. An obstacle-avoidance algorithm, rapidly-exploring random tree (RRT), was implemented for establishing collision-free paths to reach the target pruning points. In addition, path smoothing and optimization algorithms were used to reduce the path length and calculate the optimized path. Two series of simulations were conducted: (1) performance and comparison of the RRT algorithm with and without smoothing and optimization, and (2) performance of collision-free path planning considering different approach poses of the end-effector relative to the target branch. The simulations showed that the RRT algorithm successfully avoided obstacles and allowed the manipulator to reach the target point with 23 s average path finding time. The RRT path length was reduced by about 28% with smoothing and by 25% with optimization. The RRT smoothing algorithm generated the shortest path lengths but required about 1 to 3 s of additional computation time. The lowest coefficient of variation and standard deviation values were found for the optimization method, which confirmed the repeatability of the method. Considering the different end-effector approach poses, the simulations suggested that successfully finding a collision-free path was possible for branches with no existing path using the ideal (perpendicular cutter) approach pose. This study provides a foundation for future work on the development of robotic pruning systems. Keywords: Agricultural robotics, Collision-free path, Manipulator, Path planning, Robotic pruning, Virtual tree environment.

Development of an integrated 3R end-effector with a cartesian manipulator for pruning apple trees

Robotic pruning is a potential solution to address the issues of labor shortages and high associated costs, but it has challenges due to the unstructured working environment. For successful robotic pruning, target branches have to be reached with fewer spatial requirements for the end-effector cutter and the manipulator. A three-rotational (3R) degrees of freedom (DoF) end-effector was designed considering maneuvering, spatial, mechanical, and horticultural requirements. Simulations were conducted with the end-effector to investigate the reachable workspace, the cutter frame orientation, and the manipulability index. The simulation results suggested that the proposed design has a spherical reachable workspace with a void due to the presence of a physical constraint of the linear arm. The manipulability index was determined to be independent of the rotation of the first and last joint of the end-effector. The prototype of the proposed end-effector was integrated with a cartesian manipulator. An Arduino-based control system was developed along utilizing a Matlab graphical user interface (GUI). A series of field tests were conducted on ‘Fuji’/Bud. 9 apple trees with trellis-trained architecture. The field tests validated the simulation results, and the end-effector successfully cut branches up to ~25 mm diameter at wide range of orientations. This study provides the foundation for future investigations of branch accessibility for pruning with an integrated 3R end-effector and a cartesian manipulator system following a collision free trajectory.

Development of a Robotic End-Effector for Apple Tree Pruning

HighlightsAn end-effector with two degrees of freedom (2R) was developed for pruning apple trees.A rational 2×2 relationship (R2 = 0.93) was found for ‘Fuji’ apple tree branch diameter and cutting force.Simulation showed that the cutter can be aligned in a wide range of orientations in a spherical workspace.The developed end-effector was able to cut branches up to 12 mm in diameter.Abstract. Robotics and automation technologies are now used extensively in agriculture, while production operations for tree fruit crops still largely depend on manual labor. Manual pruning is a labor-intensive and costly task in apple production. Robotic pruning is a potential solution, but it involves several challenges due to the unstructured work environment. This study focused on designing an end-effector prototype for pruning considering the maneuvering, spatial, mechanical, and horticultural requirements. Branch cutting force was measured with a thin force sensor to provide guidelines for the end-effector design. The test results indicated the relationship between the force required to cut different diameter branches with an R2 value of 0.93. The end-effector was developed using two rotary motors, a pneumatic cylinder, and a pair of bypass shear blades. A three-directional linear manipulator system and a control system were built for moving the end-effector to targeted locations. A mathematical model was developed for simulation of the workspace utilization and reachable points of the end-effector. The simulation results indicated that the end-effector can be aligned in a wide range of orientations of the cutter. Field tests were conducted for validation of the simulation results and performance assessment of the end-effector. The results indicated that the end-effector with the current parameter settings successfully cut branches up to 12 mm in diameter and was able to cut branches in a wide range of possible orientations in a given 3D space. The robotic end-effector developed in this study is a core component of an automated pruning system for fruit trees. In future work, an integrated manipulator system will be developed for branch accessibility with collision-free trajectories. Keywords: Malus × domestica (Borkh.), Pruning end-effector, Reachable-points simulation, Tree pruning.

Research on a method of fruit tree pruning based on BP neural network

The mainstream pruning robots do not have the ability to make decisions independently. The pruning schemes are all artificially generated by experts according to the collected images. In order to improve the intelligence of pruning robot and reduce the labor cost of pruning work, it is necessary to study the robot pruning decision algorithm corresponding to different fruit tree varieties.In this paper, taking apples in the early fruit period as an example, referring to the technical principle of traditional fruit tree pruning and aiming at two types of interference in the pruning process, the back branches and interfering branches, a pruning decision algorithm based on BP neural network was proposed. The algorithm formed the training set by artificially collecting the accurate data of the spatial characteristics of the fruit tree branches and performed calibration for pruning type, and the neural network model was trained according to the calibrated data set. The model trained in the first stage showed the situation that the competition branches cannot be identified. Based on this, an improved algorithm was proposed to improve the classification performance of the competition branches. The experimental results verified that the F1 score of the method for the back branches was 0913; the F1 score for the centripetal branches was 0.867; the improved algorithm has an F1score of 0.755 for the competition branches; the overall conformed to the expectation, which could provide algorithm support for the pruning robot to make artificial intelligence decision.

Research on 3D skeletal model extraction algorithm of branch based on SR4000

A branch 3D Skeleton extraction method based on SR4000 is proposed, which can be used for pruning robot visual recognition. In this method a 3D Skeleton model of the branches of apple trees in the initial fruit stage was successfully constructed. According to the experimental results, the average computational efficiency of the 2AHC(2-level Aggregation Hierarchy Clustering) is increased by 369%, the detection rate is 84.69%, and the error rate is 5.03%. The average detection rate of depth analytic hierarchy process was 64%; The overall 3D Skeleton restoration effect is better. Therefore, the algorithm can better reflect the qualitative relationship between branches, improve the computational efficiency, and provide algorithm support for automatic pruning robot visual recognition.

Optimization of Chainsaw Setting Angle in Pruning Robot

This paper presents an optimal mechanical design for a chainsaw setting angle to reduce the driving force of wheels in a pruning robot that cuts branches while spiraling up. Analysis and experimental results demonstrate the effectiveness of the optimal mechanical design for the chainsaw setting angle.

Stub Length and Stub Angle Did Not Influence Renewal Shoot Number or Branch Angle of Tall Spindle ‘Gala’/Malling 9 Apple Trees

Renewal of limbs by pruning to leave a short, angled, upward-facing stub is common practice for spindle-type apple (Malus ×domestica) training systems. A short, beveled stub cut is thought to stimulate renewal growth from latent buds present underneath the base of the excised branch, and to stimulate smaller, more fruitful renewal limbs with wide crotch angles. We conducted trials over the course of 2 years that involved dormant pruning of ‘Buckeye Gala’ with renewal cuts to compare two stub lengths, 0.5 and 2 cm, and three stub orientations, upward facing, downward facing, and vertical facing, to determine the effects on renewal shoot number, position, angle, and length. We found no clear advantages with either stub length that we evaluated, and there was no improvement in renewal shoot quality with a bevel cut at any orientation. Stub length and stub angle did not influence limb renewal and may be unimportant for training orchard-pruning crews and for machine-learning and robotic pruning.