Harvesting Success Research Articles

Design and Experiment of Nighttime Greenhouse Tomato Harvesting Robot

In response to the issue of high tomato yield, low efficiency in harvesting tomatoes grown in greenhouses, and low recognition accuracy of nighttime harvesting robots, a design was developed and a robotic system was created specifically for nighttime greenhouse tomato harvesting. The robot employs a vision system and YOLOv5+HSV fusion algorithm to recognize and locate tomatoes. It then transmits this information to the robotic arm. By coordinating the visual system, the robotic arm, the end effector, and the lifting mechanism, the robot accurately picks ripe tomatoes. The robot was subjected to simulated field tests for visual recognition and harvesting, both during daytime and nighttime conditions. The results showed that the success rate of nighttime harvesting was slightly lower than during the daytime but remained at a relatively high level. The daytime harvesting success rate and the average time to pick a single fruit were 87.78% and 15.99 seconds, respectively. The nighttime harvesting success rate and the average time to pick a single fruit were 87.55% and 17.26 seconds, respectively. This approach effectively improves the recognition accuracy and harvesting speed of the harvesting robot, reducing damage to tomatoes during harvesting, and addresses the issues of supplementary lighting and image noise reduction for nighttime harvesting robots.

A selective harvesting robot for cherry tomatoes: Design, development, field evaluation analysis

AbstractWith the aging population and increasing labor costs, traditional manual harvesting methods have become less economically efficient. Consequently, research into fully automated harvesting using selective harvesting robots for cherry tomatoes has become a hot topic. However, most of the current research is focused on individual harvesting of large tomatoes, and there is less research on the development of complete systems for harvesting cherry tomatoes in clusters. The purpose of this study is to develop a harvesting robot system capable of picking tomato clusters by cutting their fruit‐bearing pedicels and to evaluate the robot prototype in real greenhouse environments. First, to enhance the grasping stability, a novel end‐effector was designed. This end‐effector utilizes a cam mechanism to achieve asynchronous actions of cutting and grasping with only one power source. Subsequently, a visual perception system was developed to locate the cutting points of the pedicels. This system is divided into two parts: rough positioning of the fruits in the far‐range view and accurate positioning of the cutting points of the pedicels in the close‐range view. Furthermore, it possesses the capability to adaptively infer the approaching pose of the end‐effector based on point cloud features extracted from fruit‐bearing pedicels and stems. Finally, a prototype of the tomato‐harvesting robot was assembled for field trials. The test results demonstrate that in tomato clusters with unobstructed pedicels, the localization success rates for the cutting points were 88.5% and 83.7% in the two greenhouses, respectively, while the harvesting success rates reached 57.7% and 55.4%, respectively. The average cycle time to harvest a tomato cluster was 24 s. The experimental results prove the potential for commercial application of the developed tomato‐harvesting robot and through the analysis of failure cases, discuss directions for future work.

Nighttime Harvesting of OrBot (Orchard RoBot)

The Robotics Vision Lab of Northwest Nazarene University has developed the Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 s. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted in images with the lowest noise. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94% compared to 88% during the daytime operations.

Design and experiment on mechanized batch harvesting of Shiitake mushrooms

Shiitake mushroom must be harvested in time or its commercial value will be reduced. At present, manual harvesting is time-consuming and labor-intensive, and the robotic selective harvesting is far from mature and marketable. Based on the analysis of manual harvesting, a mechanized batch-based harvesting method for the stick cultivation of shiitake mushrooms is proposed as an alternative harvest method in the scenario of large-scale factory production. The feasibility of the simplified picking method was verified by a data acquisition system with integrated force sensor and inertial measurement unit, and the picking force and picking action during manual picking were measured. A mechanized harvesting scheme by picking rings with forward forces to the mushroom stalk was proposed, and its finite element analysis concluded that the mushroom could be picked without damaging the mushroom by using a 5/10 size latex tube. A new mushroom Batch Harvesting Picker (BHP) was designed, including a pushing mechanism, a guiding mechanism, and a harvesting mechanism. By analyzing the causes of harvesting failure and mushroom damage, the stick surface profiling unit were improved by multi-body dynamics simulation. The simulation results show that when the preload force of the reset spring of the harvesting mechanism is set to 30 N, the least pressure on the mushroom stick is 8.97 N, and the picking ring working closer along the surface of stick. The experiment of BHP showed that the harvesting success rate is 98.93 %, the harvesting damage rate is 3.31 %, and the average harvesting time of a single mushroom stick is 9.8 s, which improved the harvesting efficiency of single stick by 2.2–6.6 times compared with the manual harvesting process, which proved the feasibility and efficiency of this batch harvesting scheme in large-scale factory production of shiitake mushrooms.

Peduncle collision-free grasping based on deep reinforcement learning for tomato harvesting robot

Collision-free grasping of the thin, brief peduncles connecting cherry tomato clusters to the main stem was crucial for tomato harvesting robots. Recognizing that the optimal operating posture for each individual peduncle was various, this study proposed a novel peduncle grasping posture decision model using deep reinforcement learning (DRL) for tomato harvesting manipulators, to overcome the collision issue caused by fixed-posture grasping. This model could dynamically generated action sequences for the harvesting manipulator, ensuring that the end-effector approach to the peduncle along the collision-free path with the optimal grasping posture. Building upon prior research into the multi-task identification of tomato clusters, peduncles, and the main stem, a keypoint-based spatial pose description model for tomato bunches was devised. Through this, the optimal operating posture for the end-effector on the peduncle was established. An improved HER-SAC (Soft Actor Critic with Hindsight Experience Replay) algorithm was subsequently established to guide the end-effector in collision-free grasping motions. The reward function of this algorithm incorporated end-effector posture constraints obtained from the optimal posture plane. In the training phase, a heuristic strategy model, providing prior knowledge, was merged with a dynamic gain module to sidestep local optimal policies, collectively enhancing the learning efficiency. In the simulation, our method improved the success rate of the peduncle grasping by at least 14 %, compared with SAC, HER-DDPG and HER-TD3. For the identical scenarios, improved HER-SAC reached the desired posture with a minimum of 15.5 % fewer steps compared to other algorithms. In field experiments conducted in tomato greenhouses, the robot achieved a harvesting success rate of 85.5 %, which was an increase of 57.3 % and 43.0 % compared to traditional methods with fixed horizontal and parallel-to-main-stem postures, respectively. The average operation time, from identification to successful harvesting, was 11.42 s. Our findings offer a promising solution to enhancing the efficiency of tomato-harvesting robots.

Fruit growing direction recognition and nesting grasping strategies for tomato harvesting robots

AbstractIn recent years, the potential of robotic harvesting in greenhouse tomato production has garnered significant attention within the tomato industry. However, there is a lack of sufficient research on the complete replacement of manual harvesting with this technology. In this paper, we propose a tomato harvesting method that utilizes a nesting approach to simplify the process and minimize damage. The paper describes the tomato harvesting robot prototype, the visual system equipped with three vision‐based tomato detectors: YOLOv5_CBAM, which incorporates a convolutional block attention module; YOLOv5_SE, enhanced with a squeeze‐and‐excitation block; and a standard YOLOv5s model. Additionally, a novel shear gripping method for fruit bunches is presented, utilizing a bottom‐up snapping technique during harvesting. Point cloud data are utilized to determine the position of the tomato's main stem and bunch. The paper includes field tests and experimental findings, which indicate that the YOLOv5_CBAM model achieves the highest precision (82.62%) and recall (82.57%), outperforming YOLOv5_SE and standard YOLOv5s. Field experiments demonstrate that the improved end‐effector and vision system have significantly enhanced the robot's performance, achieving a 57.5% harvesting success rate in just 14.9 s.

The Monash Apple Retrieving System: A review on system intelligence and apple harvesting performance

The wide-scale adoption of selective apple harvesting robots is yet to be seen, due to their strict requirement of well-manicured canopies to perform reliably. However, maintaining this standard of canopy is labour and resource-intensive, and an unsustainable practice for farmers in the long term who are already burdened with labour shortages. Without a complete planner and intelligent decision-making, a robotic harvesting platform would struggle to adapt to, and navigate in even typical canopy environments. In this work, we introduce the Monash Apple Retrieving System (MARS), a selective apple harvesting platform that is capable of navigating and harvesting apples in complex canopy environments without the need for canopy simplification. Utilising a four-tier planning process, MARS is able to generate gentle, collision-free harvesting trajectories in complex and unstructured canopies. Extensive trials were conducted over the 2021 and 2022 apple harvesting seasons in Australia, where MARS achieved a harvesting success rate of 62.8% at a cycle time of 9.18 s per apple, with negligible damage to the fruit. Performance increased to 70.8% success rate and 7.91 s of cycle time under ideal visibility conditions. Although performance can be further enhanced with canopy simplification and hardware upgrades, the reported results were achieved under realistic canopy conditions where very limited attempts were made to modify the canopy to create idealistic conditions for robotic harvesting. This baseline performance is promising for farmers who are wanting to utilise robotic harvesting, but do not have the resources to maintain well-manicured canopies. By lowering strict canopy requirements for robotic harvesting, we can improve accessibility of this technology to more farmers who are struggling with labour shortages around the world.

A Visual Method of Hydroponic Lettuces Height and Leaves Expansion Size Measurement for Intelligent Harvesting

Harvesting is an important procedure for hydroponic lettuces in plant factories. At present, hydroponic lettuces are mainly harvested manually, and the key difficulty in mechanical harvesting is reducing the occurrence of leaf injury. Measuring the size of hydroponic lettuces using the image processing method and intelligently adjusting the operating parameters of the harvesting device are the foundation of high-quality harvesting for lettuces. The overlapped leaves of adjacent hydroponic lettuces cause difficulties in measuring lettuce size, especially the leaves expansion size. Therefore, we proposed an image processing method for measuring lettuce height and leaves expansion size according to the upper contour feature of lettuces and an image included three lettuces. Firstly, the upper contours of the lettuces were extracted and segmented via image preprocessing. Secondly, lettuce height was measured according to the maximum ordinate of the contour. Lastly, the lettuce’s upper contour was fitted to a function to measure the leaves expansion size. The measurement results showed that the maximal relative error of the lettuce height measurements was 5.58%, and the average was 2.14%. The effect of the quadratic function in fitting the upper contour was the best compared with the cubic function and sine function. The maximal relative error of the leaves expansion size measurements was 8.59%, and the average was 4.03%. According to the results of the lettuce height and leaves expansion size measurements, the grabbing parameters of each lettuce were intelligently adjusted to verify the harvesting effect. The harvesting success rates of lettuces was above 90%, and the injured leaves areas of the left, middle, and right lettuces in each image were 192.6 mm2, 228.1 mm2, and 205.6 mm2, respectively. This paper provides a reference for the design and improvement of intelligent harvesters for hydroponic lettuces.

Citrus pose estimation from an RGB image for automated harvesting

Automated fruit harvesting is promising research in the development of agricultural modernization. However, the complex and non-structural orchard environment is extremely challenging. In order to meet the needs of different end-effectors and to improve the success rate of automatic fruit harvesting, it is critical to perform fruit pose estimation before picking operations. In this study, a citrus pose estimation method through a single RGB image is introduced. The rotation of the citrus pose is defined as a vector that passes through the center of the fruit, which is perpendicular to the plane where the fruit navel point is located. Simply speaking, a multi-task learning model named FPENet is proposed to simultaneously locate the fruit navel point and predict the fruit rotation vector. And a hyperparameter is introduced in the loss function to achieve the simultaneous convergence of multiple tasks. In addition, this paper designs a 2D image annotation tool and constructs a citrus pose dataset, which contributes to model training and also the algorithm evaluation. In the experiment, we evaluate and analyze each module of the proposed network structure, and verify its performance on a harvesting robot. The experimental results show that the FPENet achieves an 88.92 AP score on fruit navel point detection, and 11.13° on the average error of the rotation vector. Over 90% of rotation vectors have an angular error of less than 22.5°. The harvesting success rate is 79.79%. This study offers a new idea for fruit pose estimation and provides the possibility and foundation for estimating fruit pose with a 2D image input.

Lab2Field transfer of a robotic raspberry harvester enabled by a soft sensorized physical twin

Robotic fruit harvesting requires dexterity to handle delicate crops and development relying upon field testing possible only during the harvesting season. Here we focus on raspberry crops, and explore how the research methodology of harvesting robots can be accelerated through soft robotic technologies. We propose and demonstrate a physical twin of the harvesting environment: a sensorized physical simulator of a raspberry plant with tunable properties, used to train a robotic harvester in the laboratory regardless of season. The sensors on the twin allow for direct comparison with human demonstrations, used to tune the robot controllers. In early field demonstrations, an 80% harvesting success rate was achieved without any modifications on the lab trained robot.

Hybrid Visual Servo Control of a Robotic Manipulator for Cherry Tomato Harvesting

This paper aims to develop a visual servo control of a robotic manipulator for cherry tomato harvesting. In the robotic manipulator, an RGB-depth camera was mounted to the end effector to acquire the poses of the target cherry tomatoes in space. The eye-in-hand-based visual servo controller guides the end effector to implement eye–hand coordination to harvest the target cherry tomatoes, in which a hybrid visual servo control method (HVSC) with the fuzzy dynamic control parameters was proposed by combining position-based visual servo (PBVS) control and image-based visual servo (IBVS) control for the tradeoff of both performances. In addition, a novel cutting and clipping integrated mechanism was designed to pick the target cherry tomatoes. The proposed tomato-harvesting robotic manipulator with HVSC was validated and evaluated in a laboratory testbed based on harvesting implementation. The results show that the developed robotic manipulator using HVSC has an average harvesting time of 9.40 s/per and an average harvesting success rate of 96.25% in picking cherry tomatoes.

YOLO기반의 객체인식을 통한 딸기 수확 로봇 시스템에 대한 연구

Recently, interest in intelligent agricultural robots has surged due to the aging of rural workers, leading to active research on agricultural automation. We designed a strawberry harvesting robot that moves along a rail, equipped with a 3-axis linear actuator, an RGB-Depth camera for object detection, and a rotary gripper for branch cutting. Our research focused on developing algorithms for strawberry maturity classification using an AI vision system, calculating cutting points for fruit acquisition, and implementing these algorithms in a robot. A convolutional neural network based on YOLO was employed for object detection, and representation learning was used to determine the picking point with an ROI(Region Of Interest) image derived from object detection. Our strawberry harvesting robot system boasts an average harvesting success rate of 90% for ripe fruits.

Development and field evaluation of a robotic harvesting system for plucking high-quality tea

The critical issue of the robotic harvesting high-quality tea is to realize tea shoot detection, plucking point localization, and motion planning. In addition, the accuracy and efficiency of the robotic plucking of high-quality tea in the field are essential. Therefore, a robotic harvesting system is proposed in this paper by combining deep learning, point cloud processing, and spatial path planning. First, the deep learning method and the compressed YOLOv3 network are used to quickly and accurately identify tea shoots. Second, an efficient point cloud processing-based 3D localization algorithm for high-quality tea plucking points was proposed. The genetic algorithm is then used to shorten the end-effector's motion path by optimizing the plucking sequences. Eventually, a harvester robot with a parallel manipulator was developed to conduct field plucking experiments and evaluate the effectiveness of the proposed harvesting system. All experimental results demonstrate that the success rates of detection, localization, and motion plucking are 85.16 %, 78.90 %, and 80.23 %, respectively. Furthermore, the overall process harvesting success rate is 53.91 %, and the average plucking time for a single shoot is 2.233 s. Therefore, the proposed harvesting approach can provide technical support for the precise and rapid harvesting of high-quality tea.

Adaptive end‐effector pose control for tomato harvesting robots

AbstractThis paper investigates the development of a tomato‐harvesting robot operating on a plant factory and primarily studies the reachable pose of tomatoes in the nondexterous workspace of manipulator. The end‐effector can only reach the tomatoes with reachable poses when the tomatoes are within the nondexterous workspace. If the grasping pose is not reachable, it will lead to grasping failure. An adaptive end‐effector pose control method based on a genetic algorithm (GA) is proposed to find a reachable pose. The inverse kinematic solution based on analysis method of the manipulator is analyzed and the objective function of whether the manipulator has a solution or not is obtained. The grasping pose is set as an individual owing to the position of the tomatoes is fixed and the grasping pose is variable. The GA is used to solve until a pose that can make the inverse kinematics have a solution is generated. This pose is the reachable grasping pose of the tomato at this position. The quintic interpolation polynomial is used to plan the trajectory to avoid damage to tomatoes owing to fast approaching speed and a distance based background filtering method is proposed. Experiments were performed to verify the effectiveness of the proposed method. The radius of the workspace of the UR3e manipulator with the end‐effector increased from 550 to 800 mm and the grasping range expanded by 208%. The harvesting success rate using the adaptive end‐effector pose control method and trajectory planning method was 88%. The cycle of harvesting a tomato was 20 s. The experimental results indicated that the proposed tomato‐recognition and end‐effector pose control method are feasible and effective.

A Method of Grasping Detection for Kiwifruit Harvesting Robot Based on Deep Learning

Kiwifruit harvesting with robotics can be troublesome due to the clustering feature. The gripper of the end effector will easily cause unstable fruit grasping, or the bending and separation action will interfere with the neighboring fruit because of an inappropriate grasping angle, which will further affect the success rate. Therefore, predicting the correct grasping angle for each fruit can guide the gripper to safely approach, grasp, bend and separate the fruit. To improve the grasping rate and harvesting success rate, this study proposed a grasping detection method for a kiwifruit harvesting robot based on the GG-CNN2. Based on the vertical downward growth characteristics of kiwifruit, the grasping configuration of the manipulator was defined. The clustered kiwifruit was mainly divided into single fruit, linear cluster, and other cluster, and the grasping dataset included depth images, color images, and grasping labels. The GG-CNN2 was improved based on focal loss to prevent the algorithm from generating the optimal grasping configuration in the background or at the edge of the fruit. The performance test of the grasping detection network and the verification test of robotic picking were carried out in orchards. The results showed that the number of parameters of GG-CNN2 was 66.7 k, the average image calculation speed was 58 ms, and the average grasping detection accuracy was 76.0%, which ensures the grasping detection can run in real time. The verification test results indicated that the manipulator combined with the position information provided by the target detection network YOLO v4 and the grasping angle provided by the grasping detection network GG-CNN2 could achieve a harvesting success rate of 88.7% and a fruit drop rate of 4.8%; the average picking time was 6.5 s. Compared with the method in which the target detection network only provides fruit position information, this method presented the advantages of harvesting rate and fruit drop rate when harvesting linear clusters, especially other cluster, and the picking time was slightly increased. Therefore, the grasping detection method proposed in this study is suitable for near-neighbor multi-kiwifruit picking, and it can improve the success rate of robotic harvesting.

Development and Evaluation of a Watermelon-Harvesting Robot Prototype: Vision System and End-Effector

Over the past decade, there have been increasing attempts to integrate robotic harvesting technology into agricultural scenarios to reduce growing labour costs and increase crop yields. In this paper, we demonstrate a prototype harvesting robot for picking watermelons in greenhouses. For robotic harvesting, we design a dedicated end-effector for grasping fruits and shearing pedicels, which mainly consists of a flexible gripper and a cutting device. The improved YOLOv5s–CBAM is employed to locate the watermelon fruits with 89.8% accuracy on the test dataset, while the K-means method is used to further refine the segmentation of the watermelon point cloud in the region of interest. Then, the ellipsoid is fitted with the segmented fruit point cloud to obtain the lowest point of the ellipsoid as the grasping point. A series of tests conducted in a laboratory simulation scenario proved that the overall harvesting success rate was 93.3% with a positioning error of 8.7 mm when the watermelon was unobstructed. The overall harvesting success rate was 85.0% with a positioning error of 14.6 mm when the watermelon was partially obscured by leaves.

Fruit pose recognition and directional orderly grasping strategies for tomato harvesting robots

The prospect of robotic harvesting in greenhouse tomato production has received much attention in the tomato industry in recent years, but research into the replacement of manual harvesting with this technology is still inadequate. There is still a large gap between robotic harvesting accuracy and harvesting efficiency compared to manual harvesting. The aim of this study is to improve the success rate and efficiency of robotic harvesting by accurately identifying the position of the tomato fruit and estimating the grasping pose. Based on a specially designed adsorption and clamping integrated manipulator, we develop the optimal sorting algorithm and fruit nearest neighbour positioning algorithm and design directional grasping and sequential picking control strategies that can reduce the impact of dense fruit on the accurate grasping of the manipulator. The results evaluated in the industrial greenhouse show that the YOLOv5m-based fruit bunch and fruit recognition accuracies are 90.2 % and 97.3 %, respectively. Based on the optimized picking strategy, the impact of collision on the manipulator's grasp was substantially reduced, and the fruit harvesting success rate increased to 72.1 %, while the average harvesting time reached 14.6 s for a single fruit. The research suggests that with moderate improvements in harvesting technology, the use of robotic systems in the tomato industry will gradually become a reality.

Development of a dual-arm rapid grape-harvesting robot for horizontal trellis cultivation.

It is extremely necessary to achieve the rapid harvesting of table grapes planted with a standard trellis in the grape industry. The design and experimental analysis of a dual-arm high-speed grape-harvesting robot were carried out to address the limitations of low picking efficiency and high grape breakage rate of multijoint robotic arms. Based on the characteristics of the harvesting environment, such as the small gap between grape clusters, standard trellis, and vertical suspension of clusters, the configuration of the dual-arm harvesting robot is reasonably designed and analyzed, and the overall configuration of the machine and the installation position of key components are derived. Robotic arm and camera view analysis of the workspace harvesting process was performed using MATLAB, and it can be concluded that the structural design of this robot meets the grape harvesting requirements with a standard trellis. To improve the harvesting efficiency, some key high-speed harvesting technologies were adopted, such as the harvesting sequence decision based on the “sequential mirroring method” of grape cluster depth information, “one-eye and dual-arm” high-speed visual servo, dual arm action sequence decision, and optimization of the “visual end effector” large tolerance combination in a natural environment. The indoor accuracy experiment shows that when the degree of obscuration of grape clusters by leaves increases, the vision algorithm based on the geometric contours of grape clusters can still meet the demands of harvesting tasks. The motion positioning average errors of the left and right robotic arms were (X: 2.885 mm, Y: 3.972 mm, Z: 2.715 mm) and (X: 2.471 mm, Y: 3.289 mm, Z: 3.775 mm), respectively, and the average dual-arm harvesting time in one grape cluster was 8.45 s. The field performance test verifies that the average harvesting cycle of the robot with both arms reached 9 s/bunch, and the success rate of bunch identification and harvesting success rate reached 88 and 83%, respectively, which were significantly better than those of existing harvesting robots worldwide.

Tomato harvesting robotic system based on Deep-ToMaToS: Deep learning network using transformation loss for 6D pose estimation of maturity classified tomatoes with side-stem

This paper presents the development of an autonomous harvesting robot system for tomato, a representative crop cultivated in the facility horticulture smart farm. The automated harvesting work using a robotic system is very challenging because of the appearance, environmental features, such as the atypical directions of the peduncles or their growing form in a bunch. Also, the robot system should enable the harvesting of the target fruit only without damaging other fruits, stems, and branches. Hence, this paper presents a deep learning network pipeline, Deep-ToMaToS, capable of three-level maturity classification and 6D pose (3D translation + 3D rotation) estimation of the target fruit simultaneously. Due to the difficulties encountered in building a large-scale dataset to train and test the deep learning model for the 6D pose estimation in the real world, we presented an automatic data collection scheme based on a photo-realistic 3D simulator environment. The robotic harvesting system includes a harvesting motion control algorithm based on the result of the 6D pose estimation. The overall process of the motion control phase is described along with the decision way of the appropriate final posture of the harvesting module mounted at the end-effector of the robot manipulator via removal of invalid motions getting out of the valid workspace or redundant motions. We conducted experiments on the 6D pose estimation based on the Deep-ToMaToS and the harvesting motion control in virtual and real smart farm environments. The experimental results showed a 6D pose estimation accuracy of 96 % based on the ADD_S metric, and the proposed harvesting motion control algorithm achieves the harvesting success rate of 84.5 % on average. The experimental results reveal that the harvesting robot system has significant potential to extend to harvesting works for other fruits and crops.

Research on Multiobjective Optimization Algorithm for Cooperative Harvesting Trajectory Optimization of an Intelligent Multiarm Straw-Rotting Fungus Harvesting Robot

In view of the difficulties of fruit cluster identification, the specific harvesting sequence constraints of aggregated fruits, and the balanced harvesting task assignment for the multiple arms with a series-increasing symmetric shared (SISS) region, this paper proposes a multi-objective optimization algorithm, which combines genetic algorithm (GA) and ant colony optimization (ACO) stepwise, to optimize the multiarm cooperative harvesting trajectory of straw-rotting fungus to effectively improve the harvesting efficiency and the success rate of non-destructive harvesting. In this approach, firstly, the multiarm trajectory optimization problem is abstracted as a multiple travelling salesman problem (MTSP). Secondly, an improved local density clustering algorithm is designed to identify the cluster fruits to prepare data for harvesting aggregated fruits in a specific order later. Thirdly, the MTSP has been decomposed into M independent TSP (traveling salesman problem) problems by using GA, in which a new DNA (deoxyribonucleic acid) assignment rule is designed to resolve the problem of the average distribution of multiarm harvesting tasks with the SISS region. Then, the improved ant colony algorithm, combined with the auction mechanism, is adopted to achieve the shortest trajectory of each arm, which settles the difficulty that the clustered mature fruits should be harvested in a specified order. The experiments show that it can search for a relatively stable optimal solution in a relatively short time. The average harvesting efficiency is up to 1183 pcs/h and the average harvesting success rate is about 97%. Therefore, the proposed algorithm can better plan the harvesting trajectory for multiarm intelligent harvesting, especially for areas with many aggregated fruits.