Harvesting Success Rate Research Articles

Design and Experiment of Nighttime Greenhouse Tomato Harvesting Robot

In response to the issue of high tomato yield, low efficiency in harvesting tomatoes grown in greenhouses, and low recognition accuracy of nighttime harvesting robots, a design was developed and a robotic system was created specifically for nighttime greenhouse tomato harvesting. The robot employs a vision system and YOLOv5+HSV fusion algorithm to recognize and locate tomatoes. It then transmits this information to the robotic arm. By coordinating the visual system, the robotic arm, the end effector, and the lifting mechanism, the robot accurately picks ripe tomatoes. The robot was subjected to simulated field tests for visual recognition and harvesting, both during daytime and nighttime conditions. The results showed that the success rate of nighttime harvesting was slightly lower than during the daytime but remained at a relatively high level. The daytime harvesting success rate and the average time to pick a single fruit were 87.78% and 15.99 seconds, respectively. The nighttime harvesting success rate and the average time to pick a single fruit were 87.55% and 17.26 seconds, respectively. This approach effectively improves the recognition accuracy and harvesting speed of the harvesting robot, reducing damage to tomatoes during harvesting, and addresses the issues of supplementary lighting and image noise reduction for nighttime harvesting robots.

Nighttime Harvesting of OrBot (Orchard RoBot)

The Robotics Vision Lab of Northwest Nazarene University has developed the Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 s. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted in images with the lowest noise. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94% compared to 88% during the daytime operations.

Design and experiment on mechanized batch harvesting of Shiitake mushrooms

Shiitake mushroom must be harvested in time or its commercial value will be reduced. At present, manual harvesting is time-consuming and labor-intensive, and the robotic selective harvesting is far from mature and marketable. Based on the analysis of manual harvesting, a mechanized batch-based harvesting method for the stick cultivation of shiitake mushrooms is proposed as an alternative harvest method in the scenario of large-scale factory production. The feasibility of the simplified picking method was verified by a data acquisition system with integrated force sensor and inertial measurement unit, and the picking force and picking action during manual picking were measured. A mechanized harvesting scheme by picking rings with forward forces to the mushroom stalk was proposed, and its finite element analysis concluded that the mushroom could be picked without damaging the mushroom by using a 5/10 size latex tube. A new mushroom Batch Harvesting Picker (BHP) was designed, including a pushing mechanism, a guiding mechanism, and a harvesting mechanism. By analyzing the causes of harvesting failure and mushroom damage, the stick surface profiling unit were improved by multi-body dynamics simulation. The simulation results show that when the preload force of the reset spring of the harvesting mechanism is set to 30 N, the least pressure on the mushroom stick is 8.97 N, and the picking ring working closer along the surface of stick. The experiment of BHP showed that the harvesting success rate is 98.93 %, the harvesting damage rate is 3.31 %, and the average harvesting time of a single mushroom stick is 9.8 s, which improved the harvesting efficiency of single stick by 2.2–6.6 times compared with the manual harvesting process, which proved the feasibility and efficiency of this batch harvesting scheme in large-scale factory production of shiitake mushrooms.

Peduncle collision-free grasping based on deep reinforcement learning for tomato harvesting robot

Collision-free grasping of the thin, brief peduncles connecting cherry tomato clusters to the main stem was crucial for tomato harvesting robots. Recognizing that the optimal operating posture for each individual peduncle was various, this study proposed a novel peduncle grasping posture decision model using deep reinforcement learning (DRL) for tomato harvesting manipulators, to overcome the collision issue caused by fixed-posture grasping. This model could dynamically generated action sequences for the harvesting manipulator, ensuring that the end-effector approach to the peduncle along the collision-free path with the optimal grasping posture. Building upon prior research into the multi-task identification of tomato clusters, peduncles, and the main stem, a keypoint-based spatial pose description model for tomato bunches was devised. Through this, the optimal operating posture for the end-effector on the peduncle was established. An improved HER-SAC (Soft Actor Critic with Hindsight Experience Replay) algorithm was subsequently established to guide the end-effector in collision-free grasping motions. The reward function of this algorithm incorporated end-effector posture constraints obtained from the optimal posture plane. In the training phase, a heuristic strategy model, providing prior knowledge, was merged with a dynamic gain module to sidestep local optimal policies, collectively enhancing the learning efficiency. In the simulation, our method improved the success rate of the peduncle grasping by at least 14 %, compared with SAC, HER-DDPG and HER-TD3. For the identical scenarios, improved HER-SAC reached the desired posture with a minimum of 15.5 % fewer steps compared to other algorithms. In field experiments conducted in tomato greenhouses, the robot achieved a harvesting success rate of 85.5 %, which was an increase of 57.3 % and 43.0 % compared to traditional methods with fixed horizontal and parallel-to-main-stem postures, respectively. The average operation time, from identification to successful harvesting, was 11.42 s. Our findings offer a promising solution to enhancing the efficiency of tomato-harvesting robots.

Fruit growing direction recognition and nesting grasping strategies for tomato harvesting robots

AbstractIn recent years, the potential of robotic harvesting in greenhouse tomato production has garnered significant attention within the tomato industry. However, there is a lack of sufficient research on the complete replacement of manual harvesting with this technology. In this paper, we propose a tomato harvesting method that utilizes a nesting approach to simplify the process and minimize damage. The paper describes the tomato harvesting robot prototype, the visual system equipped with three vision‐based tomato detectors: YOLOv5_CBAM, which incorporates a convolutional block attention module; YOLOv5_SE, enhanced with a squeeze‐and‐excitation block; and a standard YOLOv5s model. Additionally, a novel shear gripping method for fruit bunches is presented, utilizing a bottom‐up snapping technique during harvesting. Point cloud data are utilized to determine the position of the tomato's main stem and bunch. The paper includes field tests and experimental findings, which indicate that the YOLOv5_CBAM model achieves the highest precision (82.62%) and recall (82.57%), outperforming YOLOv5_SE and standard YOLOv5s. Field experiments demonstrate that the improved end‐effector and vision system have significantly enhanced the robot's performance, achieving a 57.5% harvesting success rate in just 14.9 s.

Real-World Mobilization, Harvest, and Transplant Outcomes in Newly Diagnosed Multiple Myeloma Patients Receiving D-VTd: A UK Centre Experience

Introduction In newly diagnosed multiple myeloma (NDMM) patients eligible for autologous stem cell transplantation (ASCT), daratumumab plus, bortezomib, thalidomide, and dexamethasone (D-VTd) followed by high-dose melphalan with ASCT, is considered standard of care in Europe. The CASSIOPEIA trial reported a lower stem cell (SC) yield and higher plerixafor (PLX) use, but no impact over ASCT feasibility and safety for the D-VTd group. However, real-world data from clinical experience after its approval is scarce. In the United Kingdom, the National Institute for Health and Care Excellence approved D-VTd as frontline treatment in November 2021. Our aim was to describe the real-world experience regarding mobilization, harvest, and ASCT outcomes in NDMM patients receiving D-VTd. Methods We included all consecutive NDMM patients referred from 10 haematology centres to our institution and subsequently to apheresis, from June 2022 to May 2023. Patients received D-VTd as induction and had achieved at least partial response at time of referral. Patients underwent SC mobilisation with cyclophosphamide 1.5g/m 2 D+1, GCS-F 5µg/kg D+5 to D+10, and apheresis D+11 and D+12. Additional VTd cycles were given when length of waiting for apheresis was more than 6 weeks. Pre-emptive PLX was given for pre-apheresis CD34+count between 5-15cells/µL. After a successful harvest (≥2.0x10 6 CD34+cells/kg) ASCT was scheduled in 4-6weeks. Results A total of 95 patients were included. The median age was 63 years (range 34-74), 71.6% were male, 16.8% were ISS-III, 9.2% were R-ISS-III, and 28.9% had high-risk cytogenetics. Regarding induction treatment patients received 4 cycles of D-VTD; additionally 11 (11.6%) received radiotherapy, and 36 (37.9%) additional VTD pre-apheresis,th a median of 2 (range 1-4). MM response pre-apheresis was ≥VGPR in 80%. The median time from the start of D-VTd to apheresis was 6 months (range 4-10). A total of 87 patients underwent apheresis; 21 (22.1%) received pre-emptive PLX and 71 (81.6%) underwent two apheresis procedures. The mean pre-apheresis and total dose CD34+cells were 29.2/µL (range 0.4-220.7) and 4.5x10 6/kg (range 1.2-12.6), respectively. From the 14 patients that failed to collect ≥2.0x10 6cell/kg on first attempt, 8 underwent a second attempt. Overall harvest success rate was 92.6% ( n = 88) (Figure 1). Regarding the additional VTd versus non-VTd groups, mean total CD34+cell dose was 5.1x10 6/kg (range 1.3-10.7) versus 4.8x10 6/kg (range 1.2-12.6) ( p = 0.61) and PLX was used in 25% versus20.3% ( p = 0.28). After a median of 7 weeks (range 4-32), 68 patients (77.3%) proceeded to ASCT, 11 (12.5%) are awaiting, and 9 (10.2%) were cancelled. A 54.4% (n = 37) of patients experienced a delay for ASCT, 7 due to second harvest attempt. Mean CD34+cells infused was 3.7x10 6/kg (range 2.0-6.7). All patients achieved neutrophil and platelet engraftments after a median of 13 days (range 9-17) and 19 days (range 13-29), respectively. One patient died after engraftment, day +49, due to pneumonia. MM response at 100-days after ASCT was ≥VGPR in 97.1% and MRD-neg in 85.2% of the evaluated patients (Table 1). Conclusions Based on our results, we confirm that compared to published data the addition of daratumumab to VTd in NDMM patients, results in a lower SC yield and higher pre-emptive PLX administration. As expected, when reflecting the real-world practice timely interventions are difficult to maintain leading to delays on harvesting and ACST procedures. Herein we report the implementation of a “bridging” strategy with additional VTd cycles between D-VTd induction and apheresis, this did not influence our harvest outcomes. Although longer follow-up is desirable, our results regarding harvest success (92.6%) and ASCT outcomes, including engraftment and MM response at 100-days, are comparable to those reported by the CASSIOPEIA trial.

The Monash Apple Retrieving System: A review on system intelligence and apple harvesting performance

The wide-scale adoption of selective apple harvesting robots is yet to be seen, due to their strict requirement of well-manicured canopies to perform reliably. However, maintaining this standard of canopy is labour and resource-intensive, and an unsustainable practice for farmers in the long term who are already burdened with labour shortages. Without a complete planner and intelligent decision-making, a robotic harvesting platform would struggle to adapt to, and navigate in even typical canopy environments. In this work, we introduce the Monash Apple Retrieving System (MARS), a selective apple harvesting platform that is capable of navigating and harvesting apples in complex canopy environments without the need for canopy simplification. Utilising a four-tier planning process, MARS is able to generate gentle, collision-free harvesting trajectories in complex and unstructured canopies. Extensive trials were conducted over the 2021 and 2022 apple harvesting seasons in Australia, where MARS achieved a harvesting success rate of 62.8% at a cycle time of 9.18 s per apple, with negligible damage to the fruit. Performance increased to 70.8% success rate and 7.91 s of cycle time under ideal visibility conditions. Although performance can be further enhanced with canopy simplification and hardware upgrades, the reported results were achieved under realistic canopy conditions where very limited attempts were made to modify the canopy to create idealistic conditions for robotic harvesting. This baseline performance is promising for farmers who are wanting to utilise robotic harvesting, but do not have the resources to maintain well-manicured canopies. By lowering strict canopy requirements for robotic harvesting, we can improve accessibility of this technology to more farmers who are struggling with labour shortages around the world.

Human-centered approach for an efficient cucumber harvesting robot system: Harvest ordering, visual servoing, and end-effector

This paper presents a human-centered approach for an efficient cucumber-harvesting robot system. Specifically, harvest ordering, visual servoing, and end-effector-based manipulation functionalities were integrated to realize efficient and stable harvesting. The proposed approach involved determining the optimal harvest ordering, guiding the end-effector to the cucumber pedicel through visual servoing, and designing an end-effector to effectively harvest long cucumbers. The performance of the system was evaluated through preliminary and field experiments. The results of the preliminary experiments showed that harvest ordering decreased the harvesting time and travel length and increased the battery efficiency. The visual servoing was robust, and pedicels could be rapidly detected at a speed of 16–23 FPS through computer vision technologies. The pedicel could be accurately positioned within the cutting area of the end-effector. Furthermore, the proposed end-effector could effectively cut thin cucumber pedicels (3–6 mm), with a 100.0% success rate. Field experiments were conducted at three sites in Korea: Green Monsters, Sangju smart-valley, and Fresh-farm. The harvest success rate at the three sites ranged from 50.9% to 60.0%, with an overall value of 56.6%. The overall average harvest time of 56.0 s. The positional accuracy of the system was within the optimal range of 0–30 mm. Furthermore, the primary causes of harvest failure were analyzed, and future research directions to improve the performance of harvest robots were discussed.

Citrus pose estimation from an RGB image for automated harvesting

Automated fruit harvesting is promising research in the development of agricultural modernization. However, the complex and non-structural orchard environment is extremely challenging. In order to meet the needs of different end-effectors and to improve the success rate of automatic fruit harvesting, it is critical to perform fruit pose estimation before picking operations. In this study, a citrus pose estimation method through a single RGB image is introduced. The rotation of the citrus pose is defined as a vector that passes through the center of the fruit, which is perpendicular to the plane where the fruit navel point is located. Simply speaking, a multi-task learning model named FPENet is proposed to simultaneously locate the fruit navel point and predict the fruit rotation vector. And a hyperparameter is introduced in the loss function to achieve the simultaneous convergence of multiple tasks. In addition, this paper designs a 2D image annotation tool and constructs a citrus pose dataset, which contributes to model training and also the algorithm evaluation. In the experiment, we evaluate and analyze each module of the proposed network structure, and verify its performance on a harvesting robot. The experimental results show that the FPENet achieves an 88.92 AP score on fruit navel point detection, and 11.13° on the average error of the rotation vector. Over 90% of rotation vectors have an angular error of less than 22.5°. The harvesting success rate is 79.79%. This study offers a new idea for fruit pose estimation and provides the possibility and foundation for estimating fruit pose with a 2D image input.

Lab2Field transfer of a robotic raspberry harvester enabled by a soft sensorized physical twin

Robotic fruit harvesting requires dexterity to handle delicate crops and development relying upon field testing possible only during the harvesting season. Here we focus on raspberry crops, and explore how the research methodology of harvesting robots can be accelerated through soft robotic technologies. We propose and demonstrate a physical twin of the harvesting environment: a sensorized physical simulator of a raspberry plant with tunable properties, used to train a robotic harvester in the laboratory regardless of season. The sensors on the twin allow for direct comparison with human demonstrations, used to tune the robot controllers. In early field demonstrations, an 80% harvesting success rate was achieved without any modifications on the lab trained robot.

A multi-arm robot system for efficient apple harvesting: Perception, task plan and control

Robot harvesting has emerged as an urgent need for apple industry due to a sharp decline in agricultural labor. In the field of harvesting robots, the use of multiple robotic arms to improve operational efficiency and promote industrial applications has gained significant attention. Despite the significant progress made in multi-arm harvesting robots in recent years, their widespread application in orchard production is hindered by insufficient efficiency of operation and the accuracy of fruit positioning. This paper focuses on the precise perception and multi-arm collaborative control issues of harvesting robots, and proposes a multi-arm apple harvesting robot system. Firstly, the paper introduces the hardware and software integration method and kinematic configuration of the robot, and presents its workspace division and asynchronous sequential operation mode. Secondly, the paper proposes a stereo vision fruit recognition and localization algorithm based on multi-task deep learning to enhance the accuracy of apple fruit positioning and a method of combining multiple perspectives to acquire a global fruit map is introduced. Finally, the paper presents a multi-arm task planning method based on the Markov game framework to optimize the target harvesting order of each arm and improve the collaboration efficiency. The effectiveness of the robot and its perception and control methods are verified through multiple field experiments in orchards. The field trials showed that the proposed vision system reduces the median locating error of the robot system by up to 44.43%; the proposed task planning algorithm can reduce the average cycle time by 33.3% compared to the heuristic-based algorithm, and time taken for optimizing task planning ranged from 1.14 s to 1.21 s; and the robot’s harvest success rate varied from 71.28% to 80.45%, and the average cycle time ranged from 5.8 s to 6.7 s.

Hybrid Visual Servo Control of a Robotic Manipulator for Cherry Tomato Harvesting

This paper aims to develop a visual servo control of a robotic manipulator for cherry tomato harvesting. In the robotic manipulator, an RGB-depth camera was mounted to the end effector to acquire the poses of the target cherry tomatoes in space. The eye-in-hand-based visual servo controller guides the end effector to implement eye–hand coordination to harvest the target cherry tomatoes, in which a hybrid visual servo control method (HVSC) with the fuzzy dynamic control parameters was proposed by combining position-based visual servo (PBVS) control and image-based visual servo (IBVS) control for the tradeoff of both performances. In addition, a novel cutting and clipping integrated mechanism was designed to pick the target cherry tomatoes. The proposed tomato-harvesting robotic manipulator with HVSC was validated and evaluated in a laboratory testbed based on harvesting implementation. The results show that the developed robotic manipulator using HVSC has an average harvesting time of 9.40 s/per and an average harvesting success rate of 96.25% in picking cherry tomatoes.

YOLO기반의 객체인식을 통한 딸기 수확 로봇 시스템에 대한 연구

Recently, interest in intelligent agricultural robots has surged due to the aging of rural workers, leading to active research on agricultural automation. We designed a strawberry harvesting robot that moves along a rail, equipped with a 3-axis linear actuator, an RGB-Depth camera for object detection, and a rotary gripper for branch cutting. Our research focused on developing algorithms for strawberry maturity classification using an AI vision system, calculating cutting points for fruit acquisition, and implementing these algorithms in a robot. A convolutional neural network based on YOLO was employed for object detection, and representation learning was used to determine the picking point with an ROI(Region Of Interest) image derived from object detection. Our strawberry harvesting robot system boasts an average harvesting success rate of 90% for ripe fruits.

Development and field evaluation of a robotic harvesting system for plucking high-quality tea

The critical issue of the robotic harvesting high-quality tea is to realize tea shoot detection, plucking point localization, and motion planning. In addition, the accuracy and efficiency of the robotic plucking of high-quality tea in the field are essential. Therefore, a robotic harvesting system is proposed in this paper by combining deep learning, point cloud processing, and spatial path planning. First, the deep learning method and the compressed YOLOv3 network are used to quickly and accurately identify tea shoots. Second, an efficient point cloud processing-based 3D localization algorithm for high-quality tea plucking points was proposed. The genetic algorithm is then used to shorten the end-effector's motion path by optimizing the plucking sequences. Eventually, a harvester robot with a parallel manipulator was developed to conduct field plucking experiments and evaluate the effectiveness of the proposed harvesting system. All experimental results demonstrate that the success rates of detection, localization, and motion plucking are 85.16 %, 78.90 %, and 80.23 %, respectively. Furthermore, the overall process harvesting success rate is 53.91 %, and the average plucking time for a single shoot is 2.233 s. Therefore, the proposed harvesting approach can provide technical support for the precise and rapid harvesting of high-quality tea.

Development of a novel pull-cutting end-effector for ex-situ robotic harvesting of white asparagus based on MBD-DEM coupling simulation

The existing selective harvest of white asparagus originally imitates manual operation, belongs to in-situ harvesting mode, and improves harvesting speed by reverse motion compensation resulting in complicated mechanical and control. Asparagus spear grows underground with limited harvesting space, brittle and easily damaged. The design of robotic end-effector for efficient selective harvest is still a challenge. Inspired by ‘fishing with nets in moving boat’, this paper firstly proposed a novel ex-situ harvesting mode with sequential subprocess of move-penetrating, pull-cutting, lifting-out, and eject-throwing and the rigid-flexible coupling pull-cutting end-effector (PCEE) was designed. Secondly, the model of PCEE-soil-spear was established, and the two-way coupled MBD-DEM (Multi-Body Dynamics and Discrete Element Method) analysis was utilized to the harvesting process to clarify the interactive forces and element motions. Finally, the harvesting test bench with designed PCEE was built and tested harvesting performance. The experimental results showed that the maximum soil resistances in the moving and penetrating directions during the move-penetrating subprocess and in the moving direction during the pull-cutting subprocess were 445 N, 530 N and 624 N, and the relative errors between the test and simulation were 5.3 %, 11.7 % and 15.2 %, respectively, which validated the MBD-DEM coupling model. The harvest success rate of proposed PCEE was 90 % and the average single cycle time was 3.2 s, which demonstrated its effectiveness and feasibility for the selective harvesting robots of white asparagus.

Adaptive end‐effector pose control for tomato harvesting robots

AbstractThis paper investigates the development of a tomato‐harvesting robot operating on a plant factory and primarily studies the reachable pose of tomatoes in the nondexterous workspace of manipulator. The end‐effector can only reach the tomatoes with reachable poses when the tomatoes are within the nondexterous workspace. If the grasping pose is not reachable, it will lead to grasping failure. An adaptive end‐effector pose control method based on a genetic algorithm (GA) is proposed to find a reachable pose. The inverse kinematic solution based on analysis method of the manipulator is analyzed and the objective function of whether the manipulator has a solution or not is obtained. The grasping pose is set as an individual owing to the position of the tomatoes is fixed and the grasping pose is variable. The GA is used to solve until a pose that can make the inverse kinematics have a solution is generated. This pose is the reachable grasping pose of the tomato at this position. The quintic interpolation polynomial is used to plan the trajectory to avoid damage to tomatoes owing to fast approaching speed and a distance based background filtering method is proposed. Experiments were performed to verify the effectiveness of the proposed method. The radius of the workspace of the UR3e manipulator with the end‐effector increased from 550 to 800 mm and the grasping range expanded by 208%. The harvesting success rate using the adaptive end‐effector pose control method and trajectory planning method was 88%. The cycle of harvesting a tomato was 20 s. The experimental results indicated that the proposed tomato‐recognition and end‐effector pose control method are feasible and effective.

A Method of Grasping Detection for Kiwifruit Harvesting Robot Based on Deep Learning

Kiwifruit harvesting with robotics can be troublesome due to the clustering feature. The gripper of the end effector will easily cause unstable fruit grasping, or the bending and separation action will interfere with the neighboring fruit because of an inappropriate grasping angle, which will further affect the success rate. Therefore, predicting the correct grasping angle for each fruit can guide the gripper to safely approach, grasp, bend and separate the fruit. To improve the grasping rate and harvesting success rate, this study proposed a grasping detection method for a kiwifruit harvesting robot based on the GG-CNN2. Based on the vertical downward growth characteristics of kiwifruit, the grasping configuration of the manipulator was defined. The clustered kiwifruit was mainly divided into single fruit, linear cluster, and other cluster, and the grasping dataset included depth images, color images, and grasping labels. The GG-CNN2 was improved based on focal loss to prevent the algorithm from generating the optimal grasping configuration in the background or at the edge of the fruit. The performance test of the grasping detection network and the verification test of robotic picking were carried out in orchards. The results showed that the number of parameters of GG-CNN2 was 66.7 k, the average image calculation speed was 58 ms, and the average grasping detection accuracy was 76.0%, which ensures the grasping detection can run in real time. The verification test results indicated that the manipulator combined with the position information provided by the target detection network YOLO v4 and the grasping angle provided by the grasping detection network GG-CNN2 could achieve a harvesting success rate of 88.7% and a fruit drop rate of 4.8%; the average picking time was 6.5 s. Compared with the method in which the target detection network only provides fruit position information, this method presented the advantages of harvesting rate and fruit drop rate when harvesting linear clusters, especially other cluster, and the picking time was slightly increased. Therefore, the grasping detection method proposed in this study is suitable for near-neighbor multi-kiwifruit picking, and it can improve the success rate of robotic harvesting.

Development and Evaluation of a Watermelon-Harvesting Robot Prototype: Vision System and End-Effector

Over the past decade, there have been increasing attempts to integrate robotic harvesting technology into agricultural scenarios to reduce growing labour costs and increase crop yields. In this paper, we demonstrate a prototype harvesting robot for picking watermelons in greenhouses. For robotic harvesting, we design a dedicated end-effector for grasping fruits and shearing pedicels, which mainly consists of a flexible gripper and a cutting device. The improved YOLOv5s–CBAM is employed to locate the watermelon fruits with 89.8% accuracy on the test dataset, while the K-means method is used to further refine the segmentation of the watermelon point cloud in the region of interest. Then, the ellipsoid is fitted with the segmented fruit point cloud to obtain the lowest point of the ellipsoid as the grasping point. A series of tests conducted in a laboratory simulation scenario proved that the overall harvesting success rate was 93.3% with a positioning error of 8.7 mm when the watermelon was unobstructed. The overall harvesting success rate was 85.0% with a positioning error of 14.6 mm when the watermelon was partially obscured by leaves.

Fruit pose recognition and directional orderly grasping strategies for tomato harvesting robots

The prospect of robotic harvesting in greenhouse tomato production has received much attention in the tomato industry in recent years, but research into the replacement of manual harvesting with this technology is still inadequate. There is still a large gap between robotic harvesting accuracy and harvesting efficiency compared to manual harvesting. The aim of this study is to improve the success rate and efficiency of robotic harvesting by accurately identifying the position of the tomato fruit and estimating the grasping pose. Based on a specially designed adsorption and clamping integrated manipulator, we develop the optimal sorting algorithm and fruit nearest neighbour positioning algorithm and design directional grasping and sequential picking control strategies that can reduce the impact of dense fruit on the accurate grasping of the manipulator. The results evaluated in the industrial greenhouse show that the YOLOv5m-based fruit bunch and fruit recognition accuracies are 90.2 % and 97.3 %, respectively. Based on the optimized picking strategy, the impact of collision on the manipulator's grasp was substantially reduced, and the fruit harvesting success rate increased to 72.1 %, while the average harvesting time reached 14.6 s for a single fruit. The research suggests that with moderate improvements in harvesting technology, the use of robotic systems in the tomato industry will gradually become a reality.

Development of a dual-arm rapid grape-harvesting robot for horizontal trellis cultivation.

It is extremely necessary to achieve the rapid harvesting of table grapes planted with a standard trellis in the grape industry. The design and experimental analysis of a dual-arm high-speed grape-harvesting robot were carried out to address the limitations of low picking efficiency and high grape breakage rate of multijoint robotic arms. Based on the characteristics of the harvesting environment, such as the small gap between grape clusters, standard trellis, and vertical suspension of clusters, the configuration of the dual-arm harvesting robot is reasonably designed and analyzed, and the overall configuration of the machine and the installation position of key components are derived. Robotic arm and camera view analysis of the workspace harvesting process was performed using MATLAB, and it can be concluded that the structural design of this robot meets the grape harvesting requirements with a standard trellis. To improve the harvesting efficiency, some key high-speed harvesting technologies were adopted, such as the harvesting sequence decision based on the “sequential mirroring method” of grape cluster depth information, “one-eye and dual-arm” high-speed visual servo, dual arm action sequence decision, and optimization of the “visual end effector” large tolerance combination in a natural environment. The indoor accuracy experiment shows that when the degree of obscuration of grape clusters by leaves increases, the vision algorithm based on the geometric contours of grape clusters can still meet the demands of harvesting tasks. The motion positioning average errors of the left and right robotic arms were (X: 2.885 mm, Y: 3.972 mm, Z: 2.715 mm) and (X: 2.471 mm, Y: 3.289 mm, Z: 3.775 mm), respectively, and the average dual-arm harvesting time in one grape cluster was 8.45 s. The field performance test verifies that the average harvesting cycle of the robot with both arms reached 9 s/bunch, and the success rate of bunch identification and harvesting success rate reached 88 and 83%, respectively, which were significantly better than those of existing harvesting robots worldwide.