Sweet Pepper Harvesting Robot Research Articles

Performance improvements of a sweet pepper harvesting robot in protected cropping environments

AbstractUsing robots to harvest sweet peppers in protected cropping environments has remained unsolved despite considerable effort by the research community over several decades. In this paper, we present the robotic harvester, Harvey, designed for sweet peppers in protected cropping environments that achieved a 76.5% success rate on 68 fruit (within a modified scenario) which improves upon our prior work which achieved 58% on 24 fruit and related sweet pepper harvesting work which achieved 33% on 39 fruit (for their best tool in a modified scenario). This improvement was primarily achieved through the introduction of a novel peduncle segmentation system using an efficient deep convolutional neural network, in conjunction with three‐dimensional postfiltering to detect the critical cutting location. We benchmark the peduncle segmentation against prior art demonstrating an improvement in performance with a score of 0.564 compared to 0.302. The robotic harvester uses a perception pipeline to detect a target sweet pepper and an appropriate grasp and cutting pose used to determine the trajectory of a multimodal harvesting tool to grasp the sweet pepper and cut it from the plant. A novel decoupling mechanism enables the gripping and cutting operations to be performed independently. We perform an in‐depth analysis of the full robotic harvesting system to highlight bottlenecks and failure points that future work could address.

Planning the sequence of tasks for harvesting robots

A methodology for planning the sequence of tasks for a harvesting robot is presented. The fruit targets are situated at unknown locations and must be detected by the robot through a sequence of sensing tasks. Once the targets are detected, the robot must execute a harvest action at each target location. The traveling salesman paradigm (TSP) is used to plan the sequence of sensing and harvesting tasks taking into account the costs of the sensing and harvesting actions and the traveling times. Sensing is planned online. The methodology is validated and evaluated in both laboratory and greenhouse conditions for a case study of a sweet pepper harvesting robot. The results indicate that planning the sequence of tasks for a sweet pepper harvesting robot results in 12% cost reduction. Incorporating the sensing operation in the planning sequence for fruit harvesting is a new approach in fruit harvesting robots and is important for cycle time reduction. Furthermore, the sequence is re-planned as sensory information becomes available and the costs of these new sensing operations are also considered in the planning.

Development of a sweet pepper harvesting robot

AbstractThis paper presents the development, testing and validation of SWEEPER, a robot for harvesting sweet pepper fruit in greenhouses. The robotic system includes a six degrees of freedom industrial arm equipped with a specially designed end effector, RGB‐D camera, high‐end computer with graphics processing unit, programmable logic controllers, other electronic equipment, and a small container to store harvested fruit. All is mounted on a cart that autonomously drives on pipe rails and concrete floor in the end‐user environment. The overall operation of the harvesting robot is described along with details of the algorithms for fruit detection and localization, grasp pose estimation, and motion control. The main contributions of this paper are the integrated system design and its validation and extensive field testing in a commercial greenhouse for different varieties and growing conditions. A total of 262 fruits were involved in a 4‐week long testing period. The average cycle time to harvest a fruit was 24 s. Logistics took approximately 50% of this time (7.8 s for discharge of fruit and 4.7 s for platform movements). Laboratory experiments have proven that the cycle time can be reduced to 15 s by running the robot manipulator at a higher speed. The harvest success rates were 61% for the best fit crop conditions and 18% in current crop conditions. This reveals the importance of finding the best fit crop conditions and crop varieties for successful robotic harvesting. The SWEEPER robot is the first sweet pepper harvesting robot to demonstrate this kind of performance in a commercial greenhouse.

A Vision Servo System for Automated Harvest of Sweet Pepper in Korean Greenhouse Environment

Recently, farmers of sweet pepper suffer from the increase of its unit production costs due to the rise of labor costs. The rise of unit production costs of sweet pepper, on the other hand, decreases its productivity and causes the lack of its farming expertise, thus resulting in the quality degradation of products. In this regard, it is necessary to introduce an automated robot harvest system into the farming of sweet pepper. In this study, the authors developed an image-based closed-loop control system (a vision servo system) and an automated sweet pepper harvesting robot system and then carried out experiments to verify its efficiency. The working area of the manipulator that detects products through an imaging sensor in the farming environment of sweet pepper, decides whether to harvest it or not, and then informs the location of the product to the control center, which is set up at the distance scope of 350~600 mm from the center of the system and 1000 mm vertically. In order to confirm the performance of the sweet pepper recognition in this study, 269 sweet pepper images were used to extract fruits. Of 269 sweet pepper images, 82.16% were recognized successfully. The harvesting experiment of the system developed in this study was carried out with 100 sweet peppers. The result of experiment with 100 sweet peppers presents the fact that its approach rate to peduncle is about 86.7%, and via four sessions of repetitive harvest experiment it achieves a maximal 70% harvest rate, and its average time of harvest is 51.1 s.

Adaptive Image Thresholding of Yellow Peppers for a Harvesting Robot

The presented work is part of the H2020 project SWEEPER with the overall goal to develop a sweet pepper harvesting robot for use in greenhouses. As part of the solution, visual servoing is used to direct the manipulator towards the fruit. This requires accurate and stable fruit detection based on video images. To segment an image into background and foreground, thresholding techniques are commonly used. The varying illumination conditions in the unstructured greenhouse environment often cause shadows and overexposure. Furthermore, the color of the fruits to be harvested varies over the season. All this makes it sub-optimal to use fixed pre-selected thresholds. In this paper we suggest an adaptive image-dependent thresholding method. A variant of reinforcement learning (RL) is used with a reward function that computes the similarity between the segmented image and the labeled image to give feedback for action selection. The RL-based approach requires less computational resources than exhaustive search, which is used as a benchmark, and results in higher performance compared to a Lipschitzian based optimization approach. The proposed method also requires fewer labeled images compared to other methods. Several exploration-exploitation strategies are compared, and the results indicate that the Decaying Epsilon-Greedy algorithm gives highest performance for this task. The highest performance with the Epsilon-Greedy algorithm ( ϵ = 0.7) reached 87% of the performance achieved by exhaustive search, with 50% fewer iterations than the benchmark. The performance increased to 91.5% using Decaying Epsilon-Greedy algorithm, with 73% less number of iterations than the benchmark.

Design, Modeling and Performance Testing of End-Effector for Sweet Pepper Harvesting Robot Hand

This paper presents a new design and modeling fundamentals of gripper and cutting system for 5 degree of freedom robotic arm, designed to harvest sweet peppers in horticultural green house. The design consists of two parallel jaws mounted on gears and operated with the help of servo motor. The same servo motor was used to operate the cutting system which was composed of scissors. The complete system was designed to operate by using one servo motor only. The system model was developed in SolidWorks and tested for different kinematic and dynamic performances. The performance of the gripper and cutting tool system has been evaluated through simulation to determine the design parameters of practical prototype. Based on the design concept, practical prototype of the gripper and cutting system was developed by considering the results obtained by model developed in SolidWorks. The developed prototype was tested to verify the feasibility and reliability of the model developed in Solid-Works. The performance and practical application of the developed prototype was verified and validated by conducting experiments in the lab and greenhouse and comparing the results with simulation results.

Performance Testing of Thermal Cutting Systems for Sweet Pepper Harvesting Robot in Greenhouse Horticulture

This paper proposes design of end-effector and prototype of thermal cutting system for harvesting sweet peppers. The design consists of two parallel gripper bars mounted on a frame connected by specially designed notch plate and operated by servo motor. Based on voltage and current, two different types of thermal cutting system prototypes; electric arc and temperature arc respectively were developed and tested for performance. In electric arc, a special electric device was developed to obtain high voltage to perform cutting operation. At higher voltage, electrodes generate thermal arc which helps to cut stem of sweet pepper. In temperature arc, nichrome wire was mounted between two electrodes and current was provided directly to electrodes which results in generation of high temperature arc between two electrodes that help to perform cutting operation. In both prototypes, diameters of basic elements were varied and the effect of this variation on cutting operation was investigated. The temperature arc thermal system was found significantly suitable for cutting operation than electric arc thermal system. In temperature arc thermal cutting system, 0.5 mm nichrome wire shows significant results by accomplishing harvesting operation in 1.5 seconds. Also, thermal cutting system found suitable to increase shelf life of fruits by avoiding virus and fungal transformation during cutting process and sealing the fruit stem. The harvested sweet peppers by thermal cutting system can be preserved at normal room temperature for more than 15 days without any contamination.

1A1-G02 Development of Thermal Cutting System for Sweet Pepper Harvesting Robot in Greenhouse Horticulture(Robotics and Mechatronics in Agriculture)

This paper proposes the new design of thermal cutting system for 5 degree of freedom robotic arm, designed to harvest sweet peppers in horticultural green house. The design consists of two parallel gripper bars mounted on a frame connected by specially designed notch plate and operated by servo motor. The special electric device was developed to obtain the high frequencies with high voltage to perform the cutting operation. The output from the device was connected to the electric rods mounted at the front of the specially designed notch plate. At higher frequencies, the rods generate thermal arc which helps to cut the stem of the sweet pepper. The recognition system assists to locate the fruit.