Fruit-picking Robots Research Articles

Polynomial-based smooth trajectory planning for fruit-picking robot manipulator

Fruit-picking robot is an important form of intelligent agricultural machinery and has great impact in smart agriculture. Motion planning for its multi-Degree of Freedoms (DOFs) manipulator is significant to picking efficiency and fruit quality. However, it is difficult to plan appropriate trajectory for such a complex system. The purpose of this article is to generate smooth trajectories for fruit-picking robot manipulators using shortcuts that are constrained in velocity, acceleration and jerk. The proposed algorithm smooths the robot motion as a post process to trajectory planning. Given an input jerky path, the algorithm will first convert it into a fully synchronized trajectory, and then attempt to reduce the execution time of this trajectory as much as possible while retaining the kinematic motion constraints. The idea of the proposed method is to build shorter and collision-free shortcuts that replace the intermediate motion between two randomly-picked points on the trajectory. Experiments carried on a mobile fruit-picking robot demonstrate that this technique is able to generate smooth and collision-free trajectories for robot manipulators.

3D global mapping of large-scale unstructured orchard integrating eye-in-hand stereo vision and SLAM

Large-scale, high-accuracy, and adaptive three-dimensional (3D) perception are the basic technical requirements for constructing a practical and stable fruit-picking robot. The latest vision-based fruit-picking robots have been able to adapt to the complex background, uneven lighting and low color contrast of the orchard environment. However, most of them have, until now, been limited to a small field of view or rigid sampling manners. Although the simultaneous localization and mapping (SLAM) methods have the potential to realize large scale sensing, it was critically revealed in this study that the classic SLAM pipeline is not completely adapted to orchard picking tasks. In this study, the eye-in-hand stereo vision and SLAM system were integrated to provide detailed global map supporting long-term, flexible and large-scale orchard picking. To be specific, a mobile robot based on eye-in-hand vision was built and an effective hand-eye calibration method was proposed; a state-of-the-art object detection network was trained and used to establish a dynamic stereo matching method adapted well to complex orchard environments; a SLAM system was deployed and combined with the above eye-in-hand stereo vision system to obtain a detailed, wide 3D orchard map. The main contribution of this work is to build a new global mapping framework compatible to the nature of orchard picking tasks. Compared with the existing studies, this work pays more attention to the structural details of the orchard. Experimental results indicated that the constructed global map achieved both large-scale and high-resolution. This is an exploratory work providing theoretical and technical references for the future research on more stable, accurate and practical mobile fruit picking robots.

Research on the Application of Rigid-flexible Compound Driven Fruit Picking Robot Design in Realizing Fruit Picking

This article mainly designs an efficient fruit picking robot that combines rigidity and flexibility. The design of the robot adopts a mechanical arm with 5 degrees of freedom for the mecanum wheel. The mechanical arm is equipped with a gripper made using artificial muscle technology. The ZED2 binocular vision system is used to obtain spatial coordinates to realize the positioning and positioning of the mechanical arm. Video image acquisition. In order to realize the automatic recognition and picking of strawberries, this paper creates a strawberry training set, and uses the yolov3 algorithm to train the data and obtains a strawberry recognition model, which has a recall rate of larger strawberries and immature strawberries and the precision rate reached 90%, and the detection precision and precision rate of small target strawberries reached more than 80%. It can Meet the requirements of picking in real life.

水果采摘机器人采摘装置机研究现状

水果采摘机器人采摘装置机研究现状

Virtual Simulation of Fruit Picking Robot Based on Unity3D

With the development of the fruit farming industry, there have been breakthroughs in both scale and harvesting requirements, and the resulting problem is that the demand of picking robot is more and more high, the function, cost and quality of the harvest and picking efficiency compared with the traditional manual operation mode with strong competitiveness for market, that is to say, in terms of cost cheaper, picking at a faster rate, and can avoid damage on the fruit in the process of picking. Therefore, in terms of mechanical structure design, the transmission accuracy and efficiency of the picking robot should be improved. At the same time, the structural design of the picking robot should simplify the structure as much as possible, reduce the manufacturing cost, and ensure the feasibility of the picking robot’s functions. Picking robot combined with automatic walking system, automatic detection system and control system-centered intelligent system, can achieve accurate and efficient work in various conditions. Realize picking robot toward automation, intelligence, scientific development. The design of this topic is mainly aimed at the development of the picking robot and the use of analysis, determine the feasibility of the picking robot design and virtual simulation.

Optimization design of compliant constant-force mechanism for apple picking actuator

Low-destructive picking method is part of the key technologies for fruit-picking robots. To reduce the picking-induced fruit bruising during the clamping process, this paper realizes the constant output force required for low-destructive fruit clamping by setting the compliant mechanism on the end-effector and optimizing the mechanical characteristics of the compliant mechanism. Firstly, based on the shape function, the nonlinear ordinary differential equations of buckling deformation of compliant beams are established with boundary conditions. Secondly, the above boundary value problem is re-described as the initial value problem by using the shooting method, and the initial value optimization solution is combined with the genetic algorithm. Then, the sequence quadratic programming method is implemented to optimize the shape function of the beam to achieve constant force output within a certain deformation range. Taking apple picking as an example, the initial shape parameters of the compliant mechanism are set and the constant clamping force is obtained at about 7.9N. The above method is verified by the nonlinear finite element simulation, the force-displacement experiment and the apple clamping test. The compliant mechanism can provide the required grasping force and the successful grasping rate is about 95.3%. By properly adjusting the parameters of the compliant beam, the algorithm can also meet the requirements of constant force clamping for different types of fruit. The research provides a reference for the effective application of the compliant mechanism in low-destructive fruit-picking.

Detection of passion fruits and maturity classification using Red-Green-Blue Depth images

A machine vision algorithm was developed to detect passion fruits and identify maturity of the detected fruits using natural outdoor RGB-D images. As different passion fruits on the same branch can be in different maturity stages, detection and maturity classification on a complex background are very important for yield mapping and development of intelligent mobile fruit-picking robots. In this study, a Kinect sensor was used for data acquisition, and maturity stages of the fruits were divided into five categories: young (Y), near-young (NY), near-mature (NM), mature (M) and after-mature (AM). The algorithm involved two stages. First, by colour and depth images, passion fruits were detected using faster region-based convolutional neural networks (Faster R-CNN), and colour-based detection was integrated with depth-based detection for improving detection performance. Second, for each detected fruit region, the dense scale invariant features transform (DSIFT) algorithm combined with locality-constrained linear coding (LLC) was used to extract and represent the features of fruit maturity from R, G, and B channels, respectively. In addition, the RGB-DSIFT-LLC features were input into a linear support vector machine (SVM) classifier for identifying the maturity of fruits. By conducting an experimental study on a special dataset, we verified that the proposed method achieves 92.71% detection accuracy and 91.52% maturity classification accuracy.

Research progress on visual perception and end-effecter of undamaged operation for fruit-picking robot

Research progress on visual perception and end-effecter of undamaged operation for fruit-picking robot

Fault-Tolerant Design of a Limited Universal Fruit-Picking End-Effector Basedon Vision-Positioning Error

Difficulties lie in the study and application of fruit-picking robots. In particular, large random positioning errors can occur due to disturbance, which is difficult to compensate for using control methods. Existing end-effectors cannot conduct fault tolerance for these errors and are not widely applied. Therefore, a limited universal and fault-tolerant end-effector was designed to address binocular vision-positioning errors. The theory underlying the design of the new mechanism was also proposed based on institutions and vision positioning, in which random positioning errors are regarded as systematic “fault errors” of the end-effector, enabling the mechanism to complete operations within the error range. The relationship between the gripper and the cutter was modeled, and a mathematical model of error tolerance was established. Moreover, a new limited universal end-effector was designed. The binocular vision-positioning errors (including original and random positioning errors) were analyzed, and static and dynamic positioning experiments were conducted using a mechanism and a vision-positioning experimental platform based on binocular vision. The maximum positioning errors obtained were 60.1 mm in the z-direction and 17.2 mm in the x-direction, which were within the fault-tolerance range. Moreover, indoor and outdoor picking experiments were conducted for litchi and citrus using the picking manipulator based on binocular vision. The picking success rates were over 84% and 78% in indoor and outdoor tests, respectively. Finally, the favorable fault-tolerant capacity of the end-effector was validated with a comparison experiment that showed that the limited universal picking manipulator based on binocular vision could achieve litchi and citrus pickings within an acceptable error range. The results verified the applicability of the fault-tolerant design.

Quick Motion Path Plan for Joint-Robot to Pick Tomato under Free-Obstacle

To improve the efficiency of picking the fruit, a kind of quick motion path plan method for fruit-picking robot under free-obstacle was proposed in this paper. Firstly, the method obtained a series of points along straight path at equal internal, and the gripper center was designed to pass thorough these points. Inverse kinematics formulas of the robot arm were used to solve joint angles of the robot arm when the gripper center will pass thorough each point. To make the robot arm guide the gripper to reach quickly the object point, the joint angles were optimized to determinate according to the principle of energy optimization. The test of picking tomato showed that the method can both reproduce the motion of the gripper along linear points at constant speed and keep the shortest motion path, which are benefit to grip the fruit steadily. The method has a low amount of calculation, better real-time and may provide a reference for joint robot to pick the fruit quickly.

Remote Sensing Technique for Predicting Harvest Time of Tomatoes

Fast determination of growing stages and harvest time of fruits and vegetables is necessary to implement robotic operation for horticulture automation. This study evaluates the feasibility of using visible-near-infrared (Vis-NIR) spectroscopy to nondestructively determine the harvest time of tomatoes. A mobile, fibre-type, AgroSpec VIS-NIR spectrophotometer (Tec5, Germany) with a spectral range of 350-2200 nm, was used for spectral acquisition of tomatoes in reflection mode. A new index was used to measure the growing stages of tomatoes. Tomato plants were provided by Silsoe Horticultural Center, Bedfordshire, United Kingdom. Spectra were divided into a calibration set (70%) and an independent validation set (30%). Calibration set were subjected to a partial least squares regression (PLSR) with leave-one-out cross validation to establish calibration models respectively based on different spectral ranges, e.g., VIS(400-760 nm), NIR(760-2100 nm) and VIS-NIR(400-2100 nm). Prediction performance of these models on the independent validation set indicates that PLSR models based on entire spectral range (VIS-NIR) outperform those based on partial spectral ranges (VIS or NIR). Coupled with appropriate spectral transformation, the PLSR models can achieve excellent prediction performance of harvest time of tomatoes with coefficient of determination (R2) of 0.89 and RPD of 3.00. It is concluded that VIS-NIR spectroscopy combined with optimized PLSR models for GS prediction can be successfully adopted as a remote sensing technique for predicting harvest time of tomatoes, which allows for implementing autonomous fruit-picking robots.

On-line near minimum-time path planning and control of an industrial robot for picking fruits

To be competitive, an industrial robot picking fruits must be able to perform this task in an amount of time which compares to that needed by humans. Because the location of a fruit changes due to the picking of others, the determination of their location has to be performed on-line and also the associated path planning for the robot. This poses two major problems in the development of fruit-picking robots since locating fruits and path planning, in general, are computationally expensive operations. This paper contributes to relaxing the second problem. Using the fact that the motions of the links of an industrial robot are approximately decoupled, this paper proposes a new method for near minimum-time path planning and control in the presence of obstacles. This method is computationally cheap compared to methods that solve the more general problem. Experimental results are presented which indicate the feasibility of this approach to make the robot competitive.

Near-minimum-time task planning for fruit-picking robots

A near-minimum-time task-planning algorithm for fruit-harvesting robots having to pick fruits at N given locations is presented. For the given kinematic and inertial parameters of the manipulator, the algorithm determines the near-optimal sequence of fruit locations through which the arm should pass and finds the near-minimum-time path between these points. The sequence of motions was obtained by solving the traveling salesman problem (TSP) using the distance along the geodesics in the manipulator's inertia space, between every two fruit locations, as the cost to be minimized. The proposed algorithm was applied to define the motions of a citrus-picking robot and was tested for a cylindrical robot on fruit position data collected from 20 trees. Significant reduction in the required computing time was achieved by dividing the volume containing the fruits into subvolumes and estimating the geodesic distance rather than calculating it.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A fruit-tracking system for robotic harvesting

The fruit-tracking system described in this article estimated in real-time (60 Hz) the size and position of a valid fruit region in color images. This information was used to control the motion of a fruit-picking robot. Statistical classification of color pixels was employed to improve image processing under the variety of imaging conditions encountered in a grove. Manual specification of valid fruit colors was possible through interactive training sessions. Object-oriented aperture control allowed adjustment of image exposure based on the illumination level of a targeted fruit. This feature increased the dynamic illumination range of the tracking system by ignoring excessively dark or bright background conditions. The performance of the fruit-tracking system was evaluated with off-line tests and under actual operating conditions.