Apple Harvesting Research Articles

Vacuum suction end-effector development for robotic harvesters of fresh market apples

Timely harvesting of fresh apples faces challenges due to labour shortage, and the modern approach of robotic harvesting has the potential to address this issue. The prevailing process of apple harvest robotics could not meet the demands of practical applications, mainly due to the lack of a suitable manipulator, because the existing ones are associated with low picking rates, fruit damage, and high costs. A prototype apple harvesting manipulator was developed, which includes a vacuum three-revolute-degrees-of-freedom end-effector, a three-prismatic-degrees-of-freedom Cartesian system, an RGB-D camera, and system integration. The vision positioning system and controller were designed to realise precise positioning and detachment of the manipulator. The major contribution of the current study is the three-revolute-degrees-of-freedom vacuum suction end-effector, whose performance evaluation was conducted in a commercial apple orchard. Experimental results showed that a 33ϕ mm diameter suction cup achieved superior performance over a 43ϕ mm cup. The method of rotation followed by pull proved to be more effective than only pulling for apple detachment. The results indicated that the apple’s equatorial region was the optimal area for suction. Furthermore, the vacuum pressure should be at least −65 kPa to guarantee successful detachment. Experimental results showed that 83.1% of harvested apples had stems intact. For the developed manipulator, a 33ϕ mm diameter suction cup, a rotate-and-pull separation method, and −65 kPa were recommended for practical applications. With the integrated new manipulator, the developed apple harvest robot has been demonstrated to have the potential to realise robotic apple harvesting.

Design of and Experiment with a Dual-Arm Apple Harvesting Robot System

Robotic harvesting has become an urgent need for the development of the apple industry, due to the sharp decline in agricultural labor. At present, harvesting apples using robots in unstructured orchard environments remains a significant challenge. This paper focuses on addressing the challenges of perception, localization, and dual-arm coordination in harvesting robots and presents a dual-arm apple harvesting robot system. First, the paper introduces the integration of the robot’s hardware and software systems, as well as the control system architecture, and describes the robot’s workflow. Secondly, combining a dual-vision perception system, the paper adopts a fruit recognition method based on a multi-task network model and a frustum-based fruit localization approach to identify and localize fruits. Finally, to improve collaboration efficiency, a multi-arm task planning method based on a genetic algorithm is used to optimize the target harvesting sequence for each arm. Field experiments were conducted in an orchard to evaluate the overall performance of the robot system. The field trials demonstrated that the robot system achieved an overall harvest success rate of 76.97%, with an average fruit picking time of 7.29 s per fruit and a fruit damage rate of only 5.56%.

Mold in, mold out: Harvest bins harbor viable inoculum that can be reduced using novel sanitation methods to manage blue mold decay of apples

Postharvest diseases account for major economic losses worldwide, while also contributing to food waste and loss. Blue mold, caused by Penicillium expansum, is one of the most prevalent postharvest diseases of pome fruit. This necrotrophic fungal pathogen is mycotoxigenic and presents a danger to food safety via the production of patulin. In addition, fungicide resistance threatens blue mold management strategies, creating a need for the development of novel control strategies. The apple storage/processing industry uses wood and plastic harvest bins to harvest and store apples for weeks, months, and even up to 1 year. Although it has been hypothesized that apple harvest bins can serve as a viable inoculum source to incite decay, there is no experimental evidence that has conclusively demonstrated this concept. In this study, we report the presence of Penicillium spp. in apple storage bins (plastic) in the mid-Atlantic USA and developed small scale methods to test 1) if harvest bins can serve as a source of inoculum for blue mold decay and 2) evaluate novel bin sanitation methods as preventative and curative applications. We demonstrated that apple harvest bin materials can serve as a source of inoculum when in direct contact with wounded apples. Additionally, it was shown that UV-C irradiation, sodium hypochlorite, and 2-phenylethanol effectively reduce inoculum viability on plastic materials in a curative fashion. It is envisioned that these findings can aid in the development of large-scale bin sanitation strategies and their applications have great potential to impact a broader range of postharvest pathogens and stored commodities.

Apple recognition in complex environments based on FC-DETR

This study aims to address the challenges of apple recognition in complex environments by proposing a solution based on the FC-DETR model. Apple cultivation is an important part of agriculture in Xinjiang, but the increasing shortage of labor has driven the demand for automated harvesting technologies. Therefore, research and development of apple recognition technology in complex environments have become crucial. While CNN architecture models can accurately identify apples in conventional settings, their inherent receptive field limitations prevent them from fully capturing global features in noisy and complex environments, making it difficult to achieve the accuracy and robustness required for practical applications. To solve this problem, this study proposes a real-time object detection model, FC-DETR. This model incorporates the innovative FEMA-BasicBlock residual module, the CAFM cross-scale adaptive feature fusion module, and the novel Inner-WIoU loss function, thereby enhancing feature processing, multi-scale feature selection and integration, and detection accuracy. Experimental results show that the FC-DETR model achieves an apple recognition accuracy of 87 % and a recall rate of 82 % in complex backgrounds while maintaining a lightweight design. This study not only makes significant advances in automated apple harvesting technology but also contributes to improving the efficiency and sustainability of the apple industry in Xinjiang and globally.

A Novel Fusion Perception Algorithm of Tree Branch/Trunk and Apple for Harvesting Robot Based on Improved YOLOv8s

Aiming to accurately identify apple targets and achieve segmentation and the extraction of branch and trunk areas of apple trees, providing visual guidance for a picking robot to actively adjust its posture to avoid branch trunks for obstacle avoidance fruit picking, the spindle-shaped fruit trees, which are widely planted in standard modern apple orchards, were focused on, and an algorithm for apple tree fruit detection and branch segmentation for picking robots was proposed based on an improved YOLOv8s model design. Firstly, image data of spindle-shaped fruit trees in modern apple orchards were collected, and annotations of object detection and pixel-level segmentation were conducted on the data. Training set data were then augmented to improve the generalization performance of the apple detection and branch segmentation algorithm. Secondly, the original YOLOv8s network architecture’s design was improved by embedding the SE module visual attention mechanism after the C2f module of the YOLOv8s Backbone network architecture. Finally, the dynamic snake convolution module was embedded into the Neck structure of the YOLOv8s network architecture to better extract feature information of different apple targets and tree branches. The experimental results showed that the proposed improved algorithm can effectively recognize apple targets in images and segment tree branches and trunks. For apple recognition, the precision was 99.6%, the recall was 96.8%, and the mAP value was 98.3%. The mAP value for branch and trunk segmentation was 81.6%. The proposed improved YOLOv8s algorithm design was compared with the original YOLOv8s, YOLOv8n, and YOLOv5s algorithms for the recognition of apple targets and segmentation of tree branches and trunks on test set images. The experimental results showed that compared with the other three algorithms, the proposed algorithm increased the mAP for apple recognition by 1.5%, 2.3%, and 6%, respectively. The mAP for tree branch and trunk segmentation was increased by 3.7%, 15.4%, and 24.4%, respectively. The proposed detection and segmentation algorithm for apple tree fruits, branches, and trunks is of great significance for ensuring the success rate of robot harvesting, which can provide technical support for the development of an intelligent apple harvesting robot.

Assessment of 'Golden Delicious' Apples Using an Electronic Nose and Machine Learning to Determine Ripening Stages.

Consumers often face a lack of information regarding the quality of apples available in supermarkets. General appearance factors, such as color, mechanical damage, or microbial attack, influence consumer decisions on whether to purchase or reject the apples. Recently, devices known as electronic noses provide an easy-to-use and non-destructive assessment of ripening stages based on Volatile Organic Compounds (VOCs) emitted by the fruit. In this study, the 'Golden Delicious' apples, stored and monitored at the ambient temperature, were analyzed in the years 2022 and 2023 to collect data from four Metal Oxide Semiconductor (MOS) sensors (MQ3, MQ135, MQ136, and MQ138). Three ripening stages (less ripe, ripe, and overripe) were identified using Principal Component Analysis (PCA) and the K-means clustering approach from various datasets based on sensor measurements in four experiments. After applying the K-Nearest Neighbors (KNN) model, the results showed successful classification of apples for specific datasets, achieving an accuracy higher than 75%. For the dataset with measurements from all experiments, an impressive accuracy of 100% was achieved on specific test sets and on the evaluation set from new, completely independent experiments. Additionally, correlation and PCA analysis showed that choosing two or three sensors can provide equally successful results. Overall, the e-nose results highlight the importance of analyzing data from several experiments performed over a longer period after the harvest of apples. There are similarities and differences in investigated VOC parameters (ethylene, esters, alcohols, and aldehydes) for less or more mature apples analyzed during autumn or spring, which can improve the determination of the ripening stage with higher predicting success for apples investigated in the spring.

Apple-Harvesting Robot Based on the YOLOv5-RACF Model.

To address the issue of automated apple harvesting in orchards, we propose a YOLOv5-RACF algorithm for identifying apples and calculating apple diameters. This algorithm employs the robot operating dystem (ROS) to control the robot's locomotion system, Lidar mapping, and navigation, as well as the robotic arm's posture and grasping operations, achieving automated apple harvesting and placement. The tests were conducted in an actual orchard environment. The algorithm model achieved an average apple detection accuracy (mAP@0.5) of 98.748% and a (mAP@0.5:0.95) of 90.02%. The time to calculate the diameter of one apple was 0.13 s, with a measurement accuracy within an error range of 1-3 mm. The robot takes an average of 9 s to pick an apple and return to the initial pose. These results demonstrate the system's efficiency and reliability in real agricultural environments.

Control of a phytopathogenic infection in an apple orchard in the forest-steppe of Samara Oblast

The study evaluates a protection system by selecting and alternating fungicides having different mechanisms of action to exclude the emergence of resistant populations of apple tree pathogens. The experiments were started at Koshelevsky Posad LLC (Syzransky District, Samara Oblast) in 2022-2023. The orchard was planted in 2013-2014, with trees arranged in a 3.5×1.5 m grid; the 62-396 rootstock was used. The biological effectiveness of fungicide protection against apple scab (Venturia inaequalis (Cooke) Wint.) and fruit rot (Monilia fructigena Pers.) was analyzed using ‘Berkutovskoye,’ ‘Kulikovskoye,’ ‘Kuibyshevskoye,’ ‘Lobo,’ and ‘Northern Sinap’ varieties. The protection system including contact and systemic agents having different mechanisms of action was developed according to the FRAC diagram for assessing the combined risk of fungicide resistance developing in phytopathogens, which takes into account the adaptation rate of organisms (emergence of new resistant types) to changing environmental conditions. An analysis of the correlation between the growth rate of apple-specific phytopathogens and weather conditions shows no significant relationship (0.032-0.245). In these studies, the anthropogenic factor-chemical treatment with fungicides – had a predominant effect on the development of scab and monilia in apple orchards along with weather factors. The yield of marketable apples in the experimental quarter of the orchard differed insignificantly from year to year. In 2023, the yield was lower; however, this fact can be probably attributed to the return of late frosts during apple blossom in Samara Oblast. Noteworthy is that for the ‘Berkutovskoye’ and ‘Lobo’ varieties, unmarketable apples did not exceed 5 and 12 %, respectively, in the studied quarter of the orchard. The protection system using fungicides (GC Shans) helped to optimize the phytosanitary condition of the orchard, thus reducing the amount of fallen fruit and providing a stable harvest of marketable apples.

Using Learning from Demonstration (LfD) to perform the complete apple harvesting task

Learning from Demonstration (LfD) can be used to make it easier for robots to learn to perform tasks such as apple harvesting. The required apple harvesting motions can be split into four steps: 1. Approaching, 2. Grasping, 3. Detaching, 4. Placing. So far, only parts of the apple harvesting task have been learned using LfD. In our work, we learned an abstracted version of the complete apple harvesting task using a cube. We used two methods and tested the sensitivity to model parameters and the number of demonstrations. We selected (1) the Gaussian Mixture Model (GMM) with Gaussian Mixture Regression (GMR) and (2) the Hidden Markov Model (HMM) with Linear Quadratic Regression (LQR). Both methods make use of Task Parameterization (TP), which allows models to combine multiple phases from the apple harvesting task. Combining multiple harvesting phases is important for effective LfD. We analysed the performance of these algorithms on four performance metrics, namely the grasp pose, the detachment motion, the place pose, and the success rate when using the real robot. A high grasp pose accuracy was achieved, specifically a position error of 0.004 and an orientation error of 0.004 for the GMM with GMR, trained with 100 demonstrations and 9 Gaussian components. The detachment motion was performed but with a reduced range. The model that performed the best was the GMM with GMR, trained with 60 demonstrations and 5 Gaussian components. A high place pose accuracy was achieved, specifically a position error of 0.003 and an orientation error of 0.003 for the GMM with GMR, trained with 100 demonstrations and 15 Gaussian components. The task was executed by a real robot successfully, resulting in a success rate of 67% for HMM with LQR, trained with 40 demonstrations, 5 states, and a scaling factor of 1.0. For the number of components, no clear relation was found. The same was true for the number of states in the HMM with LQR. For the scaling factor, a clear relation could be found. For this task, a value in the range of -1.0 to 1.0 should be used. For the number of demonstrations, there was an optimum of 40 demonstrations. Further improvements are needed to deal with challenges such as apple shift and detecting detachment of fruits.

Food translation under occupation: Apples, identity, and resistance in Indian-occupied Kashmir

ABSTRACT In 2019, Indian Occupied Jammu and Kashmir was changed from an autonomous region to a union territory by the Indian government. On August 5, 2019 India cut off all internet and phone communication into and out of Kashmir and restricted movement into and out of the country. The shutdown occurred during the height of the apple harvest, and as apples rotted on trees and the harvest was lost, signs of resistance to the shutdown were written on the apples themselves. In this article we explore how apples translate as identity and resistance in the postcolonial context of Indian Occupied Kashmir. Using the communication blackout and ongoing occupation of Kashmir as a focal point, we examine how narratives about apples can be translated as a cultural discourse of agency, even as structured spaces for and means of resistance are increasingly eliminated by the Indian state. Central to our analysis are theoretical frames of translation, postcoloniality, and food system sovereignty and communication. We utilize dialogical narrative analysis as methodology to understand how the meanings of apples, a mundane but central part of everyday life for many, have communicated resistance, identity and agency during the last several years since the shutdown.

3D Camera and Single-Point Laser Sensor Integration for Apple Localization in Spindle-Type Orchard Systems.

Accurate localization of apples is the key factor that determines a successful harvesting cycle in the automation of apple harvesting for unmanned operations. In this regard, accurate depth sensing or positional information of apples is required for harvesting apples based on robotic systems, which is challenging in outdoor environments because of uneven light variations when using 3D cameras for the localization of apples. Therefore, this research attempted to overcome the effect of light variations for the 3D cameras during outdoor apple harvesting operations. Thus, integrated single-point laser sensors for the localization of apples using a state-of-the-art model, the EfficientDet object detection algorithm with an mAP@0.5 of 0.775 were used in this study. In the experiments, a RealSense D455f RGB-D camera was integrated with a single-point laser ranging sensor utilized to obtain precise apple localization coordinates for implementation in a harvesting robot. The single-point laser range sensor was attached to two servo motors capable of moving the center position of the detected apples based on the detection ID generated by the DeepSORT (online real-time tracking) algorithm. The experiments were conducted under indoor and outdoor conditions in a spindle-type apple orchard artificial architecture by mounting the combined sensor system behind a four-wheel tractor. The localization coordinates were compared between the RGB-D camera depth values and the combined sensor system under different light conditions. The results show that the root-mean-square error (RMSE) values of the RGB-D camera depth and integrated sensor mechanism varied from 3.91 to 8.36 cm and from 1.62 to 2.13 cm under 476~600 lx to 1023~1100 × 100 lx light conditions, respectively. The integrated sensor system can be used for an apple harvesting robotic manipulator with a positional accuracy of ±2 cm, except for some apples that were occluded due to leaves and branches. Further research will be carried out using changes in the position of the integrated system for recognition of the affected apples for harvesting operations.

Apple mechanical damage mechanism and harvesting test platform design

Abstract Apple is easily damaged in the process of the mechanical harvesting, which reduces the fruit’s quality. It is of great significance to study the damage principle of apple in the transport process of picking platform. In this study, the apple compression test was carried out. The compression and drop process of the fruit was analyzed by the finite element analysis (FEA). The experimental platform of apple harvesting was designed, the conveying process of apple was analyzed. The results of compression finite element analysis showed that when the compression force is greater than 15.0 N, both radial compression and axial compression will be damaged. The results of drop finite element analysis showed that when the drop direction is axial, the maximum contact stress of the peel and kernel decreases with the increase of drop angle, the maximum contact stress of the pulp increases first and then decreases. When the drop direction is radial, the maximum contact stress between pulp and kernel decreases with the increase of drop angle, the maximum contact stress of the peel first decreases and then increases. The simulation results of the harvesting platform transportation showed that the damage rate of apples is less than 10 % when the sub-conveyor belt speed is 0.02 m–0.04 m/s. This study can provide theoretical guidance for the design of the harvesting test platform and the reduction of the damage of apples during transportation.

Detection of Orchard Apples Using Improved YOLOv5s-GBR Model

The key technology of automated apple harvesting is detecting apples quickly and accurately. The traditional detection methods of apple detection are often slow and inaccurate in unstructured orchards. Therefore, this article proposes an improved YOLOv5s-GBR model for orchard apple detection under complex natural conditions. First, the researchers collected photos of apples in their natural environments from different angles; then, we enhanced the dataset by changing the brightness, rotating the images, and adding noise. In the YOLOv5s network, the following modules were introduced to improve its performance: First, the YOLOv5s model’s backbone network was swapped out for the GhostNetV2 module. The goal of this improvement was to lessen the computational burden on the YOLOv5s algorithm while increasing the detection speed. Second, the bi-level routing spatial attention module (BRSAM), which combines spatial attention (SA) with bi-level routing attention (BRA), was used in this study. By strengthening the model’s capacity to extract important characteristics from the target, its generality and robustness were enhanced. Lastly, this research replaced the original bounding box loss function with a repulsion loss function to detect overlapping targets. This model performs better in detection, especially in situations involving occluded and overlapping targets. According to the test results, the YOLOv5s-GBR model improved the average precision by 4.1% and recall by 4.0% compared to those of the original YOLOv5s model, with an impressive detection accuracy of 98.20% at a frame rate of only 101.2 fps. The improved algorithm increases the recognition accuracy by 12.7%, 10.6%, 5.9%, 2.7%, 1.9%, 0.8%, 2.6%, and 5.3% compared to those of YOLOv5-lite-s, YOLOv5-lite-e, yolov4-tiny, YOLOv5m, YOLOv5l, YOLOv8s, Faster R-CNN, and SSD, respectively, and the YOLOv5s-GBR model can be used to accurately recognize overlapping or occluded apples, which can be subsequently deployed in picked robots to meet the realistic demand of real-time apple detection.

An Apple Detection and Localization Method for Automated Harvesting under Adverse Light Conditions

The use of automation technology in agriculture has become particularly important as global agriculture is challenged by labor shortages and efficiency gains. The automated process for harvesting apples, an important agricultural product, relies on efficient and accurate detection and localization technology to ensure the quality and quantity of production. Adverse lighting conditions can significantly reduce the accuracy of fruit detection and localization in automated apple harvesting. Based on deep-learning techniques, this study aims to develop an accurate fruit detection and localization method under adverse light conditions. This paper explores the LE-YOLO model for accurate and robust apple detection and localization. The traditional YOLOv5 network was enhanced by adding an image enhancement module and an attention mechanism. Additionally, the loss function was improved to enhance detection performance. Secondly, the enhanced network was integrated with a binocular camera to achieve precise apple localization even under adverse lighting conditions. This was accomplished by calculating the 3D coordinates of feature points using the binocular localization principle. Finally, detection and localization experiments were conducted on the established dataset of apples under adverse lighting conditions. The experimental results indicate that LE-YOLO achieves higher accuracy in detection and localization compared to other target detection models. This demonstrates that LE-YOLO is more competitive in apple detection and localization under adverse light conditions. Compared to traditional manual and general automated harvesting, our method enables automated work under various adverse light conditions, significantly improving harvesting efficiency, reducing labor costs, and providing a feasible solution for automation in the field of apple harvesting.

Occluded apples orientation estimator based on deep learning model for robotic harvesting

Apple harvesting robot is in rapid development in the recent years due to the shortage of manual labor. Robotics vision system has been one of the main focus in the development to achieve successful grasping. Information such as the apple orientation is valuable for developing more effective fruit detachment strategies. Occlusions in the orchard environment and the lack of distinct features in an apple has made it challenging to identify the apple orientation. Our study presents a novel approach that aims to estimate the orientation of fruits by projecting 2D information onto a 3D space. This method specifically targets the field of robotic harvesting and leverages multiple well-established and reliable fundamental methods currently available that includes keypoint detection neural network and circle detection based on extracted line segments of occluded apple. Based on the extracted information, a unit vector that represents orientation of the apple can be computed and transformed according to the relative position between the apple and the camera. The proposed method is evaluated using a publicly available 3D point cloud data of 11 apple trees from an orchard farm that was additionally labeled manually with apple orientations. To verify the evaluation results, the proposed method is tested by integrating it into an apple harvesting robot to perform inspection. Results shows that the median angular error in the orchard setting is 17.6° whilst in the lab setting is 14.6°. The experimental results show that the performance of the proposed method is comparable to existing research when under heavy occlusion and thus it is suitable for harvesting robot operating in the orchard farm.

A New Picking Pattern of a Flexible Three-Fingered End-Effector for Apple Harvesting Robot

During the picking process of the apple harvesting robot, the attitude of the end effector holding the apple and the movement method of separating the apple directly affect the success rate of picking. In order to improve the stability of the picking process, reduce the gripping force, and avoid apple dislodgement and damage, this work studies the new apple-picking pattern of the flexible three-fingered end-effector based on the analysis of the existing apple-picking pattern. First, two new three-finger grasping postures for wrapping the apple horizontally and vertically on the inside of the fingers are proposed, and a new method of separating the stem with a circular-pull-down motion of the end-effector picking the apple is designed. Then, the pressure on the apple under different picking patterns was analyzed, and a branch–stem–apple simulation model was established. Combining the constraint conditions such as the angle between the apple stem and the vertical direction, the movement speed, the root impulse, and so on, the optimal angle of apple circular movement and the force required to realize the movement are obtained through dynamic simulation experiments. Finally, the experiments of apple picking patterns were carried out with the flexible three-fingered end-effector. The experiment shows that the best angle for apple picking is 15°~20° using the circular-pull-down movement separation method. In terms of average grasping force peaks and pressures, the combination of the vertical holding posture of the inner finger and the circular-pull-down movement separation method is the best picking pattern. In this pattern, the average peak exerts force on the inner side of a single finger is about 8.52 N, and the pressure is about 20.9 KPa.

Fruit recognition, task plan, and control for apple harvesting robots

Fruit recognition, task plan, and control for apple harvesting robots

基于改进遗传算法的苹果收获机器人三维路径规划

In recent years, the problem of “rural labor shortage” in China has become increasingly serious, with a large number of young laborers going out to work, leading to an increasing amount of idle rural land. The intensification of population aging and the reduction of agricultural labor force in China resulted in an urgent demand for agricultural robots. With the rapid development of agricultural machinery and automation technology, agricultural robots have been continuously developing. They can better adapt to the development of biotechnology in agriculture, and traditional harvesting methods may undergo significant changes, with an increased focus on the cultivation of crops. Therefore, this paper introduces a new encoding scheme on the basis of traditional genetic algorithm (GA) and proposes an improved double encoding GA. This new encoding scheme is used on the crossover link, whereas the path node sequence encoding scheme is still used on the mutation link. The selection operation is placed after the mutation, and the merging sorting and elitist selection are performed on the parent population, crossover population, and mutation population before selection, thereby accelerating the convergence speed. On the basis of the improved GA, the three-dimensional path of the apple harvesting robot is designed and planned, with the addition of adaptive adjustment function during the progress. The experimental simulation results show that the three-dimensional path planning of the apple harvesting mobile robot based on the improved GA can minimize the number of paths and loops and well meet the operational requirements of the harvesting robot.

An improved YOLOv5s model for assessing apple graspability in automated harvesting scene.

With continuously increasing labor costs, an urgent need for automated apple- Qpicking equipment has emerged in the agricultural sector. Prior to apple harvesting, it is imperative that the equipment not only accurately locates the apples, but also discerns the graspability of the fruit. While numerous studies on apple detection have been conducted, the challenges related to determining apple graspability remain unresolved. This study introduces a method for detecting multi-occluded apples based on an enhanced YOLOv5s model, with the aim of identifying the type of apple occlusion in complex orchard environments and determining apple graspability. Using bootstrap your own atent(BYOL) and knowledge transfer(KT) strategies, we effectively enhance the classification accuracy for multi-occluded apples while reducing data production costs. A selective kernel (SK) module is also incorporated, enabling the network model to more precisely identify various apple occlusion types. To evaluate the performance of our network model, we define three key metrics: APGA, APTUGA, and APUGA, representing the average detection accuracy for graspable, temporarily ungraspable, and ungraspable apples, respectively. Experimental results indicate that the improved YOLOv5s model performs exceptionally well, achieving detection accuracies of 94.78%, 93.86%, and 94.98% for APGA, APTUGA, and APUGA, respectively. Compared to current lightweight network models such as YOLOX-s and YOLOv7s, our proposed method demonstrates significant advantages across multiple evaluation metrics. In future research, we intend to integrate fruit posture and occlusion detection to f]urther enhance the visual perception capabilities of apple-picking equipment.

가시성을 표시한 사과 검출 데이터셋과 적응형 히트맵 회귀를 이용한 딥러닝 검출

In the fruit harvesting field, interest in automatic robot harvesting is increasing due to various seasonality and rising harvesting costs. Accurate apple detection is a difficult problem in complex orchard environments with changes in light, vibrations caused by wind, and occlusion of leaves and branches. In this paper, we introduce a dataset and an adaptive heatmap regression model that are advantageous for robot automatic apple harvesting. The apple dataset was labeled with not only the apple location but also the visibility. We propose a method to detect the center point of an apple using an adaptive heatmap regression model that adjusts the Gaussian shape according to visibility. The experimental results showed that the performance of the proposed method was applicable to apple harvesting robots, with MAP@K of 0.9809 and 0.9801 when K=5 and K=10, respectively.