Fruit Detection Research Articles

Green Fruit Detection with a Small Dataset under a Similar Color Background Based on the Improved YOLOv5-AT.

Green fruit detection is of great significance for estimating orchard yield and the allocation of water and fertilizer. However, due to the similar colors of green fruit and the background of images, the complexity of backgrounds and the difficulty in collecting green fruit datasets, there is currently no accurate and convenient green fruit detection method available for small datasets. The YOLO object detection model, a representative of the single-stage detection framework, has the advantages of a flexible structure, fast inference speed and excellent versatility. In this study, we proposed a model based on the improved YOLOv5 model that combined data augmentation methods to detect green fruit in a small dataset with a background of similar color. In the improved YOLOv5 model (YOLOv5-AT), a Conv-AT block and SA and CA blocks were designed to construct feature information from different perspectives and improve the accuracy by conveying local key information to the deeper layer. The proposed method was applied to green oranges, green tomatoes and green persimmons, and the mAPs were higher than those of other YOLO object detection models, reaching 84.6%, 98.0% and 85.1%, respectively. Furthermore, taking green oranges as an example, a mAP of 82.2% was obtained on the basis of retaining 50% of the original dataset (163 images), which was only 2.4% lower than that obtained when using 100% of the dataset (326 images) for training. Thus, the YOLOv5-AT model combined with data augmentation methods can effectively achieve accurate detection in small green fruit datasets under a similar color background. These research results could provide supportive data for improving the efficiency of agricultural production.

Optimization of Online Soluble Solids Content Detection Models for Apple Whole Fruit with Different Mode Spectra Combined with Spectral Correction and Model Fusion.

Soluble solids content (SSC) is one of the main quality indicators of apples, and it is important to improve the precision of online SSC detection of whole apple fruit. Therefore, the spectral pre-processing method of spectral-to-spectral ratio (S/S), as well as multiple characteristic wavelength member model fusion (MCMF) and characteristic wavelength and non-characteristic wavelength member model fusion (CNCMF) methods, were proposed for improving the detection performance of apple whole fruit SSC by diffuse reflection (DR), diffuse transmission (DT) and full transmission (FT) spectra. The modeling analysis showed that the S/S- partial least squares regression models for all three mode spectra had high prediction performance. After competitive adaptive reweighted sampling characteristic wavelength screening, the prediction performance of all three model spectra was improved. The particle swarm optimization-extreme learning machine models of MCMF and CNCMF had the most significant enhancement effect and could make all three mode spectra have high prediction performance. DR, DT, and FT spectra all had some prediction ability for apple whole fruit SSC, with FT spectra having the strongest prediction ability, followed by DT spectra. This study is of great significance and value for improving the accuracy of the online detection model of apple whole fruit SSC.

Intelligent Integrated System for Fruit Detection Using Multi-UAV Imaging and Deep Learning.

In the context of Industry 4.0, one of the most significant challenges is enhancing efficiency in sectors like agriculture by using intelligent sensors and advanced computing. Specifically, the task of fruit detection and counting in orchards represents a complex issue that is crucial for efficient orchard management and harvest preparation. Traditional techniques often fail to provide the timely and precise data necessary for these tasks. With the agricultural sector increasingly relying on technological advancements, the integration of innovative solutions is essential. This study presents a novel approach that combines artificial intelligence (AI), deep learning (DL), and unmanned aerial vehicles (UAVs). The proposed approach demonstrates superior real-time capabilities in fruit detection and counting, utilizing a combination of AI techniques and multi-UAV systems. The core innovation of this approach is its ability to simultaneously capture and synchronize video frames from multiple UAV cameras, converting them into a cohesive data structure and, ultimately, a continuous image. This integration is further enhanced by image quality optimization techniques, ensuring the high-resolution and accurate detection of targeted objects during UAV operations. Its effectiveness is proven by experiments, achieving a high mean average precision rate of 86.8% in fruit detection and counting, which surpasses existing technologies. Additionally, it maintains low average error rates, with a false positive rate at 14.7% and a false negative rate at 18.3%, even under challenging weather conditions like cloudiness. Overall, the practical implications of this multi-UAV imaging and DL-based approach are vast, particularly for real-time fruit recognition in orchards, marking a significant stride forward in the realm of digital agriculture that aligns with the objectives of Industry 4.0.

Spatial frequency domain imaging combining profile correction enables accurate real-time quantitative mapping of optical properties of apples

Spatial frequency domain imaging technology enables non-contact, wide-field quantitative analysis of optical properties in strong turbid media, thereby enhancing bruise detection in fruits based on changes in optical property values. However, precise and real-time measurement of the optical properties of curved biological materials is challenging due to the significant influence of tissue contours. This paper primarily presents a research study on real-time correction method based on the property that DC component diffuse reflectance (Rd_DC) and AC component diffuse reflectance (Rd_AC) are influenced by the contour trend of the tissue in a similar manner. The aim of this method is to eliminate the significant impact of sample contour angle on demodulation and enhance the detection of apple bruise. Sinusoidal illumination at a spatial frequency of 0.2 mm−1 was projected at 730 nm, with the experiments carried out on standard reflector, polytetrafluoroethylene sphere, and non-bruised and bruised apples. The results showed that, the relative error of the planar standard reflector decreased from the maximum value of 52.85–4.74%. The median of the Rd_AC for the polytetrafluoroethylene spherical standard, which fluctuated between 0.6 and 0.8, was corrected to the theoretical value of 1. For normal apples, the maximum fluctuation in the standard deviation of Rd_AC, after median filtering, reduced from 172.94% at 300×300 pixels to 18.69%. The boxplot of Rd_AC for bruised apples became smaller, with the lower fence being corrected from nearly 0–0.1750, which closely matched the theoretical value of 0.17. Additionally, the correlation of reduced scattering coefficient (μs′) in the ideal region of interest (100×100 pixels) remained at 0.9785 before and after correction, while other areas were corrected, making the bruising more pronounced. The correction method did not require additional experiments, with the consumed time in millisecond level.

An Apple Detection and Localization Method for Automated Harvesting under Adverse Light Conditions

The use of automation technology in agriculture has become particularly important as global agriculture is challenged by labor shortages and efficiency gains. The automated process for harvesting apples, an important agricultural product, relies on efficient and accurate detection and localization technology to ensure the quality and quantity of production. Adverse lighting conditions can significantly reduce the accuracy of fruit detection and localization in automated apple harvesting. Based on deep-learning techniques, this study aims to develop an accurate fruit detection and localization method under adverse light conditions. This paper explores the LE-YOLO model for accurate and robust apple detection and localization. The traditional YOLOv5 network was enhanced by adding an image enhancement module and an attention mechanism. Additionally, the loss function was improved to enhance detection performance. Secondly, the enhanced network was integrated with a binocular camera to achieve precise apple localization even under adverse lighting conditions. This was accomplished by calculating the 3D coordinates of feature points using the binocular localization principle. Finally, detection and localization experiments were conducted on the established dataset of apples under adverse lighting conditions. The experimental results indicate that LE-YOLO achieves higher accuracy in detection and localization compared to other target detection models. This demonstrates that LE-YOLO is more competitive in apple detection and localization under adverse light conditions. Compared to traditional manual and general automated harvesting, our method enables automated work under various adverse light conditions, significantly improving harvesting efficiency, reducing labor costs, and providing a feasible solution for automation in the field of apple harvesting.

Qualitative and quantitative analysis of pesticide isomers based on collision-induced dissociation of high-resolution mass spectrometry

Pesticide isomers are widely present in agricultural production, while their biological activity, efficacy, and toxicity may vary significantly. Due to structural similarity, chromatographic and mass spectrometric analysis of isomeric pesticides are challenging. In this experiment, representative isomeric pesticides with great differences in toxicity and maximum residue limits among countries were studied. Based on collision-induced dissociation (CID) of high-resolution mass spectrometry, the fragmentation pathways of substituent position isomer, functional group position isomer, carbon frame isomer, functional group isomer, and cis–trans isomer were analyzed to attribute fragment ions. In addition, for the isomers with the same product ions and different abundance ratios, the abundance differences at different energy-resolved mass spectrometry were analyzed, which was combined with the computational chemical density functional theory to simulate the specific process of fragmentation and to explain the reasons for the differences in fragment ions and abundance ratios. Based on the above study, a high-resolution mass spectrometry method was established for the separation and analysis of pesticide isomers. Twelve pesticide isomers were distinguished by retention time, fragment ions, and relative abundance ratio. This method was applied to the detection of real fruit and vegetable samples.

NVW-YOLOv8s: An improved YOLOv8s network for real-time detection and segmentation of tomato fruits at different ripeness stages

Tomato yield estimation relies significantly on the accurate detection of fruit quantity and size. And object detection and semantic segmentation of fruits emerge as efficacious methodologies for the realization of fruit counting and size detection. To address challenges associated with the detection and segmentation of tomato fruits in complex environments such as sample imbalance of different classes, small targets, and susceptibility to occlusion at varying stages of ripeness, this study proposed a foreground-foreground class balance method and an improved YOLOv8s network, NVW-YOLOv8s, for detecting and segmenting tomato fruits simultaneously. The foreground-foreground class balance method initially performed pixel-wise extraction on fruit samples with fewer instances. Subsequently, it synthesized them with original images containing a limited number of samples from the class, thereby augmenting the overall quantity of this specific category of fruit samples. In the NVW-YOLOv8s network, a C2f-N module, founded on the normalization-based attention module (NAM), was specifically crafted for residual feature learning. This design serves to augment the network's proficiency in extracting and integrating feature information pertaining to tomato fruits within intricate environments. Additionally, a variable focal loss (VFL) was introduced as the classification loss function to address the issue of positive and negative sample imbalance, and a regression loss function based on Wise-IoU (WIoU) was incorporated to tackle challenges related to fruit small targets and susceptibility to occlusion. The YOLOv8s models trained with the augmented balanced dataset revealed that the model's detection and segmentation performance, as indicated by mAP@0.5, improved by 4.8 % and 5.4 %, respectively, compared with the model trained with the augmented original training dataset. The test results on the augmented balanced dataset indicated that the proposed NVW-YOLOv8s network achieved a detection mAP@0.5 of 91.4 %, an F1-Score of 85.4 %, a segmentation mAP@0.5 of 90.7 %, and an F1-Score of 84.8 %. These results surpass the baseline YOLOv8s network by 4.3 % and 5.5 % for detection, 4.1 % and 5.0 % for segmentation, respectively. Additionally, the processing for concurrent detection and segmentation was measured at a speed of 60.2 FPS. Therefore, the proposed method has successfully met the precision and real-time requirements for intelligent yield estimation in horticultural crops.

Fallen apple detection as an auxiliary task: Boosting robotic apple detection performance through multi-task learning

In modern agricultural practices, advanced machine learning techniques play a pivotal role in optimizing yields and management. A significant challenge in orchard management is detecting apples on trees, which is essential for effective harvest planning and yield estimation. The YOLO series, especially the YOLOv8 model, stands out as a state-of-the-art solution for object detection, but its potential in orchards remains untapped. Addressing this, our study evaluates YOLOv8’s capability in orchard apple detection, aiming to set a benchmark. By employing image augmentation techniques like exposure, rotation, mosaic, and cutout, we lifted the model's performance to a state-of-the-art level. We further integrated multi-task learning, enhancing tree apple detection by also identifying apples on the ground. This approach resulted in a model with robust accuracy across evaluation metrics. Our results underscore that the YOLOv8 model achieves a leading standard in orchard apple detection. When trained for both tree and fallen apple detection, it outperformed the one when trained exclusively for the former. Recognizing fallen apples not only reduces waste but could also indicate pest activity, influencing strategic orchard decisions and potentially boosting economic returns. Merging cutting-edge tech with agricultural needs, our research showcases the promise of multi-task learning in fruit detection with deep learning.

Analysis of Image Data Augmentation in Smart Agriculture

The demand for agricultural production automation in smart agriculture has made deep learning widely used in tasks, e.g., disease identification, weed-crop segment and fruit detection based on image processing. However, in agricultural production environments, there are limitations in data acquisition, so data enhancement technology has also been widely used. This study introduces two data enhancement methods commonly used in smart agriculture, namely image transformation and GAN-based, and analyzes their specific applications and roles in three common task scenarios: disease identification, weed-crop segment, and fruit detection. According to the analysis, the data enhancement is mainly used to deal with class imbalance problems and over-fitting problems, and can significantly improve model performance in specific scenarios; the complexity of tasks and image structures has an impact on the application effects of different methods, and image segmentation tasks. It is necessary to increase the number of image modes, so methods such as GAN-based are adopted. However, due to the complexity of the image, fruit detection uses simple transformation methods to achieve better results. In addition, data enhancement methods designed for specific tasks, such as simulating synthetic images, can also achieve good results. However, data augmentation technology still has mode collapse problems and is too cumbersome in its application, which points out the direction for subsequent research. Overall, this research sorts out the applications of common data enhancement methods, analyzes the characteristics and limitations of these methods, and hopes to provide guidance for subsequent research.

An efficient method for detecting Camellia oleifera fruit under complex orchard environment

Efficient Camellia oleifera fruit detection in complex orchard environments is one of the most crucial technologies for automated picking and is significant for reducing labor costs. At present, Camellia oleifera fruit detection remains challenging due to the influence of fruit trait differences, cultivar differences, orchard environment differences, complex illumination, and occlusion. To address these challenges, a modified YOLOv7 model SDF-YOLO (SKConv-Decoupled Head-Focal EIoU-YOLOv7) has been developed. In SDF-YOLO, the selective kernel convolution (SKConv) is introduced to detect fruits of different sizes resulting from trait difference and occlusion with high performance, which utilizes multiple branches with different kernel sizes to fully extract and fuse the multi-scale information. To fully extract feature information of occlusion fruits, the decoupled head is used as the head network of SDF-YOLO to enhance the generalization performance of SDF-YOLO to cultivar differences and orchard environment differences. In addition, focal efficient intersection over union (Focal EIoU) loss is introduced to focus on high-quality bounding boxes, which can deal with positive and negative sample imbalance problems caused by occlusion and complex illumination. Experiment results demonstrate that SDF-YOLO can effectively detect Camellia oleifera fruits in complex orchard environments. Compared with eight mainstream models shows that SDF-YOLO achieves the highest precision of 94.91 %, recall of 93.01 %, F1 score of 93.95 %, and mAP of 96.65 % with an encouraging mean detection time of 36.57 ms. Moreover, SDF-YOLO exhibits robust adaptability to fruit trait differences, cultivar differences, orchard environment differences, complex illumination, and occlusion. These results indicated that SDF-YOLO is feasible for Camellia oleifera fruit detection in complex orchard environments, which provided a technical foundation for the automated picking of commercial orchards.

Non-destructive larval infestation detection in pear fruits using deep learning and x-ray CT generated radiographs

Non-destructive larval infestation detection in pear fruits using deep learning and x-ray CT generated radiographs

Monitoring of key Camellia Oleifera phenology features using field cameras and deep learning

A rapid and accurate yield estimation is of great significance to the management and sustainable development of Camellia Oleifera forests. Consequently, the simultaneous and accurate detection of key phenology features of Camellia Oleifera (buds, flowers, fruits) is crucial for precise yield estimation. This not only enables robotic harvesting but also allows for the prediction of peak flowering and fruit ripening periods to determine the optimal harvesting time. However, in recent studies, only Camellia Oleifera fruits have been marked and detected. Therefore, to enable rapid yield estimation, it is necessary to simultaneously detect the key phenology stages (buds, flowers, fruits) of Camellia Oleifera. In this study, we annotated, trained, and predicted Camellia Oleifera buds, flowers, and fruits collected via field cameras to observe their daily quantitative changes. Quantity change curves were generated to estimate crucial phenology stages. Phenology feature detection and transfer learning were performed using the YOLO v5 model, widely used YOLO v3 model, and CenterNet model with center point prediction, all trained on the same dataset. The best model for phenology feature detection was selected based on a comparison of average precision, with the YOLO v5 model achieving a higher mean Average Precision (mAP) value of 91.31 % compared to the CenterNet (85.43 %) and YOLO v3 (81.00 %) models. In YOLO v5, the AP values for bud, flower, and fruit detection were 82.80 %, 98.13 %, and 92.99 %, respectively, surpassing the CenterNet model by 3.97 %, 2.44 %, and 11.23 %, and the YOLO v3 model by 6.39 %, 17.13 %, and 11.67 %. The image size was adapted from 4000 × 3000 pixels to 512 × 512 pixels for model optimization. Additionally, data from the Seedling Center of Liuyang City collected at different years and times were utilized to showcase the generalizability and scalability of the selected models, resulting in mAP values of 86.14 %, 80.17 %, and 69.20 % for the three above-mentioned models respectively. The plotted phenology change curves unveiled that Camellia Oleifera undergoes four stages: fruit enlargement period, bud enlargement period, flowering period, and flower wilting period. The conclusion can be drawn that using field cameras and YOLOv5 can simultaneously achieve real-time detection of key phenology features (buds, flowers, and fruits) of Camellia Oleifera, in order to further record crucial phenology patterns (such as flowering peaks and fruit ripening periods). This study offers theoretical references and scientific evidence for monitoring changes in key phenology features of Camellia Oleifera.

Lightweight Non-Destructive Detection of Diseased Apples Based on Structural Re-Parameterization Technique

This study addresses the challenges in the non-destructive detection of diseased apples, specifically the high complexity and poor real-time performance of the classification model for detecting diseased fruits in apple grading. Research is conducted on a lightweight model for apple defect recognition, and an improved VEW-YOLOv8n method is proposed. The backbone network incorporates a lightweight, re-parameterization VanillaC2f module, reducing both complexity and the number of parameters, and it employs an extended activation function to enhance the model’s nonlinear expression capability. In the neck network, an Efficient-Neck lightweight structure, developed using the lightweight modules and augmented with a channel shuffling strategy, decreases the computational load while ensuring comprehensive feature information fusion. The model’s robustness and generalization ability are further enhanced by employing the WIoU bounding box loss function, evaluating the quality of anchor frames using outlier metrics, and incorporating a dynamically updated gradient gain assignment strategy. Experimental results indicate that the improved model surpasses the YOLOv8n model, achieving a 2.7% increase in average accuracy, a 24.3% reduction in parameters, a 28.0% decrease in computational volume, and an 8.5% improvement in inference speed. This technology offers a novel, effective method for the non-destructive detection of diseased fruits in apple grading working procedures.

A lightweight waxberry fruit detection model based on YOLOv5

A lightweight waxberry fruit detection model based on YOLOv5

Using an improved lightweight YOLOv8 model for real-time detection of multi-stage apple fruit in complex orchard environments

For the purpose of monitoring apple fruits effectively throughout the entire growth period in smart orchards. A lightweight model named YOLOv8n-ShuffleNetv2-Ghost-SE was proposed. The ShuffleNetv2 basic modules and down-sampling modules were alternately connected, replacing the Backbone of YOLOv8n model. The Ghost modules replaced the Conv modules and the C2fGhost modules replaced the C2f modules in the Neck part of the YOLOv8n. ShuffleNetv2 reduced the memory access cost through channel splitting operations. The Ghost module combined linear and non-linear convolutions to reduce the network computation cost. The Wise-IoU (WIoU) replaced the CIoU for calculating the bounding box regression loss, which dynamically adjusted the anchor box quality threshold and gradient gain allocation strategy, optimizing the size and position of predicted bounding boxes. The Squeeze-and-Excitation (SE) was embedded in the Backbone and Neck part of YOLOv8n to enhance the representation ability of feature maps. The algorithm ensured high precision while having small model size and fast detection speed, which facilitated model migration and deployment. Using 9652 images validated the effectiveness of the model. The YOLOv8n-ShuffleNetv2-Ghost-SE model achieved Precision of 94.1%, Recall of 82.6%, mean Average Precision of 91.4%, model size of 2.6 MB, parameters of 1.18 M, FLOPs of 3.9 G, and detection speed of 39.37 fps. The detection speeds on the Jetson Xavier NX development board were 3.17 fps. Comparisons with advanced models including Faster R-CNN, SSD, YOLOv5s, YOLOv7‑tiny, YOLOv8s, YOLOv8n, MobileNetv3_small-Faster, MobileNetv3_small-Ghost, ShuflleNetv2-Faster, ShuflleNetv2-Ghost, ShuflleNetv2-Ghost-CBAM, ShuflleNetv2-Ghost-ECA, and ShuflleNetv2-Ghost-CA demonstrated that the method achieved smaller model and faster detection speed. The research can provide reference for the development of smart devices in apple orchards.

Whole-cell bacterial biosensor applied to identify the presence of Thaumatotibia leucotreta larva in citrus fruits by volatile sensing

The false codling moth (FCM), Thaumatotibia leucotreta (Meyrick), is a major pest of citrus fruits and other plants and is defined as a quarantine pest in several countries. Supervision and visual monitoring are crucial for the detection of eggs and larvae. There is a growing need to implement alternative pest control strategies and early detection methodologies in the packing house before shipping. Biosensors may provide a better-suited solution, as they are portable, do not require expensive equipment, and may be used with simple methodologies. In this study, a whole-cell bacterial biosensor was developed to identify the presence of Thaumatotibia leucotreta larvae in citrus fruits. The volatile organic compounds (VOCs) released by the larvae were identified (n = 11), which were not previously tested. The change in the VOCs composition in infected and uninfected mandarin (Citrus reticulata) was examined using gas chromatography-mass spectrometry (GC-MS) and the bioluminescent bacterial panel. During this work, two fixation methods (alginate beads and tablets) were tested for the bioluminescent bacteria cells, and tablets were preferred due to their higher uniformity and consistency of the light signal from the cells. Furthermore, the response of the bacteria to the changes in the concentration of VOCs as a result of a change in the amount of the larvae was examined (larvae n = 3, 6, or 9). In addition, in order to demonstrate the ability of the whole-cell biosensor to detect the presence of larva within the fruit, infected and uninfected mandarins were tested on the 14th day of larval hatching. The results showed that the bioluminescent bacteria could differentiate between mandarins with and without larva. The differences in the light signals (Induction Factor (IF)) between the mandarin fruits can be seen in the results with the larva (cytotoxicity IF = 3.046, genotoxicity IF = 2.643, and quorum sensing IF = 1.954) and without (cytotoxicity IF = 1.589, genotoxicity IF = 1.149 and quorum sensing IF = 1.205). The findings of this study make it possible to further develop a prototype using bioluminescent bacteria in order to detect pests of citrus fruits. The early detection of infected fruits will allow their filtering from the crop and therefore reduce the damage to additional fruits. Even one larva can invalidate the whole stock and the plot for future export.

Iridium nanoparticles supported on polyaniline nanotubes for peroxidase mimicking towards total antioxidant capacity assay of fruits and vegetables

In this study, a straightforward approach is presented for the first time to anchor Ir nanoparticles on the surface of uniform polyaniline (PANi) nanotubes (NTs), which can be used as an efficient peroxidase (POD)-like catalyst. The morphology and chemical structure of the PANi-Ir nanocomposite are characterized by scanning electron microscopy (SEM), high-resolution transmission electron microscopy (HRTEM), X-ray diffractometer (XRD), Raman and X-ray photoelectron spectroscopy (XPS) measurements. Owing to the strong interaction between Ir nanoparticles and PANi, a remarkable catalytic enhancement is achieved compared to the bare Ir black catalyst and individual PANi NTs, dominating withan electron transfer mechanism. Furthermore, an efficient colorimetric sensor for ascorbic acid (AA) is developed with a low detection limit of 1.0 μM (S/N = 3), and a total antioxidant capacity (TAC) sensing platform is also constructed for the rigorous detection and analysis of fruits and vegetables.

Computer vision based deep learning approach for toxic and harmful substances detection in fruits

Formaldehyde (CH₂O) is one of the significant chemicals mixed with different perishable fruits in Bangladesh. The fruits are artificially preserved for extended periods by dishonest vendors using this dangerous chemical. Such substances are complicated to detect in appearance. Hence, a reliable and robust detection technique is required. To overcome this challenge and address the issue, we introduce comprehensive deep learning-based techniques for detecting toxic substances. Four different types of fruits, both in fresh and chemically mixed conditions, are used in this experiment. We have applied diverse data augmentation techniques to enlarge the dataset. The performance of four different pre-trained deep learning models was then assessed, and a brand-new model named “DurbeenNet,” created especially for this task, was presented. The primary objective was to gauge the efficacy of our proposed model compared to well-established deep learning architectures. Our assessment centered on the models' accuracy in detecting toxic substances. According to our research, GoogleNet detected toxic substances with an accuracy rate of 85.53 %, VGG-16 with an accuracy rate of 87.44 %, DenseNet with an impressive accuracy rate of 90.37 %, and ResNet50 with an accuracy rate of 91.66 %. Notably, the proposed model, DurbeenNet, outshone all other models, boasting an impressive accuracy rate of 96.71 % in detecting toxic substances among the sample fruits.

Application of Machine Vision Techniques in Low-Cost Devices to Improve Efficiency in Precision Farming

In the context of recent technological advancements driven by distributed work and open-source resources, computer vision stands out as an innovative force, transforming how machines interact with and comprehend the visual world around us. This work conceives, designs, implements, and operates a computer vision and artificial intelligence method for object detection with integrated depth estimation. With applications ranging from autonomous fruit-harvesting systems to phenotyping tasks, the proposed Depth Object Detector (DOD) is trained and evaluated using the Microsoft Common Objects in Context dataset and the MinneApple dataset for object and fruit detection, respectively. The DOD is benchmarked against current state-of-the-art models. The results demonstrate the proposed method’s efficiency for operation on embedded systems, with a favorable balance between accuracy and speed, making it well suited for real-time applications on edge devices in the context of the Internet of things.

Performance evaluation of newly released cameras for fruit detection and localization in complex kiwifruit orchard environments

AbstractConsumer RGB‐D and binocular stereo cameras were applied to fruit detection and localization. However, few studies are documented on performance comparison of newly released cameras under same scene in complex orchard. This study evaluates performance of consumer RGB‐D and binocular stereo cameras based on YOLOv5x for kiwifruit detection and localization and selection of optimal one with better application in complex orchard environment. Firstly, Azure Kinect, RealSense D435, and ZED 2i cameras were employed to capture images of kiwifruit canopies. Subsequently, YOLOv5x was applied to train and detect kiwifruits and calyxes in the images. Meanwhile, an overlap‐partitioning detection strategy was applied on kiwifruit and calyx detection. Additionally, spatial coordinate transformation was performed by integrating camera's extrinsic parameters and depth map generated by each camera. Finally, three‐dimensional coordinates of calyxes were calculated and compared with ground truth, followed by localization accuracy of calyxes were analyzed. Results show that YOLOv5x obtained mean average precision of 93.2%, 91.3%, and 95.8% for three cameras on kiwifruit and calyx detection, respectively. Overlap‐partitioning detection strategy improved the calyx detection and significantly increased average precision by 13.00%, 16.30%, and 7.70%, respectively. The mean absolute deviation of calyx coordinates on Y‐axis is relatively high for ZED 2i at 8.44 mm in comparison of 6.67 mm for Azure Kinect, while RealSense D435 achieved minimum of 10.42 mm on X‐axis and 18.33 mm on Z‐axis. Average spatial localization speed of calyxes in one image was 0.164 s, 0.037 s, and 0.062 s for Azure Kinect, RealSense D435, and ZED 2i, respectively. These results indicate the excellent performance of RealSense D435 than Azure Kinect and ZED 2i in kiwifruit orchard, which could be a valuable reference for other orchards to select a camera with high precision localization capacity.