Rapid Extraction of Tea Bud Phenotypic Parameters ‘In Situ’ Combining Key Point Recognition and Depth Image Fusion

Rare earth elements in tea garden soils and their bioavailability to tea buds in Taiwan

The traditional recognition algorithm is prone to miss detection targets in the complex tea garden environment, and it is difficult to satisfy the requirement for tea bud recognition accuracy and efficiency. In this study, the YOLOv7 model was developed to improve tea bud recognition accuracy for some extreme tea garden scenarios. In the improved model, a lightweight MobileNetV3 network is adopted to replace the original backbone network, which reduces the size of the model and improves detection efficiency. The convolutional block attention module is introduced to enhance the attention to the features of small and occluded tea buds, suppressing the interference of the complex tea garden environment on tea bud recognition and strengthening the feature extraction capability of the recognition model. Moreover, to further improve recognition accuracy for dense and occlusive scenarios, the soft non-maximum suppression strategy is integrated into the recognition model. Experimental results show that the improved YOLOv7 model has the precision, recall, and mean average precision (mAP) values of 88.3%, 87.4%, and 88.5%, respectively. Compared with the Faster R-CNN, SSD, and original YOLOv7 algorithms, the mAP of the improved YOLOv7 model is increased by 7.4, 7.9, and 3.9 percentage points, respectively, and its recognition speed is also promoted by 94.9%, 46.2%, and 16.9%. The proposed model can rapidly and accurately identify the tea buds in multiple complex tea garden scenarios - such as dense distribution, being close to the background color, and mutual occlusion - with high generalization and robustness, which can provide theoretical and technical support for the recognition of tea-picking robots. Keywords: tea bud recognition, YOLOv7, lightweight MobileNetV3 network, CBAM DOI: 10.25165/j.ijabe.20241706.9095 Citation: Song M X, Liu C, Chen L Q, Liu L C, Ning J M, Yu C Y. Recognition of tea buds based on an improved YOLOv7 model. Int J Agric & Biol Eng, 2024; 17(6): 238–244.

The tea buds detection and yield estimation method based on optimized YOLOv8

The detection of tea bud targets is the foundation of automated picking of premium tea. This article proposes a high-performance tea bud detection model to address issues such as complex environments, small target tea buds, and blurry device focus in tea bud detection. During the spring tea-picking stage, we collect tea bud images from mountainous tea gardens and annotate them. YOLOv5 tea is an improvement based on YOLOv5, which uses the efficient Simplified Spatial Pyramid Pooling Fast (SimSPPF) in the backbone for easy deployment on tea bud-picking equipment. The neck network adopts the Bidirectional Feature Pyramid Network (BiFPN) structure. It fully integrates deep and shallow feature information, achieving the effect of fusing features at different scales and improving the detection accuracy of focused fuzzy tea buds. It replaces the independent CBS convolution module in traditional neck networks with Omni-Dimensional Dynamic Convolution (ODConv), processing different weights from spatial size, input channel, output channel, and convolution kernel to improve the detection of small targets and occluded tea buds. The experimental results show that the improved model has improved precision, recall, and mean average precision by 4.4%, 2.3%, and 3.2%, respectively, compared to the initial model, and the inference speed of the model has also been improved. This study has theoretical and practical significance for tea bud harvesting in complex environments.

The low degree of intelligence and standardization of tea bud picking, as well as laborious and time-consuming manual harvesting, bring significant challenges to the sustainable development of the high-quality tea industry. There is an urgent need to investigate the critical technologies of intelligent picking robots for tea. The complexity of the model requires high hardware computing resources, which limits the deployment of the tea bud detection model in tea-picking robots. Therefore, in this study, we propose the YOLOV5M-SBSD tea bud lightweight detection model to address the above issues. The Fuding white tea bud image dataset was established by collecting Fuding white tea images; then the lightweight network ShuffleNetV2 was used to replace the YOLOV5 backbone network; the up-sampling algorithm of YOLOV5 was optimized by using CARAFE modular structure, which increases the sensory field of the network while maintaining the lightweight; then BiFPN was used to achieve more efficient multi-scale feature fusion; and the introduction of the parameter-free attention SimAm to enhance the feature extraction ability of the model while not adding extra computation. The improved model was denoted as YOLOV5M-SBSD and compared and analyzed with other mainstream target detection models. Then, the YOLOV5M-SBSD recognition model is experimented on with the tea bud dataset, and the tea buds are recognized using YOLOV5M-SBSD. The experimental results show that the recognition accuracy of tea buds is 88.7%, the recall rate is 86.9%, and the average accuracy is 93.1%, which is 0.5% higher than the original YOLOV5M algorithm's accuracy, the average accuracy is 0.2% higher, the Size is reduced by 82.89%, and the Params, and GFlops are reduced by 83.7% and 85.6%, respectively. The improved algorithm has higher detection accuracy while reducing the amount of computation and parameters. Also, it reduces the dependence on hardware, provides a reference for deploying the tea bud target detection model in the natural environment of the tea garden, and has specific theoretical and practical significance for the identification and localization of the intelligent picking robot of tea buds.

The segmentation and positioning of tea buds are the basis for intelligent picking robots to pick tea buds accurately. Tea images were collected in a complex environment, and median filtering was carried out to obtain tea bud images with smooth edges. Four semantic segmentation algorithms, U-Net, high-resolution network (HRNet_W18), fast semantic segmentation network (Fast-SCNN), and Deeplabv3+, were selected for processing images. The centroid of the tea buds and the image center of the minimum external rectangle were calculated. The farthest point from the centroid was extracted from the tea stalk orientation, which was the final picking point for tea buds. The experimental results showed that the mean intersection over union (mIoU) of HRNet_W18 was 0.81, and for a kernel with a median filter size of 3 × 3, the proportion of abnormal tea buds was only 11.6%. The average prediction accuracy of picking points with different tea stalk orientations was 57%. This study proposed a fresh tea bud segmentation and picking point location method based on a high-resolution network model. In addition, the cloud platform can be used for data sharing and real-time calculation of tea bud coordinates, reducing the computational burden of picking robots.

Tea bud detection technology is of great significance in realizing automated and intelligent plucking of tea buds. This study proposes a lightweight tea bud identification model based on modified Yolov5 to increase the picking accuracy and labor efficiency of intelligent tea bud picking while lowering the deployment pressure of mobile terminals. The following methods are used to make improvements: the backbone network CSPDarknet-53 of YOLOv5 is replaced with the EfficientNetV2 feature extraction network to reduce the number of parameters and floating-point operations of the model; the neck network of YOLOv5, the Ghost module is introduced to construct the ghost convolution and C3ghost module to further reduce the number of parameters and floating-point operations of the model; replacing the upsampling module of the neck network with the CARAFE upsampling module can aggregate the contextual tea bud feature information within a larger sensory field and improve the mean average precision of the model in detecting tea buds. The results show that the improved tea bud detection model has a mean average precision of 85.79%, only 4.14 M parameters, and only 5.02G of floating-point operations. The number of parameters and floating-point operations is reduced by 40.94% and 68.15%, respectively, when compared to the original Yolov5 model, but the mean average precision is raised by 1.67% points. The advantages of this paper’s algorithm in tea shot detection can be noticed by comparing it to other YOLO series detection algorithms. The improved YOLOv5 algorithm in this paper can effectively detect tea buds based on lightweight, and provide corresponding theoretical research for intelligent tea-picking robots.

For the sake of effectively identifying tea buds and improving the picking precision of mechanical picking, as a matter of fact, the traditional object detection algorithm has some problems such as poor detection effect and robustness under unstructured environment, the application of Yolo‐Tea object detection algorithm in tea bud detection under unstructured environment was explored. First, by introducing multi‐scale convolutional block attention module (MCBAM) and multi‐scale prediction layer into a network, the model gathers important information that is beneficial to tea buds classification and enhances the detection of small object tea buds in the dense scene. Then, based on the specific tea buds dataset, the anchor boxes are re‐clustered using K‐means and genetic algorithm. Finally, EIoU loss function is introduced into the boundary box regression stage to reduce the missed detection and speed up the convergence of the model. The multi‐detection box generated by the object is suppressed by soft‐non‐maximum suppression, the test effect image has a final boundary box, and the category score is output. The experiment results show that the mAP of the proposed algorithm for tea buds is 95.2%. Compared with the common object detection algorithm, the network shows superior performance in tea buds detection, which can effectively improve the recognition effect of tea buds under an unstructured environment.

Tea bud detection is the first step in the precise picking of famous teas. Accurate and fast tea bud detection is crucial for achieving intelligent tea bud picking. However, existing detection methods still exhibit limitations in both detection accuracy and speed due to the intricate background of tea buds and their small size. This study uses YOLOv5 as the initial network and utilizes attention mechanism to obtain more detailed information about tea buds, reducing false detections and missed detections caused by different sizes of tea buds; The addition of Spatial Pyramid Pooling Fast (SPPF) in front of the head to better utilize the attention module's ability to fuse information; Introducing the lightweight convolutional method Group Shuffle Convolution (GSConv) to ensure model efficiency without compromising accuracy; The Mean-Positional-Distance Intersection over Union (MPDIoU) can effectively accelerate model convergence and reduce the training time of the model. The experimental results demonstrate that our proposed method achieves precision (P), recall rate (R) and mean average precision (mAP) of 93.38%, 89.68%, and 95.73%, respectively. Compared with the baseline network, our proposed model's P, R, and mAP have been improved by 3.26%, 11.43%, and 7.68%, respectively. Meanwhile, comparative analyses with other deep learning methods using the same dataset underscore the efficacy of our approach in terms of P, R, mAP, and model size. This method can accurately detect the tea bud area and provide theoretical research and technical support for subsequent tea picking.

Throughout the entire tea bud harvest cycle, the buds undergo a continuous transition from emergence to maturity, accompanied by substantial morphological changes. These dynamic changes introduce significant intraclass variations, under which traditional tea bud detection algorithms often exhibit limited generalization capability and reduced stability. To reduce false negatives caused by morphological variations across different maturation stages, this study builds upon LW‐DETR and proposes a probabilistic framework for tea bud objectness estimation. The framework improves confidence prediction by modeling common features shared among tea buds. First, a common feature encoder is introduced to project each embedding into a tea bud common feature subspace, where the representations are encouraged to learn shared characteristics and reduce interembedding feature discrepancies. Then, probability distribution estimation and objectness likelihood maximization for known tea bud objects are alternated within the common feature subspace. This iterative process enables the estimation of objectness probabilities for different bonding boxes, which are subsequently used to refine the original model's output. Experimental results show that, on the 25_ZC108 dataset split by collection date, the proposed method achieved the best performance across all 7 days, with a maximum AP50 of 85.10%. Compared to state‐of‐the‐art detection methods, the introduced probabilistic framework demonstrates superior detection accuracy and enhanced generalization capability, particularly on datasets characterized by substantial intra‐class variations. Notably, the highest recall improved by 31.13%, effectively reducing false negatives. Furthermore, the proposed approach achieved the best results on datasets collected across different years and tea cultivars, which attests to the robustness and effectiveness of the common‐feature‐based probabilistic objectness framework.

A lightweight tea bud detection model based on Yolov5

The tea yield estimation provides information support for the harvest time and amount and serves as a decision-making basis for farmer management and picking. However, the manual counting of tea buds is troublesome and inefficient. To improve the efficiency of tea yield estimation, this study presents a deep-learning-based approach for efficiently estimating tea yield by counting tea buds in the field using an enhanced YOLOv5 model with the Squeeze and Excitation Network. This method combines the Hungarian matching and Kalman filtering algorithms to achieve accurate and reliable tea bud counting. The effectiveness of the proposed model was demonstrated by its mean average precision of 91.88% on the test dataset, indicating that it is highly accurate at detecting tea buds. The model application to the tea bud counting trials reveals that the counting results from test videos are highly correlated with the manual counting results (R 2 = 0.98), indicating that the counting method has high accuracy and effectiveness. In conclusion, the proposed method can realize tea bud detection and counting in natural light and provides data and technical support for rapid tea bud acquisition.

Accurate perception of tea buds is a fundamental prerequisite for intelligent and precise tea harvesting planning. However, in real tea plantation environments, reliable harvesting-oriented perception at the planning level remains highly challenging due to the small size of tea buds, severe occlusion, complex background clutter, and the lack of accurate three-dimensional spatial information. To address these challenges, we propose TeaNeRF, an integrated three-dimensional visual perception pipeline designed for harvesting-oriented tea bud analysis. Instead of treating detection, segmentation, and spatial analysis as independent tasks, TeaNeRF integrates sequential two-dimensional recognition, monocular depth estimation, and neural radiance field reconstruction into a coherent perception pipeline, allowing accurate spatial understanding of tea buds in complex natural scenes. It should be noted that the proposed integration is conducted at the perception-output level, where multiple modular components are connected through fixed interfaces, rather than through joint optimization or an end-to-end trainable formulation. The proposed framework combines an enhanced YOLO-based detector, prompt-guided segmentation, and monocular depth priors to guide NeRF-based three-dimensional reconstruction. By incorporating depth supervision and semantic-aware neural fields, TeaNeRF generates dense and geometrically consistent point clouds with reliable semantic separation. Quantitative evaluations show consistent improvements in reconstruction fidelity, as reflected by increased PSNR and reduced LPIPS across multiple tea tree scenes. Based on the reconstructed semantic point cloud, a three-dimensional clustering and geometric fitting strategy is further developed to enable tea bud counting and harvesting-oriented candidate point estimation at the perception level. Experiments conducted on a real-world dataset of 4,700 tea plantation images demonstrate that TeaNeRF improves detection accuracy (mAP@50 = 91.7%), segmentation quality (IoU = 0.640), and overall three-dimensional perception performance. Case-level counting results on representative tea trees indicate that the proposed 3D semantic point cloud–based approach can provide feasible tea bud counting behavior and consistent spatial guidance cues for downstream harvesting planning. By providing structured three-dimensional spatial information, including tea bud locations, counts, and harvesting-oriented candidate points, TeaNeRF offers practical perception-level outputs for downstream planning in automated tea harvesting systems.

Tea bud recognition and localization constitute a fundamental step toward enabling fine-grained tea plantation management and intelligent harvesting, offering substantial value in improving the picking quality of premium tea materials, reducing labor dependency, and accelerating the development of smart tea agriculture. However, most existing methods for detecting tea buds are built upon Artificial Neural Networks (ANNs) and rely extensively on floating-point computation, making them difficult to deploy efficiently on energy-constrained edge platforms. To address this challenge, this paper proposes an energy-efficient tea bud detection model, GAE-SpikeYOLO, which improves upon the Spiking Neural Networks (SNNs) detection framework SpikeYOLO. Firstly, Gated Attention Coding (GAC) is introduced into the input encoding stage to generate spike streams with richer spatiotemporal dynamics, strengthening shallow feature saliency while suppressing redundant background spikes. Secondly, the model incorporates the Temporal-Channel-Spatial Attention (TCSA) module into the neck network to enhance deep semantic attention on tea bud regions and effectively suppress high-level feature responses unrelated to the target. Lastly, the proposed model adopts the EIoU loss function to further improve bounding box regression accuracy. The detection capability of the model is systematically validated on a tea bud object detection dataset collected in natural tea garden environments. Experimental results show that the proposed GAE-SpikeYOLO achieves a Precision (P) of 83.0%, a Recall (R) of 72.1%, a mAP@0.5 of 81.0%, and a mAP@[0.5:0.95] of 60.4%, with an inference energy consumption of only 49.4 mJ. Compared with the original SpikeYOLO, the proposed model improves P, R, mAP@0.5, and mAP@[0.5:0.95] by 1.4%, 1.6%, 2.0%, and 3.3%, respectively, while achieving a relative reduction of 24.3% in inference energy consumption. The results indicate that GAE-SpikeYOLO provides an efficient and readily deployable solution for tea bud detection and other agricultural vision tasks in energy-limited scenarios.

IntroductionThe identification and localization of tea picking points is a prerequisite for achieving automatic picking of famous tea. However, due to the similarity in color between tea buds and young leaves and old leaves, it is difficult for the human eye to accurately identify them.MethodsTo address the problem of segmentation, detection, and localization of tea picking points in the complex environment of mechanical picking of famous tea, this paper proposes a new model called the MDY7-3PTB model, which combines the high-precision segmentation capability of DeepLabv3+ and the rapid detection capability of YOLOv7. This model achieves the process of segmentation first, followed by detection and finally localization of tea buds, resulting in accurate identification of the tea bud picking point. This model replaced the DeepLabv3+ feature extraction network with the more lightweight MobileNetV2 network to improve the model computation speed. In addition, multiple attention mechanisms (CBAM) were fused into the feature extraction and ASPP modules to further optimize model performance. Moreover, to address the problem of class imbalance in the dataset, the Focal Loss function was used to correct data imbalance and improve segmentation, detection, and positioning accuracy.Results and discussionThe MDY7-3PTB model achieved a mean intersection over union (mIoU) of 86.61%, a mean pixel accuracy (mPA) of 93.01%, and a mean recall (mRecall) of 91.78% on the tea bud segmentation dataset, which performed better than usual segmentation models such as PSPNet, Unet, and DeeplabV3+. In terms of tea bud picking point recognition and positioning, the model achieved a mean average precision (mAP) of 93.52%, a weighted average of precision and recall (F1 score) of 93.17%, a precision of 97.27%, and a recall of 89.41%. This model showed significant improvements in all aspects compared to existing mainstream YOLO series detection models, with strong versatility and robustness. This method eliminates the influence of the background and directly detects the tea bud picking points with almost no missed detections, providing accurate two-dimensional coordinates for the tea bud picking points, with a positioning precision of 96.41%. This provides a strong theoretical basis for future tea bud picking.