YOLOv11-SKP: an enhanced model for strawberry bounding box and key point detection in harvesting scenarios

Subject of Research. Image matching and classification methods, as well as synchronous location and mapping, are widely used on embedded and mobile devices. Their most resource-intensive part is the detection and description of the image key points. In case of classical methods for detection and description of key points they can be executed in real time on mobile devices but for modern neural network methods with better quality, such approach is difficult due to trading off performance. Thus, the issue of speeding for neural network models applied for the detection and description of key points is currently topical. The subject of research is distillation as one of the methods for reducing neural network models. The aim of the study is to obtain more compact model for detection and description of key points and a description of the procedure for this model design. Method. We proposed a method for pairing the original and more compact new model for its subsequent training on the output values of the original model. In this regard, the new model is learned to reconstruct the output of the original model without using image labels. Both networks accept identical images as input. Main Results. Neural network distillation method for detection and description of key points is tested. The objective function and training parameters that provide the best results in the framework of the study are proposed. A new data set is created for testing key point detection methods, and a new quality indicator of the allocated key points and their corresponding local features is added. New model training in the way suggested with the same number of parameters, shows greater accuracy in key points compared to the original model. A new model with a significantly smaller number of parameters shows the accuracy of point matching close to the accuracy of the original model. Practical Relevance. More compact model for detection and description of image key points is created applying the proposed method. The model is applicable on embedded and mobile devices for synchronous location and mapping. Such model application can also increase the service efficiency of the image search on the server side.

Animal keypoint detection holds significant applications in fields such as biological behavior research and animal health monitoring. Although related research has reached a relatively mature stage of human keypoint detection, it still faces numerous challenges in the realm of animal keypoint detection. Firstly, there is a scarcity of keypoint detection datasets related to animals in public datasets. Secondly, existing solutions have adopted large-scale deep learning models to achieve higher accuracy, but these models are costly and difficult to widely promote within the industry. On the other hand, small-scale, low-cost detection models that are used to reduce costs suffer from insufficient accuracy and cannot meet the needs of industrial production. Therefore, designing and implementing a lightweight and high-accuracy animal keypoint detection model to meet industry needs has significant theoretical and practical importance. Addressing the aforementioned issues, this thesis proposes a lightweight animal keypoint detection method, HRDS, which maintains high accuracy while significantly reducing model complexity. Firstly, by removing the fourth stage from the HRNet architecture, the number of parameters and the computational complexity of the model are successfully reduced. Secondly, to enhance the model’s performance and robustness, a new attention mechanism module, DS, is designed. This module effectively counterbalances the loss of accuracy due to the significant reduction in the parameter count and helps to strengthen the model’s keypoint detection capability in complex scenarios. Experiments were performed on the AP-10K dataset, and the results indicated that the HRDS method achieves an accuracy rate of 70.34%, with a 73.05% reduction in the number of parameters compared to HRNet and only a 2.64% decrease in accuracy, maintaining high precision. The inference time of HRDS reaches 26.58 ms, with an inference speed of 37.62 FPS, and its inference time is only 87.3% of that of HRNet. This provides a new solution for applications in resource-constrained environments.

Key point detection in football matches can provide effective support for understanding football videos and recognizing player movements. This study analyzes the position information of players in football matches using key point detection methods. A cascaded convolutional neural network for detection and regression is combined to design a key point detection model for the impact on football game commentary and player control rate. A first level key point detection network based on ResNet and a second level regression network based on heatmap are proposed by combining the fused stacked feature extraction and optimizing the original structure. The experiment confirmed that the model has an accuracy of 95.62% in detecting key points. At the same time, the coordinate error near the goal and other areas is relatively small. However, there is a significant coordinate error in the size exclusion zone. This research provides a certain reference for the analysis of future football matches.

In the greenhouse environment, factors such as variable lighting conditions, the similarity in color between fruit stems and background, and the complex growth posture of string tomatoes lead to low detection accuracy for picking points. This paper proposes a detection method for the synchronous recognition of tomatoes and their picking points based on keypoint detection. Using YOLOv8n-pose as the baseline model, we constructed the YOLOv8-TP model. To reduce the computational load of the model, we replaced the C2f module in the backbone network with the C2f-OREPA module. To enhance the model’s accuracy and performance, we introduced a PSA mechanism after the backbone network. Additionally, to strengthen the model’s feature extraction capabilities, we incorporated CGAFusion at the end of the Neck, which adaptively emphasizes important features while suppressing less important ones, thereby enhancing feature expressiveness. Experimental results show that the YOLOv8-TP model achieved an accuracy of 89.8% in synchronously recognizing tomatoes and picking points, with an inference speed of 154.7 FPS. The YOLOv8n-pose model achieves an inference speed of 148.6 FPS. Compared to the baseline model, YOLOv8-TP improved precision, mAP@.5, mAP@.5:.95, and F1-score by 0.6%, 1%, 2%, and 1%, respectively, while reducing model complexity by 8.1%. The Euclidean distance error for detecting picking points was less than 25 pixels, and the depth error was less than 3 millimeters. This method demonstrates excellent detection performance and provides a reference model for detecting string tomatoes and their picking points.

With increasing demand for scale-invariant and fast object recognition, speeded up robust features (SURF) have emerged and become a widely used feature extraction algorithm in computer vision. Nevertheless, SURF still requires high memory usage and heavy computations caused by the keypoint detection procedure that produces huge image pyramid composed of many hessian determinant (HD) data for supporting the scale-invariance. Therefore, in this paper, dynamic optimization schemes of the HD image pyramid are proposed for completely removing the redundancies of the keypoint detection with keeping the original functionality of SURF without any loss. The proposed approach has shown to reduce memory usage by 45–51%, the execution time by 37–50% and the power consumption by 18–42% when compared with the keypoint detection in the original SURF algorithm.

Body size parameters of beef cattle are crucial for assessing growth status and breeding value. In actual farming environments, the various postures of beef cattle and complex backgrounds can affect the accuracy and stability of non-contact body measurement methods. Therefore, this paper proposes a novel method called the cattle body measurement method (CBMM), which combines keypoint detection with local point cloud clustering. First, a keypoint detection model based on YOLOv8-SimBiFPN is constructed. This model enhances the feature extraction and fusion capabilities of YOLOv8-pose by introducing SimAM and BiFPN into the backbone and neck networks, respectively, and realizes 2D keypoint detection for beef cattle in various postures. Second, a 3D keypoint-locating algorithm based on Density-based spatial clustering of applications with noise (DBSCAN) is proposed. This algorithm utilizes 2D keypoints, depth maps and camera parameters to generate local point clouds, which are then clustered using DBSCAN to segment cattle body point clouds, thereby relocating the 3D keypoints based on their positional features. Finally, body size parameters are calculated based on the 3D keypoints and distance formulae. In our experiment, the mean average precision (mAP@0.5) of YOLOv8-SimBiFPN reached 99.1% on an Angus beef cattle keypoint detection dataset. The mean absolute percentage errors for measuring beef cattle withers height, hip height, body depth, body length, and oblique body length using the CBMM were 4.37%, 4.96%, 6.47%, 4.84%, and 4.14%, respectively. In summary, our method can achieve non-contact body measurement for beef cattle in a free-moving state with high accuracy and stability.

The detection of ships on the open sea is an important issue for both military and civilian fields. As an active microwave imaging sensor, synthetic aperture radar (SAR) is a useful device in marine supervision. To extract small and weak ships precisely in the marine areas, polarimetric synthetic aperture radar (PolSAR) data have been used more and more widely. We propose a new PolSAR ship detection method which is based on a keypoint detector, referred to as a PolSAR-SIFT keypoint detector, and a patch variation indicator in this paper. The PolSAR-SIFT keypoint detector proposed in this paper is inspired by the SAR-SIFT keypoint detector. We improve the gradient definition in the SAR-SIFT keypoint detector to adapt to the properties of PolSAR data by defining a new gradient based on the distance measurement of polarimetric covariance matrices. We present the application of PolSAR-SIFT keypoint detector to the detection of ship targets in PolSAR data by combining the PolSAR-SIFT keypoint detector with the patch variation indicator we proposed before. The keypoints extracted by the PolSAR-SIFT keypoint detector are usually located in regions with corner structures, which are likely to be ship regions. Then, the patch variation indicator is used to characterize the context information of the extracted keypoints, and the keypoints located on the sea area are filtered out by setting a constant false alarm rate threshold for the patch variation indicator. Finally, a patch centered on each filtered keypoint is selected. Then, the detection statistics in the patch are calculated. The detection statistics are binarized according to the local threshold set by the detection statistic value of the keypoint to complete the ship detection. Experiments on three data sets obtained from the RADARSAT-2 and AIRSAR quad-polarization data demonstrate that the proposed detector is effective for ship detection.

—Human pose estimation is essential in numerous practical applications, particularly in scenarios demanding fast processing and optimal resource usage, such as surveillance, human with computer interaction, and robotic systems. This technology aims to detect and analyze human body keypoints in images or videos, which is a critical step in understanding an individual's movements and behaviors. This article evaluates the performance of the original LeYOLO-Nano architecture, a lightweight variant of the YOLO architecture, in the application of human body keypoint detection for the purpose of pose estimation. Using MSCOCO2017 dataset, which includes a wide range of real-world conditions, this model achieved a mAP50 of 0.69 and a mAP50:95 of 0.362, demonstrating its ability to detect human poses with adequate accuracy. Moreover, the model is capable of handling data at a rate of 20.78 frames per second using a standard CPU, highlighting its effectiveness for real-time use on edge devices with restricted computing power. LeYOLO-Nano enables efficient human pose estimation on low-power devices, with the potential to further optimize speed and accuracy in real-world applications. Key words—Human pose estimation, keypoint detection, LeYOLO, real-time vision

Primary angle-closure glaucoma (PACG) is a major sub-type of glaucoma that is responsible for half of the glaucoma-related blindness worldwide. The early detection of PACG is very important, so as to provide timely treatment and prevent potential irreversible vision loss. Clinically, the diagnosis of PACG is based on the evaluation of anterior chamber angle (ACA) with anterior segment optical coherence tomography (AS-OCT). To this end, the Angle closure Glaucoma Evaluation (AGE) challenge1 held on MICCAI 2019 aims to encourage researchers to develop automated systems for angle closure classification and scleral spur (SS) localization. We participated in the competition and won the championship on both tasks. In this paper, we share some ideas adopted in our entry of the competition, which significantly improve the accuracy of scleral spur localization. There are extensive literatures on keypoint detection for the tasks such as human body keypoint and facial landmark detection. However, they are proven to fail on dealing with scleral spur localization in the experiments, due to the gap between natural and medical images. In this regard, we propose a set of constraints to encourage a two-stage keypoint detection framework to spontaneously exploit diverse information, including the image-level knowledge and contextual information around SS, from the AS-OCT for the accurate SS localization. Extensive experiments are conducted to demonstrate the effectiveness of the proposed constraints.1https://age.grand-challenge.org/

To meet the requirements of smart factories and intelligent industrial settings, it is of great significance to develop a reading recognition method for pointer meters. As existing methods lack the robustness necessary for complex environments and are overly specialized for specific meter types. In this paper, a robust and general method for pointer meter reading recognition is introduced. Our method demonstrates adaptability to a wide range of pointer meter types and maintains effectiveness under extreme conditions. Initially, a Tiny-YOLO object detection network is utilized to distinguish meters from the background. Subsequently, a multi-head key point detection network with a fusion strategy based on Swin Transformer is employed to identify four key points. The unique multi-head structure facilitates the key point detection of various image quality. The percentage indication is calculated using the angular method based on the key points. The numerical indication and the units come from the refined optical character recognition network by a large language model. Several extreme experiments have been conducted to demonstrate the robustness of our methods, with an average quoted error of less than 0.24%.

Human keypoint detection from a single image is very challenging due to occlusion, blur, illumination, and scale variance. In this paper, we address this problem from three aspects by devising an efficient network structure, proposing three effective training strategies, and exploiting four useful postprocessing techniques. First, we find that context information plays an important role in reasoning human body configuration and invisible keypoints. Inspired by this, we propose a cascaded context mixer (CCM), which efficiently integrates spatial and channel context information and progressively refines them. Then, to maximize CCM’s representation capability, we develop a hard-negative person detection mining strategy and a joint-training strategy by exploiting abundant unlabeled data. It enables CCM to learn discriminative features from massive diverse poses. Third, we present several sub-pixel refinement techniques for postprocessing keypoint predictions to improve detection accuracy. Extensive experiments on the MS COCO keypoint detection benchmark demonstrate the superiority of the proposed method over representative state-of-the-art (SOTA) methods. Our single model achieves comparable performance with the winner of the 2018 COCO Keypoint Detection Challenge. The final ensemble model sets a new SOTA on this benchmark. The source code will be released at https://github.com/chaimi2013/CCM .

Accurate and non-invasive measurement of fish phenotypic characteristics in underwater environments is crucial for advancing aquaculture. Traditional manual methods require significant labor to anesthetize and capture fish, which not only raises ethical concerns but also risks causing injury to the animals. Alternative hardware-based approaches, such as acoustic technology and active structured light techniques, are often costly and may suffer from limited measurement accuracy. In contrast, image-based methods utilizing low-cost binocular cameras present a more affordable solution, although they face challenges such as light refraction between water and the waterproof enclosure, which can cause discrepancies between image coordinates and actual positions. To address these challenges, we have developed a fish keypoint detection dataset and trained both a fish object detection model and a keypoint detection model using the RTMDet and RTMPose architectures to identify keypoints on Plectropomus leopardus. Since the binocular camera must be housed in a waterproof enclosure, we correct for birefringence caused by the water and the enclosure by applying refraction corrections to the detected keypoint coordinates. This ensures that the keypoint coordinates obtained underwater are consistent with those in air, thereby improving the accuracy of subsequent stereo matching. Once the corrected keypoint coordinates are obtained, we apply the least squares method, in conjunction with binocular stereo imaging principles, to perform stereo matching and derive the actual 3D coordinates of the keypoints. We calculate the fish body length by measuring the 3D coordinates of the snout and tail. Our model achieved 98.6% accuracy in keypoints detection (AP@0.5:0.95). Underwater tests showed an average measurement error of approximately 3.2 mm (MRPE=3.50%) for fish in a tank, with real-time processing at 28 FPS on an NVIDIA GTX 1060 GPU. These results confirm that our method effectively detects keypoints on fish bodies and measures their length without physical contact or removal from the tank. By eliminating invasive procedures, our approach not only improves measurement efficiency but also aligns with ethical standards in aquaculture. Compared to existing techniques, our method offers enhanced accuracy (reducing MRPE by 53.8% compared to baseline methods) and practicality, making it a valuable tool for the aquaculture industry.

Subject of Research. The paper presents the study of algorithms for key point detection and description, widely used in computer vision. Typically, the corner detector acts as a key point detector, including neural key point detectors. For some types of images obtained in medicine, the application of such detectors is problematic due to the small number of detected key points. The paper considers a problem of a neural network key point detector training on unlabeled images. Method. We proposed the definition of key points not depending on specific visual features. A method was considered for training of a neural network model meant for detecting and describing key points on unlabeled data. The application of homographic image transformation was basic to the method. The neural network model was trained to detect the same key points on pairs of noisy images related to a homographic transformation. Only positive examples were used for detector training, just points correctly matched with features produced by the neural network model for key point description. Main Results. The unsupervised learning algorithm is used to train the neural network model. For the ease of comparison, the proposed model has a similar architecture and the same number of parameters as the supervised model. Model evaluation is performed on the three different datasets: natural images, synthetic images, and retinal photographs. The proposed model shows similar results to the supervised model on the natural images and better results on retinal photographs. Improvement of results is demonstrated after additional training of the proposed model on images from the target domain. This is an advantage over a model trained on a labeled dataset. For comparison, the harmonic average of such metrics is used as: the accuracy and the depth of matching by descriptors, reproducibility of key points and image coverage. Practical Relevance. The proposed algorithm makes it possible to train the neural network key point detector together with the feature extraction model on images from the target domain without costly dataset labeling and reduce labor costs for the development of the system that uses the detector.

The tremendous growth in the availability of texture+depth(RGBD) content has motivated researchers to focus into the area of features that uses geometrical information given by depth. These features are more efficient and robust than the existing 2D features under large variations in camera viewing angles. In this paper, the performance evaluation of the keypoint detector for two views of a scene taken from different camera viewing angle is discussed. As a first step, an RGBD model is proposed for the keypoint detector. Using this model, a keypoint detector is implemented which is obtained from the numerical solution of a Partial Differential Equation (PDE). It is then extended to a multiscale representation from which the keypoints are extracted. This extraction technique is then compared with Speeded Up Robust Feature (SURF) extraction technique. Also the behaviour of the keypoint detector for scenes affected by different noises and for high contrast or low contrast images are analyzed.

Effective and efficient generation of keypoints from images is the first step of many computer vision applications, such as object matching. The last decade presented us with an arms race toward faster and more robust keypoint detection, feature description and matching. This resulted in several new algorithms, for example Scale Invariant Features Transform (SIFT), Speed-up Robust Feature (SURF), Oriented FAST and Rotated BRIEF (ORB) and Binary Robust Invariant Scalable Keypoints (BRISK). The keypoint detection has been improved using various techniques in most of these algorithms. However, in the search for faster computing, the accuracy of the algorithms is decreasing. In this paper, we present SICK (Scale-Invariant Corner Keypoints), which is a novel method for fast keypoint detection. Our experiment results show that SICK is faster to compute and more robust than recent state-of-the-art methods.