Keypoint Detection Method Research Articles

Human key point detection method based on enhanced receptive field and transformer

The existing key point detection network models have complex structures, so it is difficult to deploy on edge devices. Meanwhile, the convolution is localized and limited by the size of convolution kernels, which cannot effectively capture long-range dependencies. To address this problem, this paper introduces a lightweight Convolutional Transformer network (LHFormer Net) for human pose estimation. Considering that the sampling area of convolution kernels for different dimensional feature maps is fixed and the contextual information is singular, the enhanced receptive field block is designed to extract richer feature information and reduce information loss in feature maps. Based on the global modeling features of Transformer encoder, convolutional position encoding and multi-head self-attention are used to capture the spatial constraint relationship between key points in the deep feature extraction. Finally, a lightweight deconvolution module is used to generate higher resolution features to achieve multi-resolution supervision, which can effectively solve the problem of scale variation in pose estimation, more accurately locate key points of small and medium-sized people, ad further improve the detection accuracy of the network. Compared with other networks, the experimental results on the open access datasets COCO2017 and MPII show that the proposed network achieves a good balance between model complexity and detection performance.

Trajectory Recognition and Working Condition Analysis of Rod Pumping Systems Based on Pose Estimation Method with Heatmap-Free Joint Detection

Summary Rod pump systems are the primary production tools in oilfield development. Analyzing their working conditions provides a theoretical foundation for formulating production optimization plans and adjusting equipment parameters. Existing machine learning–based condition analysis methods rely on dynamometer cards and cannot capture the actual operational trajectory of the pumping unit. To address this issue, this paper proposes a keypoint detection method for pumping units based on pose estimation of heatmap-free joint detection from video data. A data annotation scheme suitable for the task of detecting pumping unit keypoints was developed, and the learning criteria for this task were optimized. An end-to-end heatmap-free pose estimation algorithm was used to process images of the pumping unit, yielding predicted keypoint positions, thereby enabling the identification of the keypoint motion trajectories of the pumping unit. Experiments validated the proposed method and compared it with general learning criteria. Results show that this method accurately captures the keypoint positions of the pumping unit, with the optimized learning criteria model improving by more than 5% compared with general methods and increasing the keypoint object keypoint similarity (OKS) by more than 30%. The model’s results can be used for the actual operational trajectory recognition of the pumping unit, automatically calculating the motion parameters of the polished rod, and intelligently assessing the balance and working condition analysis of the pumping unit. This realizes the intelligent application of video surveillance data, significantly contributing to the dynamic study of rod pump systems.

Video-based sequential Bayesian homography estimation for soccer field registration

A novel Bayesian framework is proposed, which explicitly relates the homography of one video frame to the next through an affine transformation while explicitly modelling keypoint uncertainty. The literature has previously used differential homography between subsequent frames, but not in a Bayesian setting. In cases where Bayesian methods have been applied, camera motion is not adequately modelled, and keypoints are treated as deterministic. The proposed method, Bayesian Homography Inference from Tracked Keypoints (BHITK), employs a two-stage Kalman filter and significantly improves existing methods. Existing keypoint detection methods may be easily augmented with BHITK. It enables less sophisticated and less computationally expensive methods to outperform the state-of-the-art approaches in most homography evaluation metrics. Furthermore, the homography annotations of the WorldCup and TS-WorldCup datasets have been refined using a custom homography annotation tool that has been released for public use. The refined datasets are consolidated and released as the consolidated and refined WorldCup (CARWC) dataset.

Two-Stage Method for Clothing Feature Detection

The rapid expansion of e-commerce, particularly in the clothing sector, has led to a significant demand for an effective clothing industry. This study presents a novel two-stage image recognition method. Our approach distinctively combines human keypoint detection, object detection, and classification methods into a two-stage structure. Initially, we utilize open-source libraries, namely OpenPose and Dlib, for accurate human keypoint detection, followed by a custom cropping logic for extracting body part boxes. In the second stage, we employ a blend of Harris Corner, Canny Edge, and skin pixel detection integrated with VGG16 and support vector machine (SVM) models. This configuration allows the bounding boxes to identify ten unique attributes, encompassing facial features and detailed aspects of clothing. Conclusively, the experiment yielded an overall recognition accuracy of 81.4% for tops and 85.72% for bottoms, highlighting the efficacy of the applied methodologies in garment categorization.

ALIKE-APPLE: A Lightweight Method for the Detection and Description of Minute and Similar Feature Points in Apples

Current image feature extraction methods fail to adapt to the fine features of apple image texture, resulting in image matching errors and degraded image processing accuracy. A multi-view orthogonal image acquisition system was constructed with apples as the research object. The system consists of four industrial cameras placed around the apple at different angles and one camera placed on top. Following the image acquisition through the system, synthetic image pairs—both before and after transformation—were generated as the input dataset. This generation process involved each image being subjected to random transformations. Through learning to extract more distinctive and descriptive features, the deep learning-based keypoint detection method surpasses traditional techniques by broadening the application range and enhancing detection accuracy. Therefore, a lightweight network called ALIKE-APPLE was proposed for surface feature point detection. The baseline model for ALIKE-APPLE is ALIKE, upon which improvements have been made to the image feature encoder and feature aggregation modules. It comprises an Improved Convolutional Attention Module (ICBAM) and a Boosting Resolution Sampling Module (BRSM). The proposed ICBAM replaced max pooling in the original image feature encoder for downsampling. It enhanced the feature fusion capability of the model by utilizing spatial contextual information and learning region associations in the image. The proposed BRSM replaced the bilinear interpolation in the original feature aggregator for upsampling, overcoming the apple side image’s geometric distortion and effectively preserving the texture details and edge information. The model size was shrunk by optimizing the number of downsampling operations from the image encoder of the original model. The experimental results showed that the average number of observed keypoints and the average matching accuracy were improved by 166.41% and 37.07%, respectively, compared with the baseline model. The feature detection model of ALIKE-APPLE was found to perform better than the optimal SuperPoint. The feature point distribution of ALIKE-APPLE showed an improvement of 10.29% in average standard deviation (Std), 8.62% in average coefficient of variation (CV), and 156.12% in average feature point density (AFPD). Moreover, the mean matching accuracy (MMA) of ALIKE-APPLE improved by 125.97%. Thus, ALIKE-APPLE boasts a more consistent allocation of feature points and greater precision in matching.

ANALYSIS OF IMAGE KEY POINT DETECTION, DESCRIPTION AND MATCHING METHODS

ANALYSIS OF IMAGE KEY POINT DETECTION, DESCRIPTION AND MATCHING METHODS

Point Cloud Registration Based on Local Variation of Surface Keypoints

Keypoint detection plays a pivotal role in three-dimensional computer vision, with widespread applications in improving registration precision and efficiency. However, current keypoint detection methods often suffer from poor robustness and low discriminability. In this study, a novel keypoint detection approach based on the local variation of surface (LVS) is proposed. The LVS keypoint detection method comprises three main steps. Firstly, the surface variation index for each point is calculated using the local coordinate system. Subsequently, points with a surface variation index lower than the local average are identified as initial keypoints. Lastly, the final keypoints are determined by selecting the minimum value within the neighborhood from the initial keypoints. Additionally, a sampling consensus correspondence estimation algorithm based on geometric constraints (SAC-GC) for efficient and robust estimation of optimal transformations in correspondences is proposed. By combining LVS and SAC-GC, we propose a coarse-to-fine point cloud registration algorithm. Experimental results on four public datasets demonstrate that the LVS keypoint detection algorithm offers improved repeatability and robustness, particularly when dealing with noisy, occluded, or cluttered point clouds. The proposed coarse-to-fine point cloud registration algorithm also exhibits enhanced robustness and computational efficiency.

A Novel Keypoint Detection Method for Radar Point Cloud Registration

A Novel Keypoint Detection Method for Radar Point Cloud Registration

Robust coverless video steganography based on inter-frame keypoint matching

Existing coverless video steganography schemes primarily focus on single-frame features, which manifest poor robustness. To enhance resistance against adversarial attacks, this paper proposes a coverless video steganography method based on inter-frame keypoint matching, leveraging the movement orientation of keypoints within the video for information hiding. Initially, the method divides video frames into blocks, and then constructs an inter-frame keypoint matching network by using the keypoint detection method combined with deep learning. By narrowing down the matching scope and filtering out mismatches, the network effectively discerns the keypoint matching relationships. For keypoints successfully matched within the blocks, a coordinate system is established based on the original keypoint, determining the position of the matched keypoints. The most frequently occurring orientation within the blocks is designated as the orientation of the block. Subsequently, we introduce a mapping rule based on quadrant labels, generating robust hash sequences according to the quadrant labels corresponding to the orientation of the blocks. Compared to the latest coverless video steganography schemes, this method shows superior robustness in resisting 14 common traditional attacks and video attacks. Concurrently, it exhibits commendable hiding capacity and hiding success rate.

A detection-regression based framework for fish keypoints detection

Applying computer vision technology in aquaculture can improve the efficiency of fish detection and health monitoring as well as optimize aquaculture management and profit. Keypoints on fish bodies are important biological indicators that can be used to calculate the individual size, mass, and behavior. However, only a few relevant studies have been conducted in this regard, and they mainly focus on detecting keypoints for stereo matching. Traditional keypoint detection methods exhibit low efficiency, poor accuracy, and weak robustness in underwater environments. Accordingly, this study proposes a new method based on object detection and point regression models to locate fish keypoints. First, individual fish are detected by employing a commonly used object detection model, YOLOv5. The detection accuracy is further improved by enhancing the network neck. In the second stage, a deep learning model for locating fish keypoints is constructed by implementing weight allocation and distribution-aware strategy in the matched left and right bounding boxes to improve on the previous work of Lite-HRNet, which was originally designed for capturing human body keypoints. The experimental results show that the proposed method can effectively detect individual underwater fish and accurately estimate the keypoints. The source code and the labeled datasets for fish detection and keypoint location are provided. The code is available at https://github.com/oucvisionlabsanya/fish_keypoint_detection.git.

Face Keypoint Detection Method Based on Blaze_ghost Network

The accuracy and speed of facial keypoint detection are crucial factors for effectively extracting fatigue features, such as eye blinking and yawning. This paper focuses on the improvement and optimization of facial keypoint detection algorithms, presenting a facial keypoint detection method based on the Blaze_ghost network and providing more reliable support for facial fatigue analysis. Firstly, the Blaze_ghost network is designed as the backbone network with a deeper structure and more parameters to better capture facial detail features, improving the accuracy of keypoint localization. Secondly, HuberWingloss is designed as the loss function to further reduce the training difficulty of the model and enhance its generalization ability. Compared to traditional loss functions, HuberWingloss can reduce the interference of outliers (such as noise and occlusion) in model training, improve the model’s robustness to complex situations, and further enhance the accuracy of keypoint detection. Experimental results show that the proposed method achieves significant improvements in both the NME (Normal Mean Error) and FR (Failure Rate) evaluation metrics. Compared to traditional methods, the proposed model demonstrates a considerable improvement in keypoint localization accuracy while still maintaining high detection efficiency.

Key point detection method for fish size measurement based on deep learning

AbstractAccurate fish size measurement in breeding areas is crucial for the fishing industry. Unlike acoustic methods with high equipment cost and low measurement accuracy, current image‐based methods offer a promising alternative. However, these image‐based methods still face challenges in selecting measurement points. To address this issue and achieve precise measurements of individual fish, this paper introduces an automatic fish size measurement method based on key point detection. We established a Fish‐Keypoints dataset and utilized deep learning techniques for the detection of fish and their key points. Using a binocular camera system, we reconstruct a three‐dimensional coordinate system to measure key points at the fish's head and tail, facilitating fish length calculation. The detection model achieves an accuracy of 85.1% in key point detection. The proposed method is tested in both land and underwater environments, demonstrating a relative measurement error of approximately 7% for fish in pools. This confirms the proposed method's ability to accurately detect measurement points, offering superior accuracy compared to other methods.

Improving the matching of deformable objects by learning to detect keypoints

We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. By leveraging true correspondences acquired by matching annotated image pairs with a specified descriptor extractor, we train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method, while outperforming the state-of-the-art keypoint detectors on real images of non-rigid objects by 20 p.p. We also apply our method on the complex real-world task of object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task. The source code and trained models are publicly available at https://github.com/verlab/LearningToDetect_PRL_2023.

Retracted: Key Point Detection Method of 3D Network Model Based on Virtual Reality Technology

Retracted: Key Point Detection Method of 3D Network Model Based on Virtual Reality Technology

Research on Key Point Detection Method of Mechanical Arm

Accurate keypoint localization is an important research direction in the field of robotic arms. Deep learning is being used more and more for motion capture and pose estimation. This is due to their ability to detect user-defined key points without the use of labels. We present an approach for detecting robot 3D key points coordinate based on two perspectives. Two images of different perspectives are processed by the same deep neural network to detect 2D projections of key points (such as joints) associated with the robot. Triangularization is then applied to recover 3D coordinates. In addition, the existing dataset of robot critical points is obtained by simulation. It seriously impedes the advancement of this field. We establish an actual robot key points dataset and corresponding evaluation criterion. The experimental results demonstrate that our framework achieves cutting-edge detection performance. It is reusable for other robot models.

Locating Tea Bud Keypoints by Keypoint Detection Method Based on Convolutional Neural Network

Tea is one of the most consumed beverages in the whole world. Premium tea is a kind of tea with high nutrition, quality, and economic value. This study solves the problem of detecting premium tea buds in automatic plucking by training a modified Mask R-CNN network for tea bud detection in images. A new anchor generation method by adding additional anchors and the CIoU loss function were used in this modified model. In this study, the keypoint detection branch was optimized to locate tea bud keypoints, which, containing a fully convolutional network (FCN), is also built to locate the keypoints of bud objects. The built convolutional neural network was trained through our dataset and obtained an 86.6% precision and 88.3% recall for the bud object detection. The keypoint localization had a precision of 85.9% and a recall of 83.3%. In addition, a dataset for the tea buds and picking points was constructed in study. The experiments show that the developed model can be robust for a range of tea-bud-harvesting scenarios and introduces the possibility and theoretical basis for fully automated tea bud harvesting.

Grasping and cutting points detection method for the harvesting of dome-type planted pumpkin using transformer network-based instance segmentation architecture.

An accurate and robust keypoint detection method is vital for autonomous harvesting systems. This paper proposed a dome-type planted pumpkin autonomous harvesting framework with keypoint (grasping and cutting points) detection method using instance segmentation architecture. To address the overlapping problem in agricultural environment and improve the segmenting precision, we proposed a pumpkin fruit and stem instance segmentation architecture by fusing transformer and point rendering. A transformer network is utilized as the architecture backbone to achieve a higher segmentation precision and point rendering is applied so that finer masks can be acquired especially at the boundary of overlapping areas. In addition, our keypoint detection algorithm can model the relationships among the fruit and stem instances as well as estimate grasping and cutting keypoints. To validate the effectiveness of our method, we created a pumpkin image dataset with manually annotated labels. Based on the dataset, we have carried out plenty of experiments on instance segmentation and keypoint detection. Pumpkin fruit and stem instance segmentation results show that the proposed method reaches the mask mAP of 70.8% and box mAP of 72.0%, which brings 4.9% and 2.5% gains over the state-of-the-art instance segmentation methods such as Cascade Mask R-CNN. Ablation study proves the effectiveness of each improved module in the instance segmentation architecture. Keypoint estimation results indicate that our method has a promising application prospect in fruit picking tasks.

Key Point Detection Method of 3D Network Model Based on Virtual Reality Technology

In order to meet the higher requirements of today’s society for the details and fidelity of 3D models, a key point detection method for 3D network models based on virtual reality technology is proposed. The key technical steps include model normalization preprocessing, feature extraction, and similarity calculation. The 3D model retrieval test of cars, trucks, and buses is carried out using the VTFT algorithm. The results showed that in different databases, the highest success rate was 86.4% and the lowest was 77.10%. Practice has proved that this method can effectively improve the efficiency and accuracy of current 3D model processing and can better meet social needs such as games and entertainment.

Application Research on Face Image Evaluation Algorithm of Deep Learning Mobile Terminal for Student Check-In Management.

With the increase in the number of data, the traditional shallow image features cannot meet the needs of image representation. As an important means of image research, deep learning network has been paid attention to. In the field of face image evaluation, deep learning algorithm has been introduced, and the recognition technology has gradually matured. Based on this, this paper studies the application of face image evaluation algorithm of deep learning mobile terminal for student check-in management. A face image detection model for student check-in management is constructed, and a deep learning network is used to realize face detection. A face detection algorithm based on candidate region joint deep learning network is designed, and a face key point detection method based on cascaded convolution network is proposed. Aiming at the low efficiency of face recognition and detection, the existing loss function is optimized, the extraction algorithm of face binary features is proposed, and experiments are designed to analyze the performance of the algorithm. The simulation results show that the face detection based on the improved deep learning network can shorten the retrieval time and improve the accuracy of face image classification.

A novel method for trajectory recognition and working condition diagnosis of sucker rod pumping systems based on high-resolution representation learning

The pumping unit is the most important lifting equipment in the rod pumping system, which transmits ground power to the downhole pump through its sucker rod. Trajectory identification of the rod pumping system can be used to study the motion law of the pumping unit, and it is the basis for the design of the pumping parameters and the analysis of the working conditions. Most of the studies on the dynamics of the pumping unit are to discuss the simple harmonic motion of the suspension point or the motion of the crank-slider in the ideal state, which is quite different from the actual situation. In this paper, a key point detection method for oil pumping units based on high-resolution representation learning is proposed. The model adopts a multi-resolution parallel network, which connects multiple network branches with different resolutions in parallel. Then it performs multi-stage processing on the input pumping unit images and outputs the predicted Gaussian heatmaps of key points to realize the trajectory recognition of the pumping unit. Different from the traditional series convolutional network structure, which needs to recover high resolution from low resolution, the parallel structure of this model is able to maintain the high resolution of the backbone network and thus the learned representations are more accurate in spatial location. This method is also experimentally verified and compared with the latest convolutional network structure. The results show that the model accuracy of the high-resolution representation learning method is more than 6% higher than that of the comparison methods. In addition, the results of this model can be used to judge the balance of the pumping unit, automatically calculate the maximum stroke and polished rod stroke, and have guiding significance for the working condition diagnosis of the rod pumping system.