Keypoint Extraction Research Articles

BRIEF-DOC A MULTI PDF WEB APPLICATION

In today’s fast-paced digital landscape, the sheer volume of information in PDF documents can be overwhelming and time-consuming to process. This presentation introduces an innovative system that leverages natural language processing (NLP) and voice-assistive technology to create a comprehensive PDF summarizer equipped with a voice assistant. The system not only extracts key points from lengthy PDFs, condensing them into easily digestible summaries, but also features a language-changing option that allows users to select their preferred language for both text and audio output. Users can listen to these summaries read aloud through the voice assistant, ensuring a hands-free and efficient way to stay informed. This technology is especially beneficial for professionals, students, and researchers who require quick insights without the need to read entire documents. Key features include: - Automatic text extraction and summarization. - Customizable summary lengths tailored to user preferences. - Integration with voice assistants for real-time, audible summaries. - Language selection for both text and audio output, accommodating diverse user needs. - Versatile applications across education, business, and research sectors.

Comparative Analysis of Keypoint Detection Performance in SIFT Implementations on Small-Scale Image Datasets

Scale-invariant feature transform (SIFT) is widely used as an image local feature extraction method because of its invariance to rotation, scale, and illumination change. SIFT has been implemented in different program libraries. However, studies that analyze the performance of SIFT implementations have not been conducted. This study examines the keypoint extraction of three well-known SIFT libraries, i.e., David Lowe's implementation, OpenSIFT, and vlSIFT in vlfeat. Performance analysis was conducted on multiclass small-scale image datasets to capture the sensitivity of keypoint detection. Although libraries are based on the same algorithm, their performance differs slightly. Regarding execution time and the average number of keypoints detected in each image, vlSIFT outperforms David Lowe’s library and OpenSIFT.

A Registration Method Based on Ordered Point Clouds for Key Components of Trains.

Point cloud registration is pivotal across various applications, yet traditional methods rely on unordered point clouds, leading to significant challenges in terms of computational complexity and feature richness. These methods often use k-nearest neighbors (KNN) or neighborhood ball queries to access local neighborhood information, which is not only computationally intensive but also confines the analysis within the object's boundary, making it difficult to determine if points are precisely on the boundary using local features alone. This indicates a lack of sufficient local feature richness. In this paper, we propose a novel registration strategy utilizing ordered point clouds, which are now obtainable through advanced depth cameras, 3D sensors, and structured light-based 3D reconstruction. Our approach eliminates the need for computationally expensive KNN queries by leveraging the inherent ordering of points, significantly reducing processing time; extracts local features by utilizing 2D coordinates, providing richer features compared to traditional methods, which are constrained by object boundaries; compares feature similarity between two point clouds without keypoint extraction, enhancing efficiency and accuracy; and integrates image feature-matching techniques, leveraging the coordinate correspondence between 2D images and 3D-ordered point clouds. Experiments on both synthetic and real-world datasets, including indoor and industrial environments, demonstrate that our algorithm achieves an optimal balance between registration accuracy and efficiency, with registration times consistently under one second.

Research on the movement pattern and kinematic model of the hindlegs of the water boatman.

The special hindleg structure and swimming setae of a water boatman give it a high degree of maneuverability, which plays an important role in swimming. This paper used a high-speed photography platform to extract key points from videos, obtaining the forward and turning movement patterns of the water boatman's hindlegs, as well as the transformation patterns of the setae. A Fourier series was used to establish the movement models of each hindleg joint, and a kinematic model of the hindlegs was established to study the continuous movement characteristics of the hindlegs. This paper provides basic data and theoretical support for the design of underwater bio-inspired robots.

Semantically-Based Animal Pose Estimation in the Wild

Abstract. Accurate animal pose estimation in the wild is potentially useful for many downstream applications such as wildlife conservation. Currently, the main approach to assessing animal poses is based on identifying keypoints of the body and constructing the skeleton. However, a direct application of frameworks to human pose estimation is not successful due to the features of the skeletal structure of humans and mammals. In this study, we propose a two-stage method: coarse-tuning with animal detection using a bounding box, as is done in most similar methods, and fine-tuning with semantic segmentation of animal. The YOLOv8 Pose Estimation and Pose Keypoint Classification model was chosen as the base model for keypoint extraction. Extensive training experiments were conducted using the AwA2 dataset (with a small number of samples from own dataset), the AP-10K dataset, and the Tiger-Pose dataset. The trained model was tested on own dataset collected from camera traps in the Ergaki National Park, Russia. Experimental results show that the proposed algorithm using additional semantic segmentation increases the accuracy of animal pose estimation by 3.6–4.8% on samples of the Ergaki dataset.

Multi-Biometric Feature Extraction from Multiple Pose Estimation Algorithms for Cross-View Gait Recognition.

Gait recognition is a behavioral biometric technique that identifies individuals based on their unique walking patterns, enabling long-distance identification. Traditional gait recognition methods rely on appearance-based approaches that utilize background-subtracted silhouette sequences to extract gait features. While effective and easy to compute, these methods are susceptible to variations in clothing, carried objects, and illumination changes, compromising the extraction of discriminative features in real-world applications. In contrast, model-based approaches using skeletal key points offer robustness against these covariates. Advances in human pose estimation (HPE) algorithms using convolutional neural networks (CNNs) have facilitated the extraction of skeletal key points, addressing some challenges of model-based approaches. However, the performance of skeleton-based methods still lags behind that of appearance-based approaches. This paper aims to bridge this performance gap by introducing a multi-biometric framework that extracts features from multiple HPE algorithms for gait recognition, employing feature-level fusion (FLF) and decision-level fusion (DLF) by leveraging a single-source multi-sample technique. We utilized state-of-the-art HPE algorithms, OpenPose, AlphaPose, and HRNet, to generate diverse skeleton data samples from a single source video. Subsequently, we employed a residual graph convolutional network (ResGCN) to extract features from the generated skeleton data. In the FLF approach, the features extracted from ResGCN and applied to the skeleton data samples generated by multiple HPE algorithms are aggregated point-wise for gait recognition, while in the DLF approach, the decisions of ResGCN applied to each skeleton data sample are integrated using majority voting for the final recognition. Our proposed method demonstrated state-of-the-art skeleton-based cross-view gait recognition performance on a popular dataset, CASIA-B.

Ship detection and water level measurement system based on 3D point cloud

ABSTRACT With the continuous development of the water transportation and shipping industries, the number of ships in rivers has steadily multiplied, followed by the increasing complexity of the ship routes. These changes have highlighted the growing importance of ship detection and water level measurement systems. Such systems not only enhance the management efficiency of waterborne traffic, ensure navigation safety, and reduce congestion and collision accidents, but also effectively safeguard the integrity of riverside and bridge structures. Ship detection and river elevation measurement based on 3D point clouds can directly acquire depth information without being affected by lighting conditions. So it has a good research prospect. Therefore, this paper proposes a novel ship detection algorithm based on improved PointRCNN and a novel method for riverbank line extraction and water level measurement based on 3D point clouds, respectively. For ship detection, the improved PointRCNN algorithm can increase the performance of data processing and keypoint extraction techniques, and make the network to keep more foreground point clouds and learn more effective features. This improves the recognition capability of distant ships. Compared to the original PointRCNN algorithm, the improved PointRCNN algorithm has achieved a 3.84% increase in detection precision in practical scenarios. Regarding riverbank extraction and water level measurement, the proposed method based on 3D point clouds can directly extract riverbank lines with depth information, obtaining water level height without direct contact with the river surface. Within a distance range between 15 and 45 m from the LiDAR, the average absolute error using this measurement method is less than 5 cm, demonstrating the good detection accuracy of this method.

Data augmentation via warping transforms for modeling natural variability in the corneal endothelium enhances semi-supervised segmentation.

Image segmentation of the corneal endothelium with deep convolutional neural networks (CNN) is challenging due to the scarcity of expert-annotated data. This work proposes a data augmentation technique via warping to enhance the performance of semi-supervised training of CNNs for accurate segmentation. We use a unique augmentation process for images and masks involving keypoint extraction, Delaunay triangulation, local affine transformations, and mask refinement. This approach accurately captures the natural variability of the corneal endothelium, enriching the dataset with realistic and diverse images. The proposed method achieved an increase in the mean intersection over union (mIoU) and Dice coefficient (DC) metrics of 17.2% and 4.8% respectively, for the segmentation task in corneal endothelial images on multiple CNN architectures. Our data augmentation strategy successfully models the natural variability in corneal endothelial images, thereby enhancing the performance and generalization capabilities of semi-supervised CNNs in medical image cell segmentation tasks.

An Efficient and Stable Registration Framework for Large Point Clouds at Two Different Moments.

Point cloud registration plays a great role in many application scenarios; however, the registration of large-scale point clouds for actual different moments suffers from the problems of low efficiency, low accuracy, and a lack of stability. In this paper, we propose a registration framework for large-scale point clouds at different moments, which firstly downsamples large-scale point clouds using a random sampling method, then performs a random expansion strategy to make up for the loss of information caused by the random sampling, then completes the first registration by a deep learning network based on the extraction of keypoints and feature descriptors in combination with RANSAC, and finally completes the registration using the point-to-point ICP method. We conducted validation experiments and application experiments on large-scale point clouds of key train components, and the experimental results are much higher in accuracy or efficiency than other methods, which proves the effectiveness of our framework, which can be applied to actual large-scale point clouds.

Optimizing obstacle avoidance path planning for intelligent mobile robots in multi-obstacle environments

To enhance the obstacle avoidance capabilities of intelligent mobile robots in environments with multiple obstacles, a method for optimal path planning has been developed and analysed. Multi-obstacle scene maps were generated using the safety boundary condition method and envelope method, with obstacle distribution abstractly represented by an obstacle rate. The robot's pose and orientation were determined through the application of Adaptive Monte Carlo localization. Speed sampling was conducted to identify the optimal speed combination, establishing a set of candidate trajectories. A trajectory evaluation function was constructed, and the trajectory with the highest score was selected. The improved A* algorithm was integrated into the evaluation function, incorporating the obstacle rate to refine path planning. To eliminate redundant sections, a key point extraction algorithm was employed to adjust and smooth the path, ensuring optimal obstacle avoidance. The proposed approach demonstrated superior performance, achieving a shorter planned path, a rapid obstacle avoidance reaction time of 32 milliseconds, a 100 % success rate in obstacle avoidance, and a path smoothness of 97 %. These results indicate that the method is highly effective for obstacle avoidance, with significant improvements in both safety and efficiency.

Challenges and Opportunities in Transitioning Autistic Individuals Into Adulthood.

Approximately 1.2 million autistic youth will reach the legal age of adulthood over the next decade. Given dynamic changes in the socioecological context of the transition to adulthood, we produced an updated transitions research agenda reflecting the perspectives of autistic young adults, care partners (parents), and professionals who use autism research. We conducted 9 focus groups: 4 young adult, 4 care partner, and 1 professional, with 59 total participants. Young adults were grouped by self-reported identity: (1) racial and ethnic minority, (2) lesbian, gay, bisexual, transgender, queer, intersex, asexual, and other identities, and (3) nonspeaking, in addition to a general group. We selected care partners who supported youth with varying intensity of support needs. We used rapid qualitative inquiry methods to extract key points from answers to semistructured interview questions. Delays in diagnosis and transition processes, and placement on wait lists resulted in significant obstacles to successful transitions. Parents assumed a dominant role by coordinating services, navigating systems and identifying opportunities for community participation, and providing direct supports. There was an overarching need for inclusion of autistic people in transitions research and consideration of cultural differences in priorities and values. Participants prioritized investigation of variation in transitions among autistic youth with intersecting identities, navigation support (eg, peer mentors), efficacy of services and supports (eg, transition and employment services), differences in available services by location, and problems within specific benefits programs. Research should focus on understanding population-level factors of system performance on outcomes and support needs, service delivery among marginalized groups, and transformation of complex service ecosystems.

Geometry-Aware Enhanced Mutual-Supervised Point Elimination with Overlapping Mask Contrastive Learning for Partitial Point Cloud Registration

Point cloud registration is one of the fundamental tasks in computer vision, but faces challenges under low overlap conditions. Recent approaches use transformers and overlapping masks to improve perception, but mask learning only considers Euclidean distances between features, ignores mismatches caused by fuzzy geometric structures, and is often computationally inefficient. To address these issues, we introduce a novel matching framework. Firstly, we fuse adaptive graph convolution with PPF features to obtain rich feature perception. Subsequently, we construct a PGT framework that uses GeoTransformer and combines it with location information encoding to enhance the geometry perception between source and target clouds. In addition, we improve the visibility of overlapping regions through information exchange and the AIS module, aiming at subsequent keypoint extraction, preserving points with distinct geometrical structures while suppressing the influence of non-overlapping regions to improve computational efficiency. Finally, the mask is refined through contrast learning to preserve geometric and distance similarity, which helps to compute the transformation parameters more accurately. We have conducted comprehensive experiments on synthetic and real-world scene datasets, demonstrating superior registration performance compared to recent deep learning methods. Our approach shows remarkable improvements of 68.21% in RRMSE and 76.31% in tRMSE on synthetic data, while also excelling in real-world scenarios with enhancements of 76.46% in RRMSE and 45.16% in tRMSE.

Research on Abnormal Behavior Monitoring in University Laboratories Based on Video Analysis Technology

The safety management of laboratories is of utmost importance in the construction and management of university laboratories. Abnormal behaviors such as smoking, incorrect wearing of personal protective equipment (PPE) like lab coats, hats, masks, and gloves pose significant safety hazards. In this paper, in order to improve the level of laboratory safety management and effectively provide an alert in the case of unsafe behaviors, video analysis technology is employed to achieve abnormal behavior recognition and monitoring through steps such as human key point detection, posture estimation, and behavior recognition. Firstly, the human pose estimation algorithm YOLO is used for human detection, followed by the extraction of human key points after segmentation. Finally, spatiotemporal graph convolution is used for feature detection and classification of abnormal behaviors. The experimental results show that the accuracy of abnormal behavior detection and recognition based on human key points reaches over 85%, which is of great significance for safety management and behavior warning in university laboratories, and thus, improves the efficiency and level of laboratory safety management.

An adaptive trajectory segmentation and simplification algorithm based on vessel behavioral features

Existing algorithms for trajectory simplification in AIS data processing have issues such as loss of key feature points, high subjectivity in threshold selection, and incomplete consideration of factors. In this paper, an adaptive trajectory segmentation and simplification algorithm based on vessel behavioral features is proposed. The algorithm identifies behavioral feature intervals by extracting vessel behavioral features and constructs an adaptive segmentation threshold function to achieve adaptive segmentation of trajectories and extraction of key feature points. Then, considering the quality, similarity, and spatial variability of trajectory simplification, an adaptive simplification threshold function is constructed to automatically select the best threshold for each sub-trajectory to generate the simplified trajectory. Our experiments demonstrate that the algorithm maintains a high simplification rate while enabling adaptive simplification of the trajectory, and also retains the key feature points and trajectory shapes to the maximum extent. The effectiveness and robustness of the algorithm are verified through comparison with other algorithms, effectively avoiding the distortion of navigational behavior that may be caused by traditional methods. This provides reliable data support for maritime traffic management and decision-making.

VSLAM based on deep learning in low-textured scenes

Abstract Traditional Visual Simultaneous Localization and Mapping (VSLAM) algorithms often suffer from the low positioning accuracy and robustness of the traditional feature extraction methods. These methods typically require environments with clear features, perform poorly or even fail in low-textured scenes. To solve this problem, we propose an RGB-D VSLAM algorithm that integrates a deep learning feature extraction network. First, to solve the problem in which sufficient feature points cannot be extracted, the proposed algorithm uses the R2D2 feature extraction network to extract keypoints and generate descriptors. We also perform lightweight optimization on the R2D2 network, ensuring accuracy while doubling the inference speed. Subsequently, we employ the L2 distance to measure the distance between descriptors, thereby enhancing the accuracy of keypoint matching. Ultimately, we add a dense mapping thread to obtain a dense 3D point map generated by the RGB-D information and poses of the keyframes. The experimental results on the TUM dataset show that the average absolute trajectory error is reduced by 59% and the relative trajectory error is reduced by 56%. The results demonstrate that our algorithm significantly improves localization accuracy and system robustness in low-textured environments.

Automatic geometric digital twin of box girder bridge using a laser-scanned point cloud

Geometric modeling is a pivotal step in creating a digital twin for existing bridge structures. Its deficiency of automation makes geometric modeling step time-consuming and laborious. This paper presents a solution for automatically modeling box girder bridges, including external and internal structures, based on laser-scanned point cloud. The solution includes three vital methods: component segmentation, key points extraction of cross-section, and internal structure reconstruction. The results indicate that the established segmentation model, BCR-Net, exhibited better performance than PointNet++ in component segmentation, as demonstrated by mIoU of 0.9751 on the test set. The mean absolute error on the dimension of the pier, bent cap and external and internal structure of the box girder is 0.27 %, 0.38 %, 0.47 %, and 0.48 %, respectively. It means the proposed methods possessed excellent modeling accuracy while ensuring high efficiency than manual modeling, providing a promising solution for digital twin modeling of bridge structures.

Fitcam: detecting and counting repetitive exercises with deep learning

Physical fitness is one of the most important traits a person could have for health longevity. Conducting regular exercise is fundamental to maintaining physical fitness, but with the caveat of occurring injury if not done properly. Several algorithms exists to automatically monitor and evaluate exercise using the user’s pose. However, it is not an easy task to accurately monitor and evaluate exercise poses automatically. Moreover, there are limited number of datasets exists in this area. In our work, we attempt to construct a neural network model that could be used to evaluate exercise poses based on key points extracted from exercise video frames. First, we collected several images consists of different exercise poses. We utilize the the OpenPose library to extract key points from exercise video datasets and LSTM neural network to learn exercise patterns. The result of our experiment has shown that the methods used are quite effective for exercise types of push-up, sit-up, squat, and plank. The neural-network model achieved more than 90% accuracy for the four exercise types.

Deep learning aided computer vision system for automated linear type trait evaluation in dairy cows

The assessment of traits is important in determining production potential, reproductive performance, and overall health of dairy cows. The assessment of these traits typically involves visual inspection and manual measurement, which can be time-consuming, subject to bias, and potentially distressing for the animals. To address these challenges, convolutional neural networks (CNNs)-aided non-invasive computer vision system was developed in the present study. This system consists of a depth camera to acquire the RGB images and depth information of cows. The DeepLabV3+ model, having the ResNet50 model as a backbone, was utilized to segment the body parts of cows from RGB images. Image processing-based algorithms were developed to extract key pixel locations for each trait from these segmented images. The system estimated trait dimensions utilizing 3D data of respective key points. The mean-IoU (intersection-over-union) values for the developed segmentation models were 93.46%, 91.25%, and 99.27% for side-view, back-view traits, and stature, respectively. Additionally, the vision system was able to estimate the trait dimensions with mean absolute percentage error (MAPE) below 6.0%. For a few traits, MAPE, however, exceeded 10.0%, indicating higher error. Inaccurate segmentation, imprecise key point extraction, visual overlaps of specific body parts, and variations in cow postures contribute to such errors. The developed system attained a Ratio of Performance to Deviation (RPD) above 1.2 for all traits, indicating its ability to estimate the dimensions of traits efficaciously. Thus, the present study demonstrated the potential of a CNN-based computer vision-based system for automating the trait measurement process in cows.

Deep learning model for automatic limb detection and gripping in a novel meat factory cell

The goal of this work has been the development of a deep-learning model which may be used in conjunction with a novel so-called “Meat Factory Cell” platform, namely to enable an industrial robot within the system to identify and successfully grip the limbs of an entire pig carcass. The model consists of three main components: (1) a U-Net-based deep learning model that predicts heatmaps with a probability distribution of gripping and key point locations on the limbs, within RGB-D images; (2) a post-processing element for the extraction of keypoints from heatmaps, and transferral of these points into 3D space using a pinhole camera model; and (3) gripper orientation estimation, which uses the predicted limb key points to define gripper orientation in 3D space. The proposed system demonstrates high precision and robustness in estimating gripping points on pig limbs based on a data test set, which includes two gripping definitions: Norwegian and Danish. These gripping definitions account for variation in the slaughter process in two different European countries. The Norwegian definition gives mAP(0.5…0.95) = 0.971, mAR(0.5…0.95) = 0.982, and distance error 13 mm, while the Danish definition gives mAP(0.5…0.95) = 0.985, mAR(0.5…0.95) = 0.995, and distance error 14 mm. The model was validated in practice during experimental trials at the Meat Factory Cell test facility at the Norwegian University of Life Science (Ås, Norway), with whole pig carcasses (n=25).

A Novel Video-Based Methodology for Automated Classification of Dystonia and Choreoathetosis in Dyskinetic Cerebral Palsy During a Lower Extremity Task.

Movement disorders in children and adolescents with dyskinetic cerebral palsy (CP) are commonly assessed from video recordings, however scoring is time-consuming and expert knowledge is required for an appropriate assessment. To explore a machine learning approach for automated classification of amplitude and duration of distal leg dystonia and choreoathetosis within short video sequences. Available videos of a heel-toe tapping task were preprocessed to optimize key point extraction using markerless motion analysis. Postprocessed key point data were passed to a time series classification ensemble algorithm to classify dystonia and choreoathetosis duration and amplitude classes (scores 0, 1, 2, 3, and 4), respectively. As ground truth clinical scoring of dystonia and choreoathetosis by the Dyskinesia Impairment Scale was used. Multiclass performance metrics as well as metrics for summarized scores: absence (score 0) and presence (score 1-4) were determined. Thirty-three participants were included: 29 with dyskinetic CP and 4 typically developing, age 14 years:6 months ± 5 years:15 months. The multiclass accuracy results for dystonia were 77% for duration and 68% for amplitude; for choreoathetosis 30% for duration and 38% for amplitude. The metrics for score 0 versus score 1 to 4 revealed an accuracy of 81% for dystonia duration, 77% for dystonia amplitude, 53% for choreoathetosis duration and amplitude. This methodology study yielded encouraging results in distinguishing between presence and absence of dystonia, but not for choreoathetosis. A larger dataset is required for models to accurately represent distinct classes/scores. This study presents a novel methodology of automated assessment of movement disorders solely from video data.