Skeleton Extraction Research Articles

3D mesh model segmentation based on skeleton extraction

ABSTRACT In order to solve the problem of low segmentation accuracy of 3D mesh models, a non-deep learning method based on skeleton extraction is proposed. First, the skeleton of the model is extracted. Then, the skeleton feature points are extracted by analysing the relationship between the skeleton data. The feature point with the smallest distance from the centroid in the same neighbourhood is selected as the segmentation point. Finally, based on the improved region-based algorithm, the segmented skeleton region is obtained with the segmentation points as the seeds, and the corresponding original data is segmented through inverse mapping. The experimental results show that compared with the existing non-deep learning methods, the proposed method can obtain better semantic separation results and segmentation boundaries.

Pose estimation at night in infrared images using a lightweight multi-stage attention network

Human Keypoints Detection is a relatively basic task in computer vision; it is the pre-task of human action recognition, behavior analysis and human–computer interaction. Since most abnormal actions occur at night, how to effectively extract skeleton sequence data in a low-light or completely dark environment poses a huge challenge for its identification. This paper proposes to use far infrared images to detection key points of the human body, which can solve the problem of human pose estimation under challenging weather conditions such as total darkness, smoke, inclement weather and glare. However, far-infrared images have some shortcomings, such as low resolution, noise and thermal characteristics; the skeleton data need to be provided in real time for the next stage of task. Based on the above reasons, this paper proposes a lightweight multi-stage attention network (LMANet) to detect the key points of human at night. This new network structure adds context information through the large receptive field, which helps to assist the detection of neighboring key points through this information, but for the sake of lightweight consideration, this article only extends the network to two stages. In addition, this article uses the attention module to effectively select channels with a large amount of information and highlight the features of key points, while eliminating background interference. In order to detect key points of the human in various complex environments, we use techniques such as difficult sample mining which improves the accuracy of key points with low confidence. Our network has been verified on two visible light datasets, fully demonstrating excellent performance. This paper successfully introduces far-infrared images into the field of pose estimation, because there is no public dataset for far-infrared pose estimation. In this paper, 700 images are selected for annotation from multiple public far-infrared object detection, segmentation and action recognition datasets; our algorithm is verified on this dataset; the effect is very good. After the paper is published, we will publish our key points of the human body annotated documents.

Road Extraction from High Resolution Remote Sensing Images Based on Vector Field Learning.

Accurate and up-to-date road network information is very important for the Geographic Information System (GIS) database, traffic management and planning, automatic vehicle navigation, emergency response and urban pollution sources investigation. In this paper, we use vector field learning to extract roads from high resolution remote sensing imaging. This method is usually used for skeleton extraction in nature image, but seldom used in road extraction. In order to improve the accuracy of road extraction, three vector fields are constructed and combined respectively with the normal road mask learning by a two-task network. The results show that all the vector fields are able to significantly improve the accuracy of road extraction, no matter the field is constructed in the road area or completely outside the road. The highest F1 score is 0.7618, increased by 0.053 compared with using only mask learning.

Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning

Scaffolding serves as one construction trade with high importance. However, scaffolding suffers from low productivity and high cost in Australia. Activity Analysis is a continuous procedure of assessing and improving the amount of time that craft workers spend on one single construction trade, which is a functional method for monitoring onsite operation and analyzing conditions causing delays or productivity decline. Workface assessment is an initial step for activity analysis to manually record the time that workers spend on each activity category. This paper proposes a method of automatic scaffolding workface assessment using a 2D video camera to capture scaffolding activities and the model of key joints and skeleton extraction, as well as machine learning classifiers, were used for activity classification. Additionally, a case study was conducted and showed that the proposed method is a feasible and practical way for automatic scaffolding workface assessment.

A novel approach for fracture skeleton extraction from rock surface images

The complexity and diversity of information on rock mass surfaces leads to disparities in many image properties among different rock mass surface images, including image brightness, color, contrast and pixel distribution. As a result, there is currently no unified automatic approach for extracting the rock fracture skeleton from the images. In this paper, according to common features of rock surface images and curvilinearity of fractures which are set as the extraction targets, a dark region curvilinear structure enhancement (DRCSE) algorithm for automatic rock fracture skeleton extraction is proposed. This algorithm mainly relies on the curvilinear structure enhancement filter Frangi in image pre-processing stage. Through multiple image processing, dark region contrast of rock surface images is enhanced while noise is reduced. After this process, the effect of Frangi filtering can be significantly improved. Particularly, by using Homomorphic filtering coupled Gamma transforming (HCG) to enhance dark region contrast, a new edge-entropy (EE) index is proposed to describe the contrast-enhanced image quality. Particle swarm optimization (PSO) is utilized to calculate the value of Gamma transforming parameters that maximize the EE index. The test results show that it has great advantages compared to traditional methods for rock fracture skeleton extraction.

Automated extraction and evaluation of fracture trace maps from rock tunnel face images via deep learning

This paper proposes an image-based method for automated rock fracture segmentation and fracture trace quantification. It is integrated using a CNN-based model named FraSegNet, a skeleton extraction algorithm, and a chain code-based polyline approximation algorithm. A rock tunnel fracture database with a total of 3,000 images of rock tunnel faces is established and selected to train and test the FraSegNet model. A comparison study is further conducted and shows that the FraSegNet model shows advanced performance in pixel-level fracture trace map extraction and noise reduction compared to other deep learning approaches and traditional image edge detection algorithms. Next, the skeletons of the predicted fracture trace maps are extracted and the corresponding polyline for each fracture skeleton is thus obtained and output as a text file composed of key nodes coordinates. The fracture trace characteristics (trace length, dip angle, density, and intensity) are acquired using node-based files. The quantitative evaluation of the proposed method illustrates that it can extract trace occurrences effectively and accurately. A case study of three full scale tunnel sections demonstrates the proposed method to be an efficient approach for acquiring and evaluating 2D fracture occurrences of under-construction rock tunnel faces.

The Study of Graph Convolutional Network for Activity Recognition

GCN is an excellent single-modal machine learning model for processing non-Euclidean data, such as skeleton data. Conventional GCN only uses single-modal data as its input. When we have multiple modal data, the application of GCN will be limited. To utilize multi-modal data advantages, we propose a method to combine GCN and LSTM to extract skeleton feature and acceleration feature, respectively. To test our method, we did a lot of experiments on Cooking Activity Dataset.

Traffic Police Gesture Recognition Based on Gesture Skeleton Extractor and Multichannel Dilated Graph Convolution Network

Traffic police gesture recognition is important in automatic driving. Most existing traffic police gesture recognition methods extract pixel-level features from RGB images which are uninterpretable because of a lack of gesture skeleton features and may result in inaccurate recognition due to background noise. Existing deep learning methods are not suitable for handling gesture skeleton features because they ignore the inevitable connection between skeleton joint coordinate information and gestures. To alleviate the aforementioned issues, a traffic police gesture recognition method based on a gesture skeleton extractor (GSE) and a multichannel dilated graph convolution network (MD-GCN) is proposed. To extract discriminative and interpretable gesture skeleton coordinate information, a GSE is proposed to extract skeleton coordinate information and remove redundant skeleton joints and bones. In the gesture discrimination stage, GSE-based features are introduced into the proposed MD-GCN. The MD-GCN constructs a graph convolution with a multichannel dilated to enlarge the receptive field, which extracts body topological and spatiotemporal action features from skeleton coordinates. Comparison experiments with state-of-the-art methods were conducted on a public dataset. The results show that the proposed method achieves an accuracy rate of 98.95%, which is the best and at least 6% higher than that of the other methods.

Intelligent pointer meter interconnection solution for data collection in farmlands

Some non-digitized measuring instruments such as pointer meters are utilized by some farmlands by virtue of their high stability and accuracy, and the data collection depending on the human movements requires considerable time and effort and has a low technology level. Therefore, an intelligent pointer meter indication reading and automatic collection method is proposed using machine vision and wireless sensor network (WSN) technology. An intelligent pointer meter indication reading device consisting of an image acquisition module, an intelligent control center, a wireless transmission module, and a power module was developed, which could accomplish intelligent meter indication reading and automatic data uploading. Furthermore, a pointer meter indication reading algorithm for a resource-limited environment is proposed. Firstly, the pointer and the circle axis were recognized by using edge detection and the geometrical characteristics of the pointer meter, and then, the scales and scale values were determined using the statistical characteristics of the circles around the pointer’s vertex. Next, by utilizing the density-based spatial clustering of applications with noise (DBSCAN) algorithm, we adjusted the scales and the scale values, taking full advantage of the pointer’s characteristic that the scales’ rotation angles changed linearly with the scale values, which were classified by an SVM taking in the HOG feature as the input. Finally, the indication was computed in terms of the relative position of the pointer between the adjacent scale values, and the accuracy was 94.2%. The proposed method was evaluated by comparing it with some newly published algorithms. The pointer detection method had stronger robustness and anti-interference capability than the image segmentation method and the skeleton extraction method. The Θ(n) value, indicating the complexity of the scale detection algorithm, was considerably smaller than the Θ(n2) value, denoting the modified central mapping method used mostly for scale detection. The energy consumption by each module was measured, and the results showed that the image acquisition module accounted for the largest energy consumption, 87.7%, which was more than eight times as much as the second-largest energy consumption, 10.3% (that of the wireless transmission module); the problem of power supply could be solved by using solar panels.

Crossing or Not? Context-Based Recognition of Pedestrian Crossing Intention in the Urban Environment

The booming self-driving cars need to understand the behaviors of other road users for better performance. Recognizing pedestrians’ crossing intentions is one of the most critical capabilities of self-driving cars to guarantee safe operations in the urban environment. Some researchers try to predict the future trajectories of pedestrians to avoid a potential collision. Others extract skeleton features from pedestrians to detect specific actions related to the crossing intention. All these methods either neglect the abundant appearance information of pedestrians, or work poorly under severe conditions, e.g., pedestrians standing far away, dim light, and occlusions. In this work, a pedestrian crossing intention recognition (PCIR) framework is proposed to recognize pedestrians’ crossing intention. A module for searching targets of interest is introduced to find the pedestrians who are likely to cross the streets, and perform scene perception simultaneously. An action recognition module utilizes a 3D convolutional neural network (CNN) to automatically extract spatiotemporal features that imply early actions before crossing, like limb movements. A distance encoding module makes full use of the contextual cues, e.g., distances between pedestrians and the oncoming vehicle, local traffic scenes around pedestrians, and vehicle speeds, to improve the recognition accuracy obtained from the action recognition module only. Finally, the PCIR framework fuses these factors to predict pedestrians’ crossing intentions by minimizing a focal loss. Experimental evaluations verify the effectiveness of the proposed method in recognizing pedestrians’ crossing intentions. Comparisons with the skeleton-based methods reveal the robustness of the PCIR framework in the urban environment.

DeepFlux for Skeleton Detection in the Wild

The medial axis, or skeleton, is a fundamental object representation that has been extensively used in shape recognition. Yet, its extension to natural images has been challenging due to the large appearance and scale variations of objects and complex background clutter that appear in this setting. In contrast to recent methods that address skeleton extraction as a binary pixel classification problem, in this article we present an alternative formulation for skeleton detection. We follow the spirit of flux-based algorithms for medial axis recovery by training a convolutional neural network to predict a two-dimensional vector field encoding the flux representation. The skeleton is then recovered from the flux representation, which captures the position of skeletal pixels relative to semantically meaningful entities (e.g., image points in spatial context, and hence the implied object boundaries), resulting in precise skeleton detection. Moreover, since the flux representation is a region-based vector field, it is better able to cope with object parts of large width. We evaluate the proposed method, termed DeepFlux, on six benchmark datasets, consistently achieving superior performance over state-of-the-art methods. Finally, we demonstrate an application of DeepFlux, augmented with a skeleton scale estimation module, to detect objects in aerial images. This combination yields results that are competitive with models trained specifically for object detection, showcasing the versatility and effectiveness of mid-level representations in high-level tasks. An implementation of our method is available at https://github.com/YukangWang/DeepFlux .

Human Skeleton Detection and Extraction in Dance Video Based on PSO-Enabled LSTM Neural Network.

With the significant increase of social informatization, the emerging information technology represented by machine vision has been applied to more and more scenes. Among them, the detection and extraction of human skeleton in a dance video based on this technology has a huge market demand in education and training. However, the existing detection and extraction technology has the problems of slow recognition speed and low extraction accuracy. Therefore, this paper proposes a neural network based on particle swarm optimization to detect and extract human skeletons in a dance video. Through the research and test on different data sets, it is found that the neural network based on particle swarm optimization algorithm has good detection and extraction ability and has high accuracy for the detection and recognition of human skeleton points. Among them, on all MPII data sets, the average accuracy of PSO-LSTM proposed in this paper is 3.9% higher than that of other optimal algorithms; on the PoseTrack data set, the average accuracy of detection and extraction is improved by 2.3%. The above results show that the neural network based on particle swarm optimization has fast detection speed and good extraction accuracy and can be used for the detection and extraction of human skeleton in a dance video.

Dismantling Networks by Skeleton Extraction and Greedy Tree Breaking

Network dismantling is one of the important NP-hard problems in the field of social network analysis. It aims to break down networks into many small components of limited size by only removing a small group of nodes. One feasible way is to decycle (eliminating all the cycles) the network first and then break the acyclic graph. However, existing decycling-based algorithms mainly concentrate on the decycling step, ignoring the importance of the tree breaking process. Besides, none of the algorithms try to pre-process the network, which may bring improvement in both effectiveness and efficiency. In this paper, we fill these two gaps by proposing a novel network dismantling algorithm that combines skeleton extraction and greedy tree breaking (SEGTB). Network skeleton supports the whole network structure, whose removal would leave a much looser structure and serves as an effective pre-processing for the dismantling problem. The presented tree breaking method is provided with theoretical proofs on its lower bound. Experiments on ten real-world datasets show that our proposed SEGTB algorithm is both effective and efficient, outperforming the state-of-the-art.

High-Accuracy On-Site Measurement of Wheel Tread Geometric Parameters by Line-Structured Light Vision Sensor

Railway wheels are one of the important parts of railway vehicles, and it is necessary to inspect their status frequently. To measure the wheel tread geometric parameters dynamically and accurately in a train running environment, a line-structured light vision sensor-based measurement system was demonstrated in this paper. In the system, a subpixel laser stripe centre extraction algorithm based on skeleton extraction, which can significantly reduce the interference of complex lighting environments in the field and greatly improve the extraction accuracy, was proposed. To further improve the measurement accuracy and stability, the influence of the eccentricity error caused by the dynamic measurement on the results was analyzed, a mathematical model was established, and the deformed profile was corrected. The system has been successfully applied to railway maintenance sections and has become a part of train safety inspection systems. Field tests were conducted to verify the performance of the system, and the results showed that the measurement accuracy and stability are marked improved after eccentric error correction, especially for the flange height and QR value.

Pose recognition in sports scenes based on deep learning skeleton sequence model

Human skeleton extraction is a basic problem in the field of computer vision. With the rapid progress of science and technology, it has become a hot issue in the field of target detection such as pedestrian recognition, behavior monitoring, and pedestrian gesture recognition. In recent years, due to the development of deep neural networks, modeling of human joints in acquired images has made progress in skeleton extraction. However, most models have low modeling accuracy, poor real-time performance, and poor model availability. problem. Aiming at the above-mentioned human target detection problem, this paper uses the deep learning skeleton sequence model gesture recognition method in sports scenes to study, aiming to provide a gesture recognition method with strong noise resistance, good real-time performance and accurate model. This article uses motion video frame images to train the VGG16 network. Using the network to extract skeleton information can strengthen the posture feature expression, and use HOG for feature extraction, and use the Adam algorithm to optimize the network to extract more posture features, thereby improving the posture of the network Recognition accuracy. Then adjust the hyperparameters and network structure of the basic network according to the training results, and obtain the key poses in the sports scene through the final classifier.

Analysis of Stadium Operation Risk Warning Model Based on Deep Confidence Neural Network Algorithm.

In this paper, a deep confidence neural network algorithm is used to design and deeply analyze the risk warning model for stadium operation. Many factors, such as video shooting angle, background brightness, diversity of features, and the relationship between human behaviors, make feature attribute-based behavior detection a focus of researchers' attention. To address these factors, researchers have proposed a method to extract human behavior skeleton and optical flow feature information from videos. The key of the deep confidence neural network-based recognition method is the extraction of the human skeleton, which extracts the skeleton sequence of human behavior from a surveillance video, where each frame of the skeleton contains 18 joints of the human skeleton and the confidence value estimated for each frame of the skeleton, and builds a deep confidence neural network model to classify the dangerous behavior based on the obtained skeleton feature information combined with the time vector in the skeleton sequence and determine the danger level of the behavior by setting the corresponding threshold value. The deep confidence neural network uses different feature information compared with the spatiotemporal graph convolutional network. The deep confidence neural network establishes the deep confidence neural network model based on the human optical flow information, combined with the temporal relational inference of video frames. The key of the temporal relationship network-based recognition method is to extract some frames from the video in an orderly or random way into the temporal relationship network. In this paper, we use several methods for comparison experiments, and the results show that the recognition method based on skeleton and optical flow features is significantly better than the algorithm of manual feature extraction.

MSB-FCN: Multi-Scale Bidirectional FCN for Object Skeleton Extraction.

The performance of state-of-the-art object skeleton detection (OSD) methods have been greatly boosted by Convolutional Neural Networks (CNNs). However, the most existing CNN-based OSD methods rely on a 'skip-layer' structure where low-level and high-level features are combined to gather multi-level contextual information. Unfortunately, as shallow features tend to be noisy and lack semantic knowledge, they will cause errors and inaccuracy. Therefore, in order to improve the accuracy of object skeleton detection, we propose a novel network architecture, the Multi-Scale Bidirectional Fully Convolutional Network (MSB-FCN), to better gather and enhance multi-scale high-level contextual information. The advantage is that only deep features are used to construct multi-scale feature representations along with a bidirectional structure for better capturing contextual knowledge. This enables the proposed MSB-FCN to learn semantic-level information from different sub-regions. Moreover, we introduce dense connections into the bidirectional structure to ensure that the learning process at each scale can directly encode information from all other scales. An attention pyramid is also integrated into our MSB-FCN to dynamically control information propagation and reduce unreliable features. Extensive experiments on various benchmarks demonstrate that the proposed MSB-FCN achieves significant improvements over the state-of-the-art algorithms.

Evaluation of rods deformation of metal lattice structure in additive manufacturing based on skeleton extraction technology.

The components with lattice structure as filling unit have great application potential in aerospace and other fields. The failure of the lattice structure directly affects the functional characteristics of the parts filled with the lattice structure. Aiming at the problem that it is difficult to evaluate the deformation degree of metal lattice structure after mechanical loading in additive manufacturing, firstly, the point cloud model of lattice structure is obtained by using CT scanning and three-dimensional reconstruction, and then the skeleton of lattice structure is automatically extracted based on L1 median algorithm. Finally, the deformation angle of rods is measured to evaluate the degree of deformation and damage of parts. In this paper, the deformation evaluation of the rods of the BCC lattice is discussed. The experimental results show that the proposed skeleton extraction technology achieves the evaluation of lattice structure deformation. The experimental model is extended to BCC lattice structure with unit cell number of n×n×n. When the ratio of the rods with more than 40% severe deformation to all rods in the lattice structure reaches (2n-1)/2n2 it indicates that the lattice structure has undergone a large degree of deformation and should not continue to serve.

Image Co-Skeletonization via Co-Segmentation.

Recent advances in the joint processing of a set of images have shown its advantages over individual processing. Unlike the existing works geared towards co-segmentation or co-localization, in this article, we explore a new joint processing topic: image co-skeletonization, which is defined as joint skeleton extraction of the foreground objects in an image collection. It is well known that object skeletonization in a single natural image is challenging, because there is hardly any prior knowledge available about the object present in the image. Therefore, we resort to the idea of image co-skeletonization, hoping that the commonness prior that exists across the semantically similar images can be leveraged to have such knowledge, similar to other joint processing problems such as co-segmentation. Moreover, earlier research has found that augmenting a skeletonization process with the object's shape information is highly beneficial in capturing the image context. Having made these two observations, we propose a coupled framework for co-skeletonization and co-segmentation tasks to facilitate shape information discovery for our co-skeletonization process through the co-segmentation process. While image co-skeletonization is our primary goal, the co-segmentation process might also benefit, in turn, from exploiting skeleton outputs of the co-skeletonization process as central object seeds through such a coupled framework. As a result, both can benefit from each other synergistically. For evaluating image co-skeletonization results, we also construct a novel benchmark dataset by annotating nearly 1.8 K images and dividing them into 38 semantic categories. Although the proposed idea is essentially a weakly supervised method, it can also be employed in supervised and unsupervised scenarios. Extensive experiments demonstrate that the proposed method achieves promising results in all three scenarios.

Blending Surface Segmentation and Editing for 3D Models

Recognizing and fitting shape primitives from underlying 3D models are key components of many computer graphics and computer vision applications. Although a vast number of structural recovery methods are available, they usually fail to identify blending surfaces, which corresponds to small transitional regions among relatively large primary patches. To address this issue, we present a novel approach for automatic segmentation and surface fitting with accurate geometric parameters from 3D models, especially mechanical parts. Overall, we formulate the structural segmentation as a Markov random field (MRF) labeling problem. In contrast to existing techniques, we first propose a new clustering algorithm to build superfacets by incorporating 3D local geometric information. This algorithm extracts the general quadric and rolling-ball blending regions, and improves the robustness of further segmentation. Next, we apply a specially designed MRF framework to efficiently partition the original model into different meaningful patches of known surface types by defining the multilabel energy function on the superfacets. Furthermore, we present an iterative optimization algorithm based on skeleton extraction to fit rolling-ball blending patches by recovering the parameters of the rolling center trajectories and ball radius. Experiments on different complex models demonstrate the effectiveness and robustness of the proposed method, and the superiority of our method is also verified through comparisons with state-of-the-art approaches. We further apply our algorithm in applications such as mesh editing by changing the radius of the rolling balls.