Local Feature Points Research Articles

Object Pose Estimation Incorporating Projection Loss and Discriminative Refinement

The accurate estimation of three-dimensional (3D) object pose is important in a wide range of applications, such as robotics and augmented reality. The key to estimate object poses is matching feature points in the captured image with predefined ones of the 3D model of the object. Existing learning-based pose estimation systems utilize a voting strategy to estimate the feature points in a vector space for improving the accuracy of the estimated pose. However, the loss function of such approaches only takes account of the direction of the vector, resulting in an error-prone localization of feature points. Therefore, this paper considers a projection loss function dealing with the error of the vector field and incorporates a refinement network to revise the predicted pose to obtain a good final output. Experimental results show that the proposed methods outperform the state-of-the-art methods in ADD(-S) metric on the LINEMOD and Occlusion LINEMOD datasets. Moreover, the proposed method can be applied to real-world practical scenarios in real time to simultaneously estimate the poses of multiple objects.

Automatic facial feature points location based on deep learning: a review

Automatic facial feature points location based on deep learning: a review

Detection of Chemically Matured Mango Fruits Using Laplacian Descriptors and Scale Determinants

Nowadays machine vision systems are largely used for Fruit harvesting, fruit tracking, fruit quality assessment, and detection of artificial ripening of fruits. Existing research concentrates on shape-based fruit harvesting using robots, 3D reconstruction of images for in field fruit tracking systems, non – destructive methods for quality assessment of fruits, and detection of artificially ripened fruits. Besides shape, the color of the fruit is a unique and protruding feature. Though a lot of effort is focused on harvesting, tracking, and quality assessment of mango fruits using these features, fewer efforts are made in the path of the identification of artificially matured mango fruits. Simpler, robust, and easily accessible solutions are desired by exploiting both RGB descriptors and feature point locators. Here a new context for detection of artificially matured mango fruit based on SURF descriptors is proposed. In particular, Feature Point Location (FPL) centroid values, approximated Laplacian matrix values, and Scale determinant values are used to identify artificial matured mango fruit. The results of the experiment are conducted on two different datasets which demonstrates the efficiency of the method proposed.

LPCF: Robust Correlation Tracking via Locality Preserving Tracking Validation.

In visual tracking, the tracking model must be updated online, which often leads to undesired inclusion of corrupted training samples, and hence inducing tracking failure. We present a locality preserving correlation filter (LPCF) integrating a novel and generic decontamination approach, which mitigates the model drift problem. Our decontamination approach maintains the local neighborhood feature points structures of the bounding box center. This proposed tracking-result validation approach models not only the spatial neighborhood relationship but also the topological structures of the bounding box center. Additionally, a closed-form solution to our approach is derived, which makes the tracking-result validation process could be accomplished in only milliseconds. Moreover, a dimensionality reduction strategy is introduced to improve the real-time performance of our translation estimation component. Comprehensive experiments are performed on OTB-2015, LASOT, TrackingNet. The experimental results show that our decontamination approach remarkably improves the overall performance by 6.2%, 12.6%, and 3%, meanwhile, our complete algorithm improves the baseline by 27.8%, 34.8%, and 15%. Finally, our tracker achieves the best performance among most existing decontamination trackers under the real-time requirement.

Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN.

With a focus on fatigue driving detection research, a fully automated driver fatigue status detection algorithm using driving images is proposed. In the proposed algorithm, the multitask cascaded convolutional network (MTCNN) architecture is employed in face detection and feature point location, and the region of interest (ROI) is extracted using feature points. A convolutional neural network, named EM-CNN, is proposed to detect the states of the eyes and mouth from the ROI images. The percentage of eyelid closure over the pupil over time (PERCLOS) and mouth opening degree (POM) are two parameters used for fatigue detection. Experimental results demonstrate that the proposed EM-CNN can efficiently detect driver fatigue status using driving images. The proposed algorithm EM-CNN outperforms other CNN-based methods, i.e., AlexNet, VGG-16, GoogLeNet, and ResNet50, showing accuracy and sensitivity rates of 93.623% and 93.643%, respectively.

Building-Specific Recognition Algorithms Based on Image Sequence

Building extraction, recognition and corresponding three-dimensional modeling in image is an active research field, which is mainly applied to map feature extraction, urban population survey and so on. It has important significance in military and civil areas. Aiming at the shortcoming of the classical invariant feature extraction algorithm, a building recognition algorithm based on local feature and shape contour matching is proposed. Firstly, according to the fact that the whole building is susceptible to rotation and inclination, the local feature points of known buildings are extracted, and their orientation, position and angle are determined. At the same time, the similarity criterion and mapping function of arbitrary shape contour matching are formulated according to the scale change information of buildings in different states, and the building recognition under different illumination and scales is realized. The simulation results show that the method can quickly and accurately adapt to the construction in different environments. The accuracy of this method fluctuates around 97%, while the average accuracy of 20 experiments is 96.56%. The average accuracy of the SFBR method is 94.66%. Compared with the SFBR and SIFT methods, the average accuracy of the method is improved by nearly 2%.

STACKED LOCAL FEATURE DETECTOR FOR HYPERSPECTRAL IMAGE

Abstract. Images registration is an important task in hyperspectral image processing, while almost all local feature point based image matching algorithm is designed for single band image only and miss the advantage of abundant spectral information. Therefor, in this paper we propose a novel local feature detector for hyperspectral image. This method, which is named stacked local feature detector (HSI-SFD), stack all local feature points detected from every single spectral band. With redundant information, local feature miss-detected at noise pixel can be filtered out by the stack count threshold, leading to more reliable and robust local features. To verify the algorithm, a hyperspectral image matching dataset is built. Multiview hyperspectral images of the several different flat targets are taken by scan-line hyperspectral camera in library. Each image contains 270 bands within 400–1000 nm. Groundtruth transformation matrix between images is computed from corresponding points selected by hands. Experiment on the dataset shows that features points miss-detected at texture-less area can be inhibited by HSI-SFD. Both keypoint repeatability and matching accuracy increase significantly. Precision and matching score increase up to 10% for some scene.

Feature Analysis and Target Detection of High-resolution Remote Sensing Image of Coal Mining Subsidence Area

Aiming at the phenomenon of regional ground collapse caused by mining, the site information is obtained through field investigation, and the features of typical collapse targets are described. The drone remote sensing images are used to extract and analyze the color features and shape features of typical collapse targets. Combine the Hough transform, shadow and Harris corner detection methods to achieve the target detection of collapsed pits and trough collapsed areas. The combined method uses Hough transform to detect the edge shape curve of the collapsed pits and trough collapsed areas, and use shadow detection method to extract its internal areas, and Harris corner detection method is used to locate local feature points in the image. Experimental analysis results show that the method has good application effect on target detection in collapsed pits and trough collapsed areas.

Attention-based relation and context modeling for point cloud semantic segmentation

Semantic segmentation of point cloud is a fundamental problem in scene-level understanding. Despite advancement in recent years by leveraging capabilities of Neural Networks and massive labeling datasets available, providing fine-grained semantic segmentation for point cloud is still challenging, given the fact that point cloud is usually unstructured, unordered and sparse. In this paper, we achieve semantic point cloud labeling by adaptively exploring semantic relation and aggregating contextual information between points. Specifically, we first introduce an attention-based local relation learning module for collecting local features, which can capture semantic relation in a manner of anisotropy. And we then design a novel context aggregation module guided by multi-scale supervision to obtain long-range dependencies between semantically-correlated points and enhance the distinctive ability of points in feature space. In addition, a gated propagation strategy is adopted instead of skip links to conditionally concatenate local point features in different layers. We empirically evaluate our method on public benchmarks (S3DIS and ShapeNetPart), and demonstrate our performance is on par or better than state-of-the-art methods.

Challenge and Algorithm of Face Alignment Development

Face alignment is a key step in face recognition. The location of face feature points is located in the face image, and the difference between different faces is reduced by geometric transformation. This is the basic condition of face information processing, such as expression recognition, face tracking, head pose estimation and so on. Due to the interference of expression, illumination, shading and other factors, face alignment has a great challenge and is becoming the developmental direction. Different algorithms can solve different problems at different levels. Deep learning algorithm can solve the shortcomings of traditional algorithm, improve the accuracy of face alignment, and promote the development of face alignment.

Precise vehicle ego-localization using feature matching of pavement images

Purpose Precise vehicle localization is a basic and critical technique for various intelligent transportation system (ITS) applications. It also needs to adapt to the complex road environments in real-time. The global positioning system and the strap-down inertial navigation system are two common techniques in the field of vehicle localization. However, the localization accuracy, reliability and real-time performance of these two techniques can not satisfy the requirement of some critical ITS applications such as collision avoiding, vision enhancement and automatic parking. Aiming at the problems above, this paper aims to propose a precise vehicle ego-localization method based on image matching. Design/methodology/approach This study included three steps, Step 1, extraction of feature points. After getting the image, the local features in the pavement images were extracted using an improved speeded up robust features algorithm. Step 2, eliminate mismatch points. Using a random sample consensus algorithm to eliminate mismatched points of road image and make match point pairs more robust. Step 3, matching of feature points and trajectory generation. Findings Through the matching and validation of the extracted local feature points, the relative translation and rotation offsets between two consecutive pavement images were calculated, eventually, the trajectory of the vehicle was generated. Originality/value The experimental results show that the studied algorithm has an accuracy at decimeter-level and it fully meets the demand of the lane-level positioning in some critical ITS applications.

Face pose estimation of SDM based on multi-template matching

This paper presents the face pose research of SDM algorithm and POSIT algorithm with multiple template matching. Firstly we detect the input face image, locate the feature points in the detected facial area, then compare the traditional SDM algorithm with the improved multi-template SDM algorithm, and then select the multi-template SDM algorithm. Feature point location is finally applied to the POSIT algorithm. Experimental results prove the effectiveness of the method and make it competitive.

Sparse metric-based mesh saliency

In this paper, we propose an accurate and robust approach to salient region detection for 3D polygonal surface meshes. The salient regions of a mesh are those that geometrically stand out from their contexts and therefore are semantically important for geometry processing and shape analysis. However, a suitable definition of region contexts for saliency detection remains elusive in the field, and the previous methods fail to produce saliency maps that agree well with human annotations. We address these issues by computing saliency in a global manner and enforcing sparsity for more accurate saliency detection. Specifically, we represent the geometry of a mesh using a metric that globally encodes the shape distances between every pair of local regions. We then propose a sparsity-enforcing rarity optimization problem, solving which allows us to obtain a compact set of salient regions globally distinct from each other. We build a perceptually motivated 3D eye fixation dataset and use a large-scale Schelling saliency dataset for extensive benchmarking of saliency detection methods. The results show that our computed saliency maps are closer to the ground-truth. To showcase the usefulness of our saliency maps for geometry processing, we apply them to feature point localization and achieve higher accuracy compared to established feature detectors.

Real-time tumor localization with single x-ray projection at arbitrary gantry angles using a convolutional neural network (CNN)

For tumor tracking therapy, precise knowledge of tumor position in real-time is very important. A technique using single x-ray projection based on a convolutional neural network (CNN) was recently developed which can achieve accurate tumor localization in real-time. However, this method was only validated at fixed gantry angles. In this study, an improved technique is developed to handle arbitrary gantry angles for rotational radiotherapy.To evaluate the highly complex relationship between x-ray projections at arbitrary angles and tumor motion, a special CNN was proposed. In this network, a binary region of interest (ROI) mask was applied on every extracted feature map. This avoids the overfitting problem due to gantry rotation by directing the network to neglect those irrelevant pixels whose intensity variation had nothing to do with breathing motion. In addition, an angle-dependent fully connection layer (ADFCL) was utilized to recover the mapping from extracted feature maps to tumor motion, which would vary with the gantry angles. The method was tested with images from 15 realistic patients and compared with a variant network of VGG, developed by Oxford University’s Visual Geometry Group.The tumors were clearly visible on x-ray projections for five patients only. The average tumor localization error was under 1.8 mm and 1.0 mm in superior-inferior and lateral directions. For the other ten patients whose tumors were not clearly visible in the x-ray projection, a feature point localization error was computed to evaluate the proposed method, the mean value of which was no more than 1.5 mm and 1.0 mm in both directions for all patients.A tumor localization method for single x-ray projection at arbitrary angles based on a novel CNN was developed and validated in this study for real-time operation. This greatly expanded the applicability of the tumor localization framework to the rotation therapy.

Wavelet-based real-time calculation of multiple physiological parameters on an embedded platform

Objective: This paper aims to present how physiological signals can be processed based on wavelet decomposition to calculate multiple physiological parameters in real-time on an embedded platform. Approach: An ECG and PPG are decomposed to the appropriate scale based on a quadratic spline wavelet base in order to obtain high and narrow pulse peaks at the location of the mutation points. Based on the decomposed waveforms, feature points are positioned to calculate physiological parameters in real-time, including heart rate, pulse rate, blood oxygen, and blood pressure. The proposed algorithm has been implemented on a Texas Instruments’ CC2640R2F. Main results: The misdetection rate of feature point location based on the square wavelet decomposition waveform is only 0.57% in the acquired ECG and 0.23% in the acquired PPG. Heart rate and pulse rate are both highly correlated with the reference, both having correlation coefficients of 0.99. The pulse rate and heart rate are 3.85% (51/1326) and 2.94% (39/1326) outside the 95% consistency limit, respectively. The systolic and diastolic blood pressures are significantly associated with standard equipment measurements, with correlation coefficients of 0.87 and 0.83. The systolic and diastolic blood pressures were 5.88% (21/357) and 5.32% (19/357) outside the 95% consistency limit, respectively. Significance: The real-time calculation of multiple physiological parameters based on wavelet decomposition on an embedded platform presented here shows outstanding accuracy.

TGNet: Geometric Graph CNN on 3-D Point Cloud Segmentation

Recent geometric deep learning works define convolution operations in local regions and have enjoyed remarkable success on non-Euclidean data, including graph and point clouds. However, the high-level geometric correlations between the input and its neighboring coordinates or features are not fully exploited, resulting in suboptimal segmentation performance. In this article, we propose a novel graph convolution architecture, which we term as Taylor Gaussian mixture model (GMM) network (TGNet), to efficiently learn expressive and compositional local geometric features from point clouds. The TGNet is composed of basic geometric units, TGConv, that conduct local convolution on irregular point sets and are parametrized by a family of filters. Specifically, these filters are defined as the products of the local point features and the neighboring geometric features extracted from local coordinates. These geometric features are expressed by Gaussian weighted Taylor kernels. Then, a parametric pooling layer aggregates TGConv features to generate new feature vectors for each point. TGNet employs TGConv on multiscale neighborhoods to extract coarse-to-fine semantic deep features while improving its scale invariance. Additionally, a conditional random field (CRF) is adopted within the output layer to further improve the segmentation results. Using three point cloud data sets, qualitative and quantitative experimental results demonstrate that the proposed method achieves 62.2% average accuracy on ScanNet, 57.8% and 68.17% mean intersection over union (mIoU) on Stanford Large-Scale 3D Indoor Spaces (S3DIS) and Paris-Lille-3D data sets, respectively.

Built-up area detection from high resolution remote sensing images using geometric features

Based on characteristic of High Spatial Resolution(HSR) image, an adaptive local geometric invariant feature extraction method based on Gabor transform and Tensor Voting(TV) is proposed and applied to building area extraction from HSR remote sensing images.First, image geometric features are analyzed. Second, the feasibility of extracting built-up area using local feature points and probability density estimation are determined. This study provides specific methods and steps. First, considering the abundant geometric features of high spatial resolution remote sensing images, multi-scale and multi-direction Gabor filter banks are adopted in detecting the singularity of images contain building area. In order to extract edge information of buildings, only real part Gabor coefficients are used. Meanwhile, we also measured the influence of Gabor kernels of different size on geometric feature extraction experiments, and the optimal scale parameter intervals for geometric feature extraction of remote sensing images with different spatial resolutions are thus determined. Second, it is a fact that the response of Gabor filter at each pixel is a measure for orientation certainty, thus, we introduce the orientation tensor which represents an ideal direction in the direction of the unit vector perpendicular to frequency. By weighting the orientation tensor with Gabor coefficients ans summing over it to complete the information fusion, the resulting tensor givens an estimate for local orientation and orientation uncertainty at image position. The tensors are then used as an initial estimate for global context refinement using tensor voting and the points are classified based on the their likelihoods of being part of a feature type, non-maximal suppression is used to extract geometric features. A key advantage of combining the Gabor filtering and tensor voting is that it eliminates the need for any thresholds therefore removing any data dependencies. In order to achieve a reliable extraction of local invariant feature such as corners from built-up area, three criterions are further proposed to refine the first stage result. Finally, each local building corner indicates a building be detected in image. However, only one of them is not sufficient alone to detecting a building. In fact, the more local points a buildings has, the more probable its detection becomes. Based on that fact, a probability density estimate method is generated to describe the probability that the pixel belongs to the building area, and the Otsu method is used to automatically extract the polygon area of the residential area. Experiments were carried out using image data sets such as Google and GF-2 with resolution higher than 1 meter, results showing that the proposed method can achieve higher accuracy in building area detection compared with the state-of-the-art corner detection algorithm such as Harris corner and High-speed corner detect method.

Automatic measurement of traditional Chinese costume from its silhouette through Fuzzy c-means clustering method

As the precious cultural heritage of the Chinese nation, traditional costumes are in urgent need of scientific research and protection. In particular, there are scanty studies on costume silhouettes, due to the reasons of the need for cultural relic protection, and the strong subjectivity of manual measurement, which limit the accuracy of quantitative research. This paper presents an automatic measurement method for traditional Chinese costume dimensions based on fuzzy C-means clustering and silhouette feature point location. The method is consisted of six steps: (1) costume image acquisition; (2) costume image preprocessing; (3) color space transformation; (4) object clustering segmentation; (5) costume silhouette feature point location; and (6) costume measurement. First, the relative total variation model was used to obtain the environmental robustness and costume color adaptability. Second, the FCM clustering algorithm was used to implement image segmentation to extract the outer silhouette of the costume. Finally, automatic measurement of costume silhouette was achieved by locating its feature points. The experimental results demonstrated that the proposed method could effectively segment the outer silhouette of a costume image and locate the feature points of the silhouette. The measurement accuracy could meet the requirements of industrial application, thus providing the dual value of costume culture research and industrial application.

Region-based online flaw detection of 3D printing via fringe projection

Additive manufacturing allows for the production of custom parts with complex internal features. However, a lack of online detection of the printing process is one of the challenges that has prevented further improvements in additive manufacturing. This paper proposes a novel online monitoring technique based on a fringe projector to keep the stability of the process under control on a layer-by-layer basis. To detect microdefects with a magnitude smaller than the noise, we propose a region-based defect detection method that takes each subregion, instead of each point, as a detection unit. Compared with a single point, a sub region contains hundreds of points, which represent a domain with greater stability. We combine voxel cloud connectivity segmentation (VCCS) and fast point feature histogram (FPFH) to divide the printing area into hundreds of subregions, smooth the surface to resist the noise based on moving least square (MLS) and evaluate each subregion using local point features. The projection model and normal deviation are developed to extract the sizes and locations of the flaws. The performances of the techniques are demonstrated with some experiments using different 3D printing objects, which indicate that the proposed method is able to detect various flaws and its detection accuracy is higher than the noise magnitude.

A Novel Information Fusion Method for Vision Perception and Location of Intelligent Industrial Robots

An improved SURF (Speeded-Up Robust Feature） algorithm is proposed to deal with the time-consuming and low precision of positioning of industrial robot. Hessian matrix determinant is used to extract feature points from the target image and a multi-scale spatial pyramid is constructed. The location and scale value of feature points are determined by neighbourhood non-maximum suppression method. The direction of feature points is defined as directional feature descriptors by the binary robust independent elementary feature (BRIEF). The progressive sample consensus (PROSAC) is used to carry out second precise matching and remove mismatching points based on the Hamming distance. Then, an affine transformation model is established to describe the relationship between the template and target images. Centroid coordinates of the target can be obtained based on the affine transformation. Comparative tests were carried out to demonstrate that the proposed method can effectively improve the recognition rate and positioning accuracy of the industrial robots. The average time consuming is less than 0.2 s, the matching accuracy is 96 %, and the positioning error of the robot is less than 1.5 mm. Therefore, the proposed method has practical application importance.