Salient Point Detection Research Articles

Levenberg–Marquardt deep neural watermarking for 3D mesh using nearest centroid salient point learning

Watermarking is one of the crucial techniques in the domain of information security, preventing the exploitation of 3D Mesh models in the era of Internet. In 3D Mesh watermark embedding, moderately perturbing the vertices is commonly required to retain them in certain pre-arranged relationship with their neighboring vertices. This paper proposes a novel watermarking authentication method, called Nearest Centroid Discrete Gaussian and Levenberg–Marquardt (NCDG–LV), for distortion detection and recovery using salient point detection. In this method, the salient points are selected using the Nearest Centroid and Discrete Gaussian Geometric (NC–DGG) salient point detection model. Map segmentation is applied to the 3D Mesh model to segment into distinct sub regions according to the selected salient points. Finally, the watermark is embedded by employing the Multi-function Barycenter into each spatially selected and segmented region. In the extraction process, the embedded 3D Mesh image is extracted from each re-segmented region by means of Levenberg–Marquardt Deep Neural Network Watermark Extraction. In the authentication stage, watermark bits are extracted by analyzing the geometry via Levenberg–Marquardt back-propagation. Based on a performance evaluation, the proposed method exhibits high imperceptibility and tolerance against attacks, such as smoothing, cropping, translation, and rotation. The experimental results further demonstrate that the proposed method is superior in terms of salient point detection time, distortion rate, true positive rate, peak signal to noise ratio, bit error rate, and root mean square error compared to the state-of-the-art methods.

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Learning to locate for fine-grained image recognition

In this paper, we propose an end-to-end weakly supervised method for fine-grained image recognition called bounding box-part location method(BBPL), which can locate the object and part precisely without part annotations. The proposed method includes three modules: object detection, ObjectMask, and classification. Firstly, the object detection module predicts the bounding boxes, and the predicted bounding boxes are employed to generate a mask through ObjectMask module. The generated mask can suppress the background interference during recognition. Secondly, the classification module can be further divided into two branches, which are global feature classification and local feature classification. In global feature classification branch, global feature is extracted to get global classification result. While in local feature classification branch, salient point is first detected through our novel salient point detection module, which can greatly reduce the consuming-time compared with the most existing local feature extraction methods. Further, the local feature is extracted in these detected salient points, and local classification result is obtained by local feature classification branch. Finally, we get the final result by fusing the results of two classification branches together. With experiments on three widely used fine-grained image recognition datasets (CUB-200-2011, Stanford Cars, Stanford Dogs), our method can achieve the state-of-the-art performance.

Monocular Visual-IMU Odometry: A Comparative Evaluation of Detector–Descriptor-Based Methods

Monocular visual-inertial measurement unit (IMU) odometry has been widely used in various intelligent vehicles. As a popular technique, detector–descriptor-based visual-IMU odometry is effective and efficient due to the fact that local descriptors are robust against occlusions, background clutter, and abrupt content changes. However, to our knowledge, there is not a comprehensive and comparative evaluation study on the performance of different combinations of detectors and descriptors recently developed. In order to bridge this gap, we conduct such a comparative study in a unified framework. In particular, six typical routes with different lengths, shapes, and road scenes are selected from the well-known KITTI dataset. We first evaluate the performance of different combinations of salient point detectors and local descriptors using the six routes. Then, we tune the parameters of the best detector or descriptor obtained for each route, to further augment the results. This paper provides not only comprehensive benchmarks for assessing various algorithms but also instructive guidelines and insights for developing detectors and descriptors to handle different road scenes.

Three-dimensional salient point detection based on the Laplace–Beltrami eigenfunctions

Three-dimensional (3D) salient point detection is a fundamental problem in computer graphics and computer vision. We propose a new method using the Laplace–Beltrami eigenfunctions for detecting the salient points of 3D models. We compute the extrema of the low-frequency Laplace–Beltrami eigenfunctions and remove the redundancy of the extrema. Merging the extrema of the eigenfunctions, we keep the extrema appearing simultaneously on no less than a certain number of eigenfunctions as the candidate salient points. Clustering these extrema, we consider the representatives of the clusters as the final salient points. Our experimental results demonstrate that the proposed method is effective for 3D salient point detection. Besides that, some experiments are also conducted to verify that the proposed method is insensitive to small boundary noise.

Formation of Energy Features of the Image based on Wavelet Transform

In this paper the new approach of image analysis using energy characteristics of wavelet transform is proposed. We consider a model based on the wavelet transform, which makes it possible to form features characterizing the energy estimates of the image points and reflecting their significance. To form a model of energy characteristics, a procedure is performed based on taking into account the dependencies between the wavelet coefficients of various levels. This approach based on image representation using of energy features models can be used for contours detection, salient points detection and texture features extraction. Energy features models can be used as the basis to detect and recognize the object in various systems of information processing and control, such as systems of unmanned aerial vehicle control, distant exploration of the Earth systems, access control systems for protected objects etc.

DINOSARC: Color Features Based on Selective Aggregation of Chromatic Image Components for Wireless Capsule Endoscopy.

Wireless Capsule Endoscopy (WCE) is a noninvasive diagnostic technique enabling the inspection of the whole gastrointestinal (GI) tract by capturing and wirelessly transmitting thousands of color images. Proprietary software “stitches” the images into videos for examination by accredited readers. However, the videos produced are of large length and consequently the reading task becomes harder and more prone to human errors. Automating the WCE reading process could contribute in both the reduction of the examination time and the improvement of its diagnostic accuracy. In this paper, we present a novel feature extraction methodology for automated WCE image analysis. It aims at discriminating various kinds of abnormalities from the normal contents of WCE images, in a machine learning-based classification framework. The extraction of the proposed features involves an unsupervised color-based saliency detection scheme which, unlike current approaches, combines both point and region-level saliency information and the estimation of local and global image color descriptors. The salient point detection process involves estimation of DIstaNces On Selective Aggregation of chRomatic image Components (DINOSARC). The descriptors are extracted from superpixels by coevaluating both point and region-level information. The main conclusions of the experiments performed on a publicly available dataset of WCE images are (a) the proposed salient point detection scheme results in significantly less and more relevant salient points; (b) the proposed descriptors are more discriminative than relevant state-of-the-art descriptors, promising a wider adoption of the proposed approach for computer-aided diagnosis in WCE.

3D mesh watermarking using salient points

In this paper, we propose a robust and blind watermarking algorithm for 3D mesh models. Firstly, we extract salient feature points using a robust salient point detector based on Auto Diffusion Function (ADF). Afterwards, the mesh is segmented into different regions according to the detected salient points. Finally, we embed the watermark statistically into each region. During watermark embedding, the vertex norms ρ are decomposed into normalized bins. Then, the watermark is embedded by modifying the amplitude ρ depending on the watermark bit and the mean of each bin. In the extraction process, we extract the signature from each re-segmented region. Experiments conducted with a variety of mesh models evidenced the competitive performance of our watermarking scheme, in terms of the robustness and invisibility, when compared to other state of the art methods.

Salient Point Detection in Protrusion Parts of 3D Object Robust to Isometric Variations

In this paper, a novel method is proposed to detect 3D object salient points robust to isometric variations and stable against scaling and noise. Salient points can be used as the representative points from object protrusion parts in order to improve the object matching and retrieval algorithms. The proposed algorithm is started by determining the first salient point of the model based on the average geodesic distance of several random points. Then, according to the previous salient point, a new point is added to this set of points in each iteration. By adding every salient point, decision function is updated. Hence, a condition is created for selecting the next point in which the iterative point is not extracted from the same protrusion part so that drawing out of a representative point from every protrusion part is guaranteed. This method is stable against model variations with isometric transformations, scaling, and noise with different levels of strength due to using a feature robust to isometric variations and considering the relation between the salient points. In addition, the number of points used in averaging process is decreased in this method, which leads to lower computational complexity in comparison with the other salient point detection algorithms.

Traffic Peak Period Detection from an Image Processing View

Traffic peak period detection is very important for the guidance and control of traffic flow. Most common methods for traffic peak period detection are based on data analysis. They have achieved good performance. However, the detection processes are not intuitional enough. Besides that, the accuracy of these methods needs to be improved further. From an image processing view, we introduce a concept in corner detection, sharpness, to detect the traffic peak periods in this paper. The proposed method takes the traffic peak period detection problem as a salient point detection problem and uses the image processing strategies to solve this problem. Firstly, it generates a speed curve image with the speed data. With this image, the method for detection of salient points is adopted to obtain the peak point candidates. If one candidate has the lowest speed value, this candidate is the peak point. Finally, the peak period is gotten by moving forward and backward the corresponding time of the peak point with a time interval. Experimental results show that the proposed method has achieved higher accuracy. More importantly, as the proposed method solves the traffic peak period detection problem from an image processing view, it has more intuition.

Evaluation of wavelet-based salient point detectors for image retrieval

Content-based image retrieval system based on global visual content features normally return the retrieval results according to the similarity between features extracted from the sample query image and candidate images. However, global features usually cannot capture different characteristics of different parts in the image. Therefore, the representation of local image properties is one of the most active research issues in content-based image retrieval. The method based on salient point detection is one of the typical and effective approaches. This paper proposes three improved salient point detectors based on wavelet transform, which are calculated in the three different orientations’ and scales’ subbands and weighted equally. In contrast to the former method based on salient point detection, the improved salient point detectors aim to extract the visual information in the image more effectively. We have tested the proposed schemes and compared four salient point detectors using a wide range of image samples from the Corel Image Library, and experimental results show that the improved salient point detectors have produced promising results.

A fully pipelined and parallel hardware architecture for real-time BRISK salient point extraction

Scale and rotation invariant salient point detection and matching algorithms are variously used in computer vision applications such as image matching, 3D localization and pose estimation. Recently, hardware implementation of image and video processing algorithms has emerged as a viable solution to handle the high computational complexity of applications like 3D pose estimation with several processing stages. The hardware implementation of various stages of theses algorithms can be executed in a pipelined manner to ensure the reality of time. In this paper, a new and fully pipelined hardware architecture is proposed for salient point detection using Binary Robust Invariant Scalable Keypoints (BRISK) algorithm. BRISK algorithm is a binary keypoint extractor that detects salient points by constructing a scale-space pyramid; therefore, its fixed-point hardware implementation in a pipelined manner is challenging because of the required synchronization for various layers in scale domain. The proposed hardware architecture was implemented using Verilog Hardware Description Language, and the functionality of the design was validated through several experiments. The proposed design was synthesized by using an ASIC digital design flow utilizing 180 nm CMOS technology as well as a Virtex-4 FPGA. The design is clocked at 90.91 MHz in ASIC implementation and achieves processing rate of 169.29 frames/s while running on input images with 800 × 600 resolution. The throughput of FPGA implementation is 180.44 frames/s with 96.89 MHz clock frequency for the same input image resolution. Experimental results confirm the efficiency of the proposed hardware architecture in comparison with software implementation.

Monocular visual-IMU odometry using multi-channel image patch exemplars

In this paper, we propose three sets of multi-channel image patch features for monocular visual-IMU (Inertial Measurement Unit) odometry. The proposed feature sets extract image patch exemplars from multiple feature maps of an image. We also modify an existing visual-IMU odometry framework by using different salient point detectors and feature sets and replacing the inlier selection approach with a self-adaptive scheme. The modified framework is used to examine the proposed feature sets. In addition to the Root Mean Square Error (RMSE) metric, we use the Hausdorff distance to measure the inconsistency between the estimated and ground-truth trajectories. Compared to the point-wise comparison used by RMSE, the Hausdorff distance takes the shape inconsistency of two trajectories into account and is hence more perceptually consistent. Experimental results show that the multi-channel feature sets outperform, or perform comparably to, the single gray level channel feature sets examined in this study. Particularly, the multi-channel feature set that uses integral channels, i.e., ICIMGP (Integral Channel Image Patches), outperforms two state-of-the-art feature sets: SIFT (Scale Invariant Feature Transform) and SURF (Speed Up Robust Features). Besides, ICIMGP performs better than the two multi-channel feature sets that are designed based on derivative channels and gradient channels respectively. These promising results are attributed to the fact that the multi-channel features encode richer image characteristics than their single gray level channel counterparts.

A generalised framework for saliency-based point feature detection

Here we present a novel, histogram-based salient point feature detector that may naturally be applied to both images and 3D data. Existing point feature detectors are often modality specific, with 2D and 3D feature detectors typically constructed in separate ways. As such, their applicability in a 2D-3D context is very limited, particularly where the 3D data is obtained by a LiDAR scanner. By contrast, our histogram-based approach is highly generalisable and as such, may be meaningfully applied between 2D and 3D data. Using the generalised approach, we propose salient point detectors for images, and both untextured and textured 3D data. The approach naturally allows for the detection of salient 3D points based jointly on both the geometry and texture of the scene, allowing for broader applicability. The repeatability of the feature detectors is evaluated using a range of datasets including image and LiDAR input from indoor and outdoor scenes. Experimental results demonstrate a significant improvement in terms of 2D-2D and 2D-3D repeatability compared to existing multi-modal feature detectors.

ON FPGA BASED ACCELERATION OF IMAGE PROCESSING IN MOBILE ROBOTICS

In visual navigation tasks, a lack of the computational resources is one of the main limitations of micro robotic platforms to be deployed in autonomous missions. It is because the most of nowadays techniques of visual navigation relies on a detection of salient points that is computationally very demanding. In this paper, an FPGA assisted acceleration of image processing is considered to overcome limitations of computational resources available on-board and to enable high processing speeds while it may lower the power consumption of the system. The paper reports on performance evaluation of the CPU–based and FPGA–based implementations of a visual teach-and-repeat navigation system based on detection and tracking of the FAST image salient points. The results indicate that even a computationally efficient FAST algorithm can benefit from a parallel (low–cost) FPGA–based implementation that has a competitive processing time but more importantly it is a more power efficient.

Landmarks inside the shape: Shape matching using image descriptors

In the last few decades, significant advances in image matching are provided by rich local descriptors that are defined through physical measurements such as reflectance. As such measurements are not naturally available for silhouettes, existing arsenal of image matching tools cannot be utilized in shape matching. We propose that the recently presented SPEM representation can be used analogous to image intensities to detect local keypoints using invariant image salient point detectors. We devise a shape similarity measure based on the number of matching internal regions. The performance of the similarity measure in planar shape retrieval indicates that the landmarks inside the shape silhouettes provide a strong representation of the regional characteristics of 2D planar shapes.

Evaluation of local spatial–temporal features for cross-view action recognition

Local spatial–temporal feature-based representation is extremely popular for human action recognition. Many spatial–temporal salient point detectors and descriptors have been proposed. Although the promising results have been achieved for action recognition recently, there still exist two severe problems: (1) there is lack of systematic evaluation of local spatial–temporal features on cross-view action recognition; (2) there is lack of a baseline method especially for the task of cross-view action recognition, which can adaptively bridge different feature spaces from multiple views for this cross-domain task. In this paper, we evaluate four popular spatial–temporal features (STIP, Cuboids, MoSIFT, HoG3D) with the framework of transferable dictionary pair learning. This framework can first learn one transferable dictionary pair in both unsupervised and supervised settings. Then, training samples in the source view and testing samples in the target view can be represented with corresponding source and target dictionary respectively to get sparse feature representations, which is used to training classifier for action recognition. In this way, it can map the features from different views into the same feature space to handle the cross-domain task. The evaluation of four spatial–temporal features and the framework of transferable dictionary pair learning are implemented on the popular multi-view human action dataset, IXMAS. The comparative experiments against the representative methods further demonstrate the superiority of this framework on cross-view human action recognition.

Content-based image retrieval using local visual attention feature

Content-based image retrieval (CBIR) has been an active research topic in the last decade. As one of the promising approaches, salient point based image retrieval has attracted many researchers. However, the related work is usually very time consuming, and some salient points always may not represent the most interesting subset of points for image indexing. Based on fast and performant salient point detector, and the salient point expansion, a novel content-based image retrieval using local visual attention feature is proposed in this paper. Firstly, the salient image points are extracted by using the fast and performant SURF (Speeded-Up Robust Features) detector. Then, the visually significant image points around salient points can be obtained according to the salient point expansion. Finally, the local visual attention feature of visually significant image points, including the weighted color histogram and spatial distribution entropy, are extracted, and the similarity between color images is computed by using the local visual attention feature. Experimental results, including comparisons with the state-of-the-art retrieval systems, demonstrate the effectiveness of our proposal.

Automatic image segmentation using salient key point extraction and star shape prior

In this paper, a new unsupervised segmentation method is proposed. The method integrates the star shape prior of the image object with salient point detection algorithm. In the proposed method, the Harris salient point detection is first applied to the color image to obtain the initial salient points. A regional contrast based saliency extraction method is then used to select rough object regions in the image. To restrict the distribution of salient points, an adaptive threshold segmentation is applied to the saliency map to get the saliency mask. And then the salient region points can be obtained by placing the saliency mask on the initial Harris salient points. In order to make sure the salient points which we get are inside the image object thus the star shape constraint can be applied to the graph cuts segmentation, the Affinity Propagation (AP) clustering is employed to find the salient key points among the salient region points. Finally, these salient key points are regarded as foreground seeds and the star shape prior is introduced to graph cuts segmentation framework to extract the foreground object. Extensive experiments and comparisons on public database are provided to demonstrate the good performance of the proposed method.

Detection, Localization, and Tracking of Shock Contour Salient Points in Schlieren Sequences

Computer vision methods are proposed for feature extraction of structures of interest in schlieren images. The well studied curvature scale space is used to define features at physically salient points on shock contours. These points can include shock interactions, reflections, standoff distance, etc., and are present in many schlieren data. A challenge in developing methods for feature extraction in schlieren images is the reconciliation of existing techniques with structures of interest to an aerodynamicist. Domain-specific knowledge of physics must be incorporated into the definition and detection phases. Algorithms must be designed such that incorporation of an aerodynamic knowledge base is viable and known location and physically possible structure representations form a knowledge base that provides a unique feature definition and extraction. Model tip location and a shock intersection across several thousand frames in the complex double-cone schlieren sequence are identified, localized, and tracked. From examination of the sequence, the model appears still, and motion of the shock intersect is observed. Through the application of computer vision techniques, model stillness is confirmed, and the extent and pattern of the intersect motion are demonstrated.