Keypoint-based Approaches Research Articles

LIRRN: Location-Independent Relative Radiometric Normalization of Bitemporal Remote-Sensing Images.

Relative radiometric normalization (RRN) is a critical pre-processing step that enables accurate comparisons of multitemporal remote-sensing (RS) images through unsupervised change detection. Although existing RRN methods generally have promising results in most cases, their effectiveness depends on specific conditions, especially in scenarios with land cover/land use (LULC) in image pairs in different locations. These methods often overlook these complexities, potentially introducing biases to RRN results, mainly because of the use of spatially aligned pseudo-invariant features (PIFs) for modeling. To address this, we introduce a location-independent RRN (LIRRN) method in this study that can automatically identify non-spatially matched PIFs based on brightness characteristics. Additionally, as a fast and coregistration-free model, LIRRN complements keypoint-based RRN for more accurate results in applications where coregistration is crucial. The LIRRN process starts with segmenting reference and subject images into dark, gray, and bright zones using the multi-Otsu threshold technique. PIFs are then efficiently extracted from each zone using nearest-distance-based image content matching without any spatial constraints. These PIFs construct a linear model during subject-image calibration on a band-by-band basis. The performance evaluation involved tests on five registered/unregistered bitemporal satellite images, comparing results from three conventional methods: histogram matching (HM), blockwise KAZE, and keypoint-based RRN algorithms. Experimental results consistently demonstrated LIRRN's superior performance, particularly in handling unregistered datasets. LIRRN also exhibited faster execution times than blockwise KAZE and keypoint-based approaches while yielding results comparable to those of HM in estimating normalization coefficients. Combining LIRRN and keypoint-based RRN models resulted in even more accurate and reliable results, albeit with a slight lengthening of the computational time. To investigate and further develop LIRRN, its code, and some sample datasets are available at link in Data Availability Statement.

Self-calibration of terrestrial laser scanner using a M3C2-based planar patch algorithm

Target- or intensity-feature-based calibration has been extensively explored for in-situ calibration of terrestrial laser scanners (TLSs). Those calibration approaches main drawbacks include the necessity of mounting physical targets in the scan area and the need for the presence of rich geometrical features allowing a feature-based distinction of different points in the point cloud. We propose an autonomous TLS calibration algorithm using planar patches that are ubiquitous in urban environments and can be found without manual preparation beforehand. The scanner calibration parameters are estimated and updated by minimizing the M3C2 normal distances between corresponding planar patches. Unlike target- or keypoint-based approaches, only medium-resolution scan data is needed. A comprehensive set of calibration model parameters are estimated using scans acquired from multiple positions, thereby providing a further advantage over current keypoint-based approaches that estimate only two-face sensitive parameters. To increase speed and reduce memory consumption, we propose to execute the calibration using only parts of scan data from a single station. Studies with two high-precision scanners, Leica RTC360 and Z+F Imager 5016, show results consistent with the laborious target-based approach with improved precision. Similar two-face sensitive calibration performance to the target- and keypoint-based approaches has been achieved, but with only subsets of the point cloud using the proposed algorithm.

3D hand pose and shape estimation from RGB images for keypoint-based hand gesture recognition

Estimating the 3D pose of a hand from a 2D image is a well-studied problem and a requirement for several real-life applications such as virtual reality, augmented reality, and hand gesture recognition. Currently, reasonable estimations can be computed from single RGB images, especially when a multi-task learning approach is used to force the system to consider the shape of the hand when its pose is determined. However, depending on the method used to represent the hand, the performance can drop considerably in real-life tasks, suggesting that stable descriptions are required to achieve satisfactory results. In this paper, we present a keypoint-based end-to-end framework for 3D hand and pose estimation and successfully apply it to the task of hand gesture recognition as a study case. Specifically, after a pre-processing step in which the images are normalized, the proposed pipeline uses a multi-task semantic feature extractor generating 2D heatmaps and hand silhouettes from RGB images, a viewpoint encoder to predict the hand and camera view parameters, a stable hand estimator to produce the 3D hand pose and shape, and a loss function to guide all of the components jointly during the learning phase. Tests were performed on a 3D pose and shape estimation benchmark dataset to assess the proposed framework, which obtained state-of-the-art performance. Our system was also evaluated on two hand-gesture recognition benchmark datasets and significantly outperformed other keypoint-based approaches, indicating that it is an effective solution that is able to generate stable 3D estimates for hand pose and shape.

Mapping and localization from planar markers

Squared planar markers are a popular tool for fast, accurate and robust camera localization, but its use is frequently limited to a single marker, or at most, to a small set of them for which their relative pose is known beforehand. Mapping and localization from a large set of planar markers is yet a scarcely treated problem in favour of keypoint-based approaches. However, while keypoint detectors are not robust to rapid motion, large changes in viewpoint, or significant changes in appearance, fiducial markers can be robustly detected under a wider range of conditions. This paper proposes a novel method to simultaneously solve the problems of mapping and localization from a set of squared planar markers. First, a quiver of pairwise relative marker poses is created, from which an initial pose graph is obtained. The pose graph may contain small pairwise pose errors, that when propagated, leads to large errors. Thus, we distribute the rotational and translational error along the basis cycles of the graph so as to obtain a corrected pose graph. Finally, we perform a global pose optimization by minimizing the reprojection errors of the planar markers in all observed frames. The experiments conducted show that our method performs better than Structure from Motion and visual SLAM techniques.

Fusion of block and keypoints based approaches for effective copy-move image forgery detection

Keypoint-based and block-based methods are two main categories of techniques for detecting copy-move forged images, one of the most common digital image forgery schemes. In general, block-based methods suffer from high computational cost due to the large number of image blocks used and fail to handle geometric transformations. On the contrary, keypoint-based approaches can overcome these two drawbacks yet are found difficult to deal with smooth regions. As a result, fusion of these two approaches is proposed for effective copy-move forgery detection. First, our scheme adaptively determines an appropriate initial size of regions to segment the image into non-overlapped regions. Feature points are extracted as keypoints using the scale invariant feature transform (SIFT) from the image. The ratio between the number of keypoints and the total number of pixels in that region is used to classify the region into smooth or non-smooth (keypoints) regions. Accordingly, block based approach using Zernike moments and keypoint based approach using SIFT along with filtering and post-processing are respectively applied to these two kinds of regions for effective forgery detection. Experimental results show that the proposed fusion scheme outperforms the keypoint-based method in reliability of detection and the block-based method in efficiency.

Combining keypoint-based and segment-based features for counting people in crowded scenes

The counting of the number of people within a scene is a practical machine vision task, and it has been considered as an important application for security purposes. Most of the people counting algorithms generally extract the foreground segments and map the number of people to some features such as foreground area, texture, or edge count. Keypoint-based approaches, on the other hand, have also been proposed, which involves the use of statistical features of keypoints, such as the number of moving keypoints to estimate the crowd size. In contrast to the foreground segment-based methods, keypoint-based approaches are not sensitive to background changes, illuminations, occlusions, and shadows. However, they have limited performance due to the lack of sufficient features. In this paper, in order to estimate the crowd count, the combination of keypoint-based and segment-based (foreground) features is proposed. However, the whole approach is based on the keypoints and not all the image pixels. The proposed method, firstly, extracts the salient keypoints in the scene. Then, foreground segments are obtained by a simple morphological operation on the moving keypoints and hence the system does not suffer from difficulties associated with foreground/background segmentation. Various features are extracted from each foreground segment together with the corresponding keypoints which are highly correlated with the size, density, and occlusion level of the crowd. Finally, a combination of the segment-based and keypoint-based features is used to estimate the number of people in crowds. The experiment demonstrates that the proposed method achieves lower counting error rates compared to the existing approaches.

Keypoint-Based Analysis of Sonar Images: Application to Seabed Recognition

In this paper, we address seabed characterization and recognition in sonar images using keypoint-based approaches. Keypoint-based texture recognition has recently emerged as a powerful framework to address invariances to contrast change and geometric distortions. We investigate here to which extent keypoint-based techniques are relevant for sonar texture analysis which also involves such invariance issues. We deal with both the characterization of the visual signatures of the keypoints and the spatial patterns they form. In this respect, spatial statistics are considered. We report a quantitative evaluation for sonar seabed texture data sets comprising six texture classes such as mud, rock, and gravely sand. We clearly demonstrate the improvement brought by keypoint-based techniques compared to classical features used for sonar texture analysis such as cooccurrence and Gabor features. In this respect, we demonstrate that the joint characterization of the visual signatures of the visual keypoints and their spatial organization reaches the best recognition performances (about 97% of correct classification w.r.t. 70% and 81% using cooccurrence and Gabor features). Furthermore, the combination of difference of Gaussian keypoints and scale-invariant feature transform descriptors is recommended as the most discriminating keypoint-based framework for the analysis of sonar seabed textures.