Ground Truth Transformations Research Articles

Comparison of Point Cloud Registration Techniques on Scanned Physical Objects.

This paper presents a comparative analysis of six prominent registration techniques for solving CAD model alignment problems. Unlike the typical approach of assessing registration algorithms with synthetic datasets, our study utilizes point clouds generated from the Cranfield benchmark. Point clouds are sampled from existing CAD models and 3D scans of physical objects, introducing real-world complexities such as noise and outliers. The acquired point cloud scans, including ground-truth transformations, are made publicly available. This dataset includes several cleaned-up scans of nine 3D-printed objects. Our main contribution lies in assessing the performance of three classical (GO-ICP, RANSAC, FGR) and three learning-based (PointNetLK, RPMNet, ROPNet) methods on real-world scans, using a wide range of metrics. These include recall, accuracy and computation time. Our comparison shows a high accuracy for GO-ICP, as well as PointNetLK, RANSAC and RPMNet combined with ICP refinement. However, apart from GO-ICP, all methods show a significant number of failure cases when applied to scans containing more noise or requiring larger transformations. FGR and RANSAC are among the quickest methods, while GO-ICP takes several seconds to solve. Finally, while learning-based methods demonstrate good performance and low computation times, they have difficulties in training and generalizing. Our results can aid novice researchers in the field in selecting a suitable registration method for their application, based on quantitative metrics. Furthermore, our code can be used by others to evaluate novel methods.

Novel groundtruth transformations for the aesthetic assessment problem

Aesthetic assessment evaluates the quality of a given image using subjective annotations, commonly user ratings, as a knowledge base. Rating complexity is usually relaxed in state-of-the-art works by employing a binary high/low quality label computed from the mean value of rating votes. Nevertheless, this approach introduces uncertainty to average-quality images, which may affect the performance of machine learning models trained from annotated data.In this work, we present a novel approach to aesthetic assessment based on redefining the rating-based groundtruths present in most datasets. Our intent is twofold: to reduce the rating uncertainty and to automatically group them into clusters reflecting high and low quality patterns, thus avoiding an arbitrary threshold like 5 in 1–10 ratings. The experimentation uses the well-known AVA dataset, which consists of more than 255,000 images, and we train several CNN models to test our new groundtruths against the baseline ones. The results show that our approach achieves significant performance gains, between 3% and 9% more balanced accuracy than the baseline groundtruths.

Iterative BTreeNet: Unsupervised learning for large and dense 3D point cloud registration

3D point cloud registration is a computational process that aligns two 3D point clouds through transformation. i.e. finding matching translation and rotation. Existing state-of-the-art learning-based methods require ground-truth transformation as supervision and often perform poorly in dealing with partial point clouds and large scenes that are not trained, resulting in poor generalization for neural networks.In this paper, we propose a novel unsupervised deep learning network - Binary Tree Network (BTreeNet) that consists of a novel forward propagation, which learns features for the rotation separately from the translation and avoids the interference between the estimations of rotation and translation in one single matrix. We then propose an Iterative Binary Tree Network (IBTreeNet) to continuously improve the registration accuracy for large and dense 3D point clouds. The Chamfer Distance and the Earth Mover’s Distance are adopted as the loss function for unsupervised learning. We show that BTreeNet and IBTreeNet outperform state-of-the-art learning-based and traditional methods on partial and noisy point clouds without training them in such scenarios. Most importantly, the proposed methods exhibit remarkable generalization and robustness to unseen large and dense scenes that are never trained.

Unsupervised Point Cloud Registration by Learning Unified Gaussian Mixture Models

Sampling noise and density variation widely exist in the point cloud acquisition process, leading to few accurate point-to-point correspondences. Since they rely on point-to-point correspondence search, existing state-of-the-art point cloud registration methods face difficulty in overcoming the sampling noise and density variation accurately or efficiently. Moreover, the recent state-of-the-art learning-based methods requires ground-truth transformation as supervised information which lead to large labor costs in real scenes. In this paper, our motivation is that two point-clouds are considered two samples from a unified Gaussian Mixture Model (UGMM). Then, we leverage the advantage of the statistic model to overcome the noise and density variants, and uses the alignment score in the UGMM to supervise the network training. To achieve this motivation, we propose a new unsupervised learning-based probabilistic registration algorithm to reconstruct the unified GMM and solve the registration problem simultaneously. The proposed method formulates the registration problem into a clustering problem, which estimates the posterior probability that classifies the points of two input point clouds to components of the unified GMM. A new feature interaction module is designed to learn the posterior probability using both the self and cross point cloud information. Then, two differential modules are proposed to calculate the GMM parameters and transformation matrices. Experimental results on synthetic and real-world point cloud datasets demonstrate that our unsupervised method achieves better registration accuracy and efficiency than the state-of-the-art supervised and semi-supervised methods in handling noisy and density variant point clouds.

A new 2D-3D registration gold-standard dataset for the hip joint based on uncertainty modeling.

PurposeEstimation of the accuracy of 2D‐3D registration is paramount for a correct evaluation of its outcome in both research and clinical studies. Publicly available datasets with standardized evaluation methodology are necessary for validation and comparison of 2D‐3D registration techniques. Given the large use of 2D‐3D registration in biomechanics, we introduced the first gold standard validation dataset for computed tomography (CT)‐to‐x‐ray registration of the hip joint, based on fluoroscopic images with large rotation angles. As the ground truth computed with fiducial markers is affected by localization errors in the image datasets, we proposed a new methodology based on uncertainty propagation to estimate the accuracy of a gold standard dataset.MethodsThe gold standard dataset included a 3D CT scan of a female hip phantom and 19 2D fluoroscopic images acquired at different views and voltages. The ground truth transformations were estimated based on the corresponding pairs of extracted 2D and 3D fiducial locations. These were assumed to be corrupted by Gaussian noise, without any restrictions of isotropy. We devised the multiple projective points criterion (MPPC) that jointly optimizes the transformations and the noisy 3D fiducial locations for all views. The accuracy of the transformations obtained with the MPPC was assessed in both synthetic and real experiments using different formulations of the target registration error (TRE), including a novel formulation of the TRE (uTRE) derived from the uncertainty analysis of the MPPC.ResultsThe proposed MPPC method was statistically more accurate compared to the validation methods for 2D‐3D registration that did not optimize the 3D fiducial positions or wrongly assumed the isotropy of the noise. The reported results were comparable to previous published works of gold standard datasets. However, a formulation of the TRE commonly found in these gold standard datasets was found to significantly miscalculate the true TRE computed in synthetic experiments with known ground truths. In contrast, the uncertainty‐based uTRE was statistically closer to the true TRE.ConclusionsWe proposed a new gold standard dataset for the validation of CT‐to‐X‐ray registration of the hip joint. The gold standard transformations were derived from a novel method modeling the uncertainty in extracted 2D and 3D fiducials. Results showed that considering possible noise anisotropy and including corrupted 3D fiducials in the optimization resulted in improved accuracy of the gold standard. A new uncertainty‐based formulation of the TRE also appeared as a good alternative to the unknown true TRE that has been replaced in previous works by an alternative TRE not fully reflecting the gold standard accuracy.

Auto-Encoding Transformations in Reparameterized Lie Groups for Unsupervised Learning

Unsupervised training of deep representations has demonstrated remarkable potentials in mitigating the prohibitive expenses on annotating labeled data recently. Among them is predicting transformations as a pretext task to self-train representations, which has shown great potentials for unsupervised learning. However, existing approaches in this category learn representations by either treating a discrete set of transformations as separate classes, or using the Euclidean distance as the metric to minimize the errors between transformations. None of them has been dedicated to revealing the vital role of the geometry of transformation groups in learning representations. Indeed, an image must continuously transform along the curved manifold of a transformation group rather than through a straight line in the forbidden ambient Euclidean space. This suggests the use of geodesic distance to minimize the errors between the estimated and groundtruth transformations. Particularly, we focus on homographies, a general group of planar transformations containing the Euclidean, similarity and affine transformations as its special cases. To avoid an explicit computing of intractable Riemannian logarithm, we project homographies onto an alternative group of rotation transformations SR(3) with a tractable form of geodesic distance. Experiments demonstrate the proposed approach to Auto-Encoding Transformations exhibits superior performances on a variety of recognition problems.

STACKED LOCAL FEATURE DETECTOR FOR HYPERSPECTRAL IMAGE

Abstract. Images registration is an important task in hyperspectral image processing, while almost all local feature point based image matching algorithm is designed for single band image only and miss the advantage of abundant spectral information. Therefor, in this paper we propose a novel local feature detector for hyperspectral image. This method, which is named stacked local feature detector (HSI-SFD), stack all local feature points detected from every single spectral band. With redundant information, local feature miss-detected at noise pixel can be filtered out by the stack count threshold, leading to more reliable and robust local features. To verify the algorithm, a hyperspectral image matching dataset is built. Multiview hyperspectral images of the several different flat targets are taken by scan-line hyperspectral camera in library. Each image contains 270 bands within 400–1000 nm. Groundtruth transformation matrix between images is computed from corresponding points selected by hands. Experiment on the dataset shows that features points miss-detected at texture-less area can be inhibited by HSI-SFD. Both keypoint repeatability and matching accuracy increase significantly. Precision and matching score increase up to 10% for some scene.

Gum-Net: Unsupervised Geometric Matching for Fast and Accurate 3D Subtomogram Image Alignment and Averaging.

We propose a Geometric unsupervised matching Network (Gum-Net) for finding the geometric correspondence between two images with application to 3D subtomogram alignment and averaging. Subtomogram alignment is the most important task in cryo-electron tomography (cryo-ET), a revolutionary 3D imaging technique for visualizing the molecular organization of unperturbed cellular landscapes in single cells. However, subtomogram alignment and averaging are very challenging due to severe imaging limits such as noise and missing wedge effects. We introduce an end-to-end trainable architecture with three novel modules specifically designed for preserving feature spatial information and propagating feature matching information. The training is performed in a fully unsupervised fashion to optimize a matching metric. No ground truth transformation information nor category-level or instance-level matching supervision information is needed. After systematic assessments on six real and nine simulated datasets, we demonstrate that Gum-Net reduced the alignment error by 40 to 50% and improved the averaging resolution by 10%. Gum-Net also achieved 70 to 110 times speedup in practice with GPU acceleration compared to state-of-the-art subtomogram alignment methods. Our work is the first 3D unsupervised geometric matching method for images of strong transformation variation and high noise level. The training code, trained model, and datasets are available in our open-source software AITom.

Region-based image registration for remote sensing imagery

We propose an automatic region-based registration method for remote sensing imagery. In this method, we aim to register two images by matching region properties to address possible errors caused by local feature estimators. We apply automated image segmentation to identify the regions and calculate regional Fourier descriptors and standardized regional intensity descriptors for each region. We define a joint matching cost, as a linear combination of Euclidean distances, to establish and extract correspondences between regions. The segmentation technique utilizes kernel density estimators for edge localization, followed by morphological reconstruction and the watershed transform. We evaluated the registration performance of our method on synthetic and real datasets. We measured the registration accuracy by calculating the root-mean-squared error (RMSE) between the estimated transformation and the ground truth transformation. The results obtained using the joint intensity-Fourier descriptor were compared to the results obtained using Harris, Minimum eigenvalue, Features accelerated segment test (FAST), speeded-up robust features (SURF), binary robust invariant scalable keypoints (BRISK) and KAZE keypoint descriptors. The joint intensity-Fourier descriptor yielded average RMSE of 0.446±0.359 pixels and 1.152±0.488 pixels on two satellite imagery datasets consisting of 35 image pairs in total. These results indicate the capacity of the proposed technique for high accuracy. Our method also produces a lower registration error than the compared feature-based methods.

Evaluation of Local Feature Detectors for the Comparison of Thermal and Visual Low Altitude Aerial Images

Local features are key regions of an image suitable for applications such as image matching, and fusion. Detection of targets under varying atmospheric conditions, via aerial images is a typical defence application where multi spectral correlation is essential. Focuses on local features for the comparison of thermal and visual aerial images in this study. The state of the art differential and intensity comparison based features are evaluated over the dataset. An improved affine invariant feature is proposed with a new saliency measure. The performances of the existing and the proposed features are measured with a ground truth transformation estimated for each of the image pairs. Among the state of the art local features, Speeded Up Robust Feature exhibited the highest average repeatability of 57 per cent. The proposed detector produces features with average repeatability of 64 per cent. Future works include design of techniques for retrieval of corresponding regions.

Volumetric 3D stitching of optical coherence tomography volumes

Abstract Optical coherence tomography (OCT) is a noninvasive medical imaging modality, which provides highresolution transectional images of biological tissue. However, its potential is limited due to a relatively small field of view. To overcome this drawback, we describe a scheme for fully automated stitching of multiple 3D OCT volumes for panoramic imaging. The voxel displacements between two adjacent images are calculated by extending the Lucas-Kanade optical flow a lgorithm to dense volumetric images. A RANSAC robust estimator is used to obtain rigid transformations out of the resulting flow v ectors. T he i mages a re t ransformed into the same coordinate frame and overlapping areas are blended. The accuracy of the proposed stitching scheme is evaluated on two datasets of 7 and 4 OCT volumes, respectively. By placing the specimens on a high-accuracy motorized translational stage, ground truth transformations are available. This results in a mean translational error between two adjacent volumes of 16.6 ± 0.8 μm (2.8 ± 0.13 voxels). To the author’s knowledge, this is the first reported stitching of multiple 3D OCT volumes by using dense voxel information in the registration process. The achieved results are sufficient for providing high accuracy OCT panoramic images. Combined with a recently available high-speed 4D OCT, our method enables interactive stitching of hand-guided acquisitions.

4-Plane congruent sets for automatic registration of as-is 3D point clouds with 3D BIM models

Abstract Construction quality and progress control are demanding, yet critical construction activities. Building Information Models and as-built scanned data can be used in Scan-vs-BIM processes to effectively and comprehensively support these activities. This however requires accurate registration of scanned point clouds with 3D (BIM) models. Automating such registration remains a challenge in the context of the built environment, because as-built can be incomplete and/or contain data from non-model objects, and construction buildings and other structures often present symmetries and self-similarities that are very challenging to registration. In this paper, we present a novel automatic coarse registration method that is an adaptation of the ‘4 Points Congruent Set’ algorithm to the use of planes; we call it the ‘4-Plane Congruent Set’ (4-PlCS) algorithm. The approach is further integrated in a software system that delivers not one but a ranked list of the most likely transformations, so to allow the user to quickly select the correct transformation, if need be. Two variants of the method are also considered, in particular one in the case when the vertical axis is known a priori; we call that method the 4.5-PlCS method. The proposed algorithm is tested using five different datasets, including three simulated and two real-life ones. The results show the effectiveness of the proposed method, where the correct transformation always ranks very high (in our experiments, first or second), and is extremely close to the ground-truth transformation. Experimental comparison of the proposed approach with a standard, more intuitive approach based on finding 3-plane congruent sets shows the discriminatory power of 4-plane bases over 3-plane bases, albeit at no clear benefits in terms of computational time. The experimental results for the 4.5-PlCS method show that it delivers a non-negligible reduction in computational time (approx. 20%), but at no additional benefit in terms of effectiveness in finding the correct transformation.

Tool-supported Interactive Correction and Semantic Annotation of Narrative Clinical Reports

Our main objective is to design a method of, and supporting software for, interactive correction and semantic annotation of narrative clinical reports, which would allow for their easier and less erroneous processing outside their original context: first, by physicians unfamiliar with the original language (and possibly also the source specialty), and second, by tools requiring structured information, such as decision-support systems. Our additional goal is to gain insights into the process of narrative report creation, including the errors and ambiguities arising therein, and also into the process of report annotation by clinical terms. Finally, we also aim to provide a dataset of ground-truth transformations (specific for Czech as the source language), set up by expert physicians, which can be reused in the future for subsequent analytical studies and for training automated transformation procedures. A three-phase preprocessing method has been developed to support secondary use of narrative clinical reports in electronic health record. Narrative clinical reports are narrative texts of healthcare documentation often stored in electronic health records. In the first phase a narrative clinical report is tokenized. In the second phase the tokenized clinical report is normalized. The normalized clinical report is easily readable for health professionals with the knowledge of the language used in the narrative clinical report. In the third phase the normalized clinical report is enriched with extracted structured information. The final result of the third phase is a semi-structured normalized clinical report where the extracted clinical terms are matched to codebook terms. Software tools for interactive correction, expansion and semantic annotation of narrative clinical reports has been developed and the three-phase preprocessing method validated in the cardiology area. The three-phase preprocessing method was validated on 49 anonymous Czech narrative clinical reports in the field of cardiology. Descriptive statistics from the database of accomplished transformations has been calculated. Two cardiologists participated in the annotation phase. The first cardiologist annotated 1500 clinical terms found in 49 narrative clinical reports to codebook terms using the classification systems ICD 10, SNOMED CT, LOINC and LEKY. The second cardiologist validated annotations ofthe first cardiologist. The correct clinical terms and the codebook terms have been stored in a database. We extracted structured information from Czech narrative clinical reports by the proposed three-phase preprocessing method and linked it to electronic health records. The software tool, although generic, is tailored for Czech as the specific language of electronic health record pool under study. This will provide a potential etalon for porting this approach to dozens of other less-spoken languages. Structured information can support medical decision making, quality assurance tasks and further medical research.

Evaluation of Distance Functions for the Comparison of Gradient Orientation Histograms

Local features of an image are used in many computer vision applications such as object detection and scene matching. The gradient orientation histogram is used by many local features such as Scale Invariant Feature Transform (SIFT), a widely used image local feature. This paper discusses various distance functions that can be used to measure the similarity between the local features described by the gradient orientation histogram. A distance function, based on the quadratic form is proposed for the SIFT descriptor. The state of the art distance functions - Euclidean, Chi-square, Manhattan and the proposed quadratic form based distance function are calculated between the features extracted from the images. Nearest neighborhood ratio strategy is used to find the corresponding features based on the distance measure. Correct matches are estimated using the ground truth transformation function between the images, present in the form of homograph matrix. It is experimentally found that the proposed distance function has an execution time reduced by 21% compared to the Euclidean distance for a similar accuracy performance. The proposed distance retrieves more number of correct matches compared to the modified Earth Mover distance which is fastest among the evaluated distance functions. The future work will be aimed at improving the time taken for computing the distance matrix between the feature sets and a better strategy for computing the matches.

Spectral registration of noisy sonar data for underwater 3D mapping

3D mapping is very challenging in the underwater domain, especially due to the lack of high resolution, low noise sensors. A new spectral registration method is presented that can determine the spatial 6 DOF transformation between pairs of very noisy 3D scans with only partial overlap. The approach is hence suited to cope with sonar as the predominant underwater sensor. The spectral registration method is based on Phase Only Matched Filtering (POMF) on non-trivially resampled spectra of the 3D data. Two extensive sets of experiments are presented. First, evaluations with simulated data are done where the type and amount of noise can be controlled and the ground truth transformations between scans are known. Second, real world data from a Tritech Eclipse sonar is used. Concretely, 18 sonar scans of a large structure in form of a flood gate and a lock in the river Lesum in Bremen are used for 3D mapping. In doing so, the spectral registration method is compared to two other methods suited for noisy 3D registrations, namely Iterative Closest Point (ICP) and plane-based registration. It is shown that the spectral registration method performs very well in terms of the resulting 3D map as well as its run-times.

Fast range-independent spherical subsampling of 3D laser scanner points and data reduction performance evaluation for scene registration

Three-dimensional laser range-finders are increasingly being incorporated into applications, such as mobile robotics, that require real-time registration of scene data. However, the computational costs of adaptive range-dependent data selection and point cloud matching grow significantly with the number of points. Therefore, fast range-independent subsampling by uniform or random data reduction is usually performed at a preprocessing step. The paper proposes a new range-independent subsampling algorithm that is more effective for the widely used spherical scanning mechanism. As this type of device measures the ranges by composition of two rotations, it samples certain directions with a higher density, which can distort the registration optimization process. The proposed solution uses sensor characteristics to equalize the measure-direction density of the reduced point cloud. The paper also addresses performance assessment of subsampling methods by contributing three benchmark criteria that do not rely on a particular registration technique: one considers the ground truth transformation between two scans, and the other two are based on the analysis of a single scan. The advantages of spherical subsampling are analyzed through a comparison of range-independent methods and a simple range-dependent one with real scans from three representative scenes (urban, natural, and indoors).

Dual-isotope acquisition for CT-SPECT registration of infection studies.

The registration of CT and NM images can enhance patient diagnosis since it allows for the fusion of anatomical and functional information as well as attenuation correction of NM images. However, irrespective of the methods used, registration accuracy depends heavily on the characteristics of the input images and the degree of similarity between them. This poses a challenge for registering CT and NM images as they may have very different characteristics. To address the particular problem of CT and In-111 SPECT registration, we propose to perform a dual-isotope study which involves an additional injection of Tc-99m MDP to generate two inherently registered images: In-111 SPECT and Tc-99m SPECT. As skeletal structures are visible in both CT and Tc-99m SPECT, performing registration of these images may be much more effective. The very same spatial transformation derived can be immediately applied to complete the registration of CT and the corresponding In-111 SPECT. Accordingly, we hypothesize that the registration of CT and Tc-99m SPECT can be more accurately performed than the registration of CT and In-111 SPECT and seek to compare the accuracies between the aforementioned registrations. In this paper, we have collected three clinical datasets, with the ground-truth transformations known, and tested the proposed approach by using a mutual information-based algorithm to solve for the rigid/non-rigid misalignments introduced to them. Based on the results of our experiments, we conclude that registration using Tc-99m SPECT can achieve 100% success rate, and is thus much more superior to the registration using In-111 SPECT, which at best, achieves only 38% success rate. Clearly, the introduction of a dual-isotope acquisition can substantially improve the registration of SPECT and CT images.