Local Registration Method Research Articles

Automated geometric reconstruction and cable force inference for cable-net structures using 3D point clouds

Laser scanning provides an efficient solution to digital twin construction in civil engineering. The complexity and redundancy of large-scale point clouds substantially prolong the labor-intensive model reconstruction process. This paper presents an automated and high-precision geometric reconstruction approach for cable-net structures with a complete workflow from raw points to the extraction of cable shapes and forces. The strategy involves the extraction of coordinates of key cable regions through the characterization of cable geometry and local registration methods, followed by the computation of cable forces and shapes using segmented catenary theory and nonlinear optimization. The approach is validated with a single-curvature cable-net structure, with the accuracy of cable shapes within ±5 mm and errors in cable forces <5%. The method contributes to health monitoring and rapid reconstruction of cable-net structures. The precise geometry and forces will facilitate the creation of the mechanical models reflecting the physical authenticity of structures.

A Multivariate Local Descriptor Registration Method for Surface Topography Evaluation

This paper illustrates a systematical surface topography measurement and evaluation method based on a 3D optical system. Firstly, the point cloud data of the workpiece are extracted by the use of a 3D structured light measurement system, and the STEP file of the design model is converted into point cloud data. Secondly, the local measurement point cloud (LMPC) and digital model point cloud (DMPC) are registered by a multivariate local descriptor registration scheme proposed in this study. Thirdly, the surface shapes extracted from the STEP file are applied as a reference to segment the measuring point cloud. Finally, an error analysis scheme is conducted on specific functional surfaces. An experiment was conducted to analyse the flatness, cylindricity and roughness to demonstrate the effectiveness and advantage of the method. The comparison results show that the proposed method outperforms other 3D optical surface topography analysis methods.

Application of the Iot Network and the Deep Learning Concept in Music Teaching

In order to realize students’ in-depth understanding of music teaching content, it is necessary to reasonably allocate teaching materials according to the teaching content. Therefore, this paper puts forward the application of the concept of the Iot network and deep learning in music teaching. After the teaching resources are vectorized, the distinguishing local registration method of the DLA model is used to extract their features. Based on the dimension of the teaching content, the features of the teaching content are output in the DLA model, the music teaching resources are allocated according to the minimum matching error criterion, the hyperbolic tangent function is taken as the activation function, and the feature error is filtered by maximum aggregation. The experimental results show that the design method can, according to the music teaching content, have more than 80% accuracy in the shallow distribution and deep distribution of teaching materials, and play a positive role in promoting students’ in-depth learning.

Global correction of optical distortions in multicolor single-molecule microscopy using Zernike polynomial gradients

Accurate image alignment is critical in multicolor single-molecule fluorescence microscopy. Global alignment using affine transformations leaves residual errors due to the nonlinearity of the distortions, which decreases the effective field of view. Subsequent local refinement demands either large amounts of reference data and processing time or specialized imaging techniques like active stabilization. Here, we present a global alignment method, S/T polynomial decomposition, that uses sums of Zernike polynomial gradients to decompose the distortion between two images, correcting both linear and nonlinear distortions simultaneously. With minimal reference data, we gain diagnostic information about the distortion and achieve a colocalization accuracy comparable to local registration methods across the entire field of view.

An on-machine tool path generation method based on hybrid and local point cloud registration for laser deburring of ceramic cores

For the automatic laser deburring of ceramic cores, this study proposed a method for generating deburring tool paths by combining the computer-aided design/computer-aided manufacturing (CAD/CAM) approach with two-step coarse and fine point cloud registrations method. Specifically, the CAD/CAM approach was employed to generate the ideal tool paths based on the CAD model, and the coarse registration ensured the global optimal solution of the subsequent fine registration. Considering the complex structure and the uneven shrinkage deformation of ceramic cores, the hybrid and local fine registration method was proposed to optimize the ideal tool paths in order to generate deburring tool paths. Moreover, the fine registration also eliminated the influence of inconsistent clamping errors of ceramic cores on the final accuracy of deburring tool paths. In order to select the optimal local fine registration methods, focusing on the accuracy of deburring tool paths, four rigid and non-rigid point cloud registration methods based on the iterative closest point algorithm were compared. Furthermore, a method for evaluating the accuracy of deburring tool paths was proposed. Finally, a repeatability experiment was performed to verify the accuracy and robustness of the proposed method. The generated deburring tool paths were verified and its effectiveness was proven on a five-axis laser processing equipment.

Reducing the effects of parallax in camera-based pulse-oximetry

Camera-based pulse-oximetry enables contactless estimation of peripheral oxygen saturation (SpO2). Because of the lack of readily available and affordable single-optics multi-spectral cameras, custom-made multi-camera setups with different optical filters are currently mostly used. The introduced parallax by these cameras could however jeopardise the SpO2 algorithm assumptions, especially during subject movement. In this paper we investigate the effect of parallax quantitatively by creating a large dataset consisting of 150 videos with three different parallax settings and with realistic and challenging motion scenarios. We estimate oxygen saturation values with a previously used global frame registration method and with a newly proposed adaptive local registration method to further reduce the parallax-induced image misalignment. We found that the amount of parallax has an important effect on the accuracy of the SpO2 measurement during movement and that the proposed local image registration reduces the error by more than a factor of 2 for the most common motion scenarios during screening. Extrapolation of the results suggests that the error during the most challenging motion scenario can be reduced to approximately 2 percent when using a parallax-free single-optics camera. This study provides important insights on the possible applications and use cases of remote pulse-oximetry with current affordable and readily available cameras.

Three Dimension Scanning using Infrared Coded Light

In this competitive market, reverse engineering is introduced to shorten a new product development time by digitizing an existing product for rapid redesign. In this project, a low cost and standalone 3D scanner is being developed to achieve this purpose. Research is done comparing advantages and limitation of different hardware and method for 3D scanning. Scanner is assisted by automated rotating and illumination scanning platform to improve its performance. Software is developed for scanning of object using a Red-Green-Blue-Depth (RGBD) camera. The scanned result is processed and refined using global and local registration method. Poisson surface reconstruction is performed on the processed point clouds for generating triangular mesh. This scanner able to scan the object with the accuracy up to 5mm with computational time around 25 minutes.

Automated Optical Flow Based Registration for Adaptive Optics Scanning Laser Ophthalmoscope

This study presents an automated registration method based on optical flow for an adaptive optics scanning laser ophthalmoscope. The method was designed to align and average images to obtain a higher signal-to-noise ratio image. A correlation based optical flow image registration method, which has large registration degrees of freedom, is adopted as a local registration method. By comparing the images before and after image registration, we show the effectiveness of our method. Furthermore, the advantage of our method, which is the containment of large registration degrees of freedom, is confirmed.

Effective Super-Resolution Methods for Paired Electron Microscopic Images

This paper is concerned with investigating super-resolution algorithms and solutions for handling electron microscopic images. We note two main aspects differentiating the problem discussed here from those considered in the literature. The first difference is that in the electron imaging setting. We have a pair of physical high-resolution and low-resolution images, rather than a physical image with its downsampled counterpart. The high-resolution image covers about 25% of the view field of the low-resolution image, and the objective is to enhance the area of the low-resolution image where there is no high-resolution counterpart. The second difference is that the physics behind electron imaging is different from that of optical (visible light) photos. The implication is that super-resolution models trained by optical photos are not effective when applied to electron images. Focusing on the unique properties, we devise a global and local registration method to match the high- and low-resolution image patches and explore training strategies for applying deep learning super-resolution methods to the paired electron images. We also present a simple, non-local-mean approach as an alternative. This alternative performs as a close runner-up to the deep learning approaches, but it takes less time to train and entertains a simpler model structure.

Learning to Reconstruct and Understand Indoor Scenes From Sparse Views.

This paper proposes a new method for simultaneous 3D reconstruction and semantic segmentation for indoor scenes. Unlike existing methods that require recording a video using a color camera and/or a depth camera, our method only needs a small number of (e.g., 3~5) color images from uncalibrated sparse views, which significantly simplifies data acquisition and broadens applicable scenarios. To achieve promising 3D reconstruction from sparse views with limited overlap, our method first recovers the depth map and semantic information for each view, and then fuses the depth maps into a 3D scene. To this end, we design an iterative deep architecture, named IterNet, to estimate the depth map and semantic segmentation alternately. To obtain accurate alignment between views with limited overlap, we further propose a joint global and local registration method to reconstruct a 3D scene with semantic information. We also make available a new indoor synthetic dataset, containing photorealistic high-resolution RGB images, accurate depth maps and pixel-level semantic labels for thousands of complex layouts. Experimental results on public datasets and our dataset demonstrate that our method achieves more accurate depth estimation, smaller semantic segmentation errors, and better 3D reconstruction results over state-of-the-art methods.

Shape Descriptor-Based Local Contour Profile Registration and Measurement for Flexible Automotive Sealing Strips

For vision-based measurement, there are few research or professional tools for local contour positional errors of flexible automotive rubber strips. To support the automatic measurement of contour positional errors, a novel local contour registration and measurement method based on shape descriptors is proposed. In this method, a shape descriptor is proposed to find correspondence between a reference local contour and a desired local contour. First, a shape descriptor that includes the shape representation and restrictions of the local contour is extracted from the reference contour. Second, several tolerable shape descriptors for a desired actual local contour are constructed by adding some loosening factors to the ideal descriptor, and an angular similarity-based searching strategy is used to find the best actual local contour. Finally, from the matched local point sets, a quantitative calculation step provides the desired deviation values. This method is implemented in a sealing strip cross section measurement system, and numerous cross-sectional profiles are tested. The experimental results verify the stability and effectiveness of the proposed method. Important progress toward the automatic measurement of flexible products is demonstrated.

Respiratory motion correction for free-breathing 3D abdominal MRI using CNN-based image registration: a feasibility study.

Free-breathing abdomen imaging requires non-rigid motion registration of unavoidable respiratory motion in three-dimensional undersampled data sets. In this work, we introduce an image registration method based on the convolutional neural network (CNN) to obtain motion-free abdominal images throughout the respiratory cycle. Abdominal data were acquired from 10 volunteers using a 1.5 T MRI system. The respiratory signal was extracted from the central-space spokes, and the acquired data were reordered in three bins according to the corresponding breathing signal. Retrospective image reconstruction of the three near-motion free respiratory phases was performed using non-Cartesian iterative SENSE reconstruction. Then, we trained a CNN to analyse the spatial transform among the different bins. This network could generate the displacement vector field and be applied to perform registration on unseen image pairs. To demonstrate the feasibility of this registration method, we compared the performance of three different registration approaches for accurate image fusion of three bins: non-motion corrected (NMC), local affine registration method (LREG) and CNN. Visualization of coronal images indicated that LREG had caused broken blood vessels, while the vessels of the CNN were sharper and more consecutive. As shown in the sagittal view, compared to NMC and CNN, distorted and blurred liver contours were caused by LREG. At the same time, zoom-in axial images presented that the vessels were delineated more clearly by CNN than LREG. The statistical results of the signal-to-noise ratio, visual score, vessel sharpness and registration time over all volunteers were compared among the NMC, LREG and CNN approaches. The SNR indicated that the CNN acquired the best image quality (207.42 ± 96.73), which was better than NMC (116.67 ± 44.70) and LREG (187.93 ± 96.68). The image visual score agreed with SNR, marking CNN (3.85 ± 0.12) as the best, followed by LREG (3.43 ± 0.13) and NMC (2.55 ± 0.09). A vessel sharpness assessment yielded similar values between the CNN (0.81 ± 0.03) and LREG (0.80 ± 0.04), differentiating them from the NMC (0.78 ± 0.06). When compared with the LREG-based reconstruction, the CNN-based reconstruction reduces the registration time from 1 h to 1 min. Our preliminary results demonstrate the feasibility of the CNN-based approach, and this scheme outperforms the NMC- and LREG-based methods. Advances in knowledge: This method reduces the registration time from ~1 h to ~1 min, which has promising prospects for clinical use. To the best of our knowledge, this study shows the first convolutional neural network-based registration method to be applied in abdominal images.

Indoor Scene Point Cloud Registration Algorithm Based on RGB-D Camera Calibration.

With the increasing popularity of RGB-depth (RGB-D) sensor, research on the use of RGB-D sensors to reconstruct three-dimensional (3D) indoor scenes has gained more and more attention. In this paper, an automatic point cloud registration algorithm is proposed to efficiently handle the task of 3D indoor scene reconstruction using pan-tilt platforms on a fixed position. The proposed algorithm aims to align multiple point clouds using extrinsic parameters of the RGB-D camera obtained from every preset pan-tilt control point. A computationally efficient global registration method is proposed based on transformation matrices formed by the offline calibrated extrinsic parameters. Then, a local registration method, which is an optional operation in the proposed algorithm, is employed to refine the preliminary alignment result. Experimental results validate the quality and computational efficiency of the proposed point cloud alignment algorithm by comparing it with two state-of-the-art methods.

Intraoperative localized constrained registration in navigated bronchoscopy.

One of the major challenges in electromagnetic navigated bronchoscopy is the navigation accuracy. An initial rigid image-to-patient registration may not be optimal for the entire lung volume, as the lung tissue anatomy is likely to have shifted since the time of computer tomography (CT) acquisition. The accuracy of the initial rigid registration will also be affected throughout the procedure by breathing, coughing, patient movement and tissue displacements due to pressure from bronchoscopy tools. A method to minimize the negative impact from these factors by updating the registration locally during the procedure is needed and suggested in this paper. The intraoperative local registration method updates the initial registration by optimization in an area of special interest, for example, close to a biopsy position. The local registration was developed through an adaptation of a previously published registration method used for the initial registration of CT to the patient anatomy. The method was tested in an experimental breathing phantom setup, where respiratory movements were induced by a robotic arm. Deformations were also applied to the phantom to see if the local registration could compensate for these. The local registration was successfully applied in all 15 repetitions, five in each of the three parts of the airway phantom. The mean registration accuracy was improved from 11.8-19.4 mm to 4.0-6.7 mm, varying to some degree in the different segments of the airway model. A local registration method, to update and improve the initial image-to patient registration during navigated bronchoscopy, was developed. The method was successfully tested in a breathing phantom setup. Further development is needed to make the method more automatic. It must also be verified in human studies.

An ASIFT-Based Local Registration Method for Satellite Imagery

Imagery registration is a fundamental step, which greatly affects later processes in image mosaic, multi-spectral image fusion, digital surface modelling, etc., where the final solution needs blending of pixel information from more than one images. It is highly desired to find a way to identify registration regions among input stereo image pairs with high accuracy, particularly in remote sensing applications in which ground control points (GCPs) are not always available, such as in selecting a landing zone on an outer space planet. In this paper, a framework for localization in image registration is developed. It strengthened the local registration accuracy from two aspects: less reprojection error and better feature point distribution. Affine scale-invariant feature transform (ASIFT) was used for acquiring feature points and correspondences on the input images. Then, a homography matrix was estimated as the transformation model by an improved random sample consensus (IM-RANSAC) algorithm. In order to identify a registration region with a better spatial distribution of feature points, the Euclidean distance between the feature points is applied (named the S criterion). Finally, the parameters of the homography matrix were optimized by the Levenberg–Marquardt (LM) algorithm with selective feature points from the chosen registration region. In the experiment section, the Chang’E-2 satellite remote sensing imagery was used for evaluating the performance of the proposed method. The experiment result demonstrates that the proposed method can automatically locate a specific region with high registration accuracy between input images by achieving lower root mean square error (RMSE) and better distribution of feature points.

A Structure-Based Registration Method for Terrestrial Laser Scanning Data

This paper presents a novel structure-based registration method for terrestrial laser scanning (TLS) data. The line support region (LSR), which fits the 3D line segment, is adopted to describe the scene structure and reduce geometric complexity. Then we employ an evolution computation method to solve the optimization problem of global registration. Our method can be further enhanced by iterative closest points (ICP) or other local registration methods. We demonstrate the robustness of our algorithm on several point cloud sets with varying extent of overlap and degree of noise.

PD-0042: Automated detection of setup errors in carbon ion therapy using particle therapy PET: feasibility study

derived from the treatment plan, geometric evaluation is typically in three orthogonal directions. The purpose of this study was to retrospectively evaluate the clinically applied compromises in terms of dose,using a dose estimation method that is fast enough to be used online. Materials and Methods: 15 NSCLC patients treated with SBRT (3 fractions,54Gy), were selected where the classical geometric tolerance limits were exceeded during treatment and a compromise was made for the tumor alignment. A multiple local rigid registration method was implemented to estimate the patient deformation. 3D rectangular regions of interested (ROI) were used for the following structures: spinal cord, sternum, trachea, carina, heart and the OARs receiving a high dose, while the tumor was registered using a shaped region of interest. Thin-plate spline interpolation was used to calculate the deformation vector field, allowing a deformation of the planning CT delineations onto the CBCT image. Subsequently, the maximum dose was estimated (assuming local dose shift invariance) for the OAR, comparing the setup correction purely based on the tumor position and the clinical compromise. For the GTV the difference in D99 was calculated. Results: Results are summarized in table 1 for each fraction separately (44 in total). For only two thirds of the OARs a decrease of de max dose was achieved after compromise. For 6 out of 113 OARs (2 patients), the max dose tolerance limit was exceeded without compromise (range 0.1 – 1.6 Gy), meaning that only for 2 out of 15 patients a compromise was actually needed. After compromise 2 OARs out of 6 were still above their maximum dose tolerance limit (range 0.1 – 0.2 Gy). For the tumor dose 44 GTVs were evaluated. In general, the compromise showed a decrease in D99 for the GTV. In one case, the decrease in tumor dose was extreme (almost 10 Gy), due to a compromise made to spare a nearby vessel.

New multi-resolution image stitching with local and global alignment

Three main problems affect the alignment quality of existing studies on multi-resolution image stitching: (i) the initial motion obtained is sometimes incorrect; (ii) the local motion is hard to be estimated and (iii) the widely used global bundle adjustment is difficult to converge. The authors propose a new multi-resolution image mosaic method that combines three corresponding tactics to solve these problems. The first problem is solved by introducing an additional motion refinement strategy, which consists of the low-contrast filter and RANSAC. The former removes flatly textured surface pixels and thus eliminates the falsely matched features. The latter removes outliers and finds a robust initial motion for the next layer. The second problem is resolved by a new iteratively local registration method, which calibrates the current camera parameters based on those from previous image with robust non-linear optimisation methods. It improves the convergence efficiency and eliminates error minimisation. For the last problem, the authors introduce a five-parameter bundle adjustment method based on the axis-angle decomposition of the rotation matrix. Comparing with existing bundle adjustment methods, this method is more stable because of an accurate and simple rotation decomposition. The authors show the efficiency of the method with qualitative and quantitative experiments.

Local landmark-based registration for fMRI group studies of nonprimary auditory cortex

Interindividual functional and structural brain variability is a major problem in group studies, in which very focal activations are expected. Architectonic studies have shown that the human primary auditory area, which is located with a great constancy on Heschl's gyrus, is surrounded by several nonprimary auditory areas with surface areas of 40–310 mm 2. The small size of the latter makes them only partially accessible to fMRI group studies, because of imprecision in realignment when using currently available registration procedures. We describe here a new method for sulcal realignment using a non-rigid local landmark-based registration and show its application to the registration of fMRI acquisitions on the supratemporal plane. After an affine global voxel-based registration, which transforms all brains into the same standard space, we propose a non-rigid local landmark-based registration method based on thin-plate splines for matching the two sulci delimiting Heschl's gyrus of a given brain to the corresponding sulci of a reference brain. We show here that, in comparison with global affine and non-rigid approaches, our method leads in group studies to i) a much more precise alignment of Heschl's gyrus; and ii) a putatively optimal superposition of functionally corresponding areas on and around Heschl's gyrus.