Automatic Registration Research Articles

Registration of photoacoustic tomography vascular images: Comparison and analysis of automatic registration approaches

Image registration is crucial in the clinical application of photoacoustic tomography (PAT) for vascular growth monitoring. Aiming to find an optimized registration scheme for PAT vascular images acquired at different times and with varying imaging conditions, we compared and analyzed different commonly used intensity-based and feature-based automatic registration schemes. To further improve the registration performance, we proposed a new scheme that combines phase correlation with these commonly used intensity-based registration methods and compared their performances. The objective evaluation measures: peak signal-to-noise ratio (PSNR), structural similarity index metric (SSIM), root mean square error (RMSE), and quantitative visual perception (jump percentage P), as well as subjective evaluation using mean opinion score (MOS), were combined to evaluate the registration performance. Results show that the feature-based approaches in this study were not suitable for PAT image registration. And by adding phase correlation as rough registration, the overall registration performance was improved significantly. Among these methods, the proposed scheme of phase correlation combined with mean square error (MSE) similarity measure and regular-step-gradient-descent optimizer provides the best visual effect, accuracy, and efficiency in PAT vascular image registration.

Supervoxel-based targetless registration and identification of stable areas for deformed point clouds

Abstract Accurate and robust 3D point cloud registration is the crucial part of the processing chain in terrestrial laser scanning (TLS)-based deformation monitoring that has been widely investigated in the last two decades. For the scenarios without signalized targets, however, automatic and robust point cloud registration becomes more challenging, especially when significant deformations and changes exist between the sequence of scans which may cause erroneous registrations. In this contribution, a fully automatic registration algorithm for point clouds with partially unstable areas is proposed, which does not require artificial targets or extracted feature points. In this method, coarsely registered point clouds are firstly over-segmented and represented by supervoxels based on the local consistency assumption of deformed objects. A confidence interval based on an approximate assumption of the stochastic model is considered to determine the local minimum detectable deformation for the identification of stable areas. The significantly deformed supervoxels between two scans can be detected progressively by an efficient iterative process, solely retaining the stable areas to be utilized for the fine registration. The proposed registration method is demonstrated on two datasets (both with two-epoch scans): An indoor scene simulated with different kinds of changes, including rigid body movement and shape deformation, and the Nesslrinna landslide close to Obergurgl, Austria. The experimental results show that the proposed algorithm exhibits a higher registration accuracy and thus a better detection of deformations in TLS point clouds compared with the existing voxel-based method and the variants of the iterative closest point (ICP) algorithm.

Divergent Electoral Policies: Why Some States Increase Ballot Access

AbstractWhy have some states adopted policies expanding ballot access while others have restricted access to the ballot? Since the 1990s, some states have been adopting policies restricting access to the ballot such as requiring identification. At the same time, states have been adopting a variety of registration reforms that lower the barriers to registration and voting. Using an original, 45-state dataset, we examine state innovation within the policy domain of electoral reforms in US states. We find reforms have an independent and, sometimes, negative effect on the innovation of states in electoral reforms. Next, we use dyad analysis to examine the spread of a single policy: automatic voter registration. We find that the propensity to innovate both within and across a state makes the spread of automatic voter registration more likely. Our paper contributes to the broader understanding of why states adopt electoral reforms.

A hierarchical multiview registration framework of TLS point clouds based on loop constraint

Automatic registration of multiple point clouds is a significant preprocessing step for 3D computer vision tasks including semantic segmentation, 3D modelling, change detection, etc. Many methods were proposed to deal with this problem and yet most of them are not fully utilizing the redundant information offered by multiple common overlaps among point clouds. The existing methods are also inefficient when dealing with large-scale point clouds. In this paper, a novel automatic registration framework is presented to align point clouds efficiently and robustly. First, the overall number of scans is grouped into several scan-blocks by a proposed blocking strategy, and we build the pairwise relationship among scans through a fully connected graph in each scan-block. Second, perform loop-based coarse registration in each scan-block using a proposed false matches removal strategy. The proposed strategy can effectively identify grossly wrong scan-to-scan matches. Third, the minimum spanning tree is extracted from the graph, and ICP is applied along its edges. Moreover, the Lu–Milios algorithm is used to further optimize all poses at once by utilizing all redundant information in each scan-block. Finally, global block-to-block registration aligns all scan-blocks into a uniform coordinate reference. We test our framework on challenging WHU-TLS datasets, ETH datasets, and Robotic 3D Scan datasets to evaluate the efficiency, accuracy, as well as robustness. The experiment results show that our method achieves the state-of-the-art accuracy, while the time performance is improved by more than 30% compared with the state-of-the-art algorithms. Our source code is made available at https://github.com/WuHao-WHU/HL-MRF for benchmarking purposes.

CD/CV: Blockchain-based schemes for continuous verifiability and traceability of IoT data for edge–fog–cloud

This paper presents a continuous delivery/continuous verifiability (CD/CV) method for IoT dataflows in edge–fog–cloud. A CD model based on extraction, transformation, and load (ETL) mechanism as well as a directed acyclic graph (DAG) construction, enable end-users to create efficient schemes for the continuous verification and validation of the execution of applications in edge–fog–cloud infrastructures. This scheme also verifies and validates established execution sequences and the integrity of digital assets. CV model converts ETL and DAG into business model, smart contracts in a private blockchain for the automatic and transparent registration of transactions performed by each application in workflows/pipelines created by CD model without altering applications nor edge–fog–cloud workflows. This model ensures that IoT dataflows delivers verifiable information for organizations to conduct critical decision-making processes with certainty. A containerized parallelism model solves portability issues and reduces/compensates the overhead produced by CD/CV operations. We developed and implemented a prototype to create CD/CV schemes, which were evaluated in a case study where user mobility information is used to identify interest points, patterns, and maps. The experimental evaluation revealed the efficiency of CD/CV to register the transactions performed in IoT dataflows through edge–fog–cloud in a private blockchain network in comparison with state-of-art solutions.

Accuracy of markerless registration methods of DICOM and STL files used for computerized surgical guides in mandibles with metal restorations: An in vitro study

Statement of problemDigital imaging and communications in medicine (DICOM) files together with surface scans must be accurately registered in virtual implant planning software programs to match real-life dimensions and ensure correct plan transfer through computer-aided manufactured surgical guides. PurposeThe purpose of this in vitro study was to evaluate the accuracy of 3 different registration methods of DICOM data with and without metal restorations and a metal artifact reduction (MAR) tool for surface scans. Material and methodsThirteen dentate mandibles were assigned to each group of this study (n=39). Baseplate wax was adapted to the bone surfaces of each mandible, and 5 radiopaque markers were attached. A desktop scanner was used to obtain control scans. The groups of metal-free mandibles (MFM) and mandibles with metal restorations (MRM) were scanned to obtain DICOM data without a MAR tool. Additional DICOM data for the MRM were obtained with the MAR tool (MRM-MAR). Point-based registration (PBR), best-fit registration (BFR), and automatic registration (AR) were used to align standard tessellation language (STL) and DICOM data, and 3 data sets were exported. Radiographic markers on each data set were compared with those on the control scan, and positional deviations were calculated and statistically evaluated with 1-way ANOVA followed by multiple pairwise comparisons, independent samples t test, and 2-way ANOVA (α=.05). ResultsWithin each group, PBR had the lowest deviation values with statistical significance in the MFM and the MRM-MAR groups (P<.001). AR showed failure in the MRM and the MRM-MAR groups. Statistically significant differences were found on comparing the average deviations among the 3 groups for PBR only (P<.001). No association was found between deviation values and the presence or absence of metal restoration, while a positive association was found with the type of registration method (P<.001). ConclusionsPBR had the highest accuracy level compared with AR and BFR methods. An increase in the number of calculations resulted in more deviation values. The MAR tool had a positive effect on PBR in mandibles with metal restorations.

Image-to-Image Subpixel Registration Based on Template Matching of Road Network Extracted by Deep Learning

The vast digital archives collected by optical remote sensing observations over a long period of time can be used to determine changes in the land surface and this information can be very useful in a variety of applications. However, accurate change extraction requires highly accurate image-to-image registration, which is especially true when the target is urban areas in high-resolution remote sensing images. In this paper, we propose a new method for automatic registration between images that can be applied to noisy images such as old aerial photographs taken with analog film, in the case where changes in man-made objects such as buildings in urban areas are extracted from multitemporal high-resolution remote sensing images. The proposed method performs image-to-image registration by applying template matching to road masks extracted from images using a two-step deep learning model. We applied the proposed method to multitemporal images, including images taken more than 36 years before the reference image. As a result, the proposed method achieved registration accuracy at the subpixel level, which was more accurate than the conventional area-based and feature-based methods, even for image pairs with the most distant acquisition times. The proposed method is expected to provide more robust image-to-image registration for differences in sensor characteristics, acquisition time, resolution and color tone of two remote sensing images, as well as to temporal variations in vegetation and the effects of building shadows. These results were obtained with a road extraction model trained on images from a single area, single time period and single platform, demonstrating the high versatility of the model. Furthermore, the performance is expected to be improved and stabilized by using images from different areas, time periods and platforms for training.

A Fast 3D/3D Registration Method Based on Water Equivalent Path Length for Heavy-Ion Radiotherapy

<h3>Purpose/Objective(s)</h3> The accuracy of patient setup needs to be high in heavy-ion radiotherapy. Accordingly, there is an increasing demand for three-dimensional (3D) image-guided systems because of the limitations of the present two-dimensional imaging registration technology. One of the limitations is the inability to directly compare with planning computed tomography (CT) images. Due to this uncertainty in conformation, high dose per fraction radiotherapy is difficult. This study proposes a fast 3D/3D registration method comparing with water equivalent path length (WEPL) that uses planning and treatment CT images to minimize range variations by using the Lucas-Kanade (LK) algorithm. <h3>Materials/Methods</h3> A six-degrees-of-freedom (6DOF) couch correction was simulated for patient positioning at our center by performing automatic rigid image registration (RIR) and using the resultant translation/rotation. Our automatic RIR algorithm was used to align the planning CT and treatment CT images, simulating 6DOF. The alignment values (Δ<i>V</i>), which represents a six-dimensional vector of the amounts of 6DOF couch movement, were calculated by minimizing the following error function: E(<i>ΔV</i>)=Σ <i>_i</i>_∈Ω [<i>T_i</i> (<i>V+ΔV</i>)-<i>P_i</i> (<i>V</i>)]², where <i>V</i> is a six-dimensional vector that denotes a CT position and rotation in a treatment room; <i>T_i</i> (<i>V</i>) and <i>P_i</i> (<i>V</i>) are treatment and planning CT images respectively, whose position in the room is represented by <i>V; i</i> is a 3D vector which denotes a point in the room; and Ω is a set of calculation points, which was assigned a planning target volume (PTV) of Ω in this study. We minimized the error function by using the LK algorithm. To minimize the range variations, each pixel value of the CT images was changed into WEPL values and the alignment values were calculated in the same manner as the above algorithm. The method was evaluated using a dataset containing eight prostate cancer patient images, including a planning CT image with PTV and treatment CT images of four fraction with a fan-beam CT. The range variations were measured by calculating WEPL variations between the planning CT and treatment CT at (1) before registration and after registration based on (2) CT pixel values and (3) WEPL values, respectively. In addition, the calculation time was measured on a workstation with a single GPU. <h3>Results</h3> The mean WEL variation across the 32 cases (eight patients of four fractions) was (1) 1.8 ± 1.16 mm, (2) 1.08 ± 0.52 mm, and (3) 0.22 ± 0.17 mm. The worst WEPL variation were (1) 6.23 mm, (2) 3.06 mm, and (3) 1.17 mm. The mean calculation time were (2) 89 ms and (3) 1066 ms. <h3>Conclusion</h3> We developed a fast 3D/3D RIR method for reducing range variations to realize more accurate heavy-ion radiotherapy. We performed experiments involving eight prostate cancer patients, and our proposed method obtained high accuracy and short computation time.

A unified framework for automated registration of point clouds, mesh surfaces and 3D models by using planar surfaces

AbstractRegistration of 3D spatial data and models is a fundamental task in applications such as mapping, positioning and virtual/augmented reality. Most of the existing 3D registration methods such as iterative closest point (ICP) and recent learning‐based methods are dedicated to point cloud registration, and rely heavily on point‐wise correspondences, which limits their ability to address registration problems across different data types. Since man‐made objects and buildings usually contain many planar surfaces, it is possible to use the planes for accurate registration of different data and models. In this paper, a unified registration framework is proposed consisting of a plane extraction module, which can extract planes from various forms of spatial data such as point clouds or surface‐based 3D models, and a registration module, which performs automatic registration based on the extracted planes. Tests show that the proposed method can handle small‐overlap registration across all these data types with high success rates. The result of point cloud registration also indicates that the method achieves better accuracy as compared to ICP.

Automatic Voter Reregistration as a Housewarming Gift: Quantifying Causal Effects on Turnout Using Movers

How much can automatic voter registration (AVR) increase turnout? Unlike in many democracies, most American voters face the additional cost of registration, resulting in potential disenfranchisement. Automatic voter registration is naturally expected to promote turnout, but its causal effects have rarely been quantified due to violations of crucial assumptions. I show that a variation of AVR that targets existing registrants as opposed to eligible nonregistrants—termed automatic reregistration (ARR)—increases turnout by 5.8 percentage points. I exploit a natural experiment in a novel administrative dataset; election officials in Orange County, California, notified existing registrants who moved within-county that their residential addresses were automatically updated. The treatment alleviated registrants of reregistration burdens, but only for those who moved before the legal cutoff date, enabling a quasi-random treatment assignment. Contrary to the popular narrative, ARR had no significant effect on the turnout of registered Democrats, but Republicans’ and nonpartisans’ turnout increased by 8.1 and 7.4 percentage points, respectively.

Geometric accuracy assessment and a framework for automatic sub-pixel registration of WFI images from CBERS-4, CBERS-4A, and Amazonia-1 satellites over Brazil

Brazil faces the challenge of conciliating food production with natural vegetation preservation, and the government has been investing in Earth Observation satellites to assist in this task. In this context, images from the Wide-Field Imager (WFI) sensor onboard the China-Brazil Earth Resources Satellite (CBERS-4), CBERS-4A, and Amazonia-1 have been used in remote sensing applications in Brazil. However, the geometric accuracy of the WFI data has not yet been documented in the literature, as it was done for other satellites/sensors such as TERRA/MODIS, Landsat-8/OLI, and Sentinel-2/MSI. This work presents the first study to assess the geometric accuracy of WFI images along different spectral channels, from multi-temporal images, and taking Sentinel-2/MSI images as a reference. Surface reflectance WFI images, Level-4 (orthorectified) with spatial resolution of 55 m (CBERS-4A) and 64 m (CBERS-4 and Amazonia-1) were analysed. Thousands of control points (CPs) were used to quantify the geometric accuracy of more than 300 WFI images and to construct multiple mapping functions to correct the displacements found. The NIR band was taken as a reference in the inter-channel (band-to-band) registration evaluation. In this case, all tiles had an RMSE average of fewer than 0.44 pixels. For the multi-temporal case, the CBERS-4A/WFI had the highest average RMSE (0.87 ± 0.34 pixels). Considering the Sentinel-2/MSI as a reference, there were observed shifts up to 1.45 pixels for CBERS-4/WFI and Amazonia-1/WFI and 2.5 pixels for CBERS-4A/WFI. Given the displacements found, image registration is needed for further time series analysis of WFI images for several applications that requiring precise sub-pixel registration (i.e., land use cover change). As a result, we propose a framework for the automatic registration of WFI images to properly foster the use of WFI images. This framework substantially reduced the misregistrations, decreasing the RMSE average to less than 0.29 pixels for the WFI images from the three satellites. The use of the 2nd degree polynomial as a mapping function proved to be appropriate for registering the WFI data.

Conditional GANs based system for fibrosis detection and quantification in Hematoxylin and Eosin whole slide images.

Assessing the degree of liver fibrosis is fundamental for the management of patients with chronic liver disease, in liver transplants procedures, and in general liver disease research. The fibrosis stage is best assessed by histopathologic evaluation, and Masson's Trichrome stain (MT) is the stain of choice for this task in many laboratories around the world. However, the most used stain in histopathology is Hematoxylin Eosin (HE) which is cheaper, has a faster turn-around time and is the primary stain routinely used for evaluation of liver specimens. In this paper, we propose a novel digital pathology system that accurately detects and quantifies the footprint of fibrous tissue in HE whole slide images (WSI). The proposed system produces virtual MT images from HE using a deep learning model that learns deep texture patterns associated with collagen fibers. The training pipeline is based on conditional generative adversarial networks (cGAN), which can achieve accurate pixel-level transformation. Our comprehensive training pipeline features an automatic WSI registration algorithm, which qualifies the HE/MT training slides for the cGAN model. Using liver specimens collected during liver transplantation procedures, we conducted a range of experiments to evaluate the detected footprint of selected anatomical features. Our evaluation includes both image similarity and semantic segmentation metrics. The proposed system achieved enhanced results in the experiments with significant improvement over the state-of-the-art CycleGAN learning style, and over direct prediction of fibrosis in HE without having the virtual MT step.

DEMOCRATIZATION AND MATURE ELECTIONS: A STUDY OF YOUTH POLITICAL PARTICIPATION IN VOTERS 18 AND AUTOMATIC VOTER REGISTRATION

The discussion in this paper pertains to the issue of Voters 18 and Automatic Voter Registration (AVR). The issues refined in this debate include: (1) The efforts of democratization that is mature, fair, and equitable elections to advance social and political development; (2) Studies on the impact of youth political participation in the General Election (GE); (3) Justification of the enforcement &amp; implementation of Vote 18; (4) Improvisation of the work of the election series; and (5) Jurisdiction and responsibility of the Election Commission (EC). This paper uses secondary sources, such as research results published in journals, scholarly writing, online news sources, and public forums, to employ a qualitative methodology. The results of the study indicate that the participation of 18-year-old voters is an important factor in the formation of a government capable of representing the interests of all age groups and the community as a whole. It is hoped that the young generation or youth will now exercise their rights with maturity, discernment, and efficiency. Voting is a personal decision, and there is no incorrect vote, but all young Malaysians should have the right to vote. The future of the nation can be influenced by the rationale behind efforts to involve the youth in political power determination. Thus, the elaboration research of this study should concentrate on the significance of political system, state government, and the federal constitution education for the young generation in order to prepare them to participate in general elections.

Automatic Registration for Panoramic Images and Mobile LiDAR Data Based on Phase Hybrid Geometry Index Features

The registration of panoramic images and mobile light detection and ranging (LiDAR) data is quite challenging because different imaging mechanisms and viewing angle differences generate significant geometric and radiation distortions between the two multimodal data sources. To address this problem, we propose a registration method for panoramic images and mobile LiDAR data based on the hybrid geometric structure index feature of phase. We use the initial GPS/IMU to transform the mobile LiDAR data into an intensity map and align the two images to complete registration. Firstly, a novel feature descriptor called a hybrid geometric structure index of phase (HGIFP) is built to capture the structural information of the images. Then, a set of corresponding feature points is obtained from the two images using the constructed feature descriptor combined with a robust false-match elimination algorithm. The average pixel distance of the corresponding feature points is used as the error function. Finally, in order to complete the accurate registration of the mobile LiDAR data and panoramic images and improve computational efficiency, we propose the assumption of local motion invariance of 3D–2D corresponding feature points and minimize the error function through multiple reprojections to achieve the best registration parameters. The experimental results show that the method in this paper can complete the registration of panoramic images and the mobile LiDAR data under a rotation error within 12° and a translation error within 2 m. After registration, the average error of rotation is about 0.15°, and the average error of translation is about 1.27 cm. Moreover, it achieves a registration accuracy of less than 3 pixels in all cases, which outperforms the current five state-of-the-art methods, demonstrating its superior registration performance.

Use of Neuronavigation and Augmented Reality in Transsphenoidal Pituitary Adenoma Surgery.

The aim of this study was to report on the clinical experience with microscope-based augmented reality (AR) in transsphenoidal surgery compared to the classical microscope-based approach. AR support was established using the head-up displays of the operating microscope, with navigation based on fiducial-/surface- or automatic intraoperative computed tomography (iCT)-based registration. In a consecutive single surgeon series of 165 transsphenoidal procedures, 81 patients underwent surgery without AR support and 84 patients underwent surgery with AR support. AR was integrated straightforwardly within the workflow. ICT-based registration increased AR accuracy significantly (target registration error, TRE, 0.76 ± 0.33 mm) compared to the landmark-based approach (TRE 1.85 ± 1.02 mm). The application of low-dose iCT protocols led to a significant reduction in applied effective dosage being comparable to a single chest radiograph. No major vascular or neurological complications occurred. No difference in surgical time was seen, time to set-up patient registration prolonged intraoperative preparation time on average by twelve minutes (32.33 ± 13.35 vs. 44.13 ± 13.67 min), but seems justifiable by the fact that AR greatly and reliably facilitated surgical orientation and increased surgeon comfort and patient safety, not only in patients who had previous transsphenoidal surgery but also in cases with anatomical variants. Automatic intraoperative imaging-based registration is recommended.

Automatic Registration Algorithm for the Point Clouds Based on the Optimized RANSAC and IWOA Algorithms for Robotic Manufacturing

In order to solve the problems of low accuracy and low efficiency of point cloud registration for stereo camera systems, we propose a binocular stereo camera point cloud registration method based on IWOA and Improved ICP. We propose the following approaches in this paper—the registration process is divided into two steps to complete the initial coarse registration and the exact registration. In the initial registration stage, an improved Whale Optimization Algorithm (IWOA) based on nonlinear convergence factor and adaptive weight coefficients was proposed to realize the initial registration in combination with the RANSAC algorithm, and the obtained transformation matrix was used as the initial estimate of the subsequent exact registration algorithm. In the second step of the exact registration stage, an IICP algorithm with the introduction of normal vector weighting constraints at key points was proposed for achieving point cloud exact registration. This algorithm was verified by using Stanford point clouds (bunnies and monkeys) and our own point clouds algorithm, and the proposed algorithm in this paper has high registration accuracy, improved registration speed, and convergence speed.

Automatic registration of optical image and airborne LiDAR data based on centers and corner points of building boundaries

ABSTRACT Applications based on the fusion of aerial imagery and Light Detection and Ranging (LiDAR) data have attracted more and more interest in recent years. Registration is the crucial prerequisite of data integration because of the spatial shift between two data modalities. However, there are many challenges for registration due to the different mechanisms of LiDAR and camera, especially in complex urban scenes. To address the problem, this paper proposes an automatic and robust registration method by using the centres and corner points of building boundaries as tie points. Primary registration is achieved by using centers of building boundaries as control points. To increase the robustness and accuracy of the registration, corner points of building boundaries are utilized to estimate the transformation model repeatedly. Two datasets of optical imagery and LiDAR data with different scene coverage are used to verify the performance of the proposed method. Experimental results show that the proposed method can achieve the registration accuracy of less than 2 pixels for both cases based on an assessment of check lines. This result can basically meet the requirements of data fusion.

Design and Construction of Intelligent Corridor of Tourist Attractions Based on Sensor Network

This paper presents an in-depth analysis and study of traditional corridor construction and tourism system planning using intelligent sensing networks. This paper proposes a node deployment and coverage optimization method for HWSNs based on particle swarm optimization emperor butterfly algorithm and extreme learning machine, with the objectives of reducing node deployment cost, reducing coverage redundancy and voids, improving monitoring area coverage, and increasing the agility of data processing by 15.03%, efficiency by 12.15%, and stability by 11.31%. In the paper, the static parameter table of the sensor is stored inside the smart sensor or stored on the IoT cloud platform to achieve automatic registration of the static parameters of the smart sensor. According to the characteristics of dynamic parameters of smart sensors, an autonegotiation algorithm is designed to realize the dynamic parameters of smart sensors to be determined by autonegotiation and complete registration without human intervention, which includes four parts: utility function, concussion protocol, proposal generation strategy, and response mechanism. The intelligent sensing system is composed of three parts: smart sensor, IoT gateway, and IoT cloud platform; the functions of each part in the system are analyzed, and MAR and STM32F4 are selected as the core board of smart sensor and IoT gateway. And for the problems of time-consuming and tedious access to smart sensors in the system, we propose to study the adaptive sensor UART serial port parameters and automatic registration of sensor parameters. Based on the analysis of the current environment and resource characteristics of the traditional corridor, the current situation of ecotourism resources of the traditional corridor is sorted out, an evaluation index system is established to evaluate ecotourism resources, and the current development basis is analyzed from four aspects: natural environment, tourism resources, product development, and infrastructure. Finally, to promote the sustainable development of the ecotourism corridor in the Chengdu section of the southern foot of the Qinling Mountains, the planning and development strategies to match the ecotourism corridor in the Chengdu section of the southern foot of the Qinling Mountains are proposed in four aspects: transportation organization optimization, ecological landscape enhancement, support system construction, and regional tourism development.

Interactive 3D touch and gesture capable holographic light field display with automatic registration between user and content

AbstractThis paper presents an interactive three‐dimensional (3D) touch and gesture user interface that utilizes a light‐field display and a 3D sensor. In contrast to conventional gesture interfaces, in which usually the image space and gesture space are separated, a user can directly interact with the 3D image in this interface system. To achieve this trait, the 3D registration between the sensor and display coordinate systems is necessary. In this study, we propose a method to automatically match the position of the reconstructed 3D image with the position of the hand of the user based on scattered light detection. The proposed calibration process is automatic and enables self‐calibration during practical operation without laborious measurement work. The method allows us to use the gesture detection capabilities of the leap motion controller to develop an interface where content and gesture positions match. We show some examples of 3D touch and gesture interactions in which the gesture position and the generated content match on a light field display.

A robust registration method for UAV thermal infrared and visible images taken by dual-cameras

Automatic registration of unmanned aerial vehicle (UAV) thermal infrared and visible (TIR&V) images is fundamental for subsequent applications. However, few studies address this issue due to significant radiation gap, shape gap, and texture gap among TIR&V images. The area-based methods are not able to satisfy the accuracy and robustness of location at the same time, while the image pyramid-based methods are computationally expensive. To alleviate these problems, we proposed a so-called TWMM method for the registration of UAV TIR&V images taken by the camera equipped with both thermal infrared and visible sensors. TWMM is realized by combining Template matching with Weights, Multilevel local max-pooling, and Max index backtracking. TWMM consists of four steps: (1) computing similarity maps of the atomic patches using template matching with weights; (2) building pyramid similarity maps using multilevel local max-pooling; (3) deducing the corresponding points (CPs) from top to bottom using max index backtracking; and (4) eliminating outliers and estimating homography. Among the four steps, step 1 and step 2 are used to compute the similarity maps of patches with different sizes; step 3 and step 4 are used to deduce CPs and estimate homography with multiple similarity maps. TWMM was comprehensively evaluated with 600 UAV image pairs under four different scenes and also compared with current methods (i.e. SIFT, SURF, RIFT, RCB, TFeat, HardNet, RANSAC_Flow, HOPC, and CFOG). These image pairs have multiple features, i.e., different land covers, spatial resolutions, and illumination conditions, etc. Results indicate that TWMM achieves an 86.0% correct CP ratio (RCP) and a 96.0% correct matching rate (CMR) for all test images, which is a 15.1% improvement and 11.6% improvement, respectively, over the best state-of-the-art methods. TWMM also shows better robustness than other methods for weak-light images, achieving a 20.7% improvement in RCP and a 28.1% improvement in CMR. Therefore, TWMM is an effective and robust method for UAV TIR&V image registration and has good ability under different scenes.