Automatic Registration Research Articles

Robust Markerless Registration of Point Clouds for Terrestrial Laser Scanning-Based Measurement of Bulk Grains Stockpiled in Storehouses

HighlightsFully automated registration free from artificial markers for multi-scan point clouds aimed for TLS-based measurement of bulk grains in large storehouses.The geometric structure of the large grain storehouse is explored to derive geometrical features as the structurally semantic information for scene understanding.The geometrical features are modeled as a small ordered set and correspondences are established by performing trials for all possible matching pairs of two sets extracted from two different scans.Significant improvements have been achieved in registration accuracy, computational efficiency, and robustness against scenes with symmetric structures as well as the immunity to noises and varying point density.Abstract. Point clouds collected by terrestrial laser scanning (TLS) in the application of bulk grain measurement and quantification contain a vast amount of data, relatively low-textured surfaces and highly symmetric structures. All of these challenges make it a difficult task to automatically register multiple scans from different viewpoints needed to fully cover a large-scale scene. To address the challenges, this article presents a robust automatic marker-free registration method dedicated for multi-scan TLS point cloud data captured in large grain storehouses. The framework of the dedicated method follows the common procedure to split the entire registration into coarse alignment and fine registration, and uses the iterative closest point (ICP) algorithm for the latter. The main contribution of the proposed dedicated method is an efficient way to find a global coarse alignment that is robust across individual scans in a TLS-based bulk grain measurement project. To tackle the correspondence problem, which is at the core of a registration task, the geometric information inherent in grain storehouses is explored in the stage of global coarse alignment. The derived semantic feature points are modeled as a small ordered set and reliable correspondences are established by performing trials for all possible matching pairs of two sets extracted from two different scans. Experimental results show the dedicated method outperforms the existing generic markless registration approaches in terms of accuracy, robustness and computational efficiency. With robustness, efficiency and accuracy, the proposed markless point cloud registration method dedicated for bulk grain measurement can cover a gap between the TLS technology and various granary field applications. Especially, its applicability to the dominant storage structure in Chinese huge grain reserve system implies remarkable efficiency improvements and will facilitate the application of TLS-based measurement in the national grain inventory of China. Keywords: Bulk grain measurement, Feature extraction, Grain storehouse, Markerless registration, Point cloud, Terrestrial laser scanning.

A Generalized Tool for Accurate and Efficient Image Registration of UAV Multi-lens Multispectral Cameras by N-SURF Matching

The original multispectral (MS) images obtained from multi-lens multispectral cameras (MSCs) have significant misregistration errors, which require image registration for precise spectral measurement. However, due to the nonlinearity intensity differences among MS images, performing image matching is difficult to find sufficient correct matches (CMs) for image registration, and results in a complex coarse-to-fine solution. Based on the modification of speed-up robust feature (SURF), we proposed a normalized SURF (N-SURF) that can significantly increase the amount of CMs among different pairs of MS images and make one-step image registration possible. In this study, we first introduce N-SURF and adopt different MS datasets acquired from three representative MSCs (MCA-12, Altum, and Sequoia) to evaluate its matching ability. Meanwhile, we utilized three image transformation models—affine transform (AT), projective transform (PT), and an extended projective transform (EPT) to correct the misregistration errors of MSCs and evaluate their co-registration correctness. The results show that N-SURF can obtain 6–20 times more CMs than SURF and can successfully match all pairs of MS images, while SURF failed in the cases of significant spectral differences. Moreover, visual comparison, accuracy assessment, and residual analysis show that EPT can more accurately correct the viewpoint and lens distortion differences of MSCs than AT and PT, and it can obtain co-registration accuracy of 0.2–0.4 pixels. Subsequently, using the successful N-SURF matching and EPT model, we developed an automatic MS image registration tool that is suitable for various multilens MSCs.

Democracy's Destiny

From its beginning, America has had a paradoxical democracy, where all men are created equal while simultaneously denying the right to vote to anyone who was not White, male, or owned property. The pandemic exposed the fault lines of our democratic form of government. Those imposing barriers to the ballot are facing off against the advocates of access. It is not a new battle. In America, we seek ways to limit who can participate instead of expanding opportunities. We have dedicated our resources to dancing around the edges of democracy-by advocating for vote by mail or automatic voter registration, for example-while allowing states to develop blockades to the ballot that are confusing and quite effective. Without a doubt, America is at a crossroads. The shenanigans that this country has used to prevent access to the ballot box, such as the poll tax, grandfather clause, restrictive voter ID laws, voter purges, and felon disenfranchisement, are antidemocratic and harmful to our system. COVID-19 exposed the fault lines. We must repair them. A free, fair, inclusive, nondiscriminatory right to vote is essential to a healthy democracy. We are in the position to craft a true democratic system of government. Will this country live up to its democratic destiny or continue to deny our journey to a more perfect union?

Automatic 3D image registration for nano-resolution chemical mapping using synchrotron spectro-tomography

Nano-resolution synchrotron X-ray spectro-tomography has been demonstrated as a powerful tool for probing the three-dimensional (3D) structural and chemical heterogeneity of a sample. By reconstructing a number of tomographic data sets recorded at different X-ray energy levels, the energy-dependent intensity variation in every given voxel fingerprints the corresponding local chemistry. The resolution and accuracy of this method, however, could be jeopardized by non-ideal experimental conditions, e.g. instability in the hardware system and/or in the sample itself. Herein is presented one such case, in which unanticipated sample deformation severely degrades the data quality. To address this issue, an automatic 3D image registration method is implemented to evaluate and correct this effect. The method allows the redox heterogeneity in partially delithiated LixTa0.3Mn0.4O2 battery cathode particles to be revealed with significantly improved fidelity.

Multimodal Urban Remote Sensing Image Registration Via Roadcross Triangular Feature

Automatic image registration of multimodal urban remote sensing images remains a critical challenging task in remote sensing image analysis due to significant nonlinear radiation distortions between multimodal image pairs; most of the traditional methods focus on the feature point detection and its local description and ignore the robust road information in multimodal urban remote sensing images. Motivated by this, we propose a fast and robust registration method for multimodal urban remote sensing images via road intersection triangular features. The proposed method obtains three main stages: Road lines extraction from images, intersection triangular feature construction, and triangular feature matching. The qualitative and quantitative experimental results show that the proposed method significantly outperforms other state-of-the-art methods, even when others completely fail to achieve the registration task of cross-modal images, our method still maintains good robustness and matching efficiency.

Проблемные аспекты в реализации информационной технологии «Автоагент» на железнодорожном транспорте (на примере Восточно-Сибирской железной дороги)

The article presents the technology of the railway divisions in handling the automatic registration of shipping documents for transportation of empty gondola cars in domestic traffic (“Autoagent”). Regulatory and technical documents defining the procedure for the implementation of the technology are studied in detail, and some setbacks in the registration of transportation documents based on paper medium are identified. The “Autoagent” system failures after updating the AS ETRAN program, which is in constant operation on the East Siberian Railway, are determined and analysed. Conclusion on the implementation and improvement of the experimental technologies and various programs that are in constant use are made.

ПОВЫШЕНИЕ КАЧЕСТВА ТАМОЖЕННОГО АДМИНИСТРИРОВАНИЯ ПУТЕМ ВНЕДРЕНИЯ СОВРЕМЕННЫХ ИНФОРМАЦИОННЫХ ТЕХНОЛОГИЙ

The problems of improving the quality of customs administration are relevant for the Federal Customs Service of Russia. The development of integration processes and the activation of foreign economic relations in Russia necessitates the improvement of the activities of customs authorities, improving the efficiency of customs administration through the introduction of modern information technologies, as well as improving the entire customs infrastructure. The priority in organizing the activities of the Federal Customs Service of the Russian Federation should be the creation and implementation of information and software tools and technologies that provide full automation of customs operations, including: electronic declaration of goods, electronic preliminary information, automatic registration of customs declarations, automatic release of goods, etc. When writing the work, research methods were used: expert assessments, analysis, synthesis, system analysis, statistical analysis, formallegal, comparative-legal. Based on the analysis of the statistical data of the Federal Customs Service of Russia, the opinions of scientists and experts in the field of customs, the measures that increase the efficiency of the customs authorities are proposed.

Automatic registration of optical and SAR remote sensing image based on phase feature

Automatic registration of optical and SAR remote sensing image based on phase feature

Urban Flood Mapping With Bitemporal Multispectral Imagery Via a Self-Supervised Learning Framework

Near realtime flood mapping in densely populated urban areas is critical for emergency response. The strong heterogeneity of urban areas poses a big challenge for accurate near realtime flood mapping. However, previous studies on automatic methods for urban flood mapping perform infeasible in near realtime or fail to generalize well to other floods, for several reasons. First, multitemporal pixel-wise flood mapping requires accurate image registration, hindering the efficiency of large-scale processing. Although automatic image registration has been investigated, precisely coregistered multitemporal image sequence requires time-consuming fine tuning. Additionally, the floods may lead to the loss of many corresponding image points across multitemporal images for accurate coregistration. Second, existing unsupervised methods generally rely on hand-crafted features for floodwater detection. Such features may not well represent the patterns of floodwaters in different areas due to inconsistent weather conditions, illumination, and floodwater spectra. This article proposes a self-supervised learning framework for patch-wise urban flood mapping using bitemporal multispectral satellite imagery. Patch-wise change vector analysis is used with patch features learned through a self-supervised autoencoder to produce patch-wise change maps showing potentially flood-affected areas. Postprocessing including spectral and spatial filtering is applied to these patch-wise change maps to remove nonflood related changes. Final flood maps and parameter sensitivities were evaluated using several performance metrics. Two flood events from areas with differing degrees of urbanization were considered: Hurricane Harvey flood (2017) in Houston, Texas, and Hurricane Florence flood (2018) in Lumberton, North Carolina. The proposed method shows strong performance for self-supervised urban flood mapping.

High‐resolution optical‐to‐SAR image registration using mutual information and SPSA optimisation

The high-resolution optical and synthetic aperture radar (SAR) images are widely used in many remote sensing application areas such as image fusion and change detection where image registration is a fundamental step. The latest high-resolution optical and SAR satellites and airborne systems provide geometrically corrected images which do not contain global deformations. Though the images do not have global differences, still, registration differences exist between these optical and SAR images. These registration differences should be minimised through an automatic registration method before using the images for the aforementioned applications. However, an automatic optical-to-SAR image registration is a challenging task due to the presence of significant nonlinear intensity differences as well as local geometric distortions between the images. In order to solve these problems, an automatic optical-to-SAR image registration method is proposed which can effectively handle the registration differences between the globally corrected high-resolution images. In the proposed method, initially, a coarse registration is performed by using a discrete simultaneous perturbation stochastic approximation (SPSA) optimisation. Then, a smooth continuous SPSA optimisation is utilised for the fine registration of the images. Experiments are performed on six sets of high-resolution optical-SAR image pairs and the results show the effectiveness of the proposed method.

Blockchain-Driven Trusted Data Sharing With Privacy Protection in IoT Sensor Network

The development of the Internet of Things (IoT) technology has achieved remarkable results in recent years. A large number of sensors and machinery are connected, and combined with the Internet, to achieve intelligent management and operation. However, the traditional centralized IoT data management solution inevitably encounters challenges of data trust, security, sustainability, and user privacy is difficult to guarantee. This paper takes the intelligent transportation sensor network as an example and proposes a blockchain-based Internet of Vehicles (IoV) data secure sharing scheme (called IoVChain), which implements automatic registration, rapid authentication, and reliable sharing method of IoV data through smart contract. The smart contract performs homomorphic encryption and zero-knowledge proof processing on the sensitive part of the data, and this part exists in the form of ciphertext on the blockchain. We adopt PBFT consensus mechanism to ensure the consistency of the entire network ledger, and all the IoV data processing and usage procedure is stored in the Merkle-tree-based block, which cannot be tampered with. Compared with the centralized IoV data management solution, the IoVChain scheme avoids the risk of single point of failure, and keeps the IoV data credible, available, and tamper-resistant under the premise of privacy and security, so that the data can be traced when needed. We have implemented the IoVChain scheme based on the consortium blockchain for trusted and secure sharing of IoV data. Finally, analysis and evaluation show that the proposed IoVChain scheme is feasible, safe, and extensible for the secure sharing of IoV data.

Enhancing Lateral Contrast Using Multi-perspective Ultrasound Imaging of Abdominal Aortas

Vascular ultrasound imaging is inherently hampered by low lateral resolution and contrast. Steering of the ultrasound beams can be used to overcome these limitations in superficial artery imaging because the aperture-to-depth ratio is relatively high. However, in arteries located at larger depths, the steered beams do not overlap for larger steering angles. In this study, the ultrasound probe is physically translated over the abdomen to create large angles between acquisitions, while maintaining overlap on the abdominal aorta. In one phantom setup and 11 volunteers, 2-D cross-sectional multi-perspective ultrasound images of the abdominal aorta were acquired using seven angles between –45° and +45°. Automatic registration of the recorded images was performed by automatic feature detection of the aorta and spine. This automatic detection was successful in 62 out of 77 image sets. Compounded multi-perspective images showed an increase of contrast-to-noise ratios from 0.6 ± 0.1 to 1.2 ± 0.2 over the entire heart cycle in volunteers.

OCR Captcha Alternative for Visually Impaired Users

The internet has been playing an increasingly important role in our daily life, with the availability of many web services such as email and search engines. However, these are often threatened by attacks from computer programs such as bots. To address this problem, CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart) was developed to distinguish between computer programs and human users. Although this mechanism offers good security and limits automatic registration to web services, but many CAPTCHAs are not user friendly and sometimes can pose some challenge to the users, especially visually impaired users. This paper provides an alternative OCR based captcha that can be used by all the users (including visually impaired users). It would provide better user experience, easy to solve and hard for a bot to crack.

Robot‐assisted detection of subclinical dementia: Progress report and preliminary findings

AbstractBackgroundEarlier identification of an underlying AD pathology could increase chances that preventive or curative treatment will be more successful. Human limitations in sensory capacity, attention and parallel processing could mean that automatic and simultaneous registration from several information channels in combination with artificial intelligence processing could advance diagnostic precision and accuracy. Here we report the progress of an interdisciplinary project to develop robot‐assisted diagnostics to detect early neurocognitive disorder.MethodBehavior in 100 patients without previous evidence of cognitive disorder is recorded during their first clinical examination visit. We record gaze, pupil dilations, skin temperature changes, speech characteristics, mimics, pulse, heart rate variability, and motor activity (see image 1). We then use machine learning to identify behavior signals and signal patterns that can distinguish between persons with and without an underlying neuropathology. In addition, we perform human analysis of recorded video and audio on each patient. Over 40 behavior categories are carefully checked, among them repetitions, irrelevant deviations, hesitations, degree of problem awareness, restlessness and inadequate word usage. This analysis is performed by a semi‐blinded interdisciplinary expert group of clinically experienced physicians, neurologists, speech therapists, psychologists, nurses, and experts in automatic verbal and non‐verbal behavior analysis.The results from this study will be implemented to further develop a social robot platform (Furhat, see image 2) with adequate sensors and algorithms so that it can interact with patients and assist in the diagnostic process.ResultApproximately 25 patients have so far been recorded and analyzed. Preliminary results are that the diagnostic prediction by the expert group, solely based on observation of patient behavior from audio and video, fully has coincided with the diagnostic outcome from the subsequent clinical evaluation. Another observation has been the amount of clinically relevant behavior revealed through careful human behavior observation that the interviewing physician did not observe during the actual interview.ConclusionArtificial intelligence could represent an untapped potential for improving sensitivity and accuracy in diagnostic procedures to detect early neurocognitive disorders.

Semi-automated analysis of dot blots using ImageJ/Fiji

Commercially available dot blots provide a set of specific antibodies spotted on membranes in a given pattern. If the target analyte is present in the solution that the membrane is incubated with, the detection reaction will result in a chemiluminescence signal which is recorded by film or scanner. In order to know which analytes were detected, the analysis consists of measuring the intensity of the recorded signal on each spot. Manually measuring the entire array (typically ~200 spots) is unreliable and tedious. Fully automatic registration of the blot membrane to the template pattern often fails since there might be only very few positive spots on the membrane. This article presents an ImageJ/Fiji macro that requires minimal user input to perform a robust iterative registration of an adjustable template mask representing the spot pattern to the recorded blot. Once the template mask is matched to the dot blot, the spot intensity of each dot is measured and reported in a results table.

Automatic Registration Between Dental Cone-Beam CT and Scanned Surface via Deep Pose Regression Neural Networks and Clustered Similarities.

Computerized registration between maxillofacial cone-beam computed tomography (CT) images and a scanned dental model is an essential prerequisite for surgical planning for dental implants or orthognathic surgery. We propose a novel method that performs fully automatic registration between a cone-beam CT image and an optically scanned model. To build a robust and automatic initial registration method, deep pose regression neural networks are applied in a reduced domain (i.e., two-dimensional image). Subsequently, fine registration is performed using optimal clusters. A majority voting system achieves globally optimal transformations while each cluster attempts to optimize local transformation parameters. The coherency of clusters determines their candidacy for the optimal cluster set. The outlying regions in the iso-surface are effectively removed based on the consensus among the optimal clusters. The accuracy of registration is evaluated based on the Euclidean distance of 10 landmarks on a scanned model, which have been annotated by experts in the field. The experiments show that the registration accuracy of the proposed method, measured based on the landmark distance, outperforms the best performing existing method by 33.09%. In addition to achieving high accuracy, our proposed method neither requires human interactions nor priors (e.g., iso-surface extraction). The primary significance of our study is twofold: 1) the employment of lightweight neural networks, which indicates the applicability of neural networks in extracting pose cues that can be easily obtained and 2) the introduction of an optimal cluster-based registration method that can avoid metal artifacts during the matching procedures.

Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis.

Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this article we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end-to-end trainable and self-supervised without any external annotations. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise a network to extract the shared latent features, which are then used for MRI synthesis. Since paired data is unavailable for our study (and rare in practice), pixel-level constraints are infeasible to apply. We instead propose to enforce the distributions to be statistically indistinguishable, by adversarial learning in both the image domain and feature space. To regularise the anatomical structures between US and MRI during synthesis, we further propose an adversarial structural constraint. A new cross-modal attention technique is proposed to utilise non-local spatial information, by encouraging multi-modal knowledge fusion and propagation. We extend the approach to consider the case where 3D auxiliary information (e.g., 3D neighbours and a 3D location index) from volumetric data is also available, and show that this improves image synthesis. The proposed approach is evaluated quantitatively and qualitatively with comparison to real fetal MR images and other approaches to synthesis, demonstrating its feasibility of synthesising realistic MR images.

Registration of Image Modalities for Analyses of Tissue Samples Using 3D Image Modelling.

Biopsies are a diagnostic tool for the diagnosis of histopathological, molecular biological, proteomic, and imaging data, to narrow down disease patterns or identify diseases. Matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) provides an emerging state-of-the-art technique for molecular imaging of biological tissue. The aim of this study is the registration of MALDI MSI data sets and data acquired from different histological stainings to create a 3D model of biopsies and whole organs. The registration of the image modalities is achieved by using a variant of the authors' global, deformable Schatten-q-Norm registration approach. Utilizing a connected-component segmentation for background removal followed by a principal-axis based linear pre-registration, the images are adjusted into a homogeneous alignment. This registration approach is accompanied by the 3D reconstruction of histological and MALDI MSI data. With this, a system of automatic registration for cross-process evaluation, as well as for creating 3D models, is developed and established. The registration of MALDI MSI data with different histological image data is evaluated by using the established global image registration system. In conclusion, this multimodal image approach offers the possibility of molecular analyses of tissue specimens in clinical research and diagnosis.

Real-time registration of remote sensing images with a Markov chain model

The automatic registration of different remote sensing images of the same scene is an important problem in the processing of remote sensing images. Such images usually have different resolutions and computing an accurate registration of them in real time is thus often a challenging task. In this paper, a new statistical approach is developed for the accurate and efficient registration of two remote sensing images with different resolutions. The proposed approach utilizes a statistical model to evaluate the probability for each possible mapping between the two images and computes the one with the maximum probability for registration. Similar to most of the state-of-the-art methods for remote sensing image registration, the proposed approach assumes the existence of an affine transformation between the images to be registered. The registration is performed in three stages. In the first stage, the pixels in each image are efficiently partitioned into two sets with the Otsu’s algorithm and four approximate equations for the parameters in the affine transformation are established. In the second stage, pixels in both images with significant edge geometric features are selected for potential pixel matching. In the third stage, based on the four approximate equations generated in the first stage, an Markov Chain Model-based method is used to efficiently compute the matching that can pair the selected pixels with the maximum likelihood. The parameters of the affine transformation can be determined from the pixel pairs with a least square regression approach. Experimental results on a number of pairs of remote sensing images show that this approach can generate registration results more accurate than those obtained with a few state-of-the-art approaches. In addition, real-time evaluation also shows that the approach is computationally efficient and can be used in real-time applications for remote sensing image processing.

Fault Tolerance of Memristor-Based Perceptron Network for Neural Interface

One of the most prospective applications of artificial neural networks based on memristors (ANNMs) is the development of more compact, fast, and efficient neural interfaces. This goal is achievable, as the results of a literature survey and our own research show. We have designed the ANNM as part of the neural interface for automatic registration and stimulation of bioelectrical activity of a living neuronal culture, but there are many challenges on the way to its hardware implementation, e.g., related to nonidealities of memristive devices, parasitic elements, the limited capabilities of OpAmps, and convertors. The listed above problems can affect the ANNM quality attributes, such as the operation accuracy, fault tolerance, and reliability, and have fundamental importance to practical operation. In this paper, we focus on the investigation of the ANNM fault tolerance and try to find single points of failure. The obtained information can be used in the ANNM reliability engineering.