Accurate Correspondence Research Articles

Nonisometric Surface Registration via Conformal Laplace–Beltrami Basis Pursuit

Surface registration is one of the most fundamental problems in geometry processing. Many approaches have been developed to tackle this problem in cases where the surfaces are nearly isometric. However, it is much more challenging to compute correspondence between surfaces which are intrinsically less similar. In this paper, we propose a variational model to align the Laplace-Beltrami (LB) eigensytems of two non-isometric genus zero shapes via conformal deformations. This method enables us to compute geometrically meaningful point-to-point maps between non-isometric shapes. Our model is based on a novel basis pursuit scheme whereby we simultaneously compute a conformal deformation of a ’target shape’ and its deformed LB eigensystem. We solve the model using a proximal alternating minimization algorithm hybridized with the augmented Lagrangian method which produces accurate correspondences given only a few landmark points. We also propose a re-initialization scheme to overcome some of the difficulties caused by the non-convexity of the variational problem. Intensive numerical experiments illustrate the effectiveness and robustness of the proposed method to handle non-isometric surfaces with large deformation with respect to both noises on the underlying manifolds and errors within the given landmarks or feature functions.

Learning to estimate smooth and accurate semantic correspondence

We tackle the problem of estimating dense semantic correspondence between two images depicting different instances of the same category. In this paper, we consider semantic context and correspondence information from the neighborhood in order to overcome the drawback of previous works that estimate the correspondence of each pixel or patch independently. To this end, a novel network, called SANet, with a trainable spatial aggregation module is proposed, which is trained in an end-to-end manner and outputs semantic flow. We train this SANet by adopting two complementary loss terms: landmark loss, focusing on keypoints with ground truth, and consistency loss, applicable to all pixels without ground truth. Qualitative and quantitative experimental results demonstrate the improved network achieves a better balance between accuracy and smoothness comparing with the baseline and warps images with better visual quality.

VSAM-Based Visual Keyword Generation for Image Caption

Image caption is to understand and describe the visual content, which is expected to be applied in automatic news reporting in future. In recent years, there has been an increasing interest in an Encoder-Decoder framework for image caption: the encoder takes the responsibility for visual semantic comprehension and the decoder is designed for sentence generation. In the Encoder-Decoder framework the translation is based on the correspondence between image feature vectors and caption vectors. Attention mechanism makes sense for a more accurate correspondence. However, the attention model works with the decoder, and the focused content changes dynamically with the generated word. It results that in many cases the salient contents are not described in the caption, or the objects described are not the salient ones. To improve the precision of image caption, to bridge the gap between image understanding and sentence generation in the Encoder-Decoder framework, and to align visual information and semantic information better, we propose a concept of visual keyword as a gang board between seeing and saying. This paper presents an image dataset derived from MSCOCO as the first collection of visual keywords: Image Visual Keyword Dataset (IVKD). Also, a Visual Semantic Attention Model(VSAM) is proposed to obtain visual keywords for generating the annotation. In VSAM, the object-level visual features are extracted by an object detector after pre-training on IVKD. Then the object features are fed in an Optimized Pointer Network(OPN) to generate visual keywords. The experiments show that the precision of visual keyword generation reaches 91.7% by the proposed model VSAM.

Learned Local Features for Structure From Motion of UAV Images: A Comparative Evaluation

Unmanned aerial vehicle (UAV) images have become the main remote sensing data sources for varying applications, and structure from motion (SfM) is the golden standard for resuming camera poses. Matching local feature descriptors is the prerequisite for the accurate and complete orientation of UAV images. Recently, some newly proposed learned methods have been shown to outperform the hand-crafted methods, such as the scale invariant feature transform (SIFT) and its variants, and almost all learned methods have been trained and evaluated by using images from the internet with varying focal lengths and varying size. It is of interest to investigate the performance of these learned methods with their pretrained models for feature detection and description in the context of the SfM-based orientation. Thus, this article conducts a comprehensive evaluation of both advanced hand-crafted and newly proposed learned detectors and descriptors by using four UAV datasets. The performance of these selected methods is compared in the context of feature matching and the SfM and (multiview stereo) MVS-based reconstruction. Experimental results demonstrate that the learned descriptors combined with the SIFT-like detectors can provide accurate and complete feature correspondences and achieve better or competitive performance in the SfM and MVS-based reconstruction. For UAV image orientation, the learned descriptors can be an alternative to the existing hand-crafted descriptors without their model retraining. The source codes of this evaluation would be made publicly available.

Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos.

This paper focuses on tackling the problem of temporal language localization in videos, which aims to identify the start and end points of a moment described by a natural language sentence in an untrimmed video. However, it is non-trivial since it requires not only the comprehensive understanding of the video and sentence query, but also the accurate semantic correspondence capture between them. Existing efforts are mainly centered on exploring the sequential relation among video clips and query words to reason the video and sentence query, neglecting the other intra-modal relations (e.g., semantic similarity among video clips and syntactic dependency among the query words). Towards this end, in this work, we propose a Multi-modal Interaction Graph Convolutional Network (MIGCN), which jointly explores the complex intra-modal relations and inter-modal interactions residing in the video and sentence query to facilitate the understanding and semantic correspondence capture of the video and sentence query. In addition, we devise an adaptive context-aware localization method, where the context information is taken into the candidate moments and the multi-scale fully connected layers are designed to rank and adjust the boundary of the generated coarse candidate moments with different lengths. Extensive experiments on Charades-STA and ActivityNet datasets demonstrate the promising performance and superior efficiency of our model.

Registration of Consecutive Frames From Wireless Capsule Endoscopy for 3D Motion Estimation

Wireless Capsule Endoscopy (WCE) is a non-invasive medical procedure devised for painless in vivo inspection of the gastrointestinal (GI) tract. It is especially valuable for the examination of the small intestine since it is difficult to reach by traditional endoscopic procedures. The setup includes a camera with an embedded light source and a circuit capable of acquiring and transmitting the video. The main challenge of this technology is the identification of the position and trajectory of the capsule as it travels through the GI tract, which is particularly relevant during the detection of anomalies in the tissue. Given only the information provided by the recorded images, it is possible to estimate the 3D motion of the camera capsule and provide a full trajectory reconstruction. A critical yet difficult step in this process is the image registration between sequential frames. Therefore, being able to determine accurate correspondences between points, regions or features in two consecutive frames is crucial for the computation of the relative rotation and translation of the capsule. This paper comprises a comparative assessment of methodologies to address this problem with a porcine colon dataset obtained with our experimental setup.

On-line Dense 3D Reconstruction Method based on ANN us-ing a Single Structured Light Pattern

Dense 3D reconstruction techniques to measuring dynamic scenes and deformable objects with little texture have been widely studied for various applications like surface deformation measurement. We assume that the actual situation of scanning is acquiring sequential shape and measuring surface deformation online at the moment when an object deforms under the action of aerodynamic forces. This paper presents such a technique for on-line acquisition of 3D surface data based on a one-shot scanning method that reconstructs 3D shape from a single image where a simple color-coded pattern using de Bruijn sequence is projected onto an object. The proposed approach has advantages that it requires no assumption of global smoothness and continuous surface. To realize 3D reconstruction from a single image, there are sever-al issues to be solved, for example, difficulty on decoding structured light pattern because of influence of chromatic aberration in both projection systems and imaging systems, and difficul-ty on establishing accurate correspondences between projector and camera pixels. This paper describes the solutions of the issues by combining two methods, that is (1) an efficient and robust pattern decoding method based on artificial neural networks (ANN), and (2) a sub-pixel matching method using phase map and multiple view geometry. Furthermore, we adopt some effective strategies to improve algorithm robustness. Practical experiments were carried out to test the accuracy and efficiency of the scanning system in typical configuration. The results show that the scanning system can reconstruct 3D shape on-line in high-resolution with the accuracy of 0.09 mm and efficiency of 10 fps.

MapTree

In this paper we propose an approach for computing multiple high-quality near-isometric dense correspondences between a pair of 3D shapes. Our method is fully automatic and does not rely on user-provided landmarks or descriptors. This allows us to analyze the full space of maps and extract multiple diverse and accurate solutions, rather than optimizing for a single optimal correspondence as done in most previous approaches. To achieve this, we propose a compact tree structure based on the spectral map representation for encoding and enumerating possible rough initializations, and a novel efficient approach for refining them to dense pointwise maps. This leads to a new method capable of both producing multiple high-quality correspondences across shapes and revealing the symmetry structure of a shape without a priori information. In addition, we demonstrate through extensive experiments that our method is robust and results in more accurate correspondences than state-of-the-art for shape matching and symmetry detection.

The Haskins pediatric atlas: a magnetic-resonance-imaging-based pediatric template and atlas.

Spatial normalization plays an essential role in multi-subject MRI and functional MRI (fMRI) experiments by facilitating a common space in which group analyses are performed. Although many prominent adult templates are available, their use for pediatric data is problematic. Generalized templates for pediatric populations are limited or constructed using older methods that result in less ideal normalization. The Haskins pediatric templates and atlases aim to provide superior registration and more precise accuracy in labeling of anatomical and functional regions essential for all fMRI studies involving pediatric populations. The Haskins pediatric templates and atlases were generated with nonlinear methods using structural MRI from 72 children (age range 7-14years, median 10years), allowing for a detailed template with corresponding parcellations of labeled atlas regions. The accuracy of these templates and atlases was assessed using multiple metrics of deformation distance and overlap. When comparing the deformation distances from normalizing pediatric data between this template and both the adult templates and other pediatric templates, we found significantly less deformation distance for the Haskins pediatric template (P<0.0001). Further, the correct atlas classification was higher using the Haskins pediatric template in 74% of regions (P<0.0001). The Haskins pediatric template results in more accurate correspondence across subjects because of lower deformation distances. This correspondence also provides better accuracy in atlas locations to benefit structural and functional imaging analyses of pediatric populations.

End-to-end multimodal image registration via reinforcement learning.

Multimodal image registration is a vital initial step in several medical image applications for providing complementary information from different data modalities. Since images with different modalities do not exhibit the same characteristics, finding their accurate correspondences remains a challenge. For convolutional multimodal registration methods, two components are quite significant: descriptive image feature as well as the suited similarity metric. However, these two components are often custom-designed and are infeasible to the high diversity of tissue appearance across modalities. In this paper, we translate image registration into a decision-making problem, where registration is achieved via an artificial agent trained by asynchronous reinforcement learning. More specifically, convolutional long-short-term-memory is incorporated after stacked convolutional layers in this method to extract spatial-temporal image features and learn the similarity metric implicitly. A customized reward function driven by landmark error is advocated to guide the agent to the correct registration direction. A Monte Carlo rollout strategy is also leveraged to perform as a look-ahead inference in the testing stage, to increase registration accuracy further. Experiments on paired CT and MR images of patients diagnosed as nasopharyngeal carcinoma demonstrate that our method achieves state-of-the-art performance in medical image registration.

Comment on "Crystallization of Supercooled Liquids: Self-Consistency Correction of the Steady-State Nucleation Rate" by Abyzov et al., Entropy 2020, 22, 558.

It is shown that in the growth region (above the critical nucleation size) the transient distributions obtained numerically from the Becker-Döring equation (BDE) by Abyzov et al., Entropy 2020, 22, 558, are in accurate correspondence with the matched asymptotic (singular perturbation) solution by Shneidman, Sov. Phys. Tech. Phys. 1988, 33, 1338. The solution is unmodified by “self-consistency” corrections which affect only the steady state rate. Sensitivity of the results to selection of a specific form of the BDE (the “nucleation model”) also is briefly discussed.

Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.

Optimization of Adaptive Lighting Technology Based on Varying-Frequency Sinusoidal Grating Projection

In order to enhance the image quality of multi-reflective 3D scenes during machine vision analysis, an active self-adaptive illumination system to solve the overexposure and underexposure problems is introduced. Based on projector-camera system, adaptive illumination was achieved by modulating the projection brightness at each pixel to be proportional to the pixel brightness of the target scenes, which should firstly establish the accurate pixel correspondence between the projector and camera. In this paper, the performance of adaptive illumination is improved by optimizing the correspondence stage compared with previous research. Different grating patterns with different frequency components are projected and compared to acquire the most effective correspondence, which improves the accuracy of corresponding depth map and the experiment results can achieve higher lighting accuracy reducing the number of iterations as well as the computing time. The image quality indexes, image contrast and number of saturated or dark pixels, are combined to analyze and judge the optimized lighting result. The results demonstrate that the developed technology in this research can achieve a rate of 20 frames per second by using a standard commercial CCD (charge coupled device) camera, where it can run in real time and becomes a viable solution for the industry.

Deep learned compact binary descriptor with a lightweight network-in-network architecture for visual description

Binary descriptors have been widely used for real-time image retrieval and correspondence matching. However, most of the learned descriptors are obtained using a large deep neural network (DNN) with several million parameters, and the learned binary codes are generally not invariant to many geometrical variances which is crucial for accurate correspondence matching. To address this problem, we proposed a new learning approach using a lightweight DNN architecture via a stack of multiple multilayer perceptrons based on the network in network (NIN) architecture, and a restricted Boltzmann machine (RBM). The latter is used for mapping the features to binary codes, and carry out the geometrically invariant correspondence matching task. Our experimental results on several benchmark datasets (e.g., Brown, Oxford, Paris, INRIA Holidays, RomePatches, HPatches, and CIFAR-10) show that the proposed approach produces the learned binary descriptor that outperforms other baseline self-supervised binary descriptors in terms of correspondence matching despite the smaller size of its DNN. Most importantly, the proposed approach does not freeze the features that are obtained while pre-training the NIN model. Instead, it fine-tunes the features while learning the features needed for binary mapping through the RBM. Additionally, its lightweight architecture makes it suitable for resource-constrained devices.

Geometric calibration for LiDAR-camera system fusing 3D-2D and 3D-3D point correspondences.

Calibrating the extrinsic parameters on a system of 3D Light Detection And Ranging (LiDAR) and the monocular camera is a challenging task, because accurate 3D-2D or 3D-3D point correspondences are hard to establish from the sparse LiDAR point clouds in the calibration procedure. In this paper, we propose a geometric calibration method for estimating the extrinsic parameters of the LiDAR-camera system. In this method, a novel combination of planar boards with chessboard patterns and auxiliary calibration objects are proposed. The planar chessboard provides 3D-2D and 3D-3D point correspondences. Auxiliary calibration objects provide extra constraints for stable calibration results. After that, a novel geometric optimization framework is proposed to utilize these point correspondences, thus leading calibration results robust to LiDAR sensor noise. Besides, we contribute an automatic approach to extract point clouds of calibration objects. In the experiments, our method has a superior performance over state-of-the-art calibration methods. Furthermore, we verify our method by computing depth map and improvements can also be found. These results demonstrate that our method performance on the LiDAR-camera system is applicable for future advanced visual applications.

STFlow: Self-Taught Optical Flow Estimation Using Pseudo Labels.

The Deep learning of optical flow has been an active area for its empirical success. For the difficulty of obtaining accurate dense correspondence labels, unsupervised learning of optical flow has drawn more and more attention, while the accuracy is still far from satisfaction. By holding the philosophy that better estimation models can be trained with betterapproximated labels, which in turn can be obtained from better estimation models, we propose a self-taught learning framework to continually improve the accuracy using self-generated pseudo labels. The estimated optical flow is first filtered by bidirectional flow consistency validation and occlusion-aware dense labels are then generated by edge-aware interpolation from selected sparse matches. Moreover, by combining reconstruction loss with regression loss on the generated pseudo labels, the performance is further improved. The experimental results demonstrate that our models achieve state-of-the-art results among unsupervised methods on the public KITTI, MPI-Sintel and Flying Chairs datasets.

Automatic Registration of Optical and SAR Images Via Improved Phase Congruency Model

In this article, we propose an automatic and efficient method to solve optical and synthetic aperture radar (SAR) image registration using the improved phase congruency (PC) model. First, evenly distributed keypoints are extracted from the optical images via the block-Harris method. Complementary grid points are then selected in image regions with poor structural information and supplemented to the keypoint set. For each keypoint, a robust feature representation that captures the local spatial relationship is proposed based on the improved PC model. Specifically, we propose to use two different PC models, the classic PC and the SAR-PC, to construct features for optical and SAR images, respectively. The PC features of several directions are aggregated to construct the feature descriptors, and a similarity metric via the phase correlation of feature descriptors is obtained. The proposed similarity metric cannot only find accurate correspondence but also present efficient results without presetting the size of the search region. We compare the proposed method with two baselines and state-of-the-art (SOTA) methods, i.e., OS-SIFT, histogram of oriented PC, and channel features of oriented gradients, in various scenarios. The results show that the proposed method outperforms the baselines and shows comparable performance with SOTA methods in regions with abundant structural information and better performance in regions with less structural information. Moreover, we build a high-resolution optical and SAR image matching dataset, which consists of 10 692 nonoverlapping patch pairs of $256\times 256$ pixels and 1-m resolution. Results of two benchmarks, Siamese deep matching network, and conditional generative adversarial networks show that this dataset is practical and challenging.

Unpaired Low-Dose CT Denoising Network Based on Cycle-Consistent Generative Adversarial Network with Prior Image Information.

The widespread application of X-ray computed tomography (CT) in clinical diagnosis has led to increasing public concern regarding excessive radiation dose administered to patients. However, reducing the radiation dose will inevitably cause server noise and affect radiologists' judgment and confidence. Hence, progressive low-dose CT (LDCT) image reconstruction methods must be developed to improve image quality. Over the past two years, deep learning-based approaches have shown impressive performance in noise reduction for LDCT images. Most existing deep learning-based approaches usually require the paired training dataset which the LDCT images correspond to the normal-dose CT (NDCT) images one-to-one, but the acquisition of well-paired datasets requires multiple scans, resulting the increase of radiation dose. Therefore, well-paired datasets are not readily available. To resolve this problem, this paper proposes an unpaired LDCT image denoising network based on cycle generative adversarial networks (CycleGAN) with prior image information which does not require a one-to-one training dataset. In this method, cyclic loss, an important trick in unpaired image-to-image translation, promises to map the distribution from LDCT to NDCT by using unpaired training data. Furthermore, to guarantee the accurate correspondence of the image content between the output and NDCT, the prior information obtained from the result preprocessed using the LDCT image is integrated into the network to supervise the generation of content. Given the map of distribution through the cyclic loss and the supervision of content through the prior image loss, our proposed method can not only reduce the image noise but also retain critical information. Real-data experiments were carried out to test the performance of the proposed method. The peak signal-to-noise ratio (PSNR) improves by more than 3 dB, and the structural similarity (SSIM) increases when compared with the original CycleGAN without prior information. The real LDCT data experiment demonstrates the superiority of the proposed method according to both visual inspection and quantitative evaluation.

Numerical Study of Square Border Ultrasonic Transducer Element Arrays for Breast Imaging in Ultrasound Computed Tomography with Waveform Inversion Method

Ultrasound computed tomography (USCT) is a promising approach for early breast cancer screening. Existing studies that estimate breast images using Waveform inversion (WI) methods utilize a Circular ultrasonic transducer element array (CUTA) to collect the measurement data. However, very accurate transducer element positioning and directivity are required for signal calibration in the hardware system of these studies, thereby causing difficulties in hardware manufacturing. The purpose of this study is to estimate high-resolution USCT images using a WI method with a Square border ultrasonic transducer element array (SUTA), which reduces the difficulty of hardware manufacturing and creates accurate correspondence between the continuous and discrete forms of the transducer element positions. Therefore, in this study, an SUTA is adopted to collect the measurement data of breast imaging using the WI method. A penalized least-squares optimization problem is constructed to obtain the numerical solution of a sound speed distribution. Computer simulations are conducted to compare images reconstructed from the measurement data that are collected using an SUTA and a CUTA. The performance is evaluated using a numerical breast phantom. Results suggest that the biases of the images reconstructed are less than 1% with the evenly distributed SUTA and 1% with the CUTA, under a noise condition.

Прогнозная оценка эффективности технологии радиального бурения для башкирских эксплуатационных объектов месторождений Пермского края

The radial drilling technology efficiency for carbonate bashkir deposits of Perm Krai is considered. The geological structure of a productive part of bashkir layer is characterized by high degree of heterogeneity that promotes while drilling radial channels involvement in development additional interlayers that earlier was not drained.During the analysis the main geological process parameters affecting drilling technology efficiency were revealed. According to the dynamics of average daily oil production growth, palettes were built to forecast additional oil production as a result of radial drilling activities. Using the pallets, it is possible to predict the total additional oil production, well operating time with the effect of radial drilling and average daily oil production growth for each year.It was found that hydrochloric acid treatments performed on wells prior to radial drilling significantly reduce the effectiveness of radial drilling technology. For such wells, the value of the correction is statistically substantiated, which reduces the predictive estimate of the increase in oil production.A model was built to assess the increase in oil production in the first year after the event and an algorithm for calculating the total additional oil production was developed using linear discriminant analysis. For the resulting model, errors are calculated that are compared with the forecast efficiency of standard methods for oil-producing enterprises. This model shows a much more accurate correspondence of forecast results to actual technology application results.The probability of the event high efficiency increases significantly with a more detailed approach to the selection of wells for radial drilling. According to the forecast methodology, the technology’s efficiency was calculated and recommendations for its implementation for the wells of the Bashkir production objects were made in the interests of an oil-producing enterprise.