Image Alignment Problem Research Articles

Epithelial Cell Adhesion Molecule-Functionalized Fe3O4@Au Nanoparticles for Coregistered Optoacoustic and Magnetic Resonance Imaging and Photothermal Therapy of Hepatocellular Carcinoma

Early diagnosis and management of hepatocellular carcinoma (HCC) is important for improving the 5-year survival rate. Multimodal imaging is of great significance for obtaining complementary information, improving the efficacy of tumor detection, and monitoring HCC recurrence. The rational development of targeted nanoprobes with excellent performance is of great significance for the accurate diagnosis of HCC and image-guided photothermal therapy (PTT). In the present study, we synthesized a hybrid nanoprobe containing epithelial cell adhesion molecule-functionalized Fe3O4@Au nanoparticles as an HCC-targeted nanoprobe, which served as a dual-modal contrast agent for optoacoustic tomography (OAT) and magnetic resonance imaging (MRI) and as a photothermal sensitizer for PTT. Using this nanoprobe, we evaluated the application of coregistered OAT–MRI for the diagnosis and therapy of HCC to resolve an image alignment problem. Body-position variation in OAT and MRI was minimized using a specially designed 3D-printed dual-modality animal imaging bed constructed from polylactic acid. The registration accuracy of OAT–MRI data was further improved by a robust image registration protocol. Dual-modality coregistered imaging was performed for the early diagnosis of HCC and monitoring of curative effects. To our knowledge, this is the first study to apply a combination of a targeted nanoprobe and coregistration of OAT–MRI for the diagnosis and management of abdominal diseases.

Robust Underwater Localization Using Acoustic Image Alignment for Autonomous Intervention Systems

Acoustic imaging sonar can be used as important navigation sensors for underwater intervention systems as they consistently provide spatial information about the surrounding environment, even in limited visibility conditions. However, acoustic imaging sonars are known for high spatial ambiguity and low resolution of their measurements, which makes it challenging to obtain precise navigational information. This paper presents a novel localization method based on sensor fusion using an integrated IMU-DVL system and an acoustic imaging sonar. A sonar simulator is implemented and used to estimate the pose of the robot based on a single acoustic image, and it is formulated as an image alignment problem between a simulated acoustic image and an actual acoustic image. For this, an approximate nearest neighbor search method is employed for initial pose estimation, and a newly developed acoustic image alignment method is applied to obtain more accurate results in a continuous pose space. These methods are then combined to construct an integrated localization system using all the navigation sensors on the robot. Its feasibility and utility of the proposed approach is shown through an experimental validation in a test tank.

Robust Low Transformed Multi-Rank Tensor Methods for Image Alignment

Aligning a group of linearly correlated images is an important task in computer vision. In this paper, we propose a combination of transformed tensor nuclear norm and tensor $$\ell _1$$ norm to deal with this image alignment problem, where the observed images, stacked into a third-order tensor, are deformed by unknown domain transformations and corrupted by sparse noise like impulse noise, partial occlusions, and illumination variation. The key advantage of the proposed method is that both spatial correlation and images variation can be captured by the use of transformed tensor nuclear norm. We show that when the underlying of correlated images is a low multi-rank tensor, an upper error bound of the estimator of the proposed model can be established and this bound can be better than the previous result. Besides the proposed convex transformed tensor model, the method can be further studied by incorporating nonconvex functions in the transformed tensor nuclear norm and the sparsity norm. Both the proposed convex and nonconvex optimization models are solved by generalized Gauss–Newton algorithms. Also the global convergence of the numerical methods for solving the subproblems of convex and nonconvex optimization models can be provided under very mild conditions. Extensive numerical experiments on real images with misalignment and sparse corruptions demonstrate the performance of our proposed methods is better than that of several state-of-the-art methods in terms of accuracy and efficiency.

Robust Alignment for Panoramic Stitching Via an Exact Rank Constraint.

We study the problem of image alignment for panoramic stitching. Unlike most existing approaches that are feature-based, our algorithm works on pixels directly, and accounts for errors across the whole images globally. Technically, we formulate the alignment problem as rank-1 and sparse matrix decomposition over transformed images, and develop an efficient algorithm for solving this challenging non-convex optimization problem. The algorithm reduces to solving a sequence of subproblems, where we analytically establish exact recovery conditions, convergence and optimality, together with convergence rate and complexity. We generalize it to simultaneously align multiple images and recover multiple homographies, extending its application scope toward vast majority of practical scenarios. The experimental results demonstrate that the proposed algorithm is capable of more accurately aligning the images and generating higher quality stitched images than the state-of-the-art methods.

Batch image alignment via subspace recovery based on alternative sparsity pursuit

The problem of robust alignment of batches of images can be formulated as a low-rank matrix optimization problem, relying on the similarity of well-aligned images. Going further, observing that the images to be aligned are sampled from a union of low-rank subspaces, we propose a new method based on subspace recovery techniques to provide more robust and accurate alignment. The proposed method seeks a set of domain transformations which are applied to the unaligned images so that the resulting images are made as similar as possible. The resulting optimization problem can be linearized as a series of convex optimization problems which can be solved by alternative sparsity pursuit techniques. Compared to existing methods like robust alignment by sparse and low-rank models, the proposed method can more effectively solve the batch image alignment problem, and extract more similar structures from the misaligned images.

Navigation: Whence Our Sense of Direction?

Behavioral data have long implied our sense of direction derives from global environmental shape; electrophysiological evidence, however, has seemed to imply it derives from salient non-geometric landmarks. Experiments on the re-establishment of place fields in disoriented mice now align the electrophysiological data with the behavioral data.

Non-expansible XOR-based visual cryptography scheme with meaningful shares

XOR-based visual cryptography (VC), a brand-new type of VC, is suitable for solving the low image quality and alignment problems in VC system. However, meaningless share and pixel expansion remain to be continuing challenges in existing XOR-based VC. To fix those mentioned defects, XOR-based VC with meaningful shares is introduced in this paper. A basic algorithm implemented by a simple 2n×n matrix for constructing a (n,n) XOR-based VC is first proposed. In the following stage, a (n,n) XOR-based VC with meaningful shares is investigated by adopting the basic algorithm, where the meaningful share can be directly generated without extra process. Extensive experimental results are demonstrated, exhibiting the effectiveness and advantages of the proposed method. Further, sufficient theoretical proofs are provided for illustrating the correctness of the proposed XOR-based VC.

Rationalizing Efficient Compositional Image Alignment

We study the issue of computational efficiency for Gauss-Newton (GN) non-linear least-squares optimization in the context of image alignment. We introduce the Constant Jacobian Gauss-Newton (CJGN) optimization, a GN scheme with constant Jacobian and Hessian matrices, and the equivalence and independence conditions as the necessary requirements that any function of residuals must satisfy to be optimized with this efficient approach. We prove that the Inverse Compositional (IC) image alignment algorithm is an instance of a CJGN scheme and formally derive the compositional and extended brightness constancy assumptions as the necessary requirements that must be satisfied by any image alignment problem so it can be solved with an efficient compositional scheme. Moreover, in contradiction with previous results, we also prove that the forward and inverse compositional algorithms are not equivalent. They are equivalent, however, when the extended brightness constancy assumption is satisfied. To analyze the impact of the satisfaction of these requirements we introduce a new image alignment evaluation framework and the concepts of short- and wide-baseline Jacobian. In wide-baseline Jacobian problems the optimization will diverge if the requirements are not satisfied. However, with a good initialization, a short-baseline Jacobian problem may converge even if the requirements are not satisfied.

An interactive method for the image alignment problem based on partially supervised correspondence

In a robotic scenario in which robots are localised through the images they obtain, it is usual that the current image registration techniques fail due to large differences between images, occlusions, shadows and so on. In these cases a human can interact on the computer vision system to help it to find a preliminary mapping between some salient points. We present a method to solve the alignment problem between two images in which a human (or oracle) imposes a partial correspondence between salient points extracted from the images through a Human–Machine Interface and the Interactive Correspondence Method computes a new image alignment and a set of point correspondences. Practical evaluation shows that in two human interactions our method finds an almost optimal alignment and correspondence set between images that any current completely automatic method is able to obtain. Moreover, we demonstrate that in situations that there is a large difference between images, adding interactivity is the only method that achieves an acceptable result.

PRIME: Probabilistic Initial 3D Model Generation for Single-Particle Cryo-Electron Microscopy

Low-dose electron microscopy of cryo-preserved individual biomolecules (single-particle cryo-EM) is a powerful tool for obtaining information about the structure and dynamics of large macromolecular assemblies. Acquiring images with low dose reduces radiation damage, preserves atomic structural details, but results in low signal-to-noise ratio of the individual images. The projection directions of the two-dimensional images are random and unknown. The grand challenge is to achieve the precise three-dimensional (3D) alignment of many (tens of thousands to millions) noisy projection images, which may then be combined to obtain a faithful 3D map. An accurate initial 3D model is critical for obtaining the precise 3D alignment required for high-resolution (<10Å) map reconstruction. We report a method (PRIME) that, in a single step and without prior structural knowledge, can generate an accurate initial 3D map directly from the noisy images.

Consistent image alignment for video mosaicing

Image mosaicing is widely used in computer vision applications. In this paper, a new globally consistent image alignment method for video mosaicing is presented. Due to various uncertainties on noise, illumination, and modeling, the problem of global image alignment is considered as a stochastic estimation problem. The augmented system state consists of the transformation parameters of video image sequences. The system motion model, the augmentation model, and the observation model are constructed. The homography parameters of image sequences are augmented and estimated recursively with augmented Kalman filter in a common state vector and covariance matrix. Some experimental results are provided to validate the performance of the proposed method. The proposed image alignment method can handle the uncertainty efficiently. Image alignment is globally consistent and accurate.

Optimal Image Alignment With Random Projections of Manifolds: Algorithm and Geometric Analysis

This paper addresses the problem of image alignment based on random measurements. Image alignment consists of estimating the relative transformation between a query image and a reference image. We consider the specific problem where the query image is provided in compressed form in terms of linear measurements captured by a vision sensor. We cast the alignment problem as a manifold distance minimization problem in the linear subspace defined by the measurements. The transformation manifold that represents synthesis of shift, rotation, and isotropic scaling of the reference image can be given in closed form when the reference pattern is sparsely represented over a parametric dictionary. We show that the objective function can then be decomposed as the difference of two convex functions (DC) in the particular case where the dictionary is built on Gaussian functions. Thus, the optimization problem becomes a DC program, which in turn can be solved globally by a cutting plane method. The quality of the solution is typically affected by the number of random measurements and the condition number of the manifold that describes the transformations of the reference image. We show that the curvature, which is closely related to the condition number, remains bounded in our image alignment problem, which means that the relative transformation between two images can be determined optimally in a reduced subspace.

Discriminative Face Alignment

This paper proposes a discriminative framework for efficiently aligning images. Although conventional Active Appearance Models (AAMs)-based approaches have achieved some success, they suffer from the generalization problem, i.e., how to align any image with a generic model. We treat the iterative image alignment problem as a process of maximizing the score of a trained two-class classifier that is able to distinguish correct alignment (positive class) from incorrect alignment (negative class). During the modeling stage, given a set of images with ground truth landmarks, we train a conventional Point Distribution Model (PDM) and a boosting-based classifier, which acts as an appearance model. When tested on an image with the initial landmark locations, the proposed algorithm iteratively updates the shape parameters of the PDM via the gradient ascent method such that the classification score of the warped image is maximized. We use the term Boosted Appearance Models (BAMs) to refer to the learned shape and appearance models, as well as our specific alignment method. The proposed framework is applied to the face alignment problem. Using extensive experimentation, we show that, compared to the AAM-based approach, this framework greatly improves the robustness, accuracy, and efficiency of face alignment by a large margin, especially for unseen data.

Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization

In this work we propose the use of a modified version of the correlation coefficient as a performance criterion for the image alignment problem. The proposed modification has the desirable characteristic of being invariant with respect to photometric distortions. Since the resulting similarity measure is a nonlinear function of the warp parameters, we develop two iterative schemes for its maximization, one based on the forward additive approach and the second on the inverse compositional method. As it is customary in iterative optimization, in each iteration, the nonlinear objective function is approximated by an alternative expression for which the corresponding optimization is simple. In our case we propose an efficient approximation that leads to a closed-form solution (per iteration) which is of low computational complexity, the latter property being particularly strong in our inverse version. The proposed schemes are tested against the Forward Additive Lucas-Kanade and the Simultaneous Inverse Compositional (SIC) algorithm through simulations. Under noisy conditions and photometric distortions, our forward version achieves more accurate alignments and exhibits faster convergence whereas our inverse version has similar performance as the SIC algorithm but at a lower computational complexity.

Protein image alignment via tensor product cubic splines

The class of tensor product cubic splines is considered in an optimization process for two-dimensional polyacrylamide gel electrophoresis (2D-PAGE) image alignment problems arising in proteomics studies. Numerical results are presented along with comparisons with previous results based on piecewise affine and bilinear transformations.

A population global optimization algorithm to solve the image alignment problem in electron crystallography

Knowledge of the structure of biological specimens is critical to understanding their function. Electron crystallography is an electron microscopy (EM) approach that derives the 3D structure of specimens at high-resolution, even at atomic detail. Prior to the tomographic reconstruction, the images taken from the microscope have to be properly aligned. Traditional alignment methods in electron crystallography are based on a phase residual function to be minimized by inefficient exhaustive search procedures. This work addresses this minimization problem from an evolutionary perspective. Universal Evolutionary Global Optimizer (UEGO), an evolutionary multimodal optimization algorithm, has been applied and evaluated for the task of image alignment in this field. UEGO has turned out to be a promising technique alternative to the standard methodology. The alignments found out by UEGO show high levels of accuracy, while reducing the number of function evaluations by a significant factor with respect to the standard method.