Large Parallax Research Articles

Seam estimation based on dense matching for parallax-tolerant image stitching

Image stitching with large parallax poses a significant challenge in the field of computer vision. Existing seam-based approaches attempt to address parallax artifacts by stitching images along seams. However, issues such as object mismatches, disappearances, and duplications still arise occasionally, primarily due to inaccurate alignment of dense pixels or inappropriate seam estimation methods. In this paper, we propose a robust seam-based parallax-tolerant image stitching method that leverages dense flow estimation from state-of-the-art approaches. Firstly, we develop a seam estimation method that does not require pre-estimation of image warping model. Instead, it directly estimates the seam by measuring the local smoothness of the optical flow field and incorporating a penalty term for duplications. Subsequently, we design an iterative algorithm that utilizes the location of estimated seam to solve a spatial smooth warping model and eliminate outlier corresponding pairs. By employing this approach, we effectively address the intertwined challenges of estimating the warping model and seam. Experiment on real-world images shows that our proposed method achieves superior local alignment accuracy near the stitching seam and outperforms other state-of-the-art techniques on visual stitching result.

Accurate image alignment based on multi-warp optimization for large parallax

Image alignment with parallax is a challenging computer vision problem. While existing methods employing local-varying smooth warps have enhanced alignment accuracy for complex spacial deformations compared to global homography estimation, they fall short in adequately representing the discontinuous transformations evident in images with large parallax. In this paper, we propose an image alignment method grounded in the estimation and fusion of multiple warping models. To effectively tackle the challenges posed by discontinuous deformations in different regions, our method comprises two key stages: principal region alignment and fine region refinement. In both stages, multiple warping models are initially estimated using feature correspondences and are then concurrently optimized by minimizing pixel-level photometric loss. For each pixel, we select the optimal model that minimizes warping error. Additionally, we introduce a feature grouping method based on Delaunay triangulation. Experiments on real-world images demonstrate the superior alignment accuracy achieved by our proposed method compared to other state-of-the-arts.

CVGSR: Stereo image Super-Resolution with Cross-View guidance

The ability to capture the complementary information of stereo images is critical to the development of stereo image super-resolution. Most existing studies have attempted to integrate reliable stereo correspondence along the polar directions through parallax attention and various fusion strategies. However, most of these approaches ignore the large parallax differences in stereo images, resulting in poor performance of convolutional-based parallax attention in capturing the long-range dependencies between images. In this paper, we propose a novel cross-view guided stereo image super-resolution network (CVGSR) for reconstructing high-resolution stereo image pairs with rich texture details by fully exploiting the complementary nature of stereo image pairs. Specifically, we first deploy a cross-view interaction module (CVIM) to explore intra/cross-view dependencies from local to global to compensate for the incomplete compatibility of information between the left and right views. This module uses a progressive cross-guiding strategy to better merge features from occluded and non-occluded regions. Based on this, an efficient attention Transformer (EAT) is improved to activate more input information and further mine the cross-view complementarity. Furthermore, we design a texture loss to optimize the visual perceptual quality of reconstructed images with sharp boundaries and rich texture details. Extensive experiments on four stereo image datasets demonstrate that the proposed CVGSR achieves a competitive and excellent performance.

Multimodel fore-/background alignment for seam-based parallax-tolerant image stitching

Image stitching with large parallax is a challenging computer vision problem. Although existing seam-based approaches were proposed to achieve pleasing results, issues like object dislocation, disappearance, and duplication can still occur. In this paper, to alleviate these problems, we propose a novel seam-based parallax-tolerant image stitching method, which relies on accurately aligning background and foreground regions using multiple warping models. To estimate various spatially smooth models based on feature correspondences from depth-varying objects, we introduce an iterative algorithm that selects inliers and solves the mesh warping model by assigning weights to data. Additionally, we construct matching confidences of foreground pixels based on selecting and grouping unaligned feature pairs, thus penalizing the duplication of seam cuts. To further improve alignment, we refine the models by minimizing pixel-level errors. We then choose the best seam among multiple candidate alignment and seam finding solutions. Finally, we re-estimate the warping model by sampling and weighting points near the seam to achieve a natural-looking stitching result. Experimental results on real-world images demonstrate the effectiveness and superiority of our proposed method over other state-of-the-arts.

Research on seamless image stitching based on fast marching method

AbstractImage stitching is an important way to achieve large‐field high‐resolution imaging. The inconsistencies in brightness and structure and defects in ghosting, blurring and misalignment between images, which are inevitable and difficult to eliminate, make a challenge to image stitching, due to the external lighting environment and changes in camera pose and parameters. Here, a novel method is proposed to search for the optimal seamline based on the fast marching method, which can stitch large parallax images with high quality. A feature weight map is first formed based on the similarity in colour, edge, texture and saliency of the images. Then it is used as the cost value of the seamline to search for the optimal seamline by fast marching method. The results show that this new method is more efficient to reduce defects, such as ghosting, misalignment and chromatic aberration, and realize high quality image stitching compared with traditional stitching tools and methods, which provides a new perspective for image stitching technology.

UAV image stitching by estimating orthograph with RGB cameras

In the field of image stitching, cases with large camera optical center movement and large parallax have been the virgin territory of research. The goal of image stitching is to overcome the parallax and stitch a natural image. We look into this problem in the context of ultra-low altitude flight of a UAV. We model the 3D world in this scenario and quickly estimate orthographic projection by pairs of homography matrices. Our stitching method can achieve precise alignment since it takes parallax well into consideration. The stitching results are natural and the extra time consumed is short.

Feature Correspondences Increase and Hybrid Terms Optimization Warp for Image Stitching.

Feature detection and correct matching are the basis of the image stitching process. Whether the matching is correct and the number of matches directly affect the quality of the final stitching results. At present, almost all image stitching methods use SIFT+RANSAC pattern to extract and match feature points. However, it is difficult to obtain sufficient correct matching points in low-textured or repetitively-textured regions, resulting in insufficient matching points in the overlapping region, and this further leads to the warping model being estimated erroneously. In this paper, we propose a novel and flexible approach by increasing feature correspondences and optimizing hybrid terms. It can obtain sufficient correct feature correspondences in the overlapping region with low-textured or repetitively-textured areas to eliminate misalignment. When a weak texture and large parallax coexist in the overlapping region, the alignment and distortion often restrict each other and are difficult to balance. Accurate alignment is often accompanied by projection distortion and perspective distortion. Regarding this, we propose hybrid terms optimization warp, which combines global similarity transformations on the basis of initial global homography and estimates the optimal warping by adjusting various term parameters. By doing this, we can mitigate projection distortion and perspective distortion, while effectively balancing alignment and distortion. The experimental results demonstrate that the proposed method outperforms the state-of-the-art in accurate alignment on images with low-textured areas in the overlapping region, and the stitching results have less perspective and projection distortion.

深度学习和APAP模型结合的大视差图像拼接算法

深度学习和APAP模型结合的大视差图像拼接算法

Orbital parallax of binary systems compared to Gaia DR3 and the parallax zero-point offset at bright magnitudes

Multiple systems for which the astrometric and spectroscopic orbit are known offer the unique possibility of determining the distance to these systems directly without any assumptions. They are therefore ideal objects for a comparison of Gaia data release 3 (GDR3) parallax data, especially since GDR3 presents the results of the non-single star (NSS) analysis that potentially results in improved parallaxes. This analysis is relevant in studying the parallax zero-point offset (PZPO) that is crucial in improving upon the distance scale. An sample of 192 orbital parallax determinations for 186 systems is compiled from the literature. The stars are also potentially in wide binary systems (WBS). A search was performed and 37 WBS (candidates) were found. Only for 21 objects does the NSS analysis provide information, including 8 from the astrometric binary pipeline, for which the parallaxes do improve significantly compared to those in the main catalogue with significant lower goodness-of-fit (GOF) parameters. It appears that most of the objects in the sample are eliminated in the pre-filtering stage of the NSS analysis. The difference between the orbital parallax and the (best) Gaia parallax was finally obtained for 170 objects. A raw comparison is meaningless, however, due to limitations in accuracy both in the orbital and in Gaia data. As many systems have been eliminated in the pre-filtering stage of the astrometric NSS pipeline, they remain in GDR3 with values for the GOF parameter in the range from several tens to several hundreds. When objects with large parallax errors or unrealistically large differences between the orbital and Gaia parallaxes are eliminated, and objects with a GOF <100 or < 8 are selected (the latter also with G < 10.5 mag selected), samples of 68 and 20 stars remain. Parallax differences in magnitude bins and for the sample are presented. Three recipes from the literature that calculate the PZPO are tested. After these corrections are applied the remaining parallax differences are formally consistent with zero within the error bar for all three recipes. In all cases, an uncertainty in these averages of about 10–15 µas remains for these samples due to the small number statistics. The proof of concept of using orbital parallaxes is shown to work, but the full potential is not reached as an improved parallax from the NSS analysis is available for only for eight systems. In the final selection, the orbital parallax of 18 of 20 stars is known to better than 5%, and the parallax determination for 6 stars is better than from Gaia. In the full sample, 148 objects reach this precision in orbital parallax and therefore the full potential of using orbital parallaxes may hopefully be reached with GDR4.

Optimizing Local Alignment along the Seamline for Parallax-Tolerant Orthoimage Mosaicking

Orthoimage mosaicking with obvious parallax caused by geometric misalignment is a challenging problem in the field of remote sensing. Because the obvious objects are not included in the digital terrain model (DTM), large parallax exists in these objects. A common strategy is to search an optimal seamline between orthoimages, avoiding the majority of obvious objects. However, stitching artifacts may remain because (1) the seamline may still cross several obvious objects and (2) the orthoimages may not be precisely aligned in geometry when the accuracy of the DTM is low. While applying general image warping methods to orthoimages can improve the local geometric consistency of adjacent images, these methods usually significantly modify the geometric properties of orthophoto maps. To the best of our knowledge, no approach has been proposed in the field of remote sensing to solve the problem of local geometric misalignments after orthoimage mosaicking with obvious parallax. In this paper, we creatively propose a method to optimize local alignment along the seamline after seamline detection. It consists of the following main processes. First, we locate regions with geometric misalignments along the seamline based on the similarity measure. Second, for any one region, we find one-dimensional (1D) feature matches along the seamline using a semi-global matching approach. The deformation vectors are calculated for these matches. Third, these deformation vectors are robustly and smoothly propagated into the buffer region centered on the seamline by minimizing the associated energy function. Finally, we directly warp the orthoimages to eliminate the local parallax under the guidance of dense deformation vectors. The experimental results on several groups of orthoimages show that our proposed approach is capable of eliminating the local parallax existing in the seamline while preserving most geometric properties of digital orthophoto maps, and that it outperforms state-of-the-art approaches in terms of both visual quality and quantitative metrics.

Image stitching by disparity-guided multi-plane alignment

Image stitching aims to warp and align two or more images with overlapping areas to generate a panorama with a larger field of view. Due to the wide baseline or the abrupt depth changes caused by the foreground objects, the phenomenon that adjacent pixels or regions in one image are not adjacent in another image happens. It is difficult to avoid severe parallax artifacts and get good alignment results when stitching such images. In this paper, focusing on the images with large parallax, we design a multi-plane alignment algorithm guided by the disparity map. The concept of average tolerable parallax is proposed to help distinguish one background plane and multiple foreground objects from the image, which are robust to small parallax. In order to obtain the reliable disparity map of the image pairs got from any camera with unknown camera parameters, we propose an optimal homography estimation method based on the relative projection biases. This helps to satisfy the common epipolar line constraint when applying stereo matching modules. Experimental results demonstrate that our algorithm provides accurate stitching results on images with large parallax, and outperforms other existing state-of-the-art methods both qualitatively and quantitatively.

Structure Preservation and Seam Optimization for Parallax-Tolerant Image Stitching

Parallax processing has long been a significant and challenging task in image stitching. In this paper, we study a new hybrid warping model based on multi-homography and structure preservation to achieve accurate alignment of regions at different depths while preserving local and global image structures. The homographies of different depth regions are estimated by dividing matching feature pairs into multiple layers. Then, layered warping is performed by determining the spatial relationships between the image mesh and these multi-homography, and then refining the local and global structural distortions through mesh optimization. Four constraints are considered during the local optimization process, including the local alignment error, global alignment error, and similarity error. In addition, we explore and introduce collinear structures into an objective function as a constraint for mesh optimization warping, which can preserve salient line structures while alleviating distortions in nonoverlapping areas. Furthermore, we develop an optimal seam search method based on seam error evaluation to improve the quality of the seams. Experimental results demonstrate that compared to existing methods, the proposed algorithm presents more accurate stitching results for images with large parallax and preserves salient image structures, and outperforming the existing methods both qualitatively and quantitatively.

기하학적 불확실성 기반 영상 관성 항법의 원거리 특징점 감지

In this paper, we propose a new method that classifies the features in a visual-inertial navigation system. The far features provide inaccurate information on the position and velocity of a moving vehicle, and thus, the information provided by the far features should be used for attitude estimation without recovering the feature depths. In contrast, the information obtained from the near features can be used to obtain a reliable depth estimate because of the large parallax in the image plane. However, the criterion for labeling the features as near or far features is ambiguous. Previously, various geometric classification methods based on a stereo camera and measurement uncertainties have been reported. Herein, we present a criterion based on the geometric method using the MSCKF (Multi-State Constraint Kalman filter ). Additionally, we define a new concept — depth uncertainty — as the criterion of feature classification in the MSCKF. Using this criterion, we can draw a limited range defined as the observable region. Implementation of this method can decrease the error caused by the low parallax of the feature. The proposed method is validated through simulations and experiments, showing a 12.7 % and 21.2 % decrease in the mean position error, respectively, using the far features classification.

Hunting wide-area optical surveys for high proper motion isolated neutron stars

ABSTRACT High-velocity neutron stars (HVNSs) that were kicked out from their birth location can be potentially identified with their large proper motions, and possibly with large parallax, when they come across the solar neighborhood. In this paper, we study the feasibility of hunting isolated HVNSs in wide-area optical surveys by modelling the evolution of NS luminosity taking into account spin-down and thermal radiation. Assuming the upcoming 10-yr Vera C. Rubin Observatory’s LSST observation, our model calculations predict that about 10 HVNSs mainly consisting of pulsars with ages of 104–$10^5\, \rm yr$ and thermally emitting NSs with 105–$10^6\, \rm yr$ are detectable. We find that a few NSs with effective temperature $\lt 5 \times 10^5\, \rm K$, which are likely missed in the current and future X-ray surveys, are also detectable, and the detectability highly depends on NS cooling models. In particular, if considering dark matter heating in NS cores, we find that such cold NSs would account for $70{{\ \rm per\ cent}}$ of the whole detectable sample, and they are typically older than $10^6\, \rm yr$ and tend to have slow transverse velocities, $\lesssim 100\, \rm km\, s^{-1}$, compared to already known pulsars. Thus, the future optical observation will give a unique NS sample, which can provide essential constraints on the NS cooling and heating mechanisms. Moreover, we suggest that providing HVNS samples with optical surveys is helpful for understanding the intrinsic kick-velocity distribution of NSs.

Topological Similarity-Based Multi-Target Correlation Localization for Aerial-Ground Systems

In this paper, an algorithm for solving the multi-target correlation and co-location problem of aerial-ground heterogeneous system is investigated. Aiming at the multi-target correlation problem, the fusion algorithm of visual axis correlation method and improved topological similarity correlation method are adopted in view of large parallax and inconsistent scale between the aerial and ground perspectives. First, the visual axis was preprocessed by the threshold method, so that the sparse targets were initially associated. Then, the improved topological similarity method was used to further associate dense targets with the relative position characteristics between targets. The shortcoming of dense target similarity with small difference was optimized by the improved topological similarity method. For the problem of co-location, combined with the multi-target correlation algorithm in this paper, the triangulation positioning model was used to complete the co-location of multiple targets. In the experimental part, simulation experiments and flight experiments were designed to verify the effectiveness of the algorithm. Experimental results show that the proposed algorithm can effectively achieve multi-target correlation positioning, and that the positioning accuracy is obviously better than other positioning methods.

Region-Based Static Video Stitching for Reduction of Parallax Distortion.

In this paper, we propose a semantic segmentation-based static video stitching method to reduce parallax and misalignment distortion for sports stadium scenes with dynamic foreground objects. First, video frame pairs for stitching are divided into segments of different classes through semantic segmentation. Region-based stitching is performed on matched segment pairs, assuming that segments of the same semantic class are on the same plane. Second, to prevent degradation of the stitching quality of plain or noisy videos, the homography for each matched segment pair is estimated using the temporally consistent feature points. Finally, the stitched video frame is synthesized by stacking the stitched matched segment pairs and the foreground segments to the reference frame plane by descending order of the area. The performance of the proposed method is evaluated by comparing the subjective quality, geometric distortion, and pixel distortion of video sequences stitched using the proposed and conventional methods. The proposed method is shown to reduce parallax and misalignment distortion in segments with plain texture or large parallax, and significantly improve geometric distortion and pixel distortion compared to conventional methods.

Large parallax image matching based on Graph Matching

In the security video surveillance system, image splicing technology faces the difficulty of achieving large parallax and small overlap area. This paper summarizes the traditional stitching technology and analyzes that the image matching algorithm is the key technology to solve the problem of large parallax stitching. Aiming at the problems that traditional matching methods with low matching accuracy cannot adapt to large parallax non-rigid transformatio, a model based on image matching is proposed in this paper for splicing tasks. The model which combines the similarity of the global structure and the local structure can not only divide the feature point on the structural level but also can pair with each other. At the same time, the framework can be trained end-to-end. The training samples composed of simulation data sets are adopted to train the designed model which is tested and verifyed by the test sample sets composed of three kinds of mixed data. Compared with PCA-GM, the model proposed in this paper adapts well to splicing tasks. The matching accuracy can achieve 93.1% and 91.9% on the simulation data set and the small parallax splicing data respectively, which can initially solve the image matching problem in the traditional image splicing scene. However, a further research aiming at the problem of the matching accuracy on the large parallax data set can only achieve 74.5% will be needed.

Fourier horizontal parallax only computer and digital holography of large size.

Registration and reconstruction of high-quality digital holograms with a large view angle are intensive computer tasks since they require the space-bandwidth product (SBP) of the order of tens of gigapixels or more. This massive use of SBP severely affects the storing and manipulation of digital holograms. In order to reduce the computer burden, this work focuses on the generation and reconstruction of very large horizontal parallax only digital holograms (HPO-DHs). It is shown that these types of holograms can preserve high quality and large view angle in x direction while keeping a low use of SBP. This work first proposes a numerical technique that allows calculating very large HPO-DHs with large pixel size by merging the Fourier holography and phase added stereogram algorithm. The generated Fourier HPO-DHs enable accurate storing of holographic data from 3D objects. To decode the information contained in these Fourier HPO-DHs (FHPO-DHs), a novel angular spectrum (AS) technique that provides an efficient use of the SBP for reconstruction is proposed. Our reconstruction technique, which is called compact space bandwidth AS (CSW-AS), makes use of cylindrical parabolic waves that solve sampling issues of FHPO-DHs and AS. Moreover, the CSW-AS allows for implementing zero-padding for accurate wavefield reconstructions. Hence, suppression of aliased components and high spatial resolution is possible. Notably, the imaging chain of Fourier HPO-DH enables efficient calculation, reconstruction and storing of HPO holograms of large size. Finally, the accuracy and utility of the developed technique is proved by both numerical and optical reconstructions.

Potential asteroid discoveries by the ESA Gaia mission

Context. Since July 2014, the Gaia mission of the European Space Agency has been surveying the entire sky down to magnitude 20.7 in the visible. In addition to the millions of daily observations of stars, thousands of Solar System objects (SSOs) are observed. By comparing their positions, as measured by Gaia, to those of known objects, a daily processing pipeline filters known objects from potential discoveries. However, owing to Gaia’s specific observing mode, which follows a predetermined scanning law designed for stars as “fixed” objects on the celestial sphere, potential newly discovered moving objects are characterized by very few observations, which are acquired over a limited time. Furthermore, these objects cannot be specifically targeted by Gaia itself after their first detection. This aspect was recognized early on in the design of the Gaia data processing. Aims. A daily processing pipeline dedicated to these candidate discoveries was set up to release calls for observations to a network of ground-based telescopes. Their aim is to acquire follow-up astrometry and to characterize these objects. Methods. From the astrometry measured by Gaia, preliminary orbital solutions are determined, allowing us to predict the position of these potentially newly discovered objects in the sky while accounting for the large parallax between Gaia and the Earth (separated by 0.01 au). A specific task within the Gaia Data Processing and Analysis Consortium has been responsible for the distribution of requests for follow-up observations of potential Gaia SSO discoveries. Since late 2016, these calls for observations (nicknamed “alerts”) have been published via a Web interface with a quasi-daily frequency, together with observing guides, which is freely available to anyone worldwide. Results. Between November 2016 and the end of the first year of the extended mission (July 2020), over 1700 alerts were published, leading to the successful recovery of more than 200 objects. Among them, six have a provisional designation assigned with the Gaia observations; the others were previously known objects with poorly characterized orbits, precluding identification at the time of Gaia observations. There is a clear trend for objects with a high inclination to be unidentified, revealing a clear bias in the current census of SSOs against high-inclination populations.

The Hawaii Infrared Parallax Program. V. New T-dwarf Members and Candidate Members of Nearby Young Moving Groups

Abstract We present a search for new planetary-mass members of nearby young moving groups (YMGs) using astrometry for 694 T and Y dwarfs, including 447 objects with parallaxes, mostly produced by recent large parallax programs from UKIRT and Spitzer. Using the BANYAN Σ and LACEwING algorithms, we identify 30 new candidate YMG members, with spectral types of T0–T9 and distances of 10–43 pc. Some candidates have unusually red colors and/or faint absolute magnitudes compared to field dwarfs with similar spectral types, providing supporting evidence for their youth, including four early-T dwarfs. We establish one of these, the variable T1.5 dwarf 2MASS J21392676+0220226, as a new planetary-mass member ( M Jup) of the Carina-Near group (200 ± 50 Myr) based on its full six-dimensional kinematics, including a new parallax measurement from CFHT. The high-amplitude variability of this object is suggestive of a young age, given the coexistence of variability and youth seen in previously known YMG T dwarfs. Our four latest-type (T8–T9) YMG candidates, WISE J031624.35+430709.1, ULAS J130217.21+130851.2, WISEPC J225540.74–311841.8, and WISE J233226.49–432510.6, if confirmed, will be the first free-floating planets (≈2–6 M Jup) whose ages and luminosities are compatible with both hot-start and cold-start evolutionary models, and thus overlap with the properties of the directly imaged planet 51 Eri b. Several of our early/mid-T candidates have peculiar near-infrared spectra, indicative of heterogenous photospheres or unresolved binarity. Radial velocity measurements needed for final membership assessment for most of our candidates await upcoming 20–30 m class telescopes. In addition, we compile all 15 known T7–Y1 benchmarks and derive a homogeneous set of their effective temperatures, surface gravities, radii, and masses.