Projection Distortion Research Articles

A Parametric Method for Remapping and Calibrating Fisheye Images for Glare Analysis

High Dynamic Range (HDR) imaging using a fisheye lens has provided new opportunities to evaluate the luminous environment in visual comfort research. For glare analysis, strict calibration is necessary to extract accurate luminous maps to achieve reliable glare results. Most studies have focused on correcting the vignetting effect in HDR imaging during post-calibration. However, the lens projection also contributes to luminous map errors because of its inherent distortion. To date, there is no simple method to correct this distortion phenomenon for glare analysis. This paper presents a parametric-based methodology to correct the projection distortion from fisheye lenses for the specific use in glare analysis. HDR images were captured to examine two devices: a 190° equisolid SIGMA 8 mm F3.5 EX DG fisheye lens mounted on a Canon 5D camera, and a 195° fisheye commercial lens with an unknown projection, mounted on the rear camera of a Samsung Galaxy S7. A mathematical and geometrical model was developed to remap each pixel to correct the projection distortion using Grasshopper and MATLAB. The parametric-based method was validated using Radiance and MATLAB through checking the accuracy of pixel remapping and measuring color distortion with Structural Similarity Index (SSIM). Glare scores were used to compare the results between both devices, which validates the use of mobile phones in photometric research. The results showed that this method can be used to correct HDR images projection distortion for more accurate evaluation of the luminous environment in glare research.

A Generalized Cubemap for Encoding 360° VR Videos using Polynomial Approximation

Abstract360° VR videos provide users with an immersive visual experience. To encode 360° VR videos, spherical pixels must be mapped onto a two‐dimensional domain to take advantage of the existing video encoding and storage standards. In VR industry, standard cubemap projection is the most widely used projection method for encoding 360° VR videos. However, it exhibits pixel density variation at different regions due to projection distortion. We present a generalized algorithm to improve the efficiency of cubemap projection using polynomial approximation. In our algorithm, standard cubemap projection can be regarded as a special form with 1st‐order polynomial. Our experiments show that the generalized cubemap projection can significantly reduce the projection distortion using higher order polynomials. As a result, pixel distribution can be well balanced in the resulting 360° VR videos. We use PSNR, S‐PSNR and CPP‐PSNR to evaluate the visual quality and the experimental results demonstrate promising performance improvement against standard cubemap projection and Google's equi‐angular cubemap.

A Novel Projective-Consistent Plane Based Image Stitching Method

When different target surfaces, in three-dimensional space, are mapped onto an image plane, they have different projections. These projections vary with the viewpoint. These local differences have influence on the accuracy of image stitching. Most of the existing image stitching methods divide an input image into a number of fixed-size cells, and the pixels within the same cell are then warped using the same local transformation model for the alignment. These methods are based on the hypothesis that the transformation models in one cell are consistent. However, this hypothesis does not hold in general. In this paper, we propose a novel projective-consistent plane based image stitching method (termed PCPS). It divides the overlapping regions of an input image into some projective-consistent planes according to the normal vectors’ orientations of local regions and the reprojection errors of aligned images. The local projective transformation model is estimated for each projective-consistent plane. And then, a hybrid warping model is estimated. For the pixels in overlapping regions, the local projective transformation models are adopted to achieve a better alignment. While for the pixels in non-overlapping regions, a global projective transformation model is estimated by using the inliers uniformly distributed in the projective-consistent planes to avoid distortion. Compared with the state-of-the-art image stitching methods, the experimental results on a number of challenging image sequences show that the projective transformation model estimated by the proposed PCPS method for each projective-consistent plane is more accurate, and the achieved stitching results have less seams and projective distortion.

Constructing and Visualizing High-Quality Classifier Decision Boundary Maps

Visualizing decision boundaries of machine learning classifiers can help in classifier design, testing and fine-tuning. Decision maps are visualization techniques that overcome the key sparsity-related limitation of scatterplots for this task. To increase the trustworthiness of decision map use, we perform an extensive evaluation considering the dimensionality-reduction (DR) projection techniques underlying decision map construction. We extend the visual accuracy of decision maps by proposing additional techniques to suppress errors caused by projection distortions. Additionally, we propose ways to estimate and visually encode the distance-to-decision-boundary in decision maps, thereby enriching the conveyed information. We demonstrate our improvements and the insights that decision maps convey on several real-world datasets.

Single-Perspective Warps in Natural Image Stitching.

Results of image stitching can be perceptually divided into single-perspective and multiple-perspective. Compared to the multiple-perspective result, the single-perspective result excels in perspective consistency but suffers from projective distortion. In this paper, we propose two single-perspective warps for natural image stitching. The first one is a parametric warp, which is an incremental combination of the dual-feature-based as-projective-as-possible warp and the quasi-homography warp. The second one is a mesh-based warp, which is determined by optimizing a total energy function that simultaneously emphasizes different characteristics of the single-perspective warp, including alignment, distortion and saliency. A comprehensive evaluation demonstrates that the proposed warp outperforms some state-of-the-art warps in urban scenes, including APAP, AutoStitch, SPHP and GSP.

A Fast Sample Adaptive Offset Algorithm for 360∘ Video

The increased demand for virtual reality brings more challenges to 360∘ video coding. 360∘ video needs to be projected as a planar video before encoding. This process will produce projection distortion. The degree of projection distortion depends on the location of the pixel. Traditional coding algorithms cannot respond to this feature efficiently enough. In this paper, a fast Sample Adaptive Offset (SAO) algorithm for 360∘ video is proposed. The proposed algorithm improves the SAO process. On the basis of retaining the whole SAO process, a simplified SAO process is added. First, the coding tree unit (CTU) of performing the simplified SAO process is filtered according to the rate distortion cost (RD-cost) of the intra- or inter-prediction and the location of the CTU. Subsequently, the CTU is sampled at intervals according to the equirectangular projection (ERP) characteristics, and the CTU of performing the simplified SAO process is determined. Experimental results show that the proposed algorithm achieves 60% time of SAO process reduction, with only 0.29% luma Bjontegaard delta rate (BD-rate) increases on average.

Factoring a Homography to Analyze Projective Distortion

We present an algorithm in homogeneous coordinates for factoring a homography $$\mathbf{h}$$ on $$\mathbb {R}P^2$$ as $$\mathbf{h}=\mathbf{p}\circ \mathbf{s}$$ , where $$\mathbf{p}$$ is a perspective collineation and $$\mathbf{s}$$ is a similarity. We use the factorization to derive a function that measures the local projective distortion of a non-affine homography $$\mathbf{h}$$ at ordinary points of $$\mathbb {R}P^2$$ , and discover interesting geometric structures associated with $$\mathbf{h}$$ , somewhat like the centers and axes of perspective collineations. These results reveal infinite families of circle pairs, and neighborhoods of certain points associated with $$\mathbf{h}$$ , that give the appearance of suffering no projective distortion, even though $$\mathbf{h}$$ is not a similarity. In fact, when the factor $$\mathbf{p}$$ is a perspective collineation, every non-empty subset $$\mathcal{S}$$ of $$\mathbb {R}P^2$$ has an “ $$\mathbf{h}$$ -conjugate” set $$\mathcal{S}^*$$ such that $$\mathbf{h}(\mathcal{S}\cup \mathcal{S}^*)=\mathbf{s}(\mathcal{S}\cup \mathcal{S}^*)$$ , even though $$\mathbf{h}$$ and $$\mathbf{s}$$ do not agree on $$\mathcal{S}\cup \mathcal{S}^*$$ . We include examples from photography as well as connections to Apollonian circles, Mobius transformations and stereographic projections.

A Virtual Globe Using a Discrete Global Grid System to Illustrate the Modifiable Areal Unit Problem

In the same way that discrete global grid systems (DGGS) are used to index data on the spherical Earth, they can aggregate point data, with their spherical polygons serving as bins. DGGS are particularly useful at multiple map scales because they are spatially hierarchical and exist on the sphere or ellipsoid, allowing large or small scale binning without projection distortion. We use DGGS in a free and open-source pedagogical tool for teaching students about the modifiable areal unit problem (MAUP). Our software application uses Dutton’s quaternary triangular mesh (QTM) to bin global data points geodesically with counts or measures of any theme at multiple levels. Users can interactively select the level to which the data are binned by the QTM, as well as translate the whole tessellation east or west so that points fall into and out of different bins. These two functions illustrate the scaling and zoning aspects of the MAUP with dynamically-drawn choropleths on the surface of a virtual globe that the user can zoom and rotate, allowing visualization at virtually any cartographic scale. Users may also select various quantile classifications to further explore issues in visualizing aggregate data. In addition to presenting this new tool, we highlight the importance, especially at smaller scales, of using geodesic point-in-polygon intersection detection, rather than the projected 2D methods typically used by geographic information systems.

Shape-Optimizing and Illumination-Smoothing Image Stitching

Image stitching is usually subject to projective distortion and color difference problems. This paper presents an illumination-smoothing image stitching method based on the shape-optimizing hybrid transformation. An automatic mesh generation strategy is especially designed to reduce the calculation of the hybrid transformation within a reasonable range and guarantee the accuracy of image alignment. We consider stitching multiple images while constraining the distortion with the hybrid transformation and alleviate the color difference. In this paper, the method of color correction is adapted according to the characteristics of image stitching so as to achieve illumination-smoothing results. Moreover, the triangulation of the matching feature points is used to partition the color regions of the image and then more abundant color information is obtained for the calculation of the color transformation model. For general horizontal images, our method can achieve better results with less distortion in comparison with state-of-the-art methods. Various experimental results validate our new method.

Optimal orientations of discrete global grids and the Poles of Inaccessibility

ABSTRACT Spatial analyses involving binning often require that every bin have the same area, but this is impossible using a rectangular grid laid over the Earth or over any projection of the Earth. Discrete global grids use hexagons, triangles, and diamonds to overcome this issue, overlaying the Earth with equally-sized bins. Such discrete global grids are formed by tiling the faces of a polyhedron. Previously, the orientations of these polyhedra have been chosen to satisfy only simple criteria such as equatorial symmetry or minimizing the number of vertices intersecting landmasses. However, projection distortion and singularities in discrete global grids mean that such simple orientations may not be sufficient for all use cases. Here, I present an algorithm for finding suitable orientations; this involves solving a nonconvex optimization problem. As a side-effect of this study I show that Fuller's Dymaxion map corresponds closely to one of the optimal orientations I find. I also give new high-accuracy calculations of the Poles of Inaccessibility, which show that Point Nemo, the Oceanic Pole of Inaccessibility, is 15 km farther from land than previously recognized.

Advanced Spherical Motion Model and Local Padding for 360-Degree Video Compression.

The 360-degree video compression has two main challenges due to projection distortions namely the geometry distortion and the face boundary discontinuity. There are some trade-offs between selecting Equi-rectangular and polyhedron projections. In the Equi-rectangular projection, the geometry distortion is severer than the face boundary discontinuity, while for the polyhedron projections the face boundary discontinuity is severer than the geometry distortion. These two distortions will have side effects on the motion compensation and undermine the compression efficiency of the 360-degree video. In this paper, an integrated framework is developed to handle these two problems to improve the coding efficiency. The proposed framework mainly has two key contributions. First, we derive a unified advanced spherical motion model to handle the geometry distortion of different projection formats for the 360-degree video. When fitting the projection between the various projection formats and the sphere into the unified framework, a specific solution can be obtained for each projection format. Second, we propose a local 3-D padding method to handle the face boundary discontinuity between the neighboring faces in various projection formats of the 360-degree video. The local 3-D padding method can be applied to different projection formats through setting different angles between neighboring faces. These two methods are independent of each other and can also be combined into an integrated framework to achieve a better rate-distortion performance. The proposed framework can be seamlessly integrated into the latest video coding standard High Efficiency Video Coding. The experimental results demonstrate introducing proposed coding tools can achieve significant bitrate savings compared with the current state-of-the-art method.

АЛГОРИТМ КЛАССИФИКАЦИИ ДОКУМЕНТОВ, УДОСТОВЕРЯЮЩИХ ЛИЧНОСТЬ, НА ПРОЕКТИВНО-ИСКАЖЕННЫХ ИЗОБРАЖЕНИЯХ НА ОСНОВЕ ОБУЧАЕМОЙ МЕТРИКИ ПОДОБИЯ

АЛГОРИТМ КЛАССИФИКАЦИИ ДОКУМЕНТОВ, УДОСТОВЕРЯЮЩИХ ЛИЧНОСТЬ, НА ПРОЕКТИВНО-ИСКАЖЕННЫХ ИЗОБРАЖЕНИЯХ НА ОСНОВЕ ОБУЧАЕМОЙ МЕТРИКИ ПОДОБИЯ

Projector distortion residual compensation in fringe projection system

In fringe projection profilometry system, the lens distortion of the projector is usually represented with the distortion coefficients, but it will result in large residual. To compensate this residual, a projector distortion residual compensation method is proposed in this paper. After system calibration, the original distortion map is generated with the distortion coefficients and the original ideal pattern can be pre-distorted with the original distortion map. Then, with pinhole model and fringe projection technique, a residual distortion map is built based on per-pixel measurement. Finally, the original distortion map can be refined with the residual distortion map, and the lens distortion residual error can be compensated. Experimental results show considerable accuracy improvement in both the full-field error of projector and the final point cloud of system. In addition, the proposed compensation method can be implemented conveniently without any sophisticated ancillary equipment or complicated procedure.

A New Uniform Format for 360 VR Videos

AbstractRecent breakthroughs in VR technologies, especially in economic VR headsets and massive smartphones are creating a fast‐growing demand for 3D immersive VR content. 360 VR videos record a surrounding environment in every direction and give users a fully immersive experience. Thanks to a ton of 360 cameras that launched in the past years, 360 video content creation is exploding and 360 VR videos are becoming a new video standard in the digital industry. When ERP and CMP are perhaps the most prevalent projection and packing layout for storing 360 VR videos, they have severe projection distortion, internal discontinuous seams or disadvantages in aspect ratio. We introduce a new format for packing and storing 360 VR videos using two stage mappings. Hemispheres are seamlessly and uniformly mapped onto squares. Two respective squares are stitched to form a rectangle with the aspect ratio 2:1. Our approach is able to avoid internal discontinuity and generate uniform pixel distribution, while keeping the aspect ratio close to the majority standard aspect ratio of 16 : 9.

Image stitching by line-guided local warping with global similarity constraint

Low-textured image stitching remains a challenging problem. It is difficult to achieve good alignment of images and it is easy to break images structures are often broken due to insufficient and unreliable point correspondences. Moreover, because of the viewpoint variations between multiple images, the stitched images suffer from projective distortions. To solve these problems, this paper presents a line-guided local warping method with a global similarity constraint for image stitching. Line features which serve well for geometric descriptions and scene constraints, are employed to guide image stitching accurately. On one hand, the line features are integrated into a local warping model through a designed weight function. On the other hand, line features are adopted to impose strong geometric constraints, including line correspondence and line colinearity, to improve the stitching performance through mesh optimization. To mitigate projective distortions, we adopt a global similarity constraint, which is integrated with the projective warps via a designed weight strategy. This constraint causes the final warp to slowly change from a projective to a similarity transformation across the image. Finally, the images undergo a two-stage alignment scheme that provides accurate alignment and reduces projective distortion. We evaluate our method on a series of images and compare it with several other methods. The experimental results demonstrate that the proposed method provides a convincing stitching performance and that it outperforms other state-of-the-art methods.

Quasi-Homography Warps in Image Stitching

The naturalness of warps is gaining extensive attention in image stitching. Recent warps, such as SPHP and AANAP, use global similarity warps to mitigate projective distortion (which enlarges regions); however, they necessarily bring in perspective distortion (which generates inconsistencies). In this paper, we propose a novel quasi-homography warp, which effectively balances the perspective distortion against the projective distortion in the non-overlapping region to create a more natural-looking panorama. Our approach formulates the warp as the solution of a bivariate system, where perspective distortion and projective distortion are characterized as slope preservation and scale linearization, respectively. Because our proposed warp only relies on a global homography, it is thus totally parameter free. A comprehensive experiment shows that a quasi-homography warp outperforms some state-of-the-art warps in urban scenes, including homography, AutoStitch and SPHP. A user study demonstrates that it wins most users' favor, compared to homography and SPHP.

Image formation in the scanning helium microscope

The scanning helium microscope (SHeM) is a new addition to the array of available microscopies, particularly for delicate materials that may suffer damage under techniques utilising light or charged particles. As with all other microscopies, the specifics of image formation within the instrument are required to gain a full understanding of the produced micrographs. We present work detailing the basics of the subject for the SHeM, including the specific nature of the projection distortions that arise due to the scattering geometry. Extension of these concepts allowed for an iterative ray tracing Monte Carlo model replicating diffuse scattering from a sample surface to be constructed. Comparisons between experimental data and simulations yielded a minimum resolvable step height of (67 ± 5) µm and a minimum resolvable planar angle of (4.3 ± 0.3)° for the instrument in question.

A METHOD FOR THE SEAMLINES NETWORK AUTOMATIC SELECTION BASED ON BUILDING VECTOR

Abstract. In order to improve the efficiency of large scale orthophoto production of city, this paper presents a method for automatic selection of seamlines network in large scale orthophoto based on the buildings’ vector. Firstly, a simple model of the building is built by combining building’s vector, height and DEM, and the imaging area of the building on single DOM is obtained. Then, the initial Voronoi network of the measurement area is automatically generated based on the positions of the bottom of all images. Finally, the final seamlines network is obtained by optimizing all nodes and seamlines in the network automatically based on the imaging areas of the buildings. The experimental results show that the proposed method can not only get around the building seamlines network quickly, but also remain the Voronoi network’ characteristics of projection distortion minimum theory, which can solve the problem of automatic selection of orthophoto seamlines network in image mosaicking effectively.

Continuous combination of viewing zones in integral three-dimensional display using multiple projectors

We propose a method for arranging multiple projectors in parallel using an image-processing technique and for enlarging the viewing zone in an integral three-dimensional image display. We have developed a method to correct the projection distortion precisely using an image-processing technique combining projective and affine transformations. To combine the multiple viewing zones formed by each projector continuously and smoothly, we also devised a technique that provides accurate adjustment by generating the elemental images of a computer graphics model at high speed. We constructed a prototype device using four projectors equivalent to 4K resolution and realized a viewing zone with measured viewing angles of 49.2 deg horizontally and 45.2 deg vertically. Compared with the use of only one projector, the prototype device expanded the viewing angles by approximately two times in both the horizontal and vertical directions.

Projected image correction technology research on autonomous-perception anisotropic surfaces.

In this paper, a projection correction method is proposed for autonomous-perception depth anisotropic surfaces, in order to improve the adaptive perceptual projection of the projection equipment under different circumstances. During the parameter calibration process of a projector-camera system with low recognition precision, as well as low noise resistance at angular points, an angular-point subpixel detection algorithm based on color information guidance was proposed to effectively improve the identification precision of the angular-point detection. Meanwhile, the projection geometry correction algorithm was proposed, which was based on the topology analysis, in order to analyze the spatial topology distribution of the depth anisotropic surface and also to solve the homography matrix in the different regions of the anisotropic surface. Eventually, the homography matrix was used for the geometric distortion correction of the projection distortion image. The experimental analysis showed that the identification accuracy of the proposed method achieved 0.25 pixels, with simultaneous high planar parallelism and liner perpendicularity. Through utilizing the method proposed by us for the projection correction of the depth anisotropic surface, it was confirmed that the geometric distortion correction precision of the proposed method had reached the subpixel level, and the consistency level of the imaging picture was also confirmed.