Catadioptric Omnidirectional Imaging Research Articles

Catadioptric hyperspectral imaging, an unmixing approach

Hyperspectral imaging systems provide dense spectral information on the scene under investigation by collecting data from a high number of contiguous bands of the electromagnetic spectrum. The low spatial resolutions of these sensors frequently give rise to the mixing problem in remote sensing applications. Several unmixing approaches are developed in order to handle the challenging mixing problem on perspective images. On the other hand, omnidirectional imaging systems provide a 360‐degree field of view in a single image at the expense of lower spatial resolution. In this study, we propose a novel imaging system which integrates hyperspectral cameras with mirrors so on to yield catadioptric omnidirectional imaging systems to benefit from the advantages of both modes. Catadioptric images, incorporating a camera with a reflecting device, introduce radial warping depending on the structure of the mirror used in the system. This warping causes a non‐uniformity in the spatial resolution which further complicates the unmixing problem. In this context, a novel spatial–contextual unmixing algorithm specifically for the large field of view of the hyperspectral imaging system is developed. The proposed algorithm is evaluated on various real‐world and simulated cases. The experimental results show that the proposed approach outperforms compared methods.

The design of a catadioptric omnidirectional thermal imaging system based on the combination of genetic algorithm and gradient descent

This kind of panoramic system is difficult to design, the designers usually start from a commonly used patent and modify the patented structure, which is time consuming to make the structures to meet the requirements. In fact, the most critical part of the design is to determine the initial structure parameters of the quadrics mirror of this system instead of the parameters of the whole system, the remaining elements of the system could be integrated as a regular optical system. In this paper, we firstly created the error function about the aberrations of the quadrics mirror and calculated the minimum value of the error function related to the initial parameters of the quadrics mirror by combining the genetic algorithm (GA) with gradient descent (GD), after that we can obtain a group of parameters of the quadrics mirror. Finally, we completed the optical design of a catadioptric omnidirectional thermal imaging system based on the determined structure of the quadrics mirror. A system was obtained with an operating waveband of 8–12 μm, the field of view is: 360° in the horizontal direction, and 20–100° in the vertical direction; F/1.4; the resolution is 0.4@33lp/mm; the effective focal length is 5 mm. It demonstrates the reliability and validity of this method.

Reconstruction of catadioptric omnidirectional images using dual alternating total variation minimization

This paper discusses the possibility to extend and apply the conventional two dimensional recovering images from blurry and noisy observation with the total variation regularization method to the catadioptric images. The principal in this special method is the stabilization of dual alternating minimization. The latter introduces two auxiliary half quadratic variables to transfer the system out of the ill-posed term. The main contribution of this paper is the use of the inverse stereographic projection and the spherical harmonics in order to adapt this proposed deconvolution with catadioptric omnidirectional images. The projection on the unit sphere of the omnidirectional image, is one way to alleviate the problem of the heterogeneous resolution and the negative effects of anamorphosis. In both anisotropic and isotropic deconvolutions, the experimental results conducted on synthetic as well as captured catadioptric omnidirectional images which are subject to various effects, confirm the performance of the proposed method to restore such images impaired by the blur and noise. Compared with several state-of-the-art approaches, the images resulted can achieve up to an acceptable and higher level of deconvolution quality.

Measurement Matrix Design for Compressed Sensing Based Catadioptric Omnidirectional Imaging

The problem of low and non-uniform resolution has long been a topic of research in catadioptric omni-directional imaging. This paper presents a compressed sensing method to solve this problem. A non-uniform measurement matrix is designed according to the distribution of the system’s resolution. Numerical simulation shows that the proposed method is feasible and effective.

恒等探测距离折反射式周视红外成像系统设计及分析

恒等探测距离折反射式周视红外成像系统设计及分析

Parameterized Distortion-Invariant Feature for Robust Tracking in Omnidirectional Vision

Central catadioptric omnidirectional images exhibit serious nonlinear distortions due to the involved quadratic mirrors. Therefore, features based on the conventional pin-hole model are hard to achieve satisfactory performances when directly applied to the distorted omnidirectional images. This paper analyzes the catadioptric geometry to facilitate modeling the nonlinear distortions of omnidirectional images. Different to the conventional imaging model, the prior information is considered in catadioptric system. A parameterized neighborhood mapping model is proposed to efficiently calculate the neighborhood of an object based on its measurable radial distance in the image plane. On the basis of the parameterized nonlinear model, a distortion-invariant fragment-based joint-feature mixture model of Gaussian is presented for human target tracking in omnidirectional vision. Under the framework of Gaussian Mixture Model, the problem of feature matching is converted into the feature clustering. The joint probability distribution of a joint-feature class is modeled by a mixture of Gaussian. A weight contribution mechanism is designed to flexibly weight the fragments contribution based on their responses, which leads to a robust tracking even under serious partial occlusion. Finally, experiments validate the advantage of the proposed algorithm over other conventional approaches.

Parametric distortion-adaptive neighborhood for omnidirectional camera.

Catadioptric omnidirectional images exhibit serious nonlinear distortion due to the involved quadratic mirror. Conventional pinhole model-based methods perform poorly when directly applied to the deformed omnidirectional images. This study constructs a catadioptric geometry system to analyze the variation of the neighborhood of an object in terms of the elevation and azimuth directions in a spherical coordinate system. To accurately represent the distorted visual information, a parametric neighborhood mapping model is proposed based on the catadioptric geometry. Unlike the conventional catadioptric models, the prior information of the system is effectively integrated into the neighborhood formulation framework. Then the distortion-adaptive neighborhood can be directly calculated based on its measurable image radial distance. This method can significantly improve the computational efficiency of algorithm since statistical neighborhood sampling is not used. On the basis of the proposed neighborhood model, a distortion-invariant Haar wavelet transform is presented to perform the robust human detection and tracking in catadioptric omnidirectional vision. The experimental results verify the effectiveness of the proposed neighborhood mapping model and prove that the distorted neighborhood in the omnidirectional image follows a nonlinear pattern.

Defocus deblurring for catadioptric omnidirectional imaging based on coded aperture and omni-total variation

The problem of catadioptric omnidirectional imaging defocus blur, which is caused by lens aperture and mirror curvature, becomes more severe when high-resolution sensors and large apertures are applied. In order to overcome this problem, a novel method based on coded aperture technique is proposed in this article. Firstly, the defocus blur of catadioptric omnidirectional imaging is analyzed to calculate the point spread function for different scene points. Then, a method of obtaining optimal focused plane is proposed. Lastly, based on the strategies of neighboring annuluses division and stitching of omnidirectional images, a deconvolution algorithm using omni-total variation prior is applied to obtain all-focused/sharp omnidirectional images. Experimental results demonstrate that the proposed method is effective for omnidirectional image defocus deblurring and can be applied to most existing catadioptric omnidirectional imaging systems.

Omni-total variation algorithm for the restoration of all-focused catadioptric image

In this paper, we present a novel omni-total variation (Omni-TV) algorithm for the restoration of defocus blur to obtain all-focused catadioptric image. Catadioptric omni-directional imaging systems usually consist of conventional cameras and curved mirrors for capturing 360° field of view. Mirror curvature in the catadioptric camera often leads to noticeable blurring artifacts in omni-directional imaging. The problem becomes more severe when high resolution sensor is introduced. In an omni-directional image, two points near each other may not be close to one another in the 3D scene. Traditional gradient computation cannot be directly applied to omni-directional image processing. Thus, an omni-gradient computing method combined with the characteristics of catadioptric imaging is proposed, in which an Omni-TV minimization is used as the constraint for deconvolution regularization. The proposed method is vital for improving catadioptric omni-directional imaging quality and promoting applications in related fields like omni-directional video and image processing.

Omnigradient Based Total Variation Minimization for Enhanced Defocus Deblurring of Omnidirectional Images

We propose a new method of image restoration for catadioptric defocus blur using omnitotal variation (Omni-TV) minimization based on omnigradient. Catadioptric omnidirectional imaging systems usually consist of conventional cameras and curved mirrors for capturing 360° field of view. The problem of catadioptric omnidirectional imaging defocus blur, which is caused by lens aperture and mirror curvature, becomes more severe when high resolution sensors and large apertures are used. In an omnidirectional image, two points near each other may not be close to one another in the 3D scene. Traditional gradient computation cannot be directly applied to omnidirectional image processing. Thus, omnigradient computing method combined with the characteristics of catadioptric omnidirectional imaging is proposed. Following this Omni-TV minimization is used as the constraint for deconvolution regularization, leading to the restoration of defocus blur in an omnidirectional image to obtain all sharp omnidirectional images. The proposed method is important for improving catadioptric omnidirectional imaging quality and promoting applications in related fields like omnidirectional video and image processing.

Complementary panoramic image fusion based on wavelet multi-scale decomposition

In order to solve the problem of low and non-uniform resolution in catadioptric omnidirectional imaging, a new image fusion method based on wavelet multi-scale decomposition was proposed in this paper according to the characteristics of complementary panoramic images. Using wavelet transform, the two complementary source images were decomposed into components with different resolutions and different directions first. And then based on specific fusion rules, low frequency was fused by average operator. With high frequency fusion, the exchanging by scales principle was utilized. At last, the fused image was reconstructed by inverse wavelet transform. The experimental results show that the fusion algorithm is simple and effective in the fusion of complementary panoramic images, and has good fusion results.

Defocus deblurring for catadioptric omnidirectional imaging based on spatially invariant point spread function

The problem of catadioptric omnidirectional imaging defocus blur, which is caused by lens aperture and mirror curvature, becomes more severe when high resolution sensors and large apertures are applied. In order to overcome this problem, a novel method based on computational photography is proposed. Firstly, the defocus blur of catadioptric omnidirectional imaging is analyzed to calculate the point spread function for different scene points. Then, the defocus blur kernel of omnidirectional image is confirmed to be spatially invariant when rotating the focus ring of camera lens during an image’s integration time. Lastly, the deconvolution algorithm using prior sparse derivatives is applied to obtain all-focused/sharp omnidirectional images. Experimental results demonstrate that the proposed method is effective for omnidirectional image deblurring and can be applied to most existing catadioptric omnidirectional imaging systems.

A Survey of Catadioptric Omnidirectional Camera Calibration

For dozen years, computer vision becomes more popular, in which omnidirectional camera has a larger field of view and widely been used in many fields, such as: robot navigation, visual surveillance, virtual reality, three-dimensional reconstruction, and so on. Camera calibration is an essential step to obtain three- dimensional geometric information from a two- dimensional image. Meanwhile, the omnidirectional camera image has catadioptric distortion, which need to be corrected in many applications, thus the study of such camera calibration method has important theoretical significance and practical applications. This paper firstly introduces the research status of catadioptric omnidirectional imaging system; then the image formation process of catadioptric omnidirectional imaging system has been given; finally a simple classification of omnidirectional imaging method is given, and we discussed the advantages and disadvantages of these methods.

基于编码孔径的折反射全向成像去散焦模糊技术

基于编码孔径的折反射全向成像去散焦模糊技术

Optimization method for ultra-wide-angle and panoramic optical systems

To an ultra-wide-angle and panoramic optical system, the aberrations of point object at any field angle are separated into two types: the aperture-ray aberrations of off-axis point object and the chief-ray aberrations. A simple form of the triangular formulae of tracing an oblique-incidence ray is derived to calculate the chief-ray parameters and their aberrations; moreover, the aperture-ray aberrations of an off-axis point object are analyzed with the plane-symmetric aberration theory. Based on the two types of aberrations, we present a merit function for ultra-wide-angle and panoramic optical systems; the optimization program with the differential-evolution algorithm is then developed. To validate the optimization method we finally optimize a fish-eye lens and a catadioptric omnidirectional imaging system.

去散焦模糊的折反射全向成像系统设计

去散焦模糊的折反射全向成像系统设计

Mirror Invariant Matching between Catadioptric Omnidirectional and Perspective Images

该文针对折反射全向摄像机的不同配置可能使全向图像和透视图像存在的镜像关系，而尺度不变特征变换算法并不对镜像保持不变性，提出将透视图像进行一次水平翻转，然后将原图与翻转后的图像分别与全向图像进行匹配，取效果好的匹配作为最终结果，从而实现镜像不变。针对全向图像的环形畸变，提出先将透视图像变换为扇形图像后再进行匹配。实验结果显示，将透视图像变换为扇形图像后再与全向图像进行匹配，匹配点总数增加，而错误匹配点数减少，匹配结果明显好于变换前的匹配结果。

Difference analysis and registration research of a pair of complementary cylindrical panoramic images

Based on the introduction of the complementary catadioptric omnidirectional imaging system,the difference between correspondence points in two complementary cylindrical panoramic images was analyzed in detail.For the coordinate difference of correspondence points in two images,Harris corner was chosen to search transform model of registration,to solve the registration problem of the two complementary cylindrical panoramic images.The experimental results cover the registration results,precision of using affine model,projective model and polynomial model for registration,which suggest that third-order polynomials model is more suitable.

Automatic 3D Reconstruction from a Single Catadioptric Omnidirectional Image

In this paper we present a method for 3D urban reconstruction from a single catadioptric omnidirectional image. Firstly, we classify the catadioptric omnidirectional image to horizontal ground, vertical building surface and vertical background surface through the registration between catadioptric omnidirectional image and remote sensing image. According to the classification results, we recover the geometry based on the catadioptric projection model. The experiment shows that our method is feasible and realizes a precise 3D reconstruction for the city scenes.

Series–parallel pipeline architecture for high-resolution catadioptric panoramic unwrapping

To implement high-speed panoramic unwrapping of high-resolution catadioptric omnidirectional images on field-programmable gate array (FPGA), a novel design technique of pipeline architecture is proposed, named a ‘series–parallel pipeline’. Based on the strategy of ‘block prefetching’, catadioptric omnidirectional images are divided into image blocks before loading into the pipeline. Multiple functional sub-modules are copied to carry out the relatively time-consuming steps in parallel whereas fewer sub-modules or only one sub-module is created to carry out those less time-consuming steps. The number of copies is determined by the proportion of execution time of each step, and several neighbouring basic units are combined to form a ‘unit package’ before loading into the pipeline. The basic units in one ‘unit package’ are processed in series while carrying out the less time-consuming steps, but processed in parallel while carrying out the more time-consuming steps. A hardware pipeline design of series–parallel architecture is implemented on Xilinx Spartan-3 FPGA, which is able to unwrap one catadioptric omnidirectional image with size 1024×1024 into one cylindrical panorama with size 3200×768 at 12.480 ms per frame when the system clock is 100 MHz.