Single Omnidirectional Image Research Articles

Stripe Sensitive Convolution for Omnidirectional Image Dehazing.

The haze in a scenario may affect the 360 photo/video quality and the immersive 360 ° virtual reality (VR) experience. The recent single image dehazing methods, to date, have been only focused on plane images. In this work, we propose a novel neural network pipeline for single omnidirectional image dehazing. To create the pipeline, we build the first hazy omnidirectional image dataset, which contains both synthetic and real-world samples. Then, we propose a new stripe sensitive convolution (SSConv) to handle the distortion problems due to the equirectangular projections. The SSConv calibrates distortion in two steps: 1) extracting features using different rectangular filters and, 2) learning to select the optimal features by a weighting of the feature stripes (a series of rows in the feature maps). Subsequently, using SSConv, we design an end-to-end network that jointly learns haze removal and depth estimation from a single omnidirectional image. The estimated depth map is leveraged as the intermediate representation and provides global context and geometric information to the dehazing module. Extensive experiments on challenging synthetic and real-world omnidirectional image datasets demonstrate the effectiveness of SSConv, and our network attains superior dehazing performance. The experiments on practical applications also demonstrate that our method can significantly improve the 3-D object detection and 3-D layout performances for hazy omnidirectional images.

Deep scene synthesis of Atlanta-world interiors from a single omnidirectional image.

We present a new data-driven approach for extracting geometric and structural information from a single spherical panorama of an interior scene, and for using this information to render the scene from novel points of view, enhancing 3D immersion in VR applications. The approach copes with the inherent ambiguities of single-image geometry estimation and novel view synthesis by focusing on the very common case of Atlanta-world interiors, bounded by horizontal floors and ceilings and vertical walls. Based on this prior, we introduce a novel end-to-end deep learning approach to jointly estimate the depth and the underlying room structure of the scene. The prior guides the design of the network and of novel domain-specific loss functions, shifting the major computational load on a training phase that exploits available large-scale synthetic panoramic imagery. An extremely lightweight network uses geometric and structural information to infer novel panoramic views from translated positions at interactive rates, from which perspective views matching head rotations are produced and upsampled to the display size. As a result, our method automatically produces new poses around the original camera at interactive rates, within a working area suitable for producing depth cues for VR applications, especially when using head-mounted displays connected to graphics servers. The extracted floor plan and 3D wall structure can also be used to support room exploration. The experimental results demonstrate that our method provides low-latency performance and improves over current state-of-the-art solutions in prediction accuracy on available commonly used indoor panoramic benchmarks.

Geometric-driven structure recovery from a single omnidirectional image based on planar depth map learning

Geometric-driven structure recovery from a single omnidirectional image based on planar depth map learning

Parallel Lines for Calibration of Non-Central Conical Catadioptric Cameras

In this paper we propose a new calibration method for non-central catadioptric cameras that use a conical mirror. This method consists of using parallel lines, extracted from a single omnidirectional image, instead of using the typical checkerboard to obtain the calibration parameters of the system.

Monocular spherical depth estimation with explicitly connected weak layout cues

Spherical cameras capture scenes in a holistic manner and have been used for room layout estimation. Recently, with the availability of appropriate datasets, there has also been progress in depth estimation from a single omnidirectional image. While these two tasks are complementary, few works have been able to explore them in parallel to advance indoor geometric perception, and those that have done so either relied on synthetic data, or used small scale datasets, as few options are available that include both layout annotations and dense depth maps in real scenes. This is partly due to the necessity of manual annotations for room layouts. In this work, we move beyond this limitation and generate a 360° geometric vision (360V) dataset that includes multiple modalities, multi-view stereo data and automatically generated weak layout cues. We also explore an explicit coupling between the two tasks to integrate them into a single-shot trained model. We rely on depth-based layout reconstruction and layout-based depth attention, demonstrating increased performance across both tasks. By using single 360° cameras to scan rooms, the opportunity for facile and quick building-scale 3D scanning arises. The project page is available at https://vcl3d.github.io/ExplicitLayoutDepth/.

Spatial layout recovery from a single omnidirectional image and its matching-free sequential propagation

The goal of this work is to recover the spatial layout of indoor environments from omnidirectional images assuming a Manhattan world structure. We propose a new method for scene structure recovery from a single image. This method is based on the line extraction for omnidirectional images, line classification, and vanishing points estimation combined with a new hierarchical expansion procedure for detecting floor and wall boundaries. Each single omnidirectional image independently provides a useful hypothesis of the 3D scene structure. In order to enhance the robustness and accuracy of this single image-based hypothesis, we extend this estimation with a new homography-based procedure applied to the various hypotheses obtained along the sequence of consecutive images. A key point in this contribution is the use of geometrical constraints for computing the homographies from a single line of the floor. The homography parametrization proposed allows the design of a matching-free method for spatial layout propagation along a sequence of images. Experimental results show single image layout recovery performance and the improvement obtained with the propagation of the hypothesis through the image sequence.

Background Estimation for a Single Omnidirectional Image Sequence Captured with a Moving Camera

Background Estimation for a Single Omnidirectional Image Sequence Captured with a Moving Camera

Background Estimation for a Single Omnidirectional Image Sequence Captured with a Moving Camera

This paper proposes a background estimation method from a single omnidirectional image sequence for removing undesired regions such as moving objects, specular regions, and uncaptured regions caused by the cam- era's blind spot without manual specification. The proposed method aligns multiple frames using a reconstructed 3D model of the environment and generates background images by minimizing an energy function for selecting a frame for each pixel. In the energy function, we introduce patch similarity and camera positions to remove undesired regions more correctly and generate high-resolution images. In experiments, we demonstrate the effectiveness of the proposed method by comparing the result given by the proposed method with those from conventional approaches.

A 2-point algorithm for 3D reconstruction of horizontal lines from a single omni-directional image

Reconstruction of 3D scenes with abundant straight line features has many applications in computer vision and robot navigation. Most approaches to this problem involve stereo techniques, in which a solution to the correspondence problem between at least two different images is required. In contrast, 3D reconstruction of straight horizontal lines from a single 2D omni-directional image is studied in this paper. The authors show that, for symmetric non-central catadioptric systems, a 3D horizontal line can be estimated using only two points extracted from a single image of the line. One of the two points is the symmetry point of the image curve of horizontal line, and the other is a generic point on the image curve. This paper improves on several prior works, including horizontal line detection in omni-directional image and line reconstruction from four viewing rays, but is simpler than those methods while being more robust. We evaluate how the precision of feature point extraction can affect line reconstruction accuracy, and discuss preliminary experimental results.

Novel method of space horizontal line localization based on single catadioptric omnidirectional image

Space horizontal line localization without prior knowledge from a single omnidirectional image was exploited.Based on the existing approaches of straight line localization,the authors demonstrated that,for symmetric non-central catadioptric systems,the equation of a 3D horizontal line could be estimated using only two points extracted from a single image of the line by exploiting the characteristics of horizontal line image in catadioptric system,and meanwhile a horizontal line reconstruction algorithm based on main-point image and non-main-point image was proposed.The experimental results justify that,under different precision of image point extraction,compared to present approaches of line reconstruction,the proposed method is constantly more efficient.

Catadioptric single-shot rangefinder for textured map building in robot navigation

The paper focuses on combining the advantages of both omnidirectional vision and structured light in order to obtain panoramic depth information for generating a complete 3D visual model of the surroundings of a mobile robot. The model is completed by mapping the imaged texture onto the objects through the central view point. This process is called single-shot scene reconstruction because a visual model of the scene is obtained from a single omnidirectional image. Several such reconstructions are then registered to build a 3D map of the scene while the robot is moving. The authors call this process range scan registration modelling. A mobile platform provided with a catadioptric camera coupled with an omnidirectional structured light projector was used for this purpose. Sensor calibration, image processing and experimental results are included.

단일 전방향 영상을 이용한 공간 정보의 측정

This paper shows the calculation of 3D geometric information such as height, direction and distance under the constraints of a catadioptric camera system. The catadioptric camera system satisfies the single viewpoint constraints adopting hyperboloidal mirror. To calculate the 3D information with a single omni-directional image, the points are assumed to lie in perpendicular to the ground. The infinite plane is also detected as a circle from the structure of the mirror and camera. The analytic experiments verify the correctness of theory using real images taken in indoor environments like rooms or corridors. Thus, the experimental results show the applicability to calculate the 3D geometric information using single omni-directional images.