Dense 3D Model Research Articles

Sparse2Dense: From Direct Sparse Odometry to Dense 3-D Reconstruction

In this paper, we proposed a new deep learning based dense monocular SLAM method. Compared to existing methods, the proposed framework constructs a dense 3D model via a sparse to dense mapping using learned surface normals. With single view learned depth estimation as prior for monocular visual odometry, we obtain both accurate positioning and high quality depth reconstruction. The depth and normal are predicted by a single network trained in a tightly coupled manner.Experimental results show that our method significantly improves the performance of visual tracking and depth prediction in comparison to the state-of-the-art in deep monocular dense SLAM.

Real-time dense map fusion for stereo SLAM

SUMMARYA robot should be able to estimate an accurate and dense 3D model of its environment (a map), along with its pose relative to it, all of it in real time, in order to be able to navigate autonomously without collisions.As the robot moves from its starting position and the estimated map grows, the computational and memory footprint of a dense 3D map increases and might exceed the robot capabilities in a short time. However, a global map is still needed to maintain its consistency and plan for distant goals, possibly out of the robot field of view.In this work, we address such problem by proposing a real-time stereo mapping pipeline, feasible for standard CPUs, which is locally dense and globally sparse and accurate. Our algorithm is based on a graph relating poses and salient visual points, in order to maintain a long-term accuracy with a small cost. Within such framework, we propose an efficient dense fusion of several stereo depths in the locality of the current robot pose.We evaluate the performance and the accuracy of our algorithm in the public datasets of Tsukuba and KITTI, and demonstrate that it outperforms single-view stereo depth. We release the code as open-source, in order to facilitate the system use and comparisons.

3D Occupancy Mapping Framework Based on Acoustic Camera in Underwater Environment

Abstract In this paper, we present a novel probabilistic three-dimensional (3D) mapping framework that uses acoustic images captured in an underwater environment. Acoustic camera is a forward-looking imaging sonar that is commonly used in underwater inspection recently; however, the loss of elevation angle information makes it difficult to get a better understanding of underwater environment. To cope with this, we apply a probabilistic occupancy mapping framework with a novel inverse sensor model suitable for the acoustic camera in order to reconstruct the underwater environment in volumetric presentation. The simulations and experimental results demonstrate that our mapping framework for the acoustic camera can reconstruct dense 3D model of underwater targets successfully.

Face Alignment in Full Pose Range: A 3D Total Solution.

Face alignment, which fits a face model to an image and extracts the semantic meanings of facial pixels, has been an important topic in the computer vision community. However, most algorithms are designed for faces in small to medium poses (yaw angle is smaller than 45 degree), which lack the ability to align faces in large poses up to 90 degree. The challenges are three-fold. First, the commonly used landmark face model assumes that all the landmarks are visible and is therefore not suitable for large poses. Second, the face appearance varies more drastically across large poses, from the frontal view to the profile view. Third, labelling landmarks in large poses is extremely challenging since the invisible landmarks have to be guessed. In this paper, we propose to tackle these three challenges in an new alignment framework termed 3D Dense Face Alignment (3DDFA), in which a dense 3D Morphable Model (3DMM) is fitted to the image via Cascaded Convolutional Neural Networks. We also utilize 3D information to synthesize face images in profile views to provide abundant samples for training. Experiments on the challenging AFLW database show that the proposed approach achieves significant improvements over the state-of-the-art methods.

3D MODELING WITH PHOTOGRAMMETRY BY UAVS AND MODEL QUALITY VERIFICATION

Abstract. This paper deals with a test lead by Geomatics laboratory (DICEAM, Mediterranea University of Reggio Calabria), concerning the application of UAV photogrammetry for survey, monitoring and checking. The study case relies with the surroundings of the Department of Agriculture Sciences. In the last years, such area was interested by landslides and survey activities carried out to take the phenomenon under control. For this purpose, a set of digital images were acquired through a UAV equipped with a digital camera and GPS. Successively, the processing for the production of a 3D georeferenced model was performed by using the commercial software Agisoft PhotoScan. Similarly, the use of a terrestrial laser scanning technique allowed to product dense cloud and 3D models of the same area. To assess the accuracy of the UAV-derived 3D models, a comparison between image and range-based methods was performed.

Using 3D face priors for depth recovery

For repairing inaccurate depth measurements from commodity RGB-D sensors, existing depth recovery methods primarily rely on low-level and rigid prior information. However, as the depth quality deteriorates, the recovered depth maps become increasingly unreliable, especially for non-rigid objects. Thus, additional high-level and non-rigid information is needed to improve the recovery quality. Taking as a starting point the human face that is the primary prior available in many high-level tasks, in this paper, we incorporate face priors into the depth recovery process. In particular, we propose a joint optimization framework that consists of two main steps: transforming the face model for better alignment and applying face priors for improved depth recovery. Face priors from both sparse and dense 3D face models are studied. By comparing with the baseline method on benchmark datasets, we demonstrate that the proposed method can achieve up to 23.8% improvement in depth recovery with more accurate face registrations, bringing inspirations to both non-rigid object modeling and analysis.

GENERATING ACCURATE 3D MODELS OF ARCHITECTURAL HERITAGE STRUCTURES USING LOW-COST CAMERA AND OPEN SOURCE ALGORITHMS

Abstract. These studies have been conductedusing non-metric digital camera and dense image matching algorithms, as non-contact methods of creating monuments documentation.In order toprocess the imagery, few open-source software and algorithms of generating adense point cloud from images have been executed. In the research, the OSM Bundler, VisualSFM software, and web application ARC3D were used. Images obtained for each of the investigated objects were processed using those applications, and then dense point clouds and textured 3D models were created. As a result of post-processing, obtained models were filtered and scaled.The research showedthat even using the open-source software it is possible toobtain accurate 3D models of structures (with an accuracy of a few centimeters), but for the purpose of documentation and conservation of cultural and historical heritage, such accuracy can be insufficient.

LOW-COST 3D LASER SCANNING IN AIR ORWATER USING SELF-CALIBRATING STRUCTURED LIGHT

Abstract. In-situ calibration of structured light scanners in underwater environments is time-consuming and complicated. This paper presents a self-calibrating line laser scanning system, which enables the creation of dense 3D models with a single fixed camera and a freely moving hand-held cross line laser projector. The proposed approach exploits geometric constraints, such as coplanarities, to recover the depth information and is applicable without any prior knowledge of the position and orientation of the laser projector. By employing an off-the-shelf underwater camera and a waterproof housing with high power line lasers an affordable 3D scanning solution can be built. In experiments the performance of the proposed technique is studied and compared with 3D reconstruction using explicit calibration. We demonstrate that the scanning system can be applied to above-the-water as well as underwater scenes.

Modeling large-scale indoor scenes with rigid fragments using RGB-D cameras

Hand-held consumer depth cameras have become a commodity tool for constructing 3D models of indoor environments in real time. Recently, many methods to fuse low quality depth images into a single dense and high fidelity 3D model have been proposed. Nonetheless, dealing with large-scale scenes remains a challenging problem. In particular, the accumulation of small errors due to imperfect camera localization becomes crucial (at large scale) and results in dramatic deformations of the built 3D model. These deformations have to be corrected whenever it is possible (when a loop exists for example). To facilitate such correction, we use a structured 3D representation where points are clustered into several planar patches that compose the scene. We then propose a two-stage framework to build in details and in real-time a large-scale 3D model. The first stage (the local mapping) generates local structured 3D models with rigidity constraints from short subsequences of RGB-D images. The second stage (the global mapping) aggregates all local 3D models into a single global model in a geometrically consistent manner. Minimizing deformations of the global model reduces to re-positioning the planar patches of the local models thanks to our structured 3D representation. This allows efficient, yet accurate computations. Our experiments using real data confirm the effectiveness of our proposed method.

Parametric Surface Representation with Bump Image for Dense 3D Modeling Using an RBG-D Camera

When constructing a dense 3D model of an indoor static scene from a sequence of RGB-D images, the choice of the 3D representation (e.g. 3D mesh, cloud of points or implicit function) is of crucial ...

Dense Semantic 3D Reconstruction.

Both image segmentation and dense 3D modeling from images represent an intrinsically ill-posed problem. Strong regularizers are therefore required to constrain the solutions from being 'too noisy'. These priors generally yield overly smooth reconstructions and/or segmentations in certain regions while they fail to constrain the solution sufficiently in other areas. In this paper, we argue that image segmentation and dense 3D reconstruction contribute valuable information to each other's task. As a consequence, we propose a mathematical framework to formulate and solve a joint segmentation and dense reconstruction problem. On the one hand knowing about the semantic class of the geometry provides information about the likelihood of the surface direction. On the other hand the surface direction provides information about the likelihood of the semantic class. Experimental results on several data sets highlight the advantages of our joint formulation. We show how weakly observed surfaces are reconstructed more faithfully compared to a geometry only reconstruction. Thanks to the volumetric nature of our formulation we also infer surfaces which cannot be directly observed for example the surface between the ground and a building. Finally, our method returns a semantic segmentation which is consistent across the whole dataset.

3D model reconstruction with common hand-held cameras

A 3D model reconstruction workflow with hand-held cameras is developed. The exterior and interior orientation models combined with the state-of-the-art structure from motion and multi-view stereo techniques are applied to extract dense point cloud and reconstruct 3D model from digital images. An overview of the presented 3D model reconstruction methods is given. The whole procedure including tie point extraction, relative orientation, bundle block adjustment, dense point production and 3D model reconstruction is all reviewed in brief. Among them, we focus on bundle block adjustment procedure; the mathematical and technical details of bundle block adjustment are introduced and discussed. Finally, four scenes of images collected by hand-held cameras are tested in this paper. The preliminary results have shown that sub-pixel (<1 pixel) accuracy can be achieved with the proposed exterior---interior orientation models and satisfactory 3D models can be reconstructed using images collected by hand-held cameras. This work can be applied in indoor navigation, crime scene reconstruction, heritage reservation and other applications in geosciences.

PHOTOGRAMMETRIC TECHNIQUES FOR ROAD SURFACE ANALYSIS

The quality and condition of a road surface is of great importance for convenience and safety of driving. So the investigations of the behaviour of road materials in laboratory conditions and monitoring of existing roads are widely fulfilled for controlling a geometric parameters and detecting defects in the road surface. Photogrammetry as accurate non-contact measuring method provides powerful means for solving different tasks in road surface reconstruction and analysis. The range of dimensions concerned in road surface analysis can have great variation from tenths of millimetre to hundreds meters and more. So a set of techniques is needed to meet all requirements of road parameters estimation. Two photogrammetric techniques for road surface analysis are presented: for accurate measuring of road pavement and for road surface reconstruction based on imagery obtained from unmanned aerial vehicle. The first technique uses photogrammetric system based on structured light for fast and accurate surface 3D reconstruction and it allows analysing the characteristics of road texture and monitoring the pavement behaviour. The second technique provides dense 3D model road suitable for road macro parameters estimation.

PHOTOGRAMMETRIC TECHNIQUES FOR ROAD SURFACE ANALYSIS

Abstract. The quality and condition of a road surface is of great importance for convenience and safety of driving. So the investigations of the behaviour of road materials in laboratory conditions and monitoring of existing roads are widely fulfilled for controlling a geometric parameters and detecting defects in the road surface. Photogrammetry as accurate non-contact measuring method provides powerful means for solving different tasks in road surface reconstruction and analysis. The range of dimensions concerned in road surface analysis can have great variation from tenths of millimetre to hundreds meters and more. So a set of techniques is needed to meet all requirements of road parameters estimation. Two photogrammetric techniques for road surface analysis are presented: for accurate measuring of road pavement and for road surface reconstruction based on imagery obtained from unmanned aerial vehicle. The first technique uses photogrammetric system based on structured light for fast and accurate surface 3D reconstruction and it allows analysing the characteristics of road texture and monitoring the pavement behaviour. The second technique provides dense 3D model road suitable for road macro parameters estimation.

Three dimension reconstruction through measure-based image selection

Three dimension (3D) reconstruction is one of the research focus of computer vision and widely applied in various fields. The main steps of 3D reconstruction include image acquisition, feature point extraction and matching, camera calibration and production of dense 3D scene models. Generally, not all the input images are useful for camera calibration because some images contain similar and redundant visual information. These images can even reduce the calibration accuracy. In this paper, we propose an effective image selection method to improve the accuracy of camera calibration. Then a new 3D reconstruction algorithm is proposed by adding the image selection step to 3D reconstruction. The image selection method uses structure-from-motion algorithm to estimate the position and attitude of each camera, first. Then the contributed value to 3D reconstruction of each image is calculated. Finally, images are selected according to the contributed value of each image and their effects on the contributed values of other images. Experimental results show that our image selection algorithm can improve the accuracy of camera calibration and the 3D reconstruction algorithm proposed in this paper can get better dense 3D models than the normal algorithm without image selection.

High Accuracy Photogrammetry of Historic Rock Art

The petroglyphs of Petroglyph Park, Peterborough, created by the Algonquin Peoples between 900 and 1400 A.D., were documented in 1983 by the Heritage Recording Directorate of theGovernment of Canada. With the aim of rerecording the glyphs again at a later date to monitor the conditions and weathering, several sets of photogrammetric stereo pairs were taken of the site using Zeiss UMK and a Wild P-31 film cameras. After this project was completed, the site became designated as sacred and photography was no longer permitted, thus rendering the completion of a second recording of the site all but impossible. Therefore, the photographs taken of these magnificent petroglyphs in the 1980s are the most recent documentation available. Using the ADAMTech Mine Mapping Suite, developed in Perth Australia for the mining industry, I was able to bring these archival photos to life by creating dense 3D models that rival those produced by LiDAR. I used the photos, digitized in Ottawa on a Wehri RM-6 photogrammetric scanner, to create 3D models of the glyphs. A similar Federal project from Writing on Stone National Park in Alberta also used film photogrammetry to record the glyphs at that site in 1982. From these images as well I was able to compile 3D models. It is hoped that by scanning the original glass-plate negatives from the 1980s, and not the film copies, as we have done thus far for both projects, measurement data of even greater accuracy (down to 60um) and density can be achieved.

3D SURVEYING &amp; MODELING OF UNDERGROUND PASSAGES IN WWI FORTIFICATIONS

Abstract. The virtual reconstruction of subterranean structures is a suitable scenario for the integration of different geomatics techniques although narrow passages, lack of light and irregular surface can arise various problems in the data acquisition as well as processing procedures. Generally the final product is a dense and detailed 3D model, whose number of triangles increases quickly according to the complexity of the object. This complexity reduces the efficient use and dissemination of the produced information therefore innovative solutions are sought. The article presents the 3D surveying and modelling of underground passages of World War I (WWI) fortifications. After the acquisition of dense point clouds by means of terrestrial scanning (TLS), a simplification and optimization workflow is performed with the aim of generating a lightweight product that keeps the maximum amount of significant information. A continuous scene representation with a 87% triangle reduction is generated, while the final precision is preserved according to a tolerance predefined by the final user. Such 3D product can be employed as basis for reconstruction, consolidation, preservation and valorisation of the WWI tunnels.

Aerial multi-camera systems: Accuracy and block triangulation issues

Oblique photography has reached its maturity and has now been adopted for several applications. The number and variety of multi-camera oblique platforms available on the market is continuously growing. So far, few attempts have been made to study the influence of the additional cameras on the behaviour of the image block and comprehensive revisions to existing flight patterns are yet to be formulated. This paper looks into the precision and accuracy of 3D points triangulated from diverse multi-camera oblique platforms. Its coverage is divided into simulated and real case studies. Within the simulations, different imaging platform parameters and flight patterns are varied, reflecting both current market offerings and common flight practices. Attention is paid to the aspect of completeness in terms of dense matching algorithms and 3D city modelling – the most promising application of such systems. The experimental part demonstrates the behaviour of two oblique imaging platforms in real-world conditions. A number of Ground Control Point (GCP) configurations are adopted in order to point out the sensitivity of tested imaging networks and arising block deformations. To stress the contribution of slanted views, all scenarios are compared against a scenario in which exclusively nadir images are used for evaluation.

Point cloud-based survey for cultural heritage – An experience of integrated use of range-based and image-based technology for the San Francesco convent in Monterubbiano

Abstract. The paper aims at presenting some results of a point cloud-based survey carried out through integrated methodologies based on active and passive 3D acquisition techniques for processing 3D models. This experiment is part of a research project still in progress conducted by an interdisciplinary team from the School of Architecture and Design of Ascoli Piceno and funded by the University of Camerino. We describe an experimentation conducted on the convent of San Francesco located in Monterubbiano town center (Marche, Italy). The whole complex has undergone a number of substantial changes since the year of its foundation in 1247. The survey was based on an approach blending range-based 3D data acquired by a TOF laser scanner and image-based 3D acquired using an UAV equipped with digital camera in order to survey some external parts difficult to reach with TLS. The integration of two acquisition methods aimed to define a workflow suitable to process dense 3D models from which to generate high poly and low poly 3D models useful to describe complex architectures for different purposes such as photorealistic representations, historical documentation, risk assessment analyses based on Finite Element Methods (FEM).

Autonomous Exploration of Large Unknown Indoor Environments for Dense 3D Model Building

Autonomous exploration and mapping of indoor environments is important task for building inspections. Mapping of large environments in 3D requires high memory and computational consumptions. In this paper, we present a 3D exploration strategy for a mobile robot equipped with a 3D laser scanner. Our strategy does not require a map of the environment and ensures on-line exploration of large unknown spaces. We propose a room detection algorithm and focus on the room-by-room exploration keeping the memory and computational requirements low. We evaluated our strategy by simulations and experimentally using a real mobile robot.