3D Reconstruction Approach Research Articles

3D Reconstruction of Indoor Scenes via Image Registration

With the development of computer vision technologies, 3D reconstruction has become a hotspot. At present, 3D reconstruction relies heavily on expensive equipment and has poor real-time performance. In this paper, we aim at solving the problem of 3D reconstruction of an indoor scene with large vertical span. In this paper, we propose a novel approach for 3D reconstruction of indoor scenes with only a Kinect. Firstly, this method uses a Kinect sensor to get color images and depth images of an indoor scene. Secondly, the combination of scale-invariant feature transform and random sample consensus algorithm is used to determine the transformation matrix of adjacent frames, which can be seen as the initial value of iterative closest point (ICP). Thirdly, we establish the relative coordinate relation between pair-wise frames which are the initial point cloud data by using ICP. Finally, we achieve the 3D visual reconstruction model of indoor scene by the top-down image registration of point cloud data. This approach not only mitigates the sensor perspective restriction and achieves the indoor scene reconstruction of large vertical span, but also develops the fast algorithm of indoor scene reconstruction with large amount of cloud data. The experimental results show that the proposed algorithm has better accuracy, better reconstruction effect, and less running time for point cloud registration. In addition, the proposed method has great potential applied to 3D simultaneous location and mapping.

Multi-pose 3D facial texture refinement for face recognition

In this paper, we propose a novel approach for 3D face reconstruction from multi-facial images. Given original pose-variant images, coarse 3D face templates are initialized to reconstruct a refined 3D face mesh in an iteration manner. Then, we warp original facial images to the 2D meshes projected from 3D using Sparse Mesh Affine Warp (SMAW). Finally, we weight the face patches in each view respectively and map the patch with higher weight to a canonical UV space. For facial images with arbitrary pose, their invisible regions are filled with the corresponding UV patches. Poisson editing is applied to blend different patches seamlessly. We evaluate the proposed method on LFW dataset in terms of texture refinement and face recognition. The results demonstrate competitive performance compared to state-of-the-art methods.

Recent trends in cultural heritage 3D survey: The photogrammetric computer vision approach

Abstract The techniques of measuring and 3D modelling based on images, as is typical in photogrammetry, grew in interest again in recent years, since a new generation of software tools has spread. These ones implement in different measure the algorithms developed by computer vision, increasing the automation of the standard photogrammetric process. This made the use of image-based approaches for 3D models reconstruction enormously increase, which is an essential part of the Cultural Heritage documentation and analysis processes. Starting from these assumptions, the aim of the paper is to evaluate what and where it is possible nowadays to find the main differences between photogrammetry and computer vision approaches and how these have to be considered in the choice of the processing technique. The analysis has been performed starting from a theoretical point of view in order to trace the main characteristics of the two methods. Moreover, in order to complete the investigation, an experimental part is reported on two particular cases study, considered as representative of two types of usually surveyed objects. The results allow to enlighten some differences between the two image processing approaches, in terms of accuracy and achieved products.

Feasibility study on 3D image reconstruction from 2D orthogonal cine-MRI for MRI-guided radiotherapy.

In-room MRI is a promising image guidance strategy in external beam radiotherapy to acquire volumetric information for moving targets. However, limitations in spatio-temporal resolution led several authors to use 2D orthogonal images for guidance. The aim of this work is to present a method to concurrently compensate for non-rigid tumour motion and provide an approach for 3D reconstruction from 2D orthogonal cine-MRI slices for MRI-guided treatments. Free-breathing sagittal/coronal interleaved 2D cine-MRI were acquired in addition to a pre-treatment 3D volume in two patients. We performed deformable image registration (DIR) between cine-MRI slices and corresponding slices in the pre-treatment 3D volume. Based on an extrapolation of the interleaved 2D motion fields, the 3D motion field was estimated and used to warp the pre-treatment volume. Due to the lack of a ground truth for patients, the method was validated on a digital 4D lung phantom. On the phantom, the 3D reconstruction method was able to compensate for tumour motion and compared favourably to the results of previously adopted strategies. The difference in the 3D motion fields between the phantom and the extrapolated motion was 0.4±0.3mm for tumour and 0.8±1.5mm for whole anatomy, demonstrating feasibility of performing a 3D volumetric reconstruction directly from 2D orthogonal cine-MRI slices. Application of the method to patient data confirmed the feasibility of utilizing this method in real world scenarios. Preliminary results on phantom and patient cases confirm the feasibility of the proposed approach in an MRI-guided scenario, especially for non-rigid tumour motion compensation.

A Low-Cost Approach to Automatically Obtain Accurate 3D Models of Woody Crops.

Crop monitoring is an essential practice within the field of precision agriculture since it is based on observing, measuring and properly responding to inter- and intra-field variability. In particular, “on ground crop inspection” potentially allows early detection of certain crop problems or precision treatment to be carried out simultaneously with pest detection. “On ground monitoring” is also of great interest for woody crops. This paper explores the development of a low-cost crop monitoring system that can automatically create accurate 3D models (clouds of coloured points) of woody crop rows. The system consists of a mobile platform that allows the easy acquisition of information in the field at an average speed of 3 km/h. The platform, among others, integrates an RGB-D sensor that provides RGB information as well as an array with the distances to the objects closest to the sensor. The RGB-D information plus the geographical positions of relevant points, such as the starting and the ending points of the row, allow the generation of a 3D reconstruction of a woody crop row in which all the points of the cloud have a geographical location as well as the RGB colour values. The proposed approach for the automatic 3D reconstruction is not limited by the size of the sampled space and includes a method for the removal of the drift that appears in the reconstruction of large crop rows.

Iterative approach for 3D reconstruction of the femur from un-calibrated 2D radiographic images

Three-dimensional reconstruction of the femur is important for surgical planning in patients with cerebral palsy. This study aimed to reconstruct the three-dimensional femur shape from un-calibrated bi-planar radiographic images using self-calibration to allow for low-dose preoperative planning. The existing self-calibration techniques require anatomical landmarks that are clearly visible on bi-planar images, which are not available on the femur. In our newly developed method, the self-calibration is performed so that the contour of the statistical shape matches the image contour while the statistical shape is concomitantly optimized. The proposed approach uses conventional radiograph systems and can be easily incorporated into existing clinical protocols, as compared to other reconstruction methods.

Multi-view photometric stereo using surface deformation

This paper presents a hybrid approach for 3D reconstruction by fusing photometric stereo and multi-view stereo. The 3D surface is obtained by capturing a set of images taken from different viewpoints under time-varying illuminations. Key factors in the reconstruction process are surface normals that are obtained from photometric stereo. The surface is initialized by integrating the normals and then refined by performing iterative deformations on the initial surface and thereby optimizing image and normal consistency in multiple views. Benefiting from the employment of the deformation approach, we are able to perform image and normal consistency optimization without using matching windows. Instead, always the complete surface is back-projected. This makes the proposed approach much simpler and more robust compared to window-based approaches, which typically require global optimization with constraints on neighboring windows. Experiments on real-world data and ground-truth data show that for diffuse midsized objects without large depth discontinuities our approach improves the accuracy of the reconstructions compared to exiting approaches.

An UAS-assisted multi-sensor approach for 3D modeling and reconstruction of cultural heritage site

Abstract Unmanned Aerial System (UAS) has been widely used to produce highly-precise orthomosaics, Digital Surface Models (DSMs), Digital Terrain Models (DTMs) and 3D models in many applications. UAS is also utilized to document cultural heritage sites using low-cost photogrammetric approach. Particularly, possibility of multi-sensor acquisition provides substantial information about both geometric features and material classification. In this study, a novel methodology using multi-sensor data acquisition is proposed in order to extract and to distinguish material features from UAS-based photogrammetry for the cultural heritages. Sensors which are able to collect visible, thermal and infrared radiations of the electromagnetic spectrum were employed to produce 3D model information of Assus Ancient Theater located in Behramkale Village, Canakkale, Turkey. The results showed that the accuracies of the 3D models were obtained as ±2–3 cm, ±10–15 cm and ±5–7 cm for the digital, thermal and multi-spectral camera systems, respectively. Beside the given high-accurate geometric model, the classification outcomes as a result of the spectral analysis revealed material features in an affordable and efficient way.

A New Approach for Realistic 3D Reconstruction of Planar Surfaces from Laser Scanning Data and Imagery Collected Onboard Modern Low-Cost Aerial Mapping Systems

Over the past few years, accurate 3D surface reconstruction using remotely-sensed data has been recognized as a prerequisite for different mapping, modelling, and monitoring applications. To fulfill the needs of these applications, necessary data are generally collected using various digital imaging systems. Among them, laser scanners have been acknowledged as a fast, accurate, and flexible technology for the acquisition of high density 3D spatial data. Despite their quick accessibility, the acquired 3D data using these systems does not provide semantic information about the nature of scanned surfaces. Hence, reliable processing techniques are employed to extract the required information for 3D surface reconstruction. Moreover, the extracted information from laser scanning data cannot be effectively utilized due to the lack of descriptive details. In order to provide a more realistic and accurate perception of the scanned scenes using laser scanning systems, a new approach for 3D reconstruction of planar surfaces is introduced in this paper. This approach aims to improve the interpretability of the extracted planar surfaces from laser scanning data using spectral information from overlapping imagery collected onboard modern low-cost aerial mapping systems, which are widely adopted nowadays. In this approach, the scanned planar surfaces using laser scanning systems are initially extracted through a novel segmentation procedure, and then textured using the acquired overlapping imagery. The implemented texturing technique, which intends to overcome the computational inefficiency of the previously-developed 3D reconstruction techniques, is performed in three steps. In the first step, the visibility of the extracted planar surfaces from laser scanning data within the collected images is investigated and a list of appropriate images for texturing each surface is established. Successively, an occlusion detection procedure is carried out to identify the occluded parts of these surfaces in the field of view of captured images. In the second step, visible/non-occluded parts of the planar surfaces are decomposed into segments that will be textured using individual images. Finally, a rendering procedure is accomplished to texture these parts using available images. Experimental results from overlapping laser scanning data and imagery collected onboard aerial mapping systems verify the feasibility of the proposed approach for efficient realistic 3D surface reconstruction.

3D precision measurements of meter sized surfaces using low cost illumination and camera techniques

Using dedicated stereo camera systems and structured light is a well-known method for measuring the 3D shape of large surfaces. However the problem is not trivial when high accuracy, in the range of few tens of microns, is needed. Many error sources need to be handled carefully in order to obtain high quality results. In this study, we present a measurement method based on low-cost camera and illumination solutions combined with high-precision image analysis and a new approach in camera calibration and 3D reconstruction. The setup consists of two ordinary digital cameras and a Gobo projector as a structured light source. A matrix of dots is projected onto the target area. The two cameras capture the images of the projected pattern on the object. The images are processed by advanced subpixel resolution algorithms prior to the application of the 3D reconstruction technique. The strength of the method lays in a different approach for calibration, 3D reconstruction, and high-precision image analysis algorithms. Using a 10 mm pitch pattern of the light dots, the method is capable of reconstructing the 3D shape of surfaces. The precision (1σ repeatability) in the measurements is <10 µm over a volume of 60 × 50 × 10 cm3 at a hardware cost of ~2% of available advanced measurement techniques. The expanded uncertainty (95% confidence level) is estimated to be 83 µm, with the largest uncertainty contribution coming from the absolute length of the metal ruler used as reference.

3D reconstruction of buildings from LiDAR data considering various types of roof structures

ABSTRACTIn this article, a different approach has been proposed to detect and reconstruct buildings in 3D space. In this regard, some potentially primary features were produced and a genetic algorithm (GA) was employed in order to select the optimum features. The selected features were utilized to detect the building by using k-nearest neighbour (k-NN) algorithm. The detection results were used as inputs of reconstruction procedure. The proposed approach for 3D reconstruction consists of three main steps: roof planes were separated in the first step. Then, the corners of each plane boundary were extracted in order to provide the roof reconstruction possibility. Finally, the walls were reconstructed and merged to roofs and the final 3D model was obtained. Results evaluation indicated that the average value of buildings detection in the test areas was 87.84% in Quality. Moreover, the average value of buildings reconstruction in the test areas was 76.95% in object based Quality, and 95.66% in Quality of building planes that were greater than 25 m2, respectively. Also, the average of altimetric and planimetric RMS of the test areas were 0.3 m and 0.75 m, respectively.

Beautification and efficiency improvement of 3D modelling with the help of geometric constraints

We emphasise on the influence of geometric constraints in 3D reconstruction. In fact, we have used a new approach of 3D reconstruction which is based on bundle adjustment optimisation and constraints. Our purpose is to achieve 3D reconstruction in a simple, fast and efficient way without any prior information about cameras and 3D space. Constraints use brings us many advantages; it helps us to reduce estimated parameters number and stabilises good results quality. We try as much as possible through a comparative analysis to prove the importance of constraints use, in terms of results reliability, process speed and convergence rate. We use uncalibrated views, so we don't have any pertinent initial values used in Levenberg-Marquardt. On the hand, and thanks to the estimation of cameras' positions with constraints during Levenberg-Marquardt optimisation, we solve with the convergence problem satisfactorily. Several data will be used with the aim to demonstrate the efficiency of our present approach under different uses and kinds of constraints.

Beautification and efficiency improvement of 3D modelling with the help of geometric constraints

We emphasise on the influence of geometric constraints in 3D reconstruction. In fact, we have used a new approach of 3D reconstruction which is based on bundle adjustment optimisation and constraints. Our purpose is to achieve 3D reconstruction in a simple, fast and efficient way without any prior information about cameras and 3D space. Constraints use brings us many advantages; it helps us to reduce estimated parameters number and stabilises good results quality. We try as much as possible through a comparative analysis to prove the importance of constraints use, in terms of results reliability, process speed and convergence rate. We use uncalibrated views, so we don't have any pertinent initial values used in Levenberg-Marquardt. On the hand, and thanks to the estimation of cameras' positions with constraints during Levenberg-Marquardt optimisation, we solve with the convergence problem satisfactorily. Several data will be used with the aim to demonstrate the efficiency of our present approach under different uses and kinds of constraints.

Cell-based approach for 3D reconstruction of lymphatic capillaries in vitro reveals distinct functions of HGF and VEGF-C in lymphangiogenesis

Regeneration of lymphatic vessels is important for treatment of various disorders of lymphatic system and for restoration of lymphatic function after surgery. We have developed a method for generating a human 3D lymphatic vascular construct. In this system, human lymphatic endothelial cells, co-cultured with fibroblasts, spontaneously organized into a stable 3D lymphatic capillary network without the use of any exogenous factors. In vitro-generated lymphatic capillaries exhibited the major molecular and ultra-structural features of native, human lymphatic microvasculature: branches in the three dimensions, wide lumen, blind ends, overlapping borders, adherens and tight junctions, anchoring filaments, lack of mural cells, and poorly developed basement membrane. Furthermore, we show that fibroblast-derived VEGF-C and HGF cooperate in the formation of lymphatic vasculature by activating ERK1/2 signaling, and demonstrate distinct functions of HGF/c-Met and VEGF-C/VEGFR-3 in lymphangiogenesis. This lymphatic vascular construct is expected to facilitate studies of lymphangiogenesis in vitro and it holds promise as a strategy for regeneration of lymphatic vessels and treatment of lymphatic disorders in various conditions.

Gaze strategies while negotiating obstacles in a virtual environment with distractors

s / Gait & Posture 42S (2015) S1–S27 S3 Table 1 Morphological and structural features in CP and TD children. Age [years] Muscle volume [cm3] Muscle length [cm] MTU [cm] Average muscle area [cm2] Normalized Muscle volume [cm2] Mean EI of muscle CP 8.6±2.3 32.4±13.8 16.2±3.9 33.5±6.0 2.1±0.5 1.1±0.3 101.0±9.7 TD 9.2±2.3 52.2±19.0 17.3±2.7 34.0±4.9 2.9±0.7 1.5±0.4 129.2±20.3 t-Test (p-value) 0.740 0.103 0.331 0.566 0.018* 0.040* 0.006* volume (=volume/MTU length) were calculated. In addition, the mean EI value (expressed in a grey-scale of 256 values), was also evaluated. Comparisons were made using the Student t-test (unpaired). Statistical significance (*) was set to p 0.79, p>0.01), but with a lower slope in CP (5.2 vs. 7.8 cm3/years). In CP, a significant negative correlation was found between EI and average muscle area (R2 =0.60, p=0.04). Discussion: We found a smaller muscle volume in CP children, as previously reported [2]. Interestingly, as a CP child ages, slower muscle growth occurs than in the TD children. Averaged muscle area was found to be more sensitive to distinguish between TD and CP children. Furthermore, the MG was hyper-echoic in the CP group relative to the TD group, supporting our hypothesis. Combining EI and average muscle area shows potential in estimating the amount of fibrotic changes occurring within a muscle. Utilising the new approach for 3D reconstruction offers an objective and efficient manner to diagnose muscle and tendon morphological and structural changes in the clinical environment.

Binocular vision calibration and 3D re-construction with an orthogonal learning neural network

A new approach for binocular vision system calibration and 3D re-construction is proposed. While the system is calibrated, the sum of square distances between the vector coordinates recombined with...

A MPB-based remote sensing image 3D reconstruction system

This paper investigates the 3D reconstruction of massive remote sensing images. We propose a novel reconstruction framework (MPB) that explicitly decomposes the reconstruction task into three parts, which include image mosaic, camera pose estimation and bundle adjustment. Image mosaic includes feature detection, image matching, image registration. All remote sensing images will be registrated into a plane to establish the plane model of terrain. Then we use EPnP algorithms by RANSAC filter to estimate the camera pose, and use iterative method to get best camera pose. Bundle adjustment the camera poses and the terrain coordinates to complete the terrain 3D reconstruction. The above ideas are correct and feasible, which are measured by massive data of different terrain. The results show that this method can greatly improve the reconstruction efficiency, which provides a fast and efficient approach for 3D reconstruction of massive remote sensing images.

3D color reconstruction based on underwater RGB laser line scanning system

Laser line scanning (LLS) as an optical 3D survey method is recently used for underwater 3D imaging. This paper presents an approach for both 3D reconstruction and color restoration based on the LLS system. The system is set up with RGB laser scanning components for acquiring 3D data and color data at the same time. Direct calibration algorithms and RGB additive color model are involved for processing 3D color data. The experiment result shows that underwater targets can be feasibly 3D reconstructed with color texture simultaneously, at millimeter scales.

A Two-Stage Framework for 3D Face Reconstruction from RGBD Images.

This paper proposes a new approach for 3D face reconstruction with RGBD images from an inexpensive commodity sensor. The challenges we face are: 1) substantial random noise and corruption are present in low-resolution depth maps; and 2) there is high degree of variability in pose and face expression. We develop a novel two-stage algorithm that effectively maps low-quality depth maps to realistic face models. Each stage is targeted toward a certain type of noise. The first stage extracts sparse errors from depth patches through the data-driven local sparse coding, while the second stage smooths noise on the boundaries between patches and reconstructs the global shape by combining local shapes using our template-based surface refinement. Our approach does not require any markers or user interaction. We perform quantitative and qualitative evaluations on both synthetic and real test sets. Experimental results show that the proposed approach is able to produce high-resolution 3D face models with high accuracy, even if inputs are of low quality, and have large variations in viewpoint and face expression.

DATA FUSION FOR BUILDING RECONSTRUCTION FROM MULTI-ASPECT INSAR DATA

Abstract. In this paper we present a novel approach for 3D reconstruction of point clouds based on single baseline Multi-Aspect InSAR (MASAR) data. The point clouds represent an intermediate result to achieve a comprehensive building reconstruction framework. The exact determination of scatterers based on SAR data is a non-trivial task since the optimal solution requires the knowledge of the number of scatterers within one range cell. In recent years many methods were proposed addressing this problem but most of them require multiple observations making them inapplicable to our task. We use a Probabilistic Graphical Model (PGM) to combine all aspects into a common framework exploiting all contradictions and redundancy in the data. The model is used iteratively together with local optimizations adjusting the hypothesis of the scatterer within one range cell to the corresponding observation. This makes it possible to find a global solution.