Automatic 3D Modeling Research Articles

DEEP LEARNING FOR LOWTEXTURED IMAGE MATCHING

Abstract. Low-textured objects pose challenges for an automatic 3D model reconstruction. Such objects are common in archeological applications of photogrammetry. Most of the common feature point descriptors fail to match local patches in featureless regions of an object. Hence, automatic documentation of the archeological process using Structure from Motion (SfM) methods is challenging. Nevertheless, such documentation is possible with the aid of a human operator. Deep learning-based descriptors have outperformed most of common feature point descriptors recently. This paper is focused on the development of a new Wide Image Zone Adaptive Robust feature Descriptor (WIZARD) based on the deep learning. We use a convolutional auto-encoder to compress discriminative features of a local path into a descriptor code. We build a codebook to perform point matching on multiple images. The matching is performed using the nearest neighbor search and a modified voting algorithm. We present a new “Multi-view Amphora” (Amphora) dataset for evaluation of point matching algorithms. The dataset includes images of an Ancient Greek vase found at Taman Peninsula in Southern Russia. The dataset provides color images, a ground truth 3D model, and a ground truth optical flow. We evaluated the WIZARD descriptor on the “Amphora” dataset to show that it outperforms the SIFT and SURF descriptors on the complex patch pairs.

A Knowledge Base for Automatic Feature Recognition from Point Clouds in an Urban Scene

LiDAR technology can provide very detailed and highly accurate geospatial information on an urban scene for the creation of Virtual Geographic Environments (VGEs) for different applications. However, automatic 3D modeling and feature recognition from LiDAR point clouds are very complex tasks. This becomes even more complex when the data is incomplete (occlusion problem) or uncertain. In this paper, we propose to build a knowledge base comprising of ontology and semantic rules aiming at automatic feature recognition from point clouds in support of 3D modeling. First, several modules for ontology are defined from different perspectives to describe an urban scene. For instance, the spatial relations module allows the formalized representation of possible topological relations extracted from point clouds. Then, a knowledge base is proposed that contains different concepts, their properties and their relations, together with constraints and semantic rules. Then, instances and their specific relations form an urban scene and are added to the knowledge base as facts. Based on the knowledge and semantic rules, a reasoning process is carried out to extract semantic features of the objects and their components in the urban scene. Finally, several experiments are presented to show the validity of our approach to recognize different semantic features of buildings from LiDAR point clouds.

COMPARISON OF UAS-BASED PHOTOGRAMMETRY SOFTWARE FOR 3D POINT CLOUD GENERATION: A SURVEY OVER A HISTORICAL SITE

Abstract. Nowadays, Unmanned Aerial System (UAS)-based photogrammetry offers an affordable, fast and effective approach to real-time acquisition of high resolution geospatial information and automatic 3D modelling of objects for numerous applications such as topography mapping, 3D city modelling, orthophoto generation, and cultural heritages preservation. In this paper, the capability of four different state-of-the-art software packages as 3DSurvey, Agisoft Photoscan, Pix4Dmapper Pro and SURE is examined to generate high density point cloud as well as a Digital Surface Model (DSM) over a historical site. The main steps of this study are including: image acquisition, point cloud generation, and accuracy assessment. The overlapping images are first captured using a quadcopter and next are processed by different software to generate point clouds and DSMs. In order to evaluate the accuracy and quality of point clouds and DSMs, both visual and geometric assessments are carry out and the comparison results are reported.

Fast, automated 3D modeling of building interiors

Fast, automated 3D modeling of building interiors

3D roof model generation and analysis supporting solar system positioning

Given the growing needs of renewable energy in urban areas, identifying suitable installation locations for solar systems is increasingly important. Existing energetic cadastres often have a limited level of detail in the analysed geometry, which needs to be rapidly updated following any changes to the buildings. The required 3D data can be generated by photogrammetry techniques and Structure-from-Motion (SfM) software. In this paper, the method was tested employing Digital Mapping Camera images (virtual images), on a case study in northwest Italy. The results are analysed for assessing the reliability of the DSM produced by SfM methods, to be employed in solar radiation analyses. The problem of this kind of automatic 3D modelling can be the excessive detail sometimes required in reconstructing the geometry of the roofs and the possible noise. The results are managed and analysed through Geographical Information System (GIS) tools. For the whole workflow, proprietary and free open source software (OSS) are used. The problems and limitations are investigated in order to evaluate and confirm the reliability and cost effectiveness of the described methods.

RETRACTED ARTICLE: Multipoint infrared laser-based detection and tracking for people counting

Laser devices have received increasing attention in numerous computer-aided applications such as automatic control, 3D modeling and virtual reality. In this paper, aiming at people counting, we propose a novel people detection and tracking method based on the multipoint infrared laser, which can further facilitate intelligent scene modeling and analysis. In our method, a camera with the infrared lens filter is utilized to capture the monitored scene where an array of infrared spots is produced by the multipoint infrared laser. We build a spatial background model based on locations of spots. Pedestrians are detected by clustering of foreground spots. Then, our method tracks and counts the detected pedestrians via inferring the forward–backward motion consistency. Both quantitative and qualitative evaluation and comparison are conducted, and the experimental results demonstrate that the proposed method achieves excellent performance in challenging scenarios.

Automatic 3D model acquisition for unknown objects based on hybrid vision technology

Three-dimensional (3D) model acquisition is the process of building a 3D model of an object. But due to the limited field of view of the scanner, this task is mainly performed by taking several scans with human intervention. In order to make the 3D modeling process efficient, a novel automatic 3D modeling method for unknown objects based on hybrid vision technology in a binocular structured light system (BSLS) is proposed. Firstly, the limit visual vacuums of the BSLS are established, and they will be used to predict the unknown area with an acquired 2.5D range image. With the 2D intensity image acquired synchronously, the coarse boundary size is recovered from Shape from Shading, and it leads the prediction of the unknown area to be more precise. Based on the combination of the predicted contours, the next best viewpoint is determined with more unknown areas visible. The proposed method can be used to obtain the 3D models of unknown objects automatically, and the experimental results illustrate the validity and efficiency of our approach.

A Shape-Adjusted Tridimensional Reconstruction of Cultural Heritage Artifacts Using a Miniature Quadrotor

The innovative automated 3D modeling procedure presented here was used to reconstruct a Cultural Heritage (CH) object by means of an unmanned aerial vehicle. Using a motion capture system, a small low-cost quadrotor equipped with a miniature low-resolution Raspberry Pi camera module was accurately controlled in the closed loop mode and made to follow a trajectory around the artifact. A two-stage process ensured the accuracy of the 3D reconstruction process. The images taken during the first circular trajectory were used to draw the artifact’s shape. The second trajectory was smartly and autonomously adjusted to match the artifact’s shape, then it provides new pictures taken close to the artifact and, thus, greatly improves the final 3D reconstruction in terms of the completeness, accuracy and quickness, in particular where the artifact’s shape is complex. The results obtained here using close-range photogrammetric methods show that the process of automated 3D model reconstruction based on a robotized quadrotor using a motion capture system is a realistic approach, which could provide a suitable new digital conservation tool in the cultural heritage field.

3D Model Reconstruction System Development Based on Laser-Vision Technology

We propose a three-dimensional (3D) scanning system based on laser-vision technique and rotary mechanism combination for automatic 3D model reconstruction. The proposed scanning system consists of a laser projector, camera, and turntable. For laser-camera calibration, a new and simple method is applied. The 3D point cloud data of the surface of the scanned object are fully collected by integrating the extracted laser profiles from the laser stripe images corresponding to the rotary angles of the rotary mechanism. The obscured laser profile problem is solved by an additional camera at another viewpoint. From the collected 3D point cloud data, the 3D model of the scanned object is reconstructed based on the facet-representation method. The reconstructed 3D models showed the effectiveness and applicability of the proposed 3D scanning system in many 3D model-based applications.

Implementation of K-means segmentation algorithm on Intel Xeon Phi and GPU: Application in medical imaging

The paper presents speed up of the k-means algorithm for image segmentation. This speed up is achieved by effective parallelization. For parallel implementation we focus on Many Integrated Core (MIC) architecture with Intel Xeon Phi coprocessors. The MIC implementation is compared with GPU, CPU and sequential implementation. To demonstrate parallel capabilities of k-means algorithm, segmentation of CT images of human body are used. Results of this work will be used for development of the software application for automatic 3D model reconstruction of heart and liver.

Automatic segmentation and 3D modeling of pipelines into constituent parts from laser-scan data of the built environment

Abstract The demand for three-dimensional (3D) modeling of as-built or as-is pipelines that occupy large areas in operating plants has been growing in recent times. In practice, although measurements can be efficiently performed by the use of laser-scanning technology, generating a pipeline model from laser-scan data remains challenging, possibly due to the data characteristics. This paper proposes a method for generating 3D models of entire pipelines, including straight pipes, elbows, reducers, and tee pipes, from laser-scan data. The proposed method comprises three main tasks: (1) identifying the existence and location of entire pipelines from laser-scan data, (2) segmenting each pipeline surface into its constituent forms (straight pipe, elbow, reducer, and tee), and (3) reconstructing the geometry of the individual pipelines and generating a 3D model of it. Field experiments were performed at two different operating industrial plants to validate the method. Evaluation of the quantitative results reveals that the proposed method can indeed be used for the automation of 3D modeling of the pipelines in industrial plants—and to be capable of producing such models with high accuracy.

Automatic 3D modeling and simulation of bone-fixator system in a novel graphical user interface

Abstract External ring fixators are widely used in orthopaedics for the purposes of fracture fixation, bone lengthening and deformity correction. In these fixators, the clinician typically brings the bone fragments to an anatomically desired position by changing the lengths of the rods connecting the fixator rings. This task is accomplished by the clinician based on experience and expertise. As an alternative, commercial systems exist where the same task is automatized with the help of an accompanying software that implements a mathematical model of the fixator. In this work, we have developed a graphical user interface (GUI) implementing a mathematical theory introduced previously. The GUI also allows visualization and simulation of the patient specific bone-fixator system, something that lacks in the available software systems. The information gathered from the bone X-ray images is used to convert a canonical bone model to the patient-specific bone model. This conversion algorithm has been tested on eight different bone models and found to be effective. The visualization tool has been used in the simulation of two orthopaedic procedures, one involving a tibia and the other a femur. In both examples, the visualization tool has provided a realistic depiction of the treatment procedure. We believe that the developed GUI equipped with the visualization module could be a useful clinical tool where the clinician can visualize the applied treatment or evaluate different treatment scenarios a priori per patient.

Active Shapes for Automatic 3D Modeling of Buildings

Recent technological developments help us to acquire high quality 3D measurements of our urban environment. However, these measurements, which come as point clouds or Digital Surface Models (DSM), do not directly give 3D geometrical models of buildings. In addition to that, they are not suitable for fast 3D rendering. Therefore, detection and 3D reconstruction of buildings is an important research topic. We introduce a new active shape fitting algorithm for generating building models. Two significant improvements of the introduced method compared to our previous active shape algorithm are: (1) here, active shapes are initialized as cubes; and (2) the new energy function is computed by measuring the distances of the vertical cube faces to the building facade points and also by measuring the mean distance between the rooftop points and the top face of the cube. The proposed method helps to obtain 3D building models automatically even when the facade borders are difficult to detect because of neighboring trees or other objects. For testing the proposed approach, we use Airborne Laser Scanning (ALS) data of an area in Delft, The Netherlands. We compare the proposed 3D active shape fitting method with a previously developed 2D method. The results show the possible usage of the algorithm when simple and easy-to-render 3D models of large cities are needed.

Automatic 3D modelling of human diaphragm from lung MDCT images.

The thoracic diaphragm separates the thorax and abdomen cavity and also performs an important function in respiration. An automatic algorithm to model the human full diaphragm from multi-detector computed tomography (MDCT) images has been developed and tested. The modelling algorithm comprises these steps: (i) diaphragm top boundary estimation (ii) diaphragm side boundary estimation and (iii) full diaphragm modelling in 3D. Diaphragm top boundary is estimated based on lungs' diaphragmatic surfaces with three different methods including: linear interpolation and fitting fourth and fifth degree polynomial surfaces. Diaphragm side boundary is assumed as the inner surfaces of the lower ribs, spinal column and costal cartilages, estimated via interpolation. As the last step, the full diaphragm is modelled by employing 3D active contours that are initiated from a predefined mesh and expand towards the estimated boundaries of the diaphragm. The proposed algorithm was tested on MDCT datasets from 15 patients, and the result were compared to reference masks provided by an experienced radiologist. Based on quantitative evaluations, the accuracy of the algorithm highly depends on the diaphragm top surface estimation, e.g., the proposed algorithm failed on two datasets, both with enlarged pericardial fat pad that cuts off the left lung from the diaphragm. The proposed algorithm was tested on the remaining 13 datasets in which lungs' lower surfaces have normal contact with the diaphragm. To perform quantitative evaluations, four slices per dataset including an axial, mid-coronal and one-fourth of the sagittal planes from left and right, were compared to the ground truth. Hausdorff distance and mean distance to the closest point were measured to be 11.61 and 3.46mm respectively, when the diaphragm top surface is modelled by a fourth degree polynomial surface. Human full diaphragm can be automatically modelled with 3D active contours bounded by the lower surfaces of the lungs and inner surfaces of the lower ribs, spinal column and costal cartilages.

An automated 3D modeling of topological indoor navigation network

Indoor navigation is important for various applications such as disaster management, building modeling, safety analysis etc. In the last decade, indoor environment has been a focus of wide research that includes development of indoor data acquisition techniques, 3D data modeling and indoor navigation. In this research, an automated method for 3D modeling of indoor navigation network has been presented. 3D indoor navigation modeling requires a valid 3D model that can be represented as a cell complex: a model without any gap or intersection such that two cells (e.g. room, corridor) perfectly touch each other. This research investigates an automated method for 3D modeling of indoor navigation network using a geometrical model of indoor building environment. In order to reduce time and cost of surveying process, Trimble LaserAce 1000 laser rangefinder was used to acquire indoor building data which led to the acquisition of an inaccurate geometry of building. The connection between surveying benchmarks was established using Delaunay triangulation. Dijkstra algorithm was used to find shortest path in between building floors. The modeling results were evaluated against an accurate geometry of indoor building environment which was acquired using highly-accurate Trimble M3 total station. This research intends to investigate and propose a novel method of topological navigation network modeling with a less accurate geometrical model to overcome the need of required an accurate geometrical model. To control the uncertainty of the calibration and of the reconstruction of the building from the measurements, interval analysis and homotopy continuation will be investigated in the near future.

Methods of laser scanning point clouds integration in precise 3D building modelling

3D city models are built on the basis of photogrammetric data, especially laser scanning (LS) data. Due to the increasing levels of accuracy required of 3D city models, terrestrial and airborne laser scanning are the most frequently used technologies. The models are often fragmentary, based on only one type of geodata, e.g. aerial photography or airborne laser scanning (ALS) data. This paper presents various aspects of integrating data from different laser scanning technologies (terrestrial and airborne), which are at the moment considered to be the most detailed and accurate systems for the direct acquiring of 3D geodata. The main issue discussed in the paper is the integration of data in the form of point clouds acquiring from different laser scanning systems. Integration will be understood here as an appropriate processing of both sets of data for orientation into a uniform spatial coordinate system. Thus, the problem of extracting characteristic points common for both sets, in a way which will allow for obtaining a basis for building a complete and accurate 3D city model, will be fundamental. The main challenge when performing this task is to correctly and accurately specify the homologous point pairs used in 3D transformation when both data sets characterise different accuracy, density and content. In the present paper, manual (direct), semi-automatic (indirect determination of characteristic points by approximation) and automatic (3D model) methods for extracting characteristic points are presented, along with the results of an accuracy assessment. Furthermore different transformation methods (isometric, conformal and affine) are also verified, compared and analysed.

레이저 비전 기술을 이용한 물체의 3D 모델 재구성 방법에 관한 연구

In this study, we proposed a three-dimensional (3D) scanning system based on laser-vision technique and rotary mechanism for automatic 3D model reconstruction. The proposed scanning system consists of a laser projector, a camera, and a turntable. For laser-camera calibration a new and simple method was proposed. 3D point cloud data of the surface of scanned object was fully collected by integrating extracted laser profiles, which were extracted from laser stripe images, corresponding to rotary angles of the rotary mechanism. The obscured laser profile problem was also solved by adding an addition camera at another viewpoint. From collected 3D point cloud data, the 3D model of the scanned object was reconstructed based on facetrepresentation. The reconstructed 3D models showed effectiveness and the applicability of the proposed 3D scanning system to 3D model-based applications.

Курс компьютерной графики для студентов строительных специальностей

The article considers the state and trends in the development
 of content and methodology of teaching of engineering
 graphics at universities, in particular at South Ural State University
 (SUSU). It is noted that the development goes in the direction of
 computerization of the course and of the expansion of the scope
 of application of 3D modeling and drawing.
 The author overviews the content and methodology of teaching
 construction engineering students of SUSU computer graphics.
 The course is designed for the AutoCAD package. The article introduces
 three graphical tasks that reveal features of construction
 drawing. The topics of the abovementioned tasks are the perspective
 and shadows, drawing of civil buildings, and drawing of truss
 nodes. The course focuses on modern design technologies in construction.
 To perform the aforementioned tasks the author implements
 both traditional 2D (computer) methods of drawing and
 modern 3D design methods which are actively developing. Work
 goals and objectives for each task are given, as well as the examples
 of drawings, 3D models and the sequence of their construction.
 The author shows methods of studying properties of geometrical
 projections and of shadowing difficult elements of building on
 the example of 3D modeling. The article gives the examples of
 photorealistic rendering done by the students for building visual
 images.
 The article also exposes shadowing in the orthographic projections
 on the example of smaller architectural projects and civil
 constructions. Shadows are built both in the automatic 3D modelling
 and visualization mode and in traditional 2D methods.
 The author dwells on modern means of automation of geometric
 construction as well as shows the example of parameterization
 in the process of determining optimal parameters of perspective
 projections. The article reveals the construction of the dynamic
 block as a means of storing graphical information of modern CAD
 systems. The example of block construction in the process of truss
 node creation is also given.
 The author provides the requirements for the contents of the
 semester report, the list of additional competencies acquired by the
 student after the development of the course.

Applying Standard Digital Map Data in Map-aided, Lane-level GNSS Location

Urban positioning using the Global Positioning System (GPS) is challenging because of multipath. Urban canyons limit open sky visibility, and cause signal reflection and diffraction, resulting in significant satellite range measurement errors. The investigations reported here have been carried out in a French project calledInturb(an acronym derived from integrity and urban positioning). So far, the project has had two phases: first, a simple Three-Dimensional (3D) geometrical city modelling, called “Urban Trench”, has been developed and engineered manually from data sets collected in different cities. Positioning improvement in terms of accuracy was quantified where the model could be applied. Second, this modelling has been automated, based on the standard national BD Topo ® map database for France, with promising results. This geometrical modelling makes it possible to distinguish between line-of-sight satellite signals and those from non-line-of-sight. The latter, apparentlybona fide, signals are caused by strong reflections, usually from buildings with a lot of steel and glass in their construction. A correction of the pseudo-range measurements of the latter is also computed and applied in the position estimator. Positioning accuracy is improved, whilst availability is kept at its maximum. In the study both manual and automatic 3D models are used in extensive experimental campaigns. Results are: first, the possibility to cover entirely any urban area in the country; second, the reduction of the median error in 3D by more than 50% on data collected in Nantes, Paris and Toulouse for a total duration of nearly ten hours; third, the compliance with standards used in most embedded maps and geographical information systems, including an assessment of the trade-off between the model simplicity and the positioning improvement.

Segmentation of building point cloud models including detailed architectural/structural features and MEP systems

Accurate and rapidly produced 3D models of the as-built environment can be significant assets for a variety of Engineering scenarios. Starting with a point cloud of a scene—generated using laser scanners or image-based reconstruction methods—the user must first identify collections of points that belong to individual surfaces, and then, fit surfaces and solid geometry objects appropriate for the analysis. When performed manually, this task is often prohibitively time consuming and, in response, several research groups have recently focused on developing methods for automating the modeling process. Due to the limitations of the data collection processes as well as the complexity of as-built scenes, automated 3D modeling still presents many challenges. To overcome existing limitations, in this paper, we propose a new region growing method for robust context-free segmentation of unordered point clouds based on geometrical continuities. In our method, the user sets a single parameter which accounts for the desired level of abstraction. We treat this parameter as a locally adaptive threshold to account for local context. Our method of segmentation starts with a multi-scale feature detection, describing surface roughness and curvature around each 3D point, and is followed by seed finding and region growing steps. Experimental results from seven challenging point clouds of the built environment demonstrate that our method can account for variability in point cloud density, surface roughness, curvature, and clutter within a single scene. • Region growing method for context-free segmentation of unordered point clouds • A single user defined parameter describes the desired level of abstraction. • Multi-scale feature detection accounts for surface roughness and local curvature. • Feature re-evaluation during region growing mitigates the presence of clutter. • Reliable performance shown for challenging point clouds from the built environment