3D Model Points Research Articles

Multi-view 3D reconstruction of seedling using 2D image contour

3D reconstruction of seedling can provide comprehensive and quantitative spatial structure information, offering an effective digital tool for breeding research. However, accurate and efficient reconstruction of seedling is still a challenging work due to limited performance of depth sensor for seedling with small-size stem and unavoidable error for multi-view point cloud registration. Therefore, in this paper, we propose an accurate multi-view 3D reconstruction method for seedling using 2D image contour to constrain 3D point cloud. The rotation axis is calibrated and optimised by minimising point-to-contour distance between 2D image contour and projected exterior points from 3D point cloud. Then, to remove outliers and noise, we introduce the seedling mask of 2D image to constrained and delete projected outlier points of 3D model from corresponding view. Furthermore, we propose a residual-guided method to recognise missing region for 3D model and complete 3D model of small-size stem. Finally, we can obtain an accurate 3D model of seedling. The reconstruction accuracy is evaluated by average distance between projected contour of 3D model and 2D image contour of all views (0.3185 mm). Then, the phenotypic parameters were calculated from 3D model and the results are close to manual measurements (Plant height: R2 = 0.98, RMSE = 2.3 mm, rRMSE = 1.52%; Petioles inclination angle: R2 = 0.99, RMSE = 0.73°, rRMSE = 1.41%; Leaf area: R2 = 0.66, RMSE = 1.05 cm2, rRMSE = 7.63%; Leaf inclination angle: R2 = 0.99, RMSE = 1.01°, rRMSE = 1.72%; Stem diameter: R2 = 0.95, RMSE = 0.12 mm, rRMSE = 5.43%). Breeders can improve the selection of more resilient varieties and cultivars to different growing conditions starting from the dynamic analysis of their phenotype.

Nested Fabric Adaptation to New Urban Heritage Development

Old urban reform usually reactivates the urban fabric in a new era of sustainable development. However, what remains of the former fabric and how it interacts with the new one often inspires curiosity. How the old residents adapt their lives to the new layout should be explored qualitatively and quantitatively. This research aimed to assess the old and new fabrics in the downtown area of Keelung, Taiwan, by considering the interactions between truncated layout, proportion, and infill orientation in the mature and immature interfaces. According to the historical reform map made in 1907, the newly constructed area occupied the old constructed area in seven downtown blocks. On average, the area composed of new buildings ranged from 135.60% to 239.20% of the old area, and the average volume of the buildings reached a maximum of 41.72 m when compared to the old buildings in place prior to the reform. It seems that the new fabric purposefully maintained the old temples at the centers of the blocks. However, the old alleys, which still remain within these blocks, have been significantly overloaded with services and have become auxiliary utility spaces for the in-block residences. With regard to the part of the fabric that was truncated or reoriented by new streets, the modification could also be easily found on the second skin. A physical model analysis used a UAV 3D cloud model and QGIS® to verify the axes, hierarchies, entrances, open spaces, and corners in the commission store block and temple blocks. We found that the 3D point model and historical maps presented a convincing explanation of the evolved fabric from the past to the present. The stepwise segmentation visualizes the enclosed block inside a block on the historical maps and according to the present sections. We found that new roles for old alleys have evolved behind the new fabric.

Dual Branch PnP Based Network for Monocular 6D Pose Estimation

Monocular 6D pose estimation is a functional task in the field of computer vision and robotics. In recent years, 2D-3D correspondence-based methods have achieved improved performance in multiview and depth data-based scenes. However, for monocular 6D pose estimation, these methods are affected by the prediction results of the 2D-3D correspondences and the robustness of the perspective-n-point (PnP) algorithm. There is still a difference in the distance from the expected estimation effect. To obtain a more effective feature representation result, edge enhancement is proposed to increase the shape information of the object by analyzing the influence of inaccurate 2D-3D matching on 6D pose regression and comparing the effectiveness of the intermediate representation. Furthermore, although the transformation matrix is composed of rotation and translation matrices from 3D model points to 2D pixel points, the two variables are essentially different and the same network cannot be used for both variables in the regression process. Therefore, to improve the effectiveness of the PnP algorithm, this paper designs a dual-branch PnP network to predict rotation and translation information. Finally, the proposed method is verified on the public LM, LM-O and YCB-Video datasets. The ADD(S) values of the proposed method are 94.2 and 62.84 on the LM and LM-O datasets, respectively. The AUC of ADD(-S) value on YCB-Video is 81.1. These experimental results show that the performance of the proposed method is superior to that of similar methods.

Integration of Thermal and RGB Data Obtained by Means of a Drone for Interdisciplinary Inventory

Thermal infrared imagery is very much gaining in importance in the diagnosis of energy losses in cultural heritage through non-destructive measurement methods. Hence, owing to the fact that it is a very innovative and, above all, safe solution, it is possible to determine the condition of the building, locate places exposed to thermal escape, and plan actions to improve the condition of the facility. The presented work is devoted to the technology of creating a dense point cloud and a 3D model, based on data obtained from UAV. It has been shown that it is possible to build a 3D point model based on thermograms with the specified accuracy by using thermal measurement marks and the dense matching method. The results achieved in this way were compared and, as the result of this work, the model obtained from color photos was integrated with the point cloud created on the basis of the thermal images. The discussed approach exploits measurement data obtained with three independent devices (tools/appliances): a Matrice 300 RTK drone (courtesy of NaviGate); a Phantom 4 PRO drone; and a KT-165 thermal imaging camera. A stone church located in the southern part of Poland was chosen as the measuring object.

Computing Salient Feature Points of 3D Model Based on Geodesic Distance and Decision Graph Clustering

Introduction:: Computing salient feature points (SFP) of 3D models has important application value in the field of computer graphics. In order to extract more effectively, a novel SFP computing algorithm based on geodesic distance and decision graph clustering is proposed. Method:: Firstly, geodesic distance of model vertices is calculated based on heat conduction equation, then average geodesic distance and importance weight of vertices are calculated. Finally, decision graph clustering method is used to calculate the decision graph of model vertices. Result and Discussion:: 3D models in SHREC 2011 dataset are selected to test the proposed algorithm. Compared with the existing algorithms, this method calculates the SFP of the 3D model from a global perspective. Results show that it is not affected by model posture and noise. Conclusion:: Our method maps the SFP of 3D model to 2D decision-making diagram, which simplifies the calculation process of SFP, improves the calculation accuracy and has strong robustness.

Generating a three-dimensional measuring scene for the forest sector as based on modern geodetic technologies

Modern geodetic technologies of gathering three-dimensional spatial data incorporate terrestrial laser scanning and aerial photo survey from unmanned aerial vehicles. The combination of these technologies and joint result of survey provide the data of 3D point model and accurate information on trunks and crowns of individual trees. The paper examines the experiment with the application of method of formation of 3D measuring scene in the form of dense cloud of points combining the results of terrestrial laser scanning and materials of photogrammetric processing of UAV-provided data. The method eliminates basic shortcomings of each technology, enhances their advantages, and opens the way to the compilation of more representative 3D measuring scenes. A specific advantage of the method is the outcropping of detailed information on the form, size and condition of individual tree crowns. This option finds a practical application in landscape evaluation and design, remote measuring of trunk parameters excluding the felling of model trees for the compilation of regional timber account tables. The closest perspectives of method development are related to increasing the accuracy of combined survey by specifying flight missions and working with the light regime under forest canopy.

The effects of camera resolution and distance on suspect height analysis using PhotoModeler

The purpose of this study was to examine how camera resolution and suspect-camera distance affect the accuracy and precision of suspect height estimations using PhotoModeler software. Sixteen individuals were measured and recorded standing at 15 pre-set distances on 7 security cameras, each with a different resolution setting. A height scale was used to measure each individual’s height prior to recording and was also used as a reference height. Height estimates were taken in PhotoModeler by extracting video frames that were calibrated using 3D point model data obtained from a laser scan of the test site. Errors were calculated for the measurements and compared using the Kruskal-Wallis H-test, which indicated significant differences for errors among different resolutions and distances (p < 0.01). Interaction plots, however, demonstrated little difference in errors for most resolutions and positions. The accuracy and precision of height estimates began to decrease with resolutions under 960H and distances over 36.5 m.

An Occlusion‐aware Edge‐Based Method for Monocular 3D Object Tracking using Edge Confidence

AbstractWe propose an edge‐based method for 6DOF pose tracking of rigid objects using a monocular RGB camera. One of the critical problem for edge‐based methods is to search the object contour points in the image corresponding to the known 3D model points. However, previous methods often produce false object contour points in case of cluttered backgrounds and partial occlusions. In this paper, we propose a novel edge‐based 3D objects tracking method to tackle this problem. To search the object contour points, foreground and background clutter points are first filtered out using edge color cue, then object contour points are searched by maximizing their edge confidence which combines edge color and distance cues. Furthermore, the edge confidence is integrated into the edge‐based energy function to reduce the influence of false contour points caused by cluttered backgrounds and partial occlusions. We also extend our method to multi‐object tracking which can handle mutual occlusions. We compare our method with the recent state‐of‐art methods on challenging public datasets. Experiments demonstrate that our method improves robustness and accuracy against cluttered backgrounds and partial occlusions.

Keyframe-Based Camera Relocalization Method Using Landmark and Keypoint Matching

Camera relocalization is a challenging task, especially based on the sparse 3D map or keyframes. In this paper, we present an accurate method for RGB camera relocalization in the case of a very sparse 3D map built by limited keyframes. The core of our approach is a top-to-down feature matching strategy to provide a set of accurate 2D-to-3D matches. Specifically, we first use the landmark-based place recognition method to generate from keyframes the images similar to the current view along with the set of pairwise matched landmarks. This step constrains the 3D model points that can be matched with the current view. Then, the points are matched within the landmark pairs and combined afterward. This is in contrast to the conventional methods of feature matching that typically match points between the entire images and the whole 3D map, which, as a result, may not be robust to large viewpoint changes, the main challenge of the relocalization based on the sparse map. After feature matching, the camera pose is calculated by an efficient novel Perspective-n-Points (PnP) algorithm. We conduct experiments on challenging datasets to demonstrate that the camera poses estimated by our method based on the sparse 3D point cloud are more accurate than the classical methods using the dense map or a large number of training images.

Low-Cost and Efficient Indoor 3D Reconstruction through Annotated Hierarchical Structure-from-Motion

With the widespread application of location-based services, the appropriate representation of indoor spaces and efficient indoor 3D reconstruction have become essential tasks. Due to the complexity and closeness of indoor spaces, it is difficult to develop a versatile solution for large-scale indoor 3D scene reconstruction. In this paper, an annotated hierarchical Structure-from-Motion (SfM) method is proposed for low-cost and efficient indoor 3D reconstruction using unordered images collected with widely available smartphone or consumer-level cameras. Although the reconstruction of indoor models is often compromised by the indoor complexity, we make use of the availability of complex semantic objects to classify the scenes and construct a hierarchical scene tree to recover the indoor space. Starting with the semantic annotation of the images, images that share the same object were detected and classified utilizing visual words and the support vector machine (SVM) algorithm. The SfM method was then applied to hierarchically recover the atomic 3D point cloud model of each object, with the semantic information from the images attached. Finally, an improved random sample consensus (RANSAC) generalized Procrustes analysis (RGPA) method was employed to register and optimize the partial models into a complete indoor scene. The proposed approach incorporates image classification in the hierarchical SfM based indoor reconstruction task, which explores the semantic propagation from images to points. It also reduces the computational complexity of the traditional SfM by avoiding exhausting pair-wise image matching. The applicability and accuracy of the proposed method was verified on two different image datasets collected with smartphone and consumer cameras. The results demonstrate that the proposed method is able to efficiently and robustly produce semantically and geometrically correct indoor 3D point models.

Dynamic 3D reconstruction improvement via intensity video guided 4D fusion

The availability of high-speed 3D video sensors has greatly facilitated 3D shape acquisition of dynamic and deformable objects, but high frame rate 3D reconstruction is always degraded by spatial noise and temporal fluctuations. This paper presents a simple yet powerful dynamic 3D reconstruction improvement algorithm based on intensity video guided multi-frame 4D fusion. Temporal tracking of intensity image points (of moving and deforming objects) allows registration of the corresponding 3D model points, whose 3D noise and fluctuations are then reduced by spatio-temporal multi-frame 4D fusion. We conducted simulated noise tests and real experiments on four 3D objects using a 1000 fps 3D video sensor. The results demonstrate that the proposed algorithm is effective at reducing 3D noise and is robust against intensity noise. It outperforms existing algorithms with good scalability on both stationary and dynamic objects.

Research on BSP Algorithm of Construction of Large-Scale 3D Point Model

For the construction of large-scale surface features 3D point model, a large number of point cloud data processing calculations is needed. Previous model construction calculation was treated non-parallel manner successively and mostly with one by one point cloud. This data processing method is complex, low efficiency and requires vast computing resource. Accordance with the BSP parallel computing ideas, we design a point cloud data modeling algorithm based on BSP and build a Hama parallel computing cluster consisted of ordinary PCs. The results indicate that, large-scale 3D point model BSP construction algorithm can improve the efficiency of modeling calculations and reduce computing resources requirements for processing construction computing.

3D laser scanning of mining machines at Nastup Tusimice mine in the Czech Republic

3D laser scanning of bucket wheel excavators is useful in identifying key geometric parameters and in the creation of mathematical models. Thus, a compact and highly accurate 3D point model of a bucket wheel excavator was obtained. The evaluation consisted of vectorising their important features. These data were used to visualise the spatial position of excavators and track their movements in real time. Knowing the position of excavators, particularly the displacement of the bucket wheel relative to the digital terrain model (opencast mine), coupled with the geological model is important for mine management and calculation of volumes of extracted materials in real time. The 3D laser scanning measurements were carried out at Severočeské Doly mines.

Geological model of the central Periadriatic basin (Apennines, Italy)

3D geological models from multi-source data (cross-sections, geological maps, borehole logs and outcrops) are a critical tool to improve the interpretation of the spatial organization of subsurface structures that are not directly accessible. In this paper, we reconstruct the main geological structures and surfaces in three dimensions through the interpolation of closely and regularly spaced 2D seismic sections, constrained by wells data and surface geology. The methodology was applied in the Marche–Abruzzi sector of the Periadriatic basin, where the more external part of the Apennines fold-and-thrust belt is mostly buried under a syn- and post-orogenic, Plio–Pleistocene, siliciclastic sequence. The 3D model allowed us to correlate the main thrust fronts and related anticlines along strike, revealing a general ramp – flat – ramp trajectory characterizing the main structural trends. This geometric organization influences the sequence of thrust-system propagation and characterizes the evolution of syntectonic basins. The obtained 3D model points out several variation occurring along strike: i) main trends geometric relationships; ii) deformation chronology and iii) displacement distribution. In the northern sector, higher displacement and structural elevation are reached out by the Nereto–Bellante structure, whereas in the southern sector the Villadegna–Costiera Structure is the prevalent. All structures show a diachronic thrusts activity along strike, younger toward the north.

Smooth adaptive fitting of 3D face model for the estimation of rigid and nonrigid facial motion in video sequences

We propose a 3D wireframe face model alignment for the task of simultaneously tracking of rigid head motion and nonrigid facial expressions in video sequences. The system integrates two levels: (i) at the low level, automatic and accurate location of facial features are obtained via a cascaded optimization algorithm of a 2D shape model, (ii) at the high level, we recover, via minimizing an energy function, the optimal motion parameters of the 3D model, namely the 3D rigid motion parameters and seven nonrigid animation (Action Unit) parameters. In this latter inference, a 3D face shape model (Candide) is automatically fitted to the image sequence via a least squares minimization of the energy, defined as the residual between the projected 3D wireframe model and the 2D shape model, meanwhile imposing temporal and spatial motion-smoothness constraints over the 3D model points. Our proposed system tackles many disadvantages of the optimization and training associated with active appearance models. Extensive fitting and tracking experiments demonstrate the feasibility, accuracy and effectiveness of the developed methods. Qualitative and quantitative performance of the proposed system on several facial sequences, indicate its potential usefulness for multimedia applications, as well as facial expression analysis.

Geometry contrast enhancement for 3D point models using histogram modification

In this paper, we propose a geometry histogram modification algorithm to increase the visibility of 3D point models. We define a weak feature as a group of neighboring points yielding the small deviations of normal directions. Geometry histogram is defined as the distribution of the signed distance between a feature point and the locally approximated plane. We equalize and stretch the geometry histogram and move the corresponding feature points accordingly. We also use the OpenGL API for simple and fast rendering of 3D point models. Experimental results show that the proposed algorithm efficiently enhances the geometry contrast of 3D point models by improving the appearance of the weak features.

Monocular vision-based iterative pose estimation algorithm from corresponding feature points

To determine the relative pose between an object and a single camera using 2D-to-3D point correspondences, a kind of iterative methods based on inverse projection ray approach is proposed. An iterative algorithm which is divided into depth recovery stage and absolute orientation stage is also proposed. In the first stage, the optimal translation vector is first computed in terms of rotation matrix via least square method, then the depths of the observed points are estimated by projecting the estimated point orthogonally to the inverse projection ray defined by the image point, and finally 3D points are reconstructed using the estimated depths from previous step. In the second stage, the optimal rotation matrix is estimated by applying Umeyama algorithm to fitting of the 3D model points and 3D estimated points. The above two stages are repeated until the result converges. The global convergence of the two-stage iterative algorithm is proven based on the global convergence theorem. Finally, a spacecraft docking application is implemented to test the effectiveness and convergence of the proposed algorithm by mathematical simulation and physical simulation.

Mixed lineage kinase 3 gene mutations in mismatch repair deficient gastrointestinal tumours

Mixed lineage kinase 3 (MLK3) is a serine/threonine kinase, regulating MAPkinase signalling, in which cancer-associated mutations have never been reported. In this study, 174 primary gastrointestinal cancers (48 hereditary and 126 sporadic forms) and 7 colorectal cancer cell lines were screened for MLK3 mutations. MLK3 mutations were significantly associated with MSI phenotype in primary tumours (P = 0.0005), occurring in 21% of the MSI carcinomas. Most MLK3 somatic mutations identified were of the missense type (62.5%) and more than 80% of them affected evolutionarily conserved residues. A predictive 3D model points to the functional relevance of MLK3 missense mutations, which cluster in the kinase domain. Further, the model shows that most of the altered residues in the kinase domain probably affect MLK3 scaffold properties, instead of its kinase activity. MLK3 missense mutations showed transforming capacity in vitro and cells expressing the mutant gene were able to develop locally invasive tumours, when subcutaneously injected in nude mice. Interestingly, in primary tumours, MLK3 mutations occurred in KRAS and/or BRAF wild-type carcinomas, although not being mutually exclusive genetic events. In conclusion, we have demonstrated for the first time the presence of MLK3 mutations in cancer and its association to mismatch repair deficiency. Further, we demonstrated that MLK3 missense mutations found in MSI gastrointestinal carcinomas are functionally relevant.

Alignment of fiducial marks in a tomographic tilt series with an unknown rotation axis

Alignment for tomography using a transmission electron microscopy frequently uses colloidal gold particles as fiducial reference marks. Typically, there is an implicit assumption that the tilt axis of the tomographic series is orthogonal to the beam direction. However, this may not be true, either intentionally, if a tilt-rotate stage is used, or unintentionally, because of mechanical errors in the rotation stage or the sample fixture. Here, we provide a computer code which takes as input a set of two-dimensional (2D) observations of fiducial reference marks at various tilt angles and the values of those tilt angles. It produces as output a three-dimensional model of the observations, 2D shifts for each view, and the tilt axis direction. 2 Commercial organizations identified in this paper are for the purpose of identification only and are not endorsed by NIST. The associated equipment or software identified is not necessarily the best available for the purpose. Program summary Title of program: particleTilt Catalogue identifier: ADYW_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/ADYW_v1_0 Program obtainable from: CPC Program Library, Queen's University of Belfast, N. Ireland Computers: IBM 2 compatible desktop PC; SGI 2 Octane Operating system: Red Hat 2 WS 3 Linux (with 2.4.21-40.EL kernel); IRIX 6.5 IP30 Program language used: Fortran 90 No. of bits in a word: 32 No. of processors used: one Has the code been vectorized: no No. of lines in distributed program, including test data, etc.: 2397 No. of bytes in distributed program, including test data, etc.: 47 017 Distribution format: tar.gz Peripherals used: one Typical running time: 350 ms (larger included example, on 2.8 GHz 32-bit PC) Nature of problem: The program is used to assist the alignment step in tomography. The samples should be prepared with spherical particles (typically gold beads) which are observed in several views. (Not every particle need be observed in every view.) The program reports coordinates of a 3D model of the particles as well as the direction of the tilt axis as a point on the unit sphere. Method of solution: Our package minimizes an objective function whose free variables are a set of 3D model points and 2D shifts of the views as well as two parameters characterizing of tilt axis as a point on the unit sphere. The objective function is decomposed into a pure quadratic form which encompasses the model points and shifts, and a more complicated form which has only two degrees of freedom. The Broyden–Fletcher–Goldfarb–Shanno algorithm is applied alternately and iteratively to minimize the objective function with respect to the two sets.

Graphtracker: A topology projection invariant optical tracker

In this paper, we describe a new optical tracking algorithm for pose estimation of interaction devices in virtual and augmented reality. Given a 3D model of the interaction device and a number of camera images, the primary difficulty in pose reconstruction is to find the correspondence between 2D image points and 3D model points. Most previous methods solved this problem by the use of stereo correspondence. Once the correspondence problem has been solved, the pose can be estimated by determining the transformation between the 3D point cloud and the model. Our approach is based on the projective invariant topology of graph structures. The topology of a graph structure does not change under projection: in this way we solve the point correspondence problem by a subgraph matching algorithm between the detected 2D image graph and the model graph. In addition to the graph tracking algorithm, we describe a number of related topics. These include a discussion on the counting of topologically different graphs, a theoretical error analysis, and a method for automatically estimating a device model. Finally, we show and discuss experimental results for the position and orientation accuracy of the tracker.