Architectural Scenes Reconstruction from Uncalibrated Photos and Map Based Model Knowledge

3D reconstruction of trees is an important task for tree analysis but the most affordable approach to capture real objects is with a camera. Although, there already exist methods for 3D reconstruction of trees from multiple photographs, they mostly handle only self-standing trees captured at narrow angles. In fact, dense feature detection and matching is in most cases only the first step of the reconstruction and requires a large set of features and high similarity between individual pictures. However, capturing trees in the orchard is in most cases possible only at wider angles between the individual pictures and with overlapping branches from other trees, which prevents reliable feature matching. We introduce a new approach for estimating projection matrices to produce 3D point clouds of trees from multiple photographs. By manually relating a smaller number of points on images to reference objects, we substitute the missing dense set of features. We assign to each image a projection matrix and minimize the projection error between the images and reference objects using simulated annealing. Thereby, we produce correct projection matrices for further steps in 3D reconstruction. Our approach is tested on a simple application for 3D reconstruction of trees to produce a 3D point cloud. We analyze convergence rates of the optimization and show that the proposed approach can produce feasible projection matrices from a sufficiently large set of feature points. In the future, this approach will be a part of a complete system for tree reconstruction and analysis.

We address the problem of recovering 3D models from uncalibrated images of architectural scenes. We propose a simple, geometrically intuitive method which exploits the strong rigidity constraints of parallelism and orthogonality present in indoor and outdoor architectural scenes. We show how these simple constraints can be used to calibrate the cameras and to recover the projection matrices for each viewpoint. The projection matrices are used to recover partial 3D models of the scene and these can be used to visualise new viewpoints. Our approach does not need any a priori information about the cameras being used. A working system called PhotoBuilder had been designed and implemented to allow a user to interactively build a VRML model of a building from uncalibrated images from arbitrary viewpoints.

We address the problem of recovering 3D models from uncalibrated images of architectural scenes. We propose a simple, geometrically intuitive method which exploits the strong rigidity constraints of parallelism and orthogonality present in indoor and outdoor architectural scenes. We show how these simple constraints can be used to calibrate the cameras and to recover the projection matrices for each viewpoint. The projection matrices are used to recover partial 3D models of the scene and these can be used to visualise new viewpoints. Our approach does not need any a priori information about the cameras being used. A working system called PhotoBuilder has been designed and implemented to allow a user to interactively build a VRML model of a building from uncalibrated images from arbitrary viewpoints.

An approach for the 3D reconstruction of architectural scenes from two uncalibrated images is described in this paper. From two views of one architectural structure, three pairs of corresponding vanishing points of three major mutual orthogonal directions can be extracted. The simple but powerful constraints of parallelism and orthogonally in architectural scenes can be used to calibrate the cameras and to recover the projection matrices for each viewpoint. The projection matrices are used to reconstruct a partial 3D model of an architectural scene from two uncalibrated photographs taken from arbitrary viewpoints. The approach is applied to the real images of architectural scenes, and a 3D model of a building in VRML format is presented which illustrates the method with successful performance. It is applied to recover 3D models using the hand held digital cameras that the camera motion can't be controlled.

To present a generic geometric calibration method for tomographic imaging systems with flat-panel detectors in a very detailed manner, in the aim to provide a useful tool to the public domain. The method is based on a projection matrix which represents a mapping from 3D object coordinate system to 2D projection image plane. The projection matrix can be determined experimentally through the imaging of a phantom of known marker geometry. Accurate implementation was accomplished through direct computation algorithms, including a novel ellipse fitting using singular value decomposition and data normalization. Benefits of the method include: (1) It is capable of being applied to systems of different scan trajectories, source-detector alignments, and detector orientations; (2) projection matrices can be utilized in image reconstructions or in the extraction of explicit geometrical parameters; and (3) the method imposes minimal limits on the design of calibration phantom. C++ programs that calculate projection matrices and extract geometric parameters from them are also provided. For validation, the calibration method was applied to the computer simulation of a cone-beam CT system, as well as to three tomosynthesis prototypes of different source-detector movement patterns: Source and detector rotating synchronizedly; source rotating and detector wobbling; and source rotating and detector staying stationary. Projection matrices were computed on a view by view basis. Geometric parameters extracted from projection matrices were consistent with actual settings. Images were reconstructed by directly using projection matrices, and were compared to virtual Shepp-Logan image for CT simulation and to central projection images of CIRS breast phantoms for tomosynthesis prototypes. They showed no obvious distortion or blurring, indicating the high quality of geometric calibration results. When the computed central ray offsets were perturbed with Gaussian noises of 1 pixel standard deviation, the reconstructed image showed apparent distortion, which further demonstrated the accuracy of the geometric calibration method. The method is suitable for tomographic imaging systems with flat-panel detectors.

Purpose: To develop a high precision geometry calibration method and an efficient image reconstruction algorithm for digital tomosynthesis system. Method and Materials: A geometry calibration phantom was constructed with 40 markers arranged in two planes parallel to the breast holder. A 3×4 projection matrix which maps the coordinates (x,y,z) of a point in the object to the coordinates (u,v) of the correspondent projection point on the detector was constructed for each projection angle based on the projection images of the calibration phantom. All information for the system geometry, such as source-to-detector distance, source to ISO distance, central ray offset on the detector (u0, v0), and detector angle offsets, can be extracted from the projection matrices. The projection matrices, not explicit geometry parameters, were used in a modified Feldkamp algorithm to reconstruct the imaged object. A prototype tomosynthesis system and a CIRS anthropomorphic breast phantom with multiple embedded structures were used to test the geometry calibration accuracy and the reconstruction algorithm. Results: 3-D image of the breast phantom was reconstructed using the projection matrices. 4 fibers, 6 masses, and all 12 speck groups were visible in the focal plane. Conclusion: Geometry calibration based on the projection matrices is accurate and reconstruction using the projection matrices is efficient.

The central problem for most existing metric learning methods is to find a suitable projection matrix on the differences of all pairs of data points. However, a single unified projection matrix can hardly characterize all data similarities accurately as the practical data are usually very complicated, and simply adopting one global projection matrix might ignore important local patterns hidden in the dataset. To address this issue, this paper proposes a novel method dubbed “Data-Adaptive Metric Learning” (DAML), which constructs a data-adaptive projection matrix for each data pair by selectively combining a set of learned candidate matrices. As a result, every data pair can obtain a specific projection matrix, enabling the proposed DAML to flexibly fit the training data and produce discriminative projection results. The model of DAML is formulated as an optimization problem which jointly learns candidate projection matrices and their sparse combination for every data pair. Nevertheless, the over-fitting problem may occur due to the large amount of parameters to be learned. To tackle this issue, we adopt the Total Variation (TV) regularizer to align the scales of data embedding produced by all candidate projection matrices, and thus the generated metrics of these learned candidates are generally comparable. Furthermore, we extend the basic linear DAML model to the kernerlized version (denoted “KDAML”) to handle the non-linear cases, and the Iterative Shrinkage-Thresholding Algorithm (ISTA) is employed to solve the optimization model. Intensive experimental results on various applications including retrieval, classification, and verification clearly demonstrate the superiority of our algorithm to other state-of-the-art metric learning methodologies.

This paper concerns the incorporation of geometric information in camera calibration and 3D modeling. Using geometric constraints enables more stable results and allows us to perform tasks with fewer images. Our approach is motivated and developed within a framework of semi-automatic 3D modeling, where the user defines geometric primitives and constraints between them. It is based on the observation that constraints, such as coplanarity, parallelism, or orthogonality, are often embedded intuitively in parallelepipeds. Moreover, parallelepipeds are easy to delineate by a user and are well adapted to model the main structure of, e.g., architectural scenes. In this paper, first a duality that exists between the shape parameters of a parallelepiped and the intrinsic parameters of a camera is described. Then, a factorization-based algorithm exploiting this relation is developed. Using images of parallelepipeds, it allows us to simultaneously calibrate cameras, recover shapes of parallelepipeds, and estimate the relative pose of all entities. Besides geometric constraints expressed via parallelepipeds, our approach simultaneously takes into account the usual self-calibration constraints on cameras. The proposed algorithm is completed by a study of the singular cases of the calibration method. A complete method for the reconstruction of scene primitives that are not modeled by parallelepipeds is also briefly described. The proposed methods are validated by various experiments with real and simulated data, for single-view as well as multiview cases.

In this paper we address the design of projection matrix for compressed sensing. In most compressed sensing applications, random projection matrices have been used but it has been shown that optimizing these projections can greatly improve the sparse signal reconstruction performance. An incoherent projection matrix can greatly reduce the recovery error for sparse signal reconstruction. With this motivation, we propose an algorithm for the construction of an incoherent projection matrix with respect to the designed equiangular tight frame (ETF) for reducing pairwise mutual coherence. The designed frame consists of a set of column vectors in a finite dimensional Hilbert space with the desired norm and reduced pairwise mutual coherence. The proposed method is based on updating ETF with inertial force and constructing incoherent frame and projection matrix using alternating minimization. We compare the performance of the proposed algorithm with state-of-the-art projection matrix design algorithms via numerical experiments and the results show that the proposed algorithm outperforms the other algorithms.

In this paper, motor imagery electroencephalograph classification problem is investigated and a method which modifies the projection matrix is proposed based on common spatial pattern analysis. Exceptional samples are detected through examining the features generated by the projection matrix in the first place, which are special in terms that the projection matrix in common spatial pattern analysis fails to extract discriminant features from them. Projection matrices for exceptional trials are re-estimated and integrated together to form the final projection model. Based on this integrated model, feature extraction is carried out and classification follows by employing support vector machine. The validity of the proposed method is verified through experiment studies. Two data sets that consist of two classes are used, and results show that the proposed method generates more discriminant features.

Reconciliation of process data using other projection matrices

Person re-identification is one of the most important and challenging problems in video analytics systems; it aims to match people across non-overlapping camera views. For person re-identification, metric learning is introduced to improve the performance by providing a metric adapted for cross-view matching. The essence of metric learning is to search for an optimal projection matrix to project the original features into a new feature space. However, most existing metric learning methods overlook the inconsistency of feature distributions in multiple cameras. In this paper, we propose a multi-projection metric learning (MPML) method to overcome the inconsistency among multiple cameras in person re-identification. Our solution is to jointly learn multiple projection matrices using paired samples from different cameras to project features from different cameras into a common feature space. To make our method adaptive to newly added cameras without affecting the learned projection matrices, we further propose an adaptive MPML method, which can learn new camera projection matrices without having to update any of the obtained projection matrices. The proposed methods are evaluated on four major person re-identification data sets, with comprehensive experiments showing the effectiveness of the proposed methods and notable improvements over the state-of-the–art approaches.

Projection matrices in variable environments: λ1 in theory and practice

The mathematics of matrix models for age- and/or stage-structured population dynamics substantiates the use of the dominant eigenvalue λ 1 of the projection matrix L as a measure of the growth potential, or of adaptation, for a given species population in modern plant or animal demography. The calibration of L = T +F on the “identified-individuals-of-unknown-parents” kind of empirical data determines precisely the transition matrix T, but admits arbitrariness in the estimation of the fertility matrix F. We propose an adaptation principle that reduces calibration to the maximization of λ 1(L) under the fixed T and constraints on F ensuing from the data and expert knowledge. A theorem has been proved on the existence and uniqueness of the maximizing solution for projection matrices of a general pattern. A conjugated maximization problem for a “potential-growth indicator” under the same constraints has appeared to be a linear-programming problem with a ready solution, the solution testing whether the data and knowledge are compatible with the population growth observed.

In this paper we present a system for the reconstruction of 3D models of architectural scenes from single or multiple uncalibrated images. The partial 3D model of a building is recovered from a single image using geometric constraints such as parallelism and orthogonality, which are likely to be found in most architectural scenes. The approximate corner positions of a building are selected interactively by a user and then further refined automatically using Hough transform. The relative depths of the corner points are calculated according to the perspective projection model. Partial 3D models recovered from different viewpoints are registered to a common coordinate system for integration. The 3D model registration process is carried out using modified ICP (iterative closest point) algorithm with the initial parameters provided by geometric constraints of the building. The integrated 3D model is then fitted with piecewise planar surfaces to generate a more geometrically consistent model. The acquired images are finally mapped onto the surface of the reconstructed 3D model to create a photo-realistic model. A working system which allows a user to interactively build a 3D model of an architectural scene from single or multiple images has been proposed and implemented.