Viewpoint Planning Research Articles

Viewpoint Planning for 3D Scanning of Structures by a Mobile Robot

Viewpoint Planning for 3D Scanning of Structures by a Mobile Robot

Predicting observation results from a new viewpoint using deep learning for mobile robot viewpoint planning

Predicting observation results from a new viewpoint using deep learning for mobile robot viewpoint planning

Co-localized augmented human and X-ray observers in collaborative surgical ecosystem.

Image-guided percutaneous interventions are safer alternatives to conventional orthopedic and trauma surgeries. To advance surgical tools in complex bony structures during these procedures with confidence, a large number of images is acquired. While image-guidance is the de facto standard to guarantee acceptable outcome, when these images are presented on monitors far from the surgical site the information content cannot be associated easily with the 3D patient anatomy. In this article, we propose a collaborative augmented reality (AR) surgical ecosystem to jointly co-localize the C-arm X-ray and surgeon viewer. The technical contributions of this work include (1) joint calibration of a visual tracker on a C-arm scanner and its X-ray source via a hand-eye calibration strategy, and (2) inside-out co-localization of human and X-ray observers in shared tracking and augmentation environments using vision-based simultaneous localization and mapping. We present a thorough evaluation of the hand-eye calibration procedure. Results suggest convergence when using 50 pose pairs or more. The mean translation and rotation errors at convergence are 5.7mm and [Formula: see text], respectively. Further, user-in-the-loop studies were conducted to estimate the end-to-end target augmentation error. The mean distance between landmarks in real and virtual environment was 10.8mm. The proposed AR solution provides a shared augmented experience between the human and X-ray viewer. The collaborative surgical AR system has the potential to simplify hand-eye coordination for surgeons or intuitively inform C-arm technologists for prospective X-ray view-point planning.

Time-space Viewpoint Planning for Guard Robot with Chance Constraint

This paper presents a novel movement planning algorithm for a guard robot in an indoor environment, imitating the job of human security. A movement planner is employed by the guard robot to continuously observe a certain person. This problem can be distinguished from the person following problem which continuously follows the object. Instead, the movement planner aims to reduce the movement and the energy while keeping the target person under its visibility. The proposed algorithm exploits the topological features of the environment to obtain a set of viewpoint candidates, and it is then optimized by a cost-based set covering problem. Both the robot and the target person are modeled using geodesic motion model which considers the environment shape. Subsequently, a particle model-based planner is employed, considering the chance constraints over the robot visibility, to choose an optimal action for the robot. Simulation results using 3D simulator and experiments on a real environment are provided to show the feasibility and effectiveness of our algorithm.

Viewpoint planning for quantitative coronary angiography.

In coronary angiography, the condition of myocardial blood supply is assessed by analyzing 2-D X-ray projections of contrasted coronary arteries. This is done using a flexible C-arm system. Due to the X-ray immanent dimensionality reduction projecting the 3-D scene onto a 2-D image, the viewpoint is critical to guarantee an appropriate view onto the affected artery and, thus, enable reliable diagnosis. In this work, we introduce an algorithm computing optimal viewpoints for the assessment of coronary arteries without the need for 3-D models. We introduce the concept of optimal viewpoint planning solely based on a single angiographic X-ray image. The subsequent viewpoint is computed such that it is rotated precisely around a vessel, while minimizing foreshortening. Our algorithm reduces foreshortening substantially compared to the input view and completely eliminates it for [Formula: see text] rotations. Rotations around isocentered foreshortening-free vessels passing the isocenter are exact. The precision, however, decreases when the vessel is off-centered or foreshortened. We evaluate worst-case boundaries, providing insight in the maximal inaccuracies to be expected. This can be utilized to design viewpoints guaranteeing desired requirements, e.g., a true rotation around the vessel of at minimum [Formula: see text]. In addition, a phantom study is performed investigating the impact of input views to 3-D quantitative coronary angiography (QCA). We introduce an algorithm for optimal viewpoint planning from a single angiographic X-ray image. The quality of the second viewpoint-i.e., vessel foreshortening and true rotation around vessel-depends on the first viewpoint selected by the physician; however, our computed viewpoint is guaranteed to reduce the initial foreshortening. Our novel approach uses fluoroscopy images only and, thus, seamlessly integrates with the current clinical workflow for coronary assessment. In addition, it can be implemented in the QCA workflow without increasing user interaction, making vessel-shape reconstruction more stable by standardizing viewpoints.

AN EFFICIENT, HIERARCHICAL VIEWPOINT PLANNING STRATEGY FOR TERRESTRIAL LASER SCANNER NETWORKS

Abstract. Terrestrial laser scanner (TLS) techniques have been widely adopted in a variety of applications. However, unlike in geodesy or photogrammetry, insufficient attention has been paid to the optimal TLS network design. It is valuable to develop a complete design system that can automatically provide an optimal plan, especially for high-accuracy, large-volume scanning networks. To achieve this goal, one should look at the “optimality” of the solution as well as the computational complexity in reaching it. In this paper, a hierarchical TLS viewpoint planning strategy is developed to solve the optimal scanner placement problems. If one targeted object to be scanned is simplified as discretized wall segments, any possible viewpoint can be evaluated by a score table representing its visible segments under certain scanning geometry constraints. Thus, the design goal is to find a minimum number of viewpoints that achieves complete coverage of all wall segments. The efficiency is improved by densifying viewpoints hierarchically, instead of a “brute force” search within the entire workspace. The experiment environments in this paper were simulated from two buildings located on University of Calgary campus. Compared with the “brute force” strategy in terms of the quality of the solutions and the runtime, it is shown that the proposed strategy can provide a scanning network with a compatible quality but with more than a 70 % time saving.

Viewpoint Planning for Human Pose Recognition from Multiple Viewpoint

Viewpoint Planning for Human Pose Recognition from Multiple Viewpoint

MODEL BASED VIEWPOINT PLANNING FOR TERRESTRIAL LASER SCANNING FROM AN ECONOMIC PERSPECTIVE

Despite the enormous popularity of terrestrial laser scanners in the field of Geodesy, economic aspects in the context of data acquisition are mostly considered intuitively. In contrast to established acquisition techniques, such as tacheometry and photogrammetry, optimisation of the acquisition configuration cannot be conducted based on assumed object coordinates, as these would change in dependence to the chosen viewpoint. Instead, a combinatorial viewpoint planning algorithm is proposed that uses a given 3D-model as an input and simulates laser scans based on predefined viewpoints. The method determines a suitably small subset of viewpoints from which the sampled object surface is preferably large. An extension of the basic algorithm is proposed that only considers subsets of viewpoints that can be registered to a common dataset. After exemplification of the method, the expected acquisition time in the field is estimated based on computed viewpoint plans.

MODEL BASED VIEWPOINT PLANNING FOR TERRESTRIAL LASER SCANNING FROM AN ECONOMIC PERSPECTIVE

Abstract. Despite the enormous popularity of terrestrial laser scanners in the field of Geodesy, economic aspects in the context of data acquisition are mostly considered intuitively. In contrast to established acquisition techniques, such as tacheometry and photogrammetry, optimisation of the acquisition configuration cannot be conducted based on assumed object coordinates, as these would change in dependence to the chosen viewpoint. Instead, a combinatorial viewpoint planning algorithm is proposed that uses a given 3D-model as an input and simulates laser scans based on predefined viewpoints. The method determines a suitably small subset of viewpoints from which the sampled object surface is preferably large. An extension of the basic algorithm is proposed that only considers subsets of viewpoints that can be registered to a common dataset. After exemplification of the method, the expected acquisition time in the field is estimated based on computed viewpoint plans.

Viewpoint planning for freeform surface inspection using plane structured light scanners

This paper proposes an automatic model-based viewpoint planning method, which can achieve high precision and high efficiency for freeform surfaces inspection using plane structured light scanners. The surface model is utilized in stereolithography format, which is widely used as an industrial standard. The proposed method consists of 4 steps: topology reconstruction, mesh refinement, scan direction determination and viewpoint generation. In the first step, the topology structure of the surface model is reconstructed according to a designed data structure, based on which a neighborhood search algorithm is developed. In the second step, big facets in the surface model are segmented into several small ones, which are suitable for viewpoint planning. In the third step, an initial scan region of a viewpoint is grouped by the neighborhood search algorithm combining with total area and normal vector restrictions. Accordingly, the scan direction is determined by the normal vectors of facets in the initial scan region. In the fourth step, the position, the orientation, and the final scan region of the viewpoint are determined by 4 scan constraints, i.e., field of view, working distance range, view angle and overlap. Experimental results verify the effectiveness and advantages of the proposed method.

A view planning method incorporating self-termination for automated surface measurement

Viewpoint planning plays an important role in automatic 3D model construction. The planning process desires not only the ability to automatically determine the next sensing position, but also the ability for self-termination. In this paper, we use the trend surface, which is the regional feature of a surface for describing the global tendency of change, to compute the next best view in two steps. The exploration direction is determined by analysing the surface curvature of the trend surface. The next view pose is obtained by imposing such constraints as resolution, focus and field of view. A self-termination criterion is proposed for judging the completion condition in the measurement and reconstruction process. The termination condition is derived based on changes in the volume obtained from the last two successive viewpoints. Experimental results show that the method is effective in practical implementation.

Automatic view planning with self-termination in 3D object reconstructions

Viewpoint planning plays an important role in automatic 3D model reconstruction. The objective of view planning in a visual sensing system is to make task-directed decisions for optimal sensing pose selection. In this paper we present a method of view planning for automatically constructing the model of an unknown object from range images. The method computes the next best view in two steps. First, the exploration direction for the next view is determined via a mass vector chain (MVC) based scheme. Then the accurate position of the next view is obtained by computing the boundary integral of the vectors fields. The position with the maximum integral value is selected as the next best view (NBV). We also present a self-termination criterion for judging the completion condition in the measurement and reconstruction process. The termination condition is derived based on changes in the volume computed from two successive viewpoints. The experimental results show that the method is effective in practical implementation.

Shape recovery and viewpoint planning for reverse engineering

In this paper, a new methodology for surface reconstruction from sets of points for reverse engineering is described. The sets of points are obtained using stereovision techniques. Techniques for minimizing the number of images required for a complete object reconstruction are explored based on characterizing the shapes to be recovered in terms of visibility and number and nature of cavities. The ultimate goal of our viewpoint planning technique is to minimize the number of images when no prior information about the objects is available.

View space representation and viewpoint planning computation for automatic 3D object modeling

AbstractViewpoint planning plays an important role in automatic 3D model generation. We have proposed an optimal viewpoint planning [9] for the purpose of model generation by merging images acquired from multiple viewpoints on a sphere that encircles object. This method uses a discretized approximate sphere which involves problems of sphere tessellation and of sphere representation. One must consider that the scanned surface of an object is projected onto a sphere, and that the computational cost increases directly with the number of images. However, there are no appropriate sphere representation methods available. This paper proposes a new representation method based on a 2D array, and develops a new viewpoint planning algorithm. The proposed system is verified by algorithmic analysis and experiments using a real modeling system. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(4): 60–71, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.1232

Transinformation of object recognition and its application to viewpoint planning

This article develops an analogy between the transmission of information through a channel and an object recognition process. This analogy is based on the statistical representation of 3D objects by several multidimensional receptive field histograms. Each histogram corresponds to a particular appearance of a 3D object. The analogy between transmission of information and object recognition allows to evaluate quantitatively the employed measurement (or feature) sets and therefore to predict the performance of the object recognition process. Furthermore one can determine the most discriminant viewpoints of objects by calculating the transinformation of each viewpoint. As an application the article develops an active object recognition algorithm which is able to resolve ambiguities inherent in a single-view recognition algorithm. Most interestingly the algorithm incorporates 3D information of the objects entirely based on 2D measurements in images of the object.

A viewpoint planning strategy for determining true angles on polyhedral objects by camera alignment

The paper presents a viewpoint planning strategy which automatically guides a movable camera from an arbitrary position to a position where the optical axis is perpendicular to a plane spanned by any two intersecting edges on a polyhedral object, i.e., a junction. In related work it is common to use the changing length of edge segments to control such alignment, but we demonstrate the use of the apparent angle between the edge segments of the junction. Based on the control on the apparent angle we achieve robustness as well as independence of distance to the object and focal length. The strategy is able to determine the true angle of a junction with an accuracy of approximately 1/spl deg/, and align with an accuracy of approximately 6/spl deg/.