Projective Texture Research Articles

Dynamic Fusion Technology of Mobile Video and 3D GIS: The Example of Smartphone Video

Mobile videos contain a large amount of data, where the information interesting to the user can either be discrete or distributed. This paper introduces a method for fusing 3D geographic information systems (GIS) and video image textures. For the dynamic fusion of video in 3DGIS where the position and pose angle of the filming device change moment by moment, it integrates GIS 3D visualization, pose resolution and motion interpolation, and proposes a projection texture mapping method for constructing a dynamic depth camera to achieve dynamic fusion. In this paper, the accuracy and time efficiency of different systems of gradient descent and complementary filtering algorithms are analyzed mainly by quantitative analysis method, and the effect of dynamic fusion is analyzed by the playback delay and rendering frame rate of video on 3DGIS as indicators. The experimental results show that the gradient descent method under the Aerial Attitude Reference System (AHRS) is more suitable for the solution of smartphone attitude, and can control the root mean square error of attitude solution within 2°; the delay of video playback on 3DGIS is within 29 ms, and the rendering frame rate is 34.9 fps, which meets the requirements of the minimum resolution of human eyes.

PROJECTIVE MULTITEXTURING OF CURRENT 3D CITY MODELS AND POINT CLOUDS WITH MANY HISTORICAL IMAGES

Abstract. Iconographic image collections are a cultural heritage that could reach a larger audience by proposing their immersive presentation in a 3D web application. Proposing a historical street view application, based on these historical images, raises issues such as the unavailability of historical 3D models of the scene and the heterogeneity and sparsity of these photographs. We propose to use the 3D city and terrain models of the current scene, as well as a 3D point cloud if available, to simultaneously reproject and blend many historical images using an image-based rendering approach. Our contributions raise significantly the number of projective textures blended per rendering pass (typically from 8 to 40) on triangular meshes (of the 3D city and terrain models) and on point clouds. As a first step to tackle diachrony artifacts, we also propose a simple point cloud classification to filter in the shader the points corresponding to building or terrain details from the points corresponding to transient objects.

Omnidirectional camera pose estimation and projective texture mapping for photorealistic 3D virtual reality experiences

<p class="Abstract">Modern applications in virtual reality require a high level of fruition of the environment as if it was real. In applications that have to deal with real scenarios, it is important to acquire both its three-dimensional (3D) structure and details to enable the users to achieve good immersive experiences. The purpose of this paper is to illustrate a method to obtain a mesh with high quality texture combining a raw 3D mesh model of the environment and 360 <em>°</em> images. The main outcome is a mesh with a high level of photorealistic details. This enables both a good depth perception thanks to the mesh model and high visualization quality thanks to the 2D resolution of modern omnidirectional cameras. The fundamental step to reach this goal is the correct alignment between the 360 ° camera and the 3D mesh model. For this reason, we propose a method that embodies two steps: 1) find the 360 <em>° </em>cameras pose within the current 3D environment; 2) project the high-quality 360 <em>° </em>image on top of the mesh. After the method description, we outline its validation in two virtual reality scenarios, a mine and city environment, respectively, which allows us to compare the achieved results with the ground truth.</p>

A user-friendly method for constructing realistic dental model based on two-dimensional/three-dimensional registration

PurposeThis paper aims to obtain a texture dental model with real images and improve the rendering effect of the dental model.Design/methodology/approachThe paper proposes a semiautomatic method to construct a realistic dental model with real images based on two-dimensional/three-dimensional (2D/3D) registration. First, a 3D digital dental model and three intraoral images are obtained by a 3D scanner and digital single-lens reflex camera. Second, the camera projection poses for every intraoral images are calculated by using the single-objective optimization algorithm. Third, with camera poses, the preliminary projection texture mapping is performed; besides, the seam between two textures is marked. Finally, the marked regions are fused based on the image pyramid to eliminate obvious seams.FindingsThe paper provides a method to construct a realistic dental model. The method can map three intraoral images to the dental model. The experimental results show that the textured dental model without obvious distortion, dislocation and seams is constructed with simple interactions.Originality/valueThe proposed method can be applied to the digital smile design system to improve the communication efficiency between doctors, patients and technicians.

Exploring interaction techniques for 360 panoramas inside a 3D reconstructed scene for mixed reality remote collaboration

Remote collaboration using mixed reality (MR) enables two separated workers to collaborate by sharing visual cues. A local worker can share his/her environment to the remote worker for a better contextual understanding. However, prior techniques were using either 360 video sharing or a complicated 3D reconstruction configuration. This limits the interactivity and practicality of the system. In this paper we show an interactive and easy-to-configure MR remote collaboration technique enabling a local worker to easily share his/her environment by integrating 360 panorama images into a low-cost 3D reconstructed scene as photo-bubbles and projective textures. This enables the remote worker to visit past scenes on either an immersive 360 panoramic scenery, or an interactive 3D environment. We developed a prototype and conducted a user study comparing the two modes of how 360 panorama images could be used in a remote collaboration system. Results suggested that both photo-bubbles and projective textures can provide high social presence, co-presence and low cognitive load for solving tasks while each have its advantage and limitations. For example, photo-bubbles are good for a quick navigation inside the 3D environment without depth perception while projective textures are good for spatial understanding but require physical efforts.

Real-time 3D reconstruction method using massive multi-sensor data analysis and fusion

This paper proposes a method to reconstruct three-dimensional (3D) objects using real-time fusion and analysis of multiple sensor data. This paper attempts to create a realistic 3D visualization with which a remote pilot can intuitively control a remote unmanned robot by utilizing the characteristics of massive sensor data. The 3D reconstruction system proposed in this paper is comprised of 3D and two-dimensional (2D) data segmentation method, a 3D reconstruction method applied to each object, and a projective texture mapping method. Specifically, we propose applying both a 2D region extraction method and a 3D mesh modeling method to each object. The proposed schemes are implemented as a real-time application to verify real-time performance. This paper proves that 3D meshes can be modeled in real time by using the proposed method. The proposed method allows the remote control of a robot for real-time 3D rendering of remote scenes, which is essential for various tasks in areas that cannot be easily accessed by humans.

Fast Texture Mapping Adjustment via Local/Global Optimization.

This paper deals with the texture mapping of a triangular mesh model given a set of calibrated images. Different from the traditional approach of applying projective texture mapping with model parameterizations, we develop an image-space texture optimization scheme that aims to reduce visible seams or misalignment at texture or depth boundaries. Our novel scheme starts with an efficient local (and parallel) texture adjustment scheme at these boundaries, followed by a global correction step to rectify potential texture distortions caused by the local movement. Our phased optimization scheme achieves 50$\sim$∼100 times speed up on GPU (or 6× on CPU) compared to previous state-of-the-art methods. Experiments on a variety of models showed that we achieve this significant speed-up without sacrificing texture quality. Our approach significantly improves resilience to modeling and calibration errors, thereby allowing fast and fully automatic creation of textured models using commodity depth sensors by untrained users.

Specification and Implementation of Projective Texturing Node in X3D

Extensible 3D (X3D) is the ISO standard for defining 3D interactive web- and broadcast-based 3D content integrated with multimedia. With the advent of this integration of interactive 3D graphics into the web, users can easily produce 3D scenes within web contents. Even though there are diverse texture nodes in X3D, projective textures are not provided. We enable X3D to provide Singular Projective Texture and Multi Projective Texture nodes by materializing independent nodes of projector nodes for a singular projector and multi-projector. Our approach takes the creation of an independent projective texture node instead of Kamburelis’s method, which requires inconvenient and duplicated specifications of two nodes, Image Texture and Texture Coordinate.

GEOMETRY AND TEXTURE MEASURES FOR INTERACTIVE VIRTUALIZED REALITY INDOOR MODELER

Abstract. This paper discusses the algorithm to detect the distorted textures in the virtualized reality indoor models and automatically generate the necessary 3D planes to hold the undistorted textures. Virtualized reality (VR) interactive indoor modeler, our previous contribution enables the user to interactively create their desired indoor VR model from a single 2D image. The interactive modeler uses the projective texture mapping for mapping the textures over the manually created 3D planes. If the user has not created the necessary 3D planes, then the texture that belong to various objects are projected to the available 3D planes, which leads to the presence of distorted textures. In this paper, those distorted textures are detected automatically by the suitable principles from the shape from texture research. The texture distortion features such as the slant, tilt and the curvature parameters are calculated from the 2D image by means of affine transformation measured between the neighboring texture patches within the single image. This kind of affine transform calculation from a single image is useful in the case of deficient multiple view images. The usage of superpixels in clustering the textures corresponding to different objects, reduces the modeling labor cost. A standby database also stores the repeated basic textures that are found in the indoor model, and provides texture choices for the distorted floor, wall and other regions. Finally, this paper documents the prototype implementation and experiments with the automatic 3D plane creation and distortion detection with the above mentioned principles in the virtualized reality indoor environment.

Projective Texturing Uncertain Geometry: silhouette-aware box-filtered blending using integral radial images

Abstract. Projective texturing is a commonly used image based rendering technique that enables the synthesis of novel views from the blended reprojection of nearby views on a coarse geometry proxy approximating the scene. When scene geometry is inexact, aliasing artefacts occur. This introduces disturbing artefacts in applications such as street-level immersive navigation in mobile mapping imagery, since a pixel-accurate modelling of the scene geometry and all its details is most of the time out of question. The filtered blending approach applies the necessary 1D low-pass filtering on the projective texture to trade out the aliasing artefacts at the cost of some radial blurring. This paper proposes extensions of the filtered blending approach. Firstly, we introduce Integral Radial Images that enable constant time radial box filtering and show how they can be used to apply box-filtered blending in constant time independently of the amount of depth uncertainty. Secondly, we show a very efficient application of filtered blending where the scene geometry is only given by a loose depth interval prior rather than an actual geometry proxy. Thirdly, we propose a silhouette-aware extension of the box-filtered blending that not only account for uncertain depth along the viewing ray but also for uncertain silhouettes that have to be blurred as well.

Image-Based Rendering of LOD1 3D City Models for traffic-augmented Immersive Street-view Navigation

Abstract. It may be argued that urban areas may now be modeled with sufficient details for realistic fly-through over the cities at a reasonable price point. Modeling cities at the street level for immersive street-view navigation is however still a very expensive (or even impossible) operation if one tries to match the level of detail acquired by street-view mobile mapping imagery. This paper proposes to leverage the richness of these street-view images with the common availability of nation-wide LOD1 3D city models, using an image-based rendering technique : projective multi-texturing. Such a coarse 3D city model may be used as a lightweight scene proxy of approximate coarse geometry. The images neighboring the interpolated viewpoint are projected onto this scene proxy using their estimated poses and calibrations and blended together according to their relative distance. This enables an immersive navigation within the image dataset that is perfectly equal to – and thus as rich as – original images when viewed from their viewpoint location, and which degrades gracefully in between viewpoint locations. Beyond proving the applicability of this preprocessing-free computer graphics technique to mobile mapping images and LOD1 3D city models, our contributions are three-fold. Firstly, image distortion is corrected online in the GPU, preventing an extra image resampling step. Secondly, externally-computed binary masks may be used to discard pixels corresponding to moving objects. Thirdly, we propose a shadowmap-inspired technique that prevents, at marginal cost, the projective texturing of surfaces beyond the first, as seen from the projected image viewpoint location. Finally, an augmented visualization application is introduced to showcase the proposed immersive navigation: images are unpopulated from vehicles using externally-computed binary masks and repopulated using a 3D visualization of a 2D traffic simulation.

GPU Geocorrection for Airborne Pushbroom Imagers

We present a method for real-time geocorrection of images from airborne pushbroom sensors using the hardware acceleration and parallel computing characteristics of modern graphics processing units. This allows very fast geocorrection without accuracy loss with respect to traditional direct methods, with very little computational load for the central processor. Our method is based on a projective texture technique originally developed for fast shadow rendering. By combining the image data with inertial navigation and positioning ancillary data, we correct the image to a digital terrain model (DTM) and produce a geocorrected and georeferenced image. The method works independently of the number of channels of the sensor. Results with an ultralight hyperspectral system show that the speed achieved with standard hardware for a 1-m grid DTM gives better than real-time performance. This allows in-flight inspection of the geocorrected image during acquisition, for more efficient coverage of large target areas. In regard to accuracy, differences with standard ray tracing direct geocorrection remained subpixel.

INCREASE OF READABILITY AND ACCURACY OF 3D MODELS USING FUSION OF CLOSE RANGE PHOTOGRAMMETRY AND LASER SCANNING

Abstract. The development of laser scanning technology has opened a new page in geodesy and enabled an entirely new way of presenting data. Products obtained by the method of laser scanning are used in many sciences, as well as in archaeology. It should be noted that 3D models of archaeological artefacts obtained by laser scanning are fully measurable, written in 1:1 scale and have high accuracy. On the other hand, texture and RGB values of the surface of the object obtained by a laser scanner have lower resolution and poorer radiometric characteristics in relation to the textures captured with a digital camera. Scientific research and the goal of this paper are to increase the accuracy and readability of the 3D model with textures obtained with a digital camera. Laser scanning was performed with triangulation scanner of high accuracy, Vivid 9i (Konica Minolta), while for photogrammetric recording digital camera Nikon D90 with a lens of fixed focal length 20 mm, was used. It is important to stress that a posteriori accuracy score of the global registration of point clouds in the form of the standard deviation was ± 0.136 mm while the average distance was only ± 0.080 mm. Also research has proven that the quality projection texture model increases readability. Recording of archaeological artefacts and making their photorealistic 3D model greatly contributes to archaeology as a science, accelerates processing and reconstruction of the findings. It also allows the presentation of findings to the general public, not just to the experts.

Simulation of Realistic Water on 3D Game Scene

Realistic water simulation has been one of the classic challenges of real-time computer graphics, especially in 3D games and animations. The paper proposed an improved water simulation method based on sine function. Using GPU programming, realized real-time water rendering on 3D game scene. At the same time, by using cube map, rendering to a texture, projective texturing and normal mapping techniques, etc, the paper implemented the reflection and refraction, and put forward a simple method to enhance waves and ripples effect. Experimental results demonstrate this simulation method is realistic, rapid and applicable in real-time 3D games.

Real-Time Ocean Water Animation in Cartoon Style

This paper presents a model for automatically generating 3D cartoon ocean water animations in real-time. The dynamic ocean water surface model is modeled by a spectral method. The cartoon rendering process is implemented by multipass on GPU: First, we code normal of ocean model and generate normal map. Next, we extract discontinuities from normal map and smooth it into edge map. Finally we combine the edge map with cartoon shading based on a projective texture mapping. Some experimental results demonstrate the prettiness and efficiency of the presented model.

Visualization of Large-Scale Urban Models through Multi-Level Relief Impostors

In this paper, we present an efficient approach for the interactive rendering of large-scale urban models, which can be integrated seamlessly with virtual globe applications. Our scheme fills the gap between standard approaches for distant views of digital terrains and the polygonal models required for close-up views. Our work is oriented towards city models with real photographic textures of the building facades. At the heart of our approach is a multi-resolution tree of the scene defining multi-level relief impostors. Key ingredients of our approach include the pre-computation of a small set of zenithal and oblique relief maps that capture the geometry and appearance of the buildings inside each node, a rendering algorithm combining relief mapping with projective texture mapping which uses only a small subset of the pre-computed relief maps, and the use of wavelet compression to simulate two additional levels of the tree. Our scheme runs considerably faster than polygonal-based approaches while producing images with higher quality than competing relief-mapping techniques. We show both analytically and empirically that multi-level relief impostors are suitable for interactive navigation through large urban models.

On-line modeling for real-time 3D target tracking

Model-based object tracking can provide autonomous mobile robotic systems with real-time 6-dof pose information, for example, enabling them to rendezvous with targets from a particular desired direction. Most existing model-based trackers, however, require the geometric model of the target to be known a priori, which may pose a practical problem in real-world environments. This paper presents a novel 3D modeler capable of building an approximate model of a target object on-line. The proposed technique rapidly constructs a 3D tessellated enveloping mesh and uses projective texture mapping to further model the target object’s surface features. Separation of the target object from background clutter is achieved via customizable interest filters. The resulting real-time object-tracking system was tested extensively via simulations and experiments.

Floating Textures

AbstractWe present a novel multi‐view, projective texture mapping technique. While previous multi‐view texturing approaches lead to blurring and ghosting artefacts if 3D geometry and/or camera calibration are imprecise, we propose a texturing algorithm that warps (“floats”) projected textures during run‐time to preserve crisp, detailed texture appearance. Our GPU implementation achieves interactive to real‐time frame rates. The method is very generally applicable and can be used in combination with many image‐based rendering methods or projective texturing applications. By using Floating Textures in conjunction with, e.g., visual hull rendering, light field rendering, or free‐viewpoint video, improved rendering results are obtained from fewer input images, less accurately calibrated cameras, and coarser 3D geometry proxies.

Projective texture atlas construction for 3D photography

The use of attribute maps for 3D surfaces is an important issue in geometric modeling, visualization and simulation. Attribute maps describe various properties of a surface that are necessary in applications. In the case of visual properties, such as color, they are also called texture maps. Usually, the attribute representation exploits a parametrization g:U⊂ℝ2→ℝ3 of a surface in order to establish a two-dimensional domain where attributes are defined. However, it is not possible, in general, to find a global parametrization without introducing distortions into the mapping. For this reason, an atlas structure is often employed. The atlas is a set of charts defined by a piecewise parametrization of a surface, which allows local mappings with small distortion. Texture atlas generation can be naturally posed as an optimization problem where the goal is to minimize both the number of charts and the distortion of each mapping. Additionally, specific applications can impose other restrictions, such as the type of mapping. An example is 3D photography, where the texture comes from images of the object captured by a camera [4]. Consequently, the underlying parametrization is a projective mapping. In this work, we investigate the problem of building and manipulating texture atlases for 3D photography applications. We adopt a variational approach to construct an atlas structure with the desired properties. For this purpose, we have extended the method of Cohen–Steiner et al. [6] to handle the texture mapping set-up by minimizing distortion error when creating local charts. We also introduce a new metric tailored to projective maps that is suited to 3D photography.

Rendering stylized highlights using projective textures

We present a method to render and control stylized highlights using projective textures for real-time applications. For each vertex of a given 3D model, we compute principal directions and their corresponding principal curvatures in the preprocessing step. We find the maximum specular intensity point on the surface of the model and compute the principal directions and curvatures at the point by bilinear interpolation. The position, orientation, and projection frustum of a texture projector are determined by the principal directions and curvatures at the point. Using a simple culling method, we reduce the number of triangles to compute the principal directions and curvatures. We can control the shape, size, position, and orientation of a stylized highlight in an easy, fast, and intuitive manner. We discuss the limitations of our method and also give approximate solutions for some limitations.