Synthetic Objects Research Articles

RPR-SORS: Real-time photorealistic rendering of synthetic objects into real scenes

This paper presents a review of the Photorealistic Augmented Reality field and proposes a solution for interactively rendering virtual objects into dynamic real scenes in a photorealistic way. This solution features a rendering pipeline that comprises techniques regarding illumination, reflectance model, shadowing, composition, and camera effects. The techniques are chained in a flexible way, allowing the user to choose which techniques are to be enabled. An environment map generation procedure was developed and allows virtual objects to exhibit coherent effects such as color bleeding and specular reflection, even when the real objects are moved. The range of materials that can be rendered was widened by extending Lafortune's Spatial BRDF. The implemented infrastructure is offered as an authoring toolkit that consists of an API and a material editor tool. The aim of this authoring toolkit is to increase development productivity of Photorealistic Augmented Reality applications. The proposed solution was evaluated by taking into account visual and performance metrics. It allowed consistent rendering of dynamic scenes and photorealistic materials. The frame rate obtained was suitable to Augmented Reality applications when there were few virtual objects in the scene.

A finite mixture model for simultaneous high-dimensional clustering, localized feature selection and outlier rejection

Model-based approaches and in particular finite mixture models are widely used for data clustering which is a crucial step in several applications of practical importance. Indeed, many pattern recognition, computer vision and image processing applications can be approached as feature space clustering problems. For complex high-dimensional data, however, the use of these approaches presents several challenges such as the presence of many irrelevant features which may affect the speed and also compromise the accuracy of the used learning algorithm. Another problem is the presence of outliers which potentially influence the resulting model’s parameters. For this purpose, we propose and discuss an algorithm that partitions a given data set without a priori information about the number of clusters, the saliency of the features or the number of outliers. We illustrate the performance of our approach using different applications involving synthetic data, real data and objects shape clustering.

Rendering synthetic objects into legacy photographs

We propose a method to realistically insert synthetic objects into existing photographs without requiring access to the scene or any additional scene measurements. With a single image and a small amount of annotation, our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene. We demonstrate in a user study that synthetic images produced by our method are confusable with real scenes, even for people who believe they are good at telling the difference. Further, our study shows that our method is competitive with other insertion methods while requiring less scene information. We also collected new illumination and reflectance datasets; renderings produced by our system compare well to ground truth. Our system has applications in the movie and gaming industry, as well as home decorating and user content creation, among others.

Searching surface orientation of microscopic objects for accurate 3D shape recovery

In this article, we propose a new shape from focus (SFF) method to estimate 3D shape of microscopic objects using surface orientation cue of each object patch. Most of the SFF algorithms compute the focus value of a pixel from the information of neighboring pixels lying on the same image frame based on an assumption that the small object patch corresponding to the small neighborhood of a pixel is a plane parallel to the focal plane. However, this assumption fails in the optics with limited depth of field where the neighboring pixels of an image have different degree of focus. To overcome this problem, we try to search the surface orientation of the small object patch corresponding to each pixel in the image sequence. Searching of the surface orientation is done indirectly by principal component analysis. Then, the focus value of each pixel is computed from the neighboring pixels lying on the surface perpendicular to the corresponding surface orientation. Experimental results on synthetic and real microscopic objects show that the proposed method produces more accurate 3D shape in comparison to the existing techniques.

3D Reconstruction using Interactive Shape from Shading

The Shape-From-Shading (SFS) problem is a fundamental and classical problem in computer vision. It is the process of computing three dimensional shape of a surface using its shading information from single image. In this paper, a new method for interactive shape from shading for reconstruction of a 3D shape is presented. Experimental results on several synthetic and real object images show that the proposed method reconstructs the 3D surface of objects very efficiently.

Modeling the Self-Assembly of Lipids and Nanotubes in Solution: Forming Vesicles and Bicelles with Transmembrane Nanotube Channels

Via dissipative particle dynamics (DPD), we simulate the self-assembly of end-functionalized, amphiphilic nanotubes and lipids in a hydrophilic solvent. Each nanotube encompasses a hydrophobic stalk and two hydrophilic ends, which are functionalized with end-tethered chains. With a relatively low number of the nanotubes in solution, the components self-assemble into stable lipid-nanotube vesicles. As the number of nanotubes is increased, the system exhibits a vesicle-to-bicelle transition, resulting in stable hybrid bicelle. Moreover, our results reveal that the nanotubes cluster into distinct tripod-like structures within the vesicles and aggregate into a ring-like assembly within the bicelles. For both the vesicles and bicelles, the nanotubes assume trans-membrane orientations, with the tethered hairs extending into the surrounding solution or the encapsulated fluid. Thus, the hairs provide a means of regulating the transport of species through the self-assembled structures. Our findings provide guidelines for creating nanotube clusters with distinctive morphologies that might be difficult to achieve through more conventional means. The results also yield design rules for creating synthetic cell-like objects or microreactors that can exhibit biomimetic functionality.

Design rules for creating sensing and self-actuating microcapsules

Using computational modeling, we design a pair of biomimetic microcapsules that exploit chemical mechanisms to communicate and alter their local environment. As a result, these synthetic objects can undergo autonomous, directed motion. In the simulations, signaling microcapsules release agonist particles, while target microcapsules release antagonist particles and the permeabilities of both capsule types depend on the local particle concentration in the surrounding solution. Additionally, the released nanoscopic particles can bind to the underlying substrate and thereby create adhesion gradients that propel the microcapsules to move. Hydrodynamic interactions and the feedback mechanism provided by the dissolved particles are both necessary to achieve the cooperative behavior exhibited by these microcapsules. Our model provides a platform for integrating both the spatial and temporal behavior of assemblies of artificial cells, and allows us to design a rich variety of structures capable of exhibiting complex dynamics. Due to the cell-like attributes of polymeric microcapsules and polymersomes, material systems are available for realizing our predictions.

Variational estimation of inhomogeneous specular reflectance and illumination from a single view

Estimating the illumination and the reflectance properties of an object surface from a few images is an important but challenging problem. The problem becomes even more challenging if we wish to deal with real-world objects that naturally have spatially inhomogeneous reflectance. In this paper, we derive a novel method for estimating the spatially varying specular reflectance properties of a surface of known geometry as well as the illumination distribution of a scene from a specular-only image, for instance, recovered from two images captured with a polarizer to separate reflection components. Unlike previous work, we do not assume the illumination to be a single point light source. We model specular reflection with a spherical statistical distribution and encode its spatial variation with a radial basis function (RBF) network of their parameter values, which allows us to formulate the simultaneous estimation of spatially varying specular reflectance and illumination as a constrained optimization based on the I-divergence measure. To solve it, we derive a variational algorithm based on the expectation maximization principle. At the same time, we estimate optimal encoding of the specular reflectance properties by learning the number, centers, and widths of the RBF hidden units. We demonstrate the effectiveness of the method on images of synthetic and real-world objects.

Synthetic foldamers

Foldamers are artificial folded molecular architectures inspired by the structures and functions of biopolymers. This highlight focuses on important developments concerning foldamers produced by chemical synthesis and on the perspectives that these new self-organized molecular scaffolds offer. Progress in the field has led to synthetic objects that resemble small proteins in terms of size and complexity yet that may not contain any α-amino acids. Foldamers have introduced new tools and concepts to develop biologically active substances, synthetic receptors and novel materials.

A graph-based framework for sub-pixel image segmentation

Many image segmentation methods utilize graph structures for representing images, where the flexibility and generality of the abstract structure is beneficial. By using a fuzzy object representation, i.e., allowing partial belongingness of elements to image objects, the unavoidable loss of information when representing continuous structures by finite sets is significantly reduced, enabling feature estimates with sub-pixel precision. This work presents a framework for object representation based on fuzzy segmented graphs. Interpreting the edges as one-dimensional paths between the vertices of a graph, we extend the notion of a graph cut to that of a located cut, i.e., a cut with sub-edge precision. We describe a method for computing a located cut from a fuzzy segmentation of graph vertices. Further, the notion of vertex coverage segmentation is proposed as a graph theoretic equivalent to pixel coverage segmentations and a method for computing such a segmentation from a located cut is given. Utilizing the proposed framework, we demonstrate improved precision of area measurements of synthetic two-dimensional objects. We emphasize that although the experiments presented here are performed on two-dimensional images, the proposed framework is defined for general graphs and thus applicable to images of any dimension.

Nearest Surrounder Queries

In this paper, we present a new type of spatial queries called Nearest Surrounder (NS) queries. An NS query determines the nearest polygon-shaped spatial objects (referred to as nearest surrounder objects) and their orientations with respect to a query point from an object set. Besides, we derive two NS query variants, namely, multitier NS (m-NS) queries and angle-constrained NS (ANS) queries. An m-NS query searches multiple layers of NS objects for the same range of angles from a query point. An ANS query searches for NS objects within a specified range of angles. To evaluate NS queries and their variants, we explore angle-based and distance-based bound properties of polygons, and devise two efficient algorithms, namely, Sweep and Ripple, based on R-tree. The algorithms access objects in an order according to their orientations and distances with respect to a given query point, respectively. They are efficient as they can finish a search with one index lookup. Besides, they can progressively deliver a query result. Through empirical studies, we evaluate the proposed algorithms and report their performance for both synthetic and real object sets.

Compositing images through light source detection

Compositing an image of an object into another image is a frequently occurring task in both image processing and augmented reality. To ensure a seamless composition, it is often necessary to infer the light conditions of the image to adjust the illumination of the inserted object. Here, we present a novel algorithm for multiple light detection that leverages the limitations of the human visual system (HVS) described in the literature and measured by our own psychophysical study. Finally, we show an application of our method to both image compositing and synthetic object insertion.

Detection and separation of generic‐shaped objects by fuzzy clustering

PurposeExisting shape‐based fuzzy clustering algorithms are all designed to explicitly segment regular geometrically shaped objects in an image, with the consequence that this restricts their capability to separate arbitrarily shaped objects. The purpose of this paper is to introduce a new detection and separation of generic‐shaped object algorithm.Design/methodology/approachWith the aim of separating arbitrary‐shaped objects in an image, this paper presents a new detection and separation of generic‐shaped objects (FKG) algorithm that analytically integrates arbitrary shape information into a fuzzy clustering framework, by introducing a shape constraint that preserves the original object shape during iterative scaling.FindingsBoth qualitative and numerical empirical results analysis corroborate the improved object segmentation performance achieved by the FKG strategy upon different image types and disparately shaped objects.Originality/valueThe proposed FKG algorithm can be highly used in applications where object segmentation is necessary. Likewise, this algorithm can be applied in Moving Picture Experts Group‐4 for real object segmentation that is already applied in synthetic object segmentation.

Spectral Symmetry Analysis

We present a spectral approach for detecting and analyzing rotational and reflectional symmetries in n-dimensions. Our main contribution is the derivation of a symmetry detection and analysis scheme for sets of points in IRn and its extension to image analysis by way of local features. Each object is represented by a set of points S 2 IRn, where the symmetry is manifested by the multiple self-alignments of S. The alignment problem is formulated as a quadratic binary optimization problem, with an efficient solution via spectral relaxation. For symmetric objects, this results in a multiplicity of eigenvalues whose corresponding eigenvectors allow the detection and analysis of both types of symmetry. We improve the scheme's robustness by incorporating geometrical constraints into the spectral analysis. Our approach is experimentally verified by applying it to 2D and 3D synthetic objects as well as real images.

Soft X-ray imaging of thick carbon-based materials using the normal incidence multilayer optics

The high transparency of carbon-containing materials in the spectral region of “carbon window” ( λ ∼ 4.5–5 nm) introduces new opportunities for various soft X-ray microscopy applications. The development of efficient multilayer coated X-ray optics operating at the wavelengths of about 4.5 nm has stimulated a series of our imaging experiments to study thick biological and synthetic objects. Our experimental set-up consisted of a laser plasma X-ray source generated with the 2nd harmonics of Nd–glass laser, scandium-based thin-film filters, Co/C multilayer mirror and X-ray film UF-4. All soft X-ray images were produced with a single nanosecond exposure and demonstrated appropriate absorption contrast and detector-limited spatial resolution. A special attention was paid to the 3D imaging of thick low-density foam materials to be used in design of laser fusion targets.

Designing communicating colonies of biomimetic microcapsules

Using computational modeling, we design colonies of biomimetic microcapsules that exploit chemical mechanisms to communicate and alter their local environment. As a result, these synthetic objects can self-organize into various autonomously moving structures and exhibit ant-like tracking behavior. In the simulations, signaling microcapsules release agonist particles, whereas target microcapsules release antagonist particles and the permeabilities of both capsule types depend on the local particle concentration in the surrounding solution. Additionally, the released nanoscopic particles can bind to the underlying substrate and thereby create adhesion gradients that propel the microcapsules to move. Hydrodynamic interactions and the feedback mechanism provided by the dissolved particles are both necessary to achieve the collective dynamics exhibited by these colonies. Our model provides a platform for integrating both the spatial and temporal behavior of assemblies of "artificial cells," and allows us to design a rich variety of structures capable of exhibiting complex, cooperative behavior. Due to the cell-like attributes of polymeric microcapsules and polymersomes, material systems are available for realizing our predictions.

MRT Letter: Optimal composite depth function for 3D shape recovery of microscopic objects

Generally, in shape from focus techniques, a single focus measure is used in estimating the three-dimensional structure of microscopic objects. However, the performance of a single focus measure is limited to estimate accurately the depth map of diverse type of objects. To cope with this problem, we propose genetic programming based novel approach by developing an optimal composite depth (OCD) function for accurate depth estimation. This OCD function optimally combines the initial depth and focus information extracted from individual focus measures. An improved performance of this function is reported for synthetic and real world microscopic objects.

Adaptable aging factory for multiple objects and colorations

The visual effects of synthetic objects and environments from modern video games and movies require an impressive amount of detail to properly duplicate their real world equivalents. Among these details, effects produced by aging are particularly hard to handle, and adding them is significantly time-consuming. Existing methods such as physically based and empirical simulations are not suitable for artists since they require the manipulation of complex physical parameters, and their results are difficult to control. Our approach offers a framework for quickly adding aging effects based on a simple example. By defining an aging recipe based on local properties, an artist can easily apply similar effects to different objects or to multiple occurrences of the same object. Also, when aging patterns consist of simple color variations, we propose a color-independent process capable of producing various colorations of the same effect from a single example.

Superresolved digital in-line holographic microscopy for high-resolution lensless biological imaging

Digital in-line holographic microscopy (DIHM) is a modern approach capable of achieving micron-range lateral and depth resolutions in three-dimensional imaging. DIHM in combination with numerical imaging reconstruction uses an extremely simplified setup while retaining the advantages provided by holography with enhanced capabilities derived from algorithmic digital processing. We introduce superresolved DIHM incoming from time and angular multiplexing of the sample spatial frequency information and yielding in the generation of a synthetic aperture (SA). The SA expands the cutoff frequency of the imaging system, allowing submicron resolutions in both transversal and axial directions. The proposed approach can be applied when imaging essentially transparent (low-concentration dilutions) and static (slow dynamics) samples. Validation of the method for both a synthetic object (U.S. Air Force resolution test) to quantify the resolution improvement and a biological specimen (sperm cells biosample) are reported showing the generation of high synthetic numerical aperture values working without lenses.

Recovering surface normal of specular object by Hough transform method

In order to recover surface normal and albedo of specular objects under the condition of large shadows and highlights by photometric stereo, a novel robust method utilising Hough transform is proposed. The photometric image sequence is taken by a fixed camera when the light source is rotated around the optical axis of the camera. Since the bidirectional reflectance distribution function of many surfaces can be approximated as a combination of a diffuse component and a specular component, the diffuse part of the intensity at a fixed point on the surface obeys a sine curve when the light source rotates. The surface normal and albedo information are hidden in the sine curve and can be revealed by means of Hough transform. The bilateral symmetry of the sine curve is used to narrow the searching range of the initial phase and to speed up the Hough transform. Experiments performed on synthetic and real objects with large shadows and strong specular reflectance show the better performance of the proposed method.