Synthetic Objects Research Articles

Dissipative Self-Assembly of Dynamic Multicompartmentalized Microsystems with Light-Responsive Behaviors

Summary Living cells are compartmentalized systems operating far from thermodynamic equilibrium and consume energy from the environment to carry out their functions. The construction of biomimetic synthetic compartments has both scientific and technological value. Despite many advances, the assembly of synthetic compartments that present nonequilibrium behaviors based on dissipative (or transient) self-assembly remains a grand challenge. Here, we describe the design and synthesis of multicompartmental systems with an active microstructure and complex behaviors. By including an artificial pathway for energy dissipation, we demonstrate the autonomous generation of active polymeric systems using a strategy based on polymerization-induced self-assembly (PISA), driven by chemical fuel. Remarkably, these self-organized synthetic systems exhibit a dynamic change in their multicompartmental structures when exposed to light. This work introduces a strategy toward the design and construction of synthetic compartments with a dynamic structure and also presents pathways to engineer materials with spatially controllable structures and functionalities.

Customizable GAN: A Method for Image Synthesis of Human Controllable

In the research of computer vision, artificial controllability of image synthesis is a significant and challenging task. At present, there are two available methods. One is to utilize a simple contour to determine the shape of the synthetic object. This method has a promising effect, but it can only control the shape information of the synthetic object, but not the specific content. The other is to employ the text description to synthesize the corresponding image, which effectively controls the specific content of the synthesis, but it cannot do anything for the synthesized shape. In this paper, we propose a highly flexible and human customizable image synthesis model based on simple contour and natural language description, in which the specific content of contour and text description can be determined artificially. The contour determines basic synthetic object shape, and the natural language describes specific object content. Based on these, highly authentic and customizable images can be synthesized. The experiments are executed in the Caltech-UCSD Birds (CUB) and Oxford-102 flower datasets, and the experimental results demonstrate the effectiveness and superiority of our method. The results not only maintain the contour but also conform to the natural language description. Simultaneously, the high-quality image synthesis results, based on artificial hand-drawing contour and text description, are displayed to illustrate the high flexibility and customizability of our model.

Removal of Non-Gaussian Jitter Noise for Shape From Focus Through Improved Maximum Correntropy Criterion Kalman Filter

Three-dimensional (3D) shape reconstruction from one or multiple observations is a primary problem of computer vision. Shape from Focus (SFF) is a passive optical method that uses multiple two-dimensional (2D) images with different focus levels. When obtaining 2D images in each step along the optical axis, mechanical vibrations, referred as jitter noise, occur. SFF techniques are vulnerable to jitter noise that can vary focus values in 2D images. In this paper, new filtering method, which provides high accuracy of 3D shape reconstruction and low computational cost, is proposed. First, jitter noise is modeled as Levy distribution. This assumption makes it possible to show the influence of proposed filtering method in real environment with non-Gaussian jitter noise. Second, focus curves are modeled as Gaussian function to compare the performance of proposed filtering method with those of the conventional filtering methods. Finally, improved maximum correntropy criterion Kalman filter (IMCC-KF) is designed as a post-processing step, and is applied to the modeled focus curves. The experiments are performed on real and synthetic objects and the results demonstrate the effectiveness of proposed method.

Domain-Translated 3D Object Pose Estimation.

Synthetic 3D object models have been proven crucial in object pose estimation, as they are utilized to generate a huge number of accurately annotated data. The object pose estimation problem is usually solved for images originating from the real data domain by employing synthetic images for training data enrichment, without fully exploiting the fact that synthetic and real images may have different data distributions. In this work, we argue that 3D object pose estimation problem is easier to solve for images originating from the synthetic domain, rather than the real data domain. To this end, we propose a 3D object pose estimation framework consisting of a two-step process, where a novel pose-oriented image-to-image translation step is first employed to translate noisy real images to clean synthetic ones and then, a 3D object pose estimation method is applied on the translated synthetic images to finally predict the 3D object poses. A novel pose-oriented objective function is employed for training the image-to-image translation network, which enforces that pose-related object image characteristics are preserved in the translated images. As a result, the pose estimation network does not require real data for training purposes. Experimental evaluation has shown that the proposed framework greatly improves the 3D object pose estimation performance, when compared to state-of-the-art methods.

Color and Object Appearance in Augmented Reality

This presentation will summarize recent work on the visual perception of color appearance and object properties in optical see-through (OST) augmented reality (AR) systems. OST systems, such as Microsoft HoloLens, use a see- through display system to superimpose virtual content onto a user’s view of the real world. With careful tracking of both display and world coordinates, synthetic objects can be added to the real world, and real objects can be manipulated via synthetic overlays. Ongoing research studies how the combination of real and virtual stimuli are perceived and how users’ visual adaptation is affected; two specific examples will be explained.

Surface Reconstruction for Three-Dimensional Rockfall Volumetric Analysis

Laser scanning is routinely being used for the characterization and management of rockfall hazards. A key component of many studies is the ability to use the high-resolution topographic datasets for detailed volume estimates. 2.5-Dimensional (2.5D) approaches exist to estimate the volume of rockfall events; however these approaches require rasterization of the point cloud. These 2.5D volume estimates are therefore sensitive to picking an appropriate cell size to preserve resolution while minimizing interpolation, especially for lower volume rockfall events. To overcome the limitations of working with 2.5D raster datasets, surface reconstruction methods originating from the field of computational geometry can be implemented to assess the volume of rockfalls in 3D. In this technical note, the authors address the methods and implications of how the surface of 3D rockfall objects, derived from sequential terrestrial laser scans (TLS), are reconstructed for volumetric analysis. The Power Crust, Convex Hull and Alpha-shape algorithms are implemented to reconstruct a synthetic rockfall object generated in Houdini, a procedural modeling and animation software package. The reconstruction algorithms are also implemented for a selection of three rockfall cases studies which occurred in the White Canyon, British Columbia, Canada. The authors find that there is a trade-off between accurate surface topology reconstruction and ensuring the mesh is watertight manifold; which is required for accurate volumetric estimates. Power Crust is shown to be the most robust algorithm, however, the iterative Alpha-shape approach introduced in the study is also shown to find a balance between hole-filling and loss of detail.

Robotic Manipulation of Deformable Cells for Orientation Control

Robotic manipulation of deformable objects has been a classic topic in robotics. Compared to synthetic deformable objects such as rubber balls and clothes, biological cells are highly deformable and more prone to damage. This article presents robotic manipulation of deformable cells for orientation control (both out-of-plane and in-plane), which is required in both clinical (e.g., in vitro fertilization) and biomedical (e.g., clone) applications. Compared to manual cell orientation control based on empirical experience, the robotic approach, based on modeling and path planning, effectively rotates a cell, while consistently maintaining minimal cell deformation to avoid cell damage. A force model is established to determine the minimal force applied by the micropipette to rotate a spherical or, more generally, ellipsoidal oocyte. The force information is translated into indentation through a contact mechanics model, and the manipulation path of the micropipette is formed by connecting the indentation positions on the oocyte. An optimal controller is designed to compensate for the variations of mechanical properties across oocytes. The polar body of an oocyte is detected by deep neural networks with robustness to shape and size differences. In experiments, the system achieved an accuracy of 97.6% in polar body detection and an accuracy of 0.7 $^{\circ }$ in oocyte orientation control with maximum oocyte deformation of 2.70 $\mu$ m throughout the orientation control process.

3D Shape Recovery by Aggregating 3D Wavelet Transform-Based Image Focus Volumes Through 3D Weighted Least Squares

This paper proposes a shape from focus method based on high-frequency components from 3D discrete wavelet transform. First, an input image sequence is decomposed into approximation and detail components up to certain levels. Then, for each level, an image focus volume is obtained from the energy of detail components. These image focus volumes contain varying sized structural information about the shape of the object. From this set of image focus volumes, a single image focus volume is obtained through cross-scale aggregation of image focus volumes by applying 3D weighted least squares. The weights for the smoothness term for each pixel have been computed from the cross-scale prior. Incorporating this cross-scale prior enables the multi-scale interaction for image focus volume aggregation. Finally, the depth map is recovered from the resultant image focus volume using the best focused pixels along the optical axis. The proposed method is evaluated using image sequences of synthetic and real objects. The experimental results demonstrate the effectiveness of the proposed method.

MPS-Net: Learning to recover surface normal for multispectral photometric stereo

Multispectral Photometric Stereo (MPS) estimates per-pixel surface normals from one single image captured under three colored (red, green and blue) light sources. Unlike traditional Photometric Stereo, MPS can therefore be used in dynamic scenes for single frame reconstruction. However, MPS is challenging due to the tangle of the illumination, surface reflectance and camera response, causing inaccurate estimation of surface normal. Existing approaches rely on either extra depth information or materials calibration strategies, thus limiting its usage in practical applications. In this paper, we propose a Multispectral Photometric Stereo Network (MPS-Net) to solve this under-determined system. The MPS-Net takes the single multispectral image and an initial surface normal estimation obtained from this image itself, and outputs an accurate surface normal map, where no extra depth or materials calibration information is required. We show that the MPS-Net is not constrained to Lambertian surfaces and can be applied to surfaces with complex reflectance. We evaluated the MPS-Net using both synthetic and real objects of various materials. Our experiment results show that the MPS-Net outperforms the state-of-the-art approaches.

Kritické gesto a záležitosti péče v souřadnicích ne/způsobilosti

Critique is one of the social sciences' most respectable tasks, especially when its aim is to emancipate people oppressed for their otherness. However, there is also a critique of critique as a disabling tool, replacing the obvious actors revealed as 'fictitious' with synthetic objects that the critic herself deems more 'factual'. This article understands the critical gesture as a pragmatic resource for re-organising the field of dis/abilities. In the first part of the article, we make three critical gestures together with Jose, a person identified as mentally ill. A paranoid vision of a secret conspiracy, a naturalising concept of disease, and the critique of stigma all seek to radically redraw the dis/ability coordinates, but their emancipatory potential is thwarted by the complex interconnectedness of their objects. Jose's recovery thus ultimately hinges on a delicate balancing act combining critique and composing. In this sense, his effort resembles the careful treading of lay and professional critics in the last part of our text, in which we try to solve problems of living with dementia together with the Hanus family. While the critical gesture has an essential role to play here as well, close ethnographic encounters are rather about jointly articulating the critical matters of care, wherein the problematic agencies of both obvious and not-so-obvious actors are acknowledged.

Sampling Based on Kalman Filter for Shape from Focus in the Presence of Noise

Recovering three-dimensional (3D) shape of an object from two-dimensional (2D) information is one of the major domains of computer vision applications. Shape from Focus (SFF) is a passive optical technique that reconstructs 3D shape of an object using 2D images with different focus settings. When a 2D image sequence is obtained with constant step size in SFF, mechanical vibrations, referred as jitter noise, occur in each step. Since the jitter noise changes the focus values of 2D images, it causes erroneous recovery of 3D shape. In this paper, a new filtering method for estimating optimal image positions is proposed. First, jitter noise is modeled as Gaussian or speckle function, secondly, the focus curves acquired by one of the focus measure operators are modeled as a quadratic function for application of the filter. Finally, Kalman filter as the proposed method is designed and applied for removing jitter noise. The proposed method is experimented by using image sequences of synthetic and real objects. The performance is evaluated through various metrics to show the effectiveness of the proposed method in terms of reconstruction accuracy and computational complexity. Root Mean Square Error (RMSE), correlation, Peak Signal-to-Noise Ratio (PSNR), and computational time of the proposed method are improved on average by about 48%, 11%, 15%, and 5691%, respectively, compared with conventional filtering methods.

Deep Learning Based Object Recognition Using Physically-Realistic Synthetic Depth Scenes

Recognizing objects and estimating their poses have a wide range of application in robotics. For instance, to grasp objects, robots need the position and orientation of objects in 3D. The task becomes challenging in a cluttered environment with different types of objects. A popular approach to tackle this problem is to utilize a deep neural network for object recognition. However, deep learning-based object detection in cluttered environments requires a substantial amount of data. Collection of these data requires time and extensive human labor for manual labeling. In this study, our objective was the development and validation of a deep object recognition framework using a synthetic depth image dataset. We synthetically generated a depth image dataset of 22 objects randomly placed in a 0.5 m × 0.5 m × 0.1 m box, and automatically labeled all objects with an occlusion rate below 70%. Faster Region Convolutional Neural Network (R-CNN) architecture was adopted for training using a dataset of 800,000 synthetic depth images, and its performance was tested on a real-world depth image dataset consisting of 2000 samples. Deep object recognizer has 40.96% detection accuracy on the real depth images and 93.5% on the synthetic depth images. Training the deep learning model with noise-added synthetic images improves the recognition accuracy for real images to 46.3%. The object detection framework can be trained on synthetically generated depth data, and then employed for object recognition on the real depth data in a cluttered environment. Synthetic depth data-based deep object detection has the potential to substantially decrease the time and human effort required for the extensive data collection and labeling.

Faux materials and aspirational identity: Celluloid combs and working class dreams in the Pennsylvania anthracite region

The introduction of synthetic and semi-synthetic materials in the late 19th and early 20th centuries opened a world of new possibilities for consumer objects. The abundance of new goods ushered in a new age of consumption: one in which identities were not only expressed through material culture, but could be actively made through material culture. One avenue that has not been thoroughly explored is the role of synthetic and semi-synthetic objects in negotiating both association with and distance from different forms of identity in the archaeological record. This article explores the cultural connotations underlying a small find recovered from a working class household in the anthracite coal company town of Eckley, Pennsylvania. Using the example of a celluloid lice comb, the author discusses the ways that mass production fundamentally changed status signaling in the Western world at the beginning of the 20th century and offers thoughts on improving our interpretations of the social consequences of objects within historical archaeological research.

NEURAL NETWORK CLASSIFICATION METHOD FOR AIRCRAFT IN ISAR IMAGES

This article offers a neural network method for automatic classification of Inverse Synthetic Aperture Radar objects represented in images with high level of post-receive optimization. A full explanation of the procedures of two-layer neural network architecture creating and training is described. The classification in the recognition stage is proposed, based on several main classes or sets of flying objects. The classification sets are designed according to distinctive specifications in the structural models of the aircrafts. The neural network is experimentally simulated in MATLAB environment. Numerical results of the experiments carried, prove the correct classification of the objects in ISAR optimized images.

Jitter Elimination in Shape Recovery by using Adaptive Neural Network Filter.

Three-dimensional (3D) cameras are expensive because they employ additional charged coupled device sensors and optical elements, e.g., lasers or complicated scanning mirror systems. One passive optical method, shape from focus (SFF), provides an efficient low cost solution for 3D cameras. However, mechanical vibration of the SFF imaging system causes jitter noise along the optical axis, which makes it difficult to obtain accurate shape information of objects. In traditional methods, this error cannot be removed and increases as the estimation of the shape recovery progresses. Therefore, the final 3D shape may be inaccurate. We introduce an accurate depth estimation method using an adaptive neural network (ANN) filter to remove the jitter noise effects. Jitter noise is modeled by both Gaussian distribution and non-Gaussian distribution. Then, focus curves are modeled by quadratic functions. The ANN filter is designed as an optimal estimator restoring the original position of each frame of the input image sequence in the modeled jitter noise, as a pre-processing step before the initial depth map is obtained. The proposed method was evaluated using image sequences of both synthetic and real objects. Experimental results demonstrate that it is reasonably efficient and that its accuracy is comparable with that of existing systems.

Radial-Based oversampling for noisy imbalanced data classification

Imbalanced data classification remains a focus of intense research, mostly due to the prevalence of data imbalance in various real-life application domains. A disproportion among objects from different classes may significantly affect the performance of standard classification models. The first problem is the high imbalance ratios that pose a serious learning difficulty and require usage of dedicated methods, capable of alleviating this issue. The second important problem which may appear is noise, which may be accompanying the training data and causing strong deterioration of the classifier performance or increase the time required for its training. Therefore, the desirable classification model should be robust to both skewed data distributions and noise. One of the most popular approaches for handling imbalanced data is oversampling of the minority objects in their neighborhood. In this work we will criticize this approach and propose a novel strategy for dealing with imbalanced data, with particular focus on the noise presence. We propose Radial-Based Oversampling (RBO) method, which can find regions in which the synthetic objects from minority class should be generated on the basis of the imbalance distribution estimation with radial basis functions. Results of experiments, carried out on a representative set of benchmark datasets, confirm that the proposed guided synthetic oversampling algorithm offers an interesting alternative to popular state-of-the-art solutions for imbalanced data preprocessing.

3D Affine: An Embedding of Local Image Features for Viewpoint Invariance Using RGB-D Sensor Data

Local image features are invariant to in-plane rotations and robust to minor viewpoint changes. However, the current detectors and descriptors for local image features fail to accommodate out-of-plane rotations larger than 25°–30°. Invariance to such viewpoint changes is essential for numerous applications, including wide baseline matching, 6D pose estimation, and object reconstruction. In this study, we present a general embedding that wraps a detector/descriptor pair in order to increase viewpoint invariance by exploiting input depth maps. The proposed embedding locates smooth surfaces within the input RGB-D images and projects them into a viewpoint invariant representation, enabling the detection and description of more viewpoint invariant features. Our embedding can be utilized with different combinations of descriptor/detector pairs, according to the desired application. Using synthetic and real-world objects, we evaluated the viewpoint invariance of various detectors and descriptors, for both standalone and embedded approaches. While standalone local image features fail to accommodate average viewpoint changes beyond 33.3°, our proposed embedding boosted the viewpoint invariance to different levels, depending on the scene geometry. Objects with distinct surface discontinuities were on average invariant up to 52.8°, and the overall average for all evaluated datasets was 45.4°. Similarly, out of a total of 140 combinations involving 20 local image features and various objects with distinct surface discontinuities, only a single standalone local image feature exceeded the goal of 60° viewpoint difference in just two combinations, as compared with 19 different local image features succeeding in 73 combinations when wrapped in the proposed embedding. Furthermore, the proposed approach operates robustly in the presence of input depth noise, even that of low-cost commodity depth sensors, and well beyond.

Signal Processing on Static and Dynamic 3D Meshes: Sparse Representations and Applications

Nowadays, real-time 3D scanning and reconstruction becomes a requirement for a variety of interactive applications in various fields, including heritage science, gaming, engineering, landscape topography, and medicine. From the introduction of 3D scanning, which allowed the representation of real world or synthetic objects into the virtual world, hardware and software advances have seen tremendous progress. However, despite the continuous improvement of the new generation image sensors and acquisition techniques, the acquired data are often corrupted by the low-frequency noise, outliers, misalignment, missing data, and variations in point density. These effects are amplified if the low-cost sensors and hardware are being used (e.g., mobile devices); thus, the acquisition and communication cost per datum is driven to a minimum. This paper provides a comprehensive review of the ongoing efforts in geometry and signal processing, describing several models from a wide range of signal processing relevant tasks, such as robust principal component analysis, compressive sampling, and matrix completion. Various scalable architectures and optimization algorithms are analyzed and reviewed, revealing significant insights into the fundamental processing operations and the involved implementation tradeoffs. Moreover, the impact of sparse modeling and optimization tools to several 3D mesh processing tasks, such as completion of missing data, feature preserving noise removal, and rejection of outliers, is illustrated via test cases with several constraints posed by the arbitrarily complex animated scenarios. Finally, the identified limitations together with the potential open research directions are also presented for future research efforts toward modeling and optimization for static and dynamic 3D models.

Improving Depth Computation From Robust Focus Approximation

Shape-from-focus (SFF) technique is to recover three-dimensional shape of an object from a sequence of two-dimensional images of the same object taken by gradually changing the focus settings of the imaging device. In SFF, for each pixel location (i, j), a focus measure operator computes sharpness (focus) value on a small neighborhood of the pixel (i, j) at each image along the z-axis (optical axis). The image sensor position z that has the maximum focus value and camera parameter information provide the distance information between the lens and the object point corresponding to the pixel (i, j). In traditional SFF methods, the final focus value of each pixel is determined by an average value of the initial focus values at the neighborhood of the pixel to remove the noise effects. However, it only reduces the noise effects instead of completely removing it. In this paper, we proposed a noise filtering technique that tries to remove noises while computing the focus values. First, an initial focus measure volume is computed by applying one of the focus measures on each pixel in the image sequence. Then, the focus value at each pixel is examined whether it is corrupted by noise or not by analyzing the neighboring focus values. The noisy focus value of the pixel is re-computed from noise-free focus values of neighboring pixels. The experimental results conducted on both the synthetic and real-world objects show the proposed method produces the better three dimensional shape in comparison to the existing methods.

Optimizing Image Focus for Shape From Focus Through Locally Weighted Non-Parametric Regression

Shape from focus is a passive optical technique that uses multiple images with different focus levels. To compute the sharpness of each pixel in the image sequence, obtained by translating the object plane at a constant step size along the optical axis, focus measure operators are used. However, the focus measure operators do not provide the accurate focus value of the center pixel due to information loss between consecutive frames. In this paper, a local regression is proposed as a new fitting method for the focus curves. This method fits the focus curves, obtained by using one of the focus measure operators, by utilizing the weighted least squares regression as non-parametric regression. The experiments are performed on synthetic and real objects to demonstrate the effectiveness of the proposed method.