Texture Generation Research Articles

Thick Clouds Removal From Multitemporal ZY-3 Satellite Images Using Deep Learning

The presence of clouds greatly reduces the ground information of high-resolution satellite data. In order to improve the utilization of high-resolution satellite data, this article presents a cloud removal method based on deep learning. This is the first end-to-end architecture that has great potential to detect and remove clouds from high-resolution satellite data. For cloud detection, a convolution neural network (CNN) architecture is used to detect them. For cloud removal, the content generation network, the texture generation network, and the spectrum generation network based on traditional CNN are proposed. The proposed CNN architecture can use multisource data (content, texture, and spectral) as an input of the unified framework. The results of both the simulated and real image experiments demonstrate that the proposed method is robust and can effectively remove thick clouds, thin clouds, and cloud shadows. In addition, compared with some existing methods, the proposed method can recover land cover information accurately.

Anisotropic diffusion creep in postperovskite provides a new model for deformation at the core−mantle boundary

The lowermost portion of Earth's mantle (D″) above the core-mantle boundary shows anomalous seismic features, such as strong seismic anisotropy, related to the properties of the main mineral MgSiO3 postperovskite. But, after over a decade of investigations, the seismic observations still cannot be explained simply by flow models which assume dislocation creep in postperovskite. We have investigated the chemical diffusivity of perovskite and postperovskite phases by experiment and ab initio simulation, and derive equations for the observed anisotropic diffusion creep. There is excellent agreement between experiments and simulations for both phases in all of the chemical systems studied. Single-crystal diffusivity in postperovskite displays at least 3 orders of magnitude of anisotropy by experiment and simulation (Da = 1,000 Db; Db ≈ Dc) in zinc fluoride, and an even more extreme anisotropy is predicted (Da = 10,000 Dc; Dc = 10,000 Db) in the natural MgSiO3 system. Anisotropic chemical diffusivity results in anisotropic diffusion creep, texture generation, and a strain-weakening rheology. The results for MgSiO3 postperovskite strongly imply that regions within the D″ region of Earth dominated by postperovskite will 1) be substantially weaker than regions dominated by perovskite and 2) develop a strain-induced crystallographic-preferred orientation with strain-weakening rheology. This leads to strain localization and the possibility to bring regions with significantly varying textures into close proximity by strain on narrow shear zones. Anisotropic diffusion creep therefore provides an attractive alternative explanation for the complexity in observed seismic anisotropy and the rapid lateral changes in seismic velocities in D″.

Void Filling of Digital Elevation Models with a Terrain Texture Learning Model Based on Generative Adversarial Networks

Digital elevation models (DEMs) are an important information source for spatial modeling. However, data voids, which commonly exist in regions with rugged topography, result in incomplete DEM products, and thus significantly degrade DEM data quality. Interpolation methods are commonly used to fill voids of small sizes. For large-scale voids, multi-source fusion is an effective solution. Nevertheless, high-quality auxiliary source information is always difficult to retrieve in rugged mountainous areas. Thus, the void filling task is still a challenge. In this paper, we proposed a method based on a deep convolutional generative adversarial network (DCGAN) to address the problem of DEM void filling. A terrain texture generation model (TTGM) was constructed based on the DCGAN framework. Elevation, terrain slope, and relief degree composed the samples in the training set to better depict the terrain textural features of the DEM data. Moreover, the resize-convolution was utilized to replace the traditional deconvolution process to overcome the staircase in the generated data. The TTGM was trained on non-void SRTM (Shuttle Radar Topography Mission) 1-arc-second data patches in mountainous regions collected across the globe. Then, information neighboring the voids was involved in order to infer the latent encoding for the missing areas approximated to the distribution of training data. This was implemented with a loss function composed of pixel-wise, contextual, and perceptual constraints during the reconstruction process. The most appropriate fill surface generated by the TTGM was then employed to fill the voids, and Poisson blending was performed as a postprocessing step. Two models with different input sizes (64 × 64 and 128 × 128 pixels) were trained, so the proposed method can efficiently adapt to different sizes of voids. The experimental results indicate that the proposed method can obtain results with good visual perception and reconstruction accuracy, and is superior to classical interpolation methods.

A novel framework for inverse procedural texture modeling

Procedural textures are powerful tools that have been used in graphics for decades. In contrast to the alternative exemplar-based texture synthesis techniques, procedural textures provide user control and fast texture generation with low-storage cost and unlimited texture resolution. However, creating procedural models for complex textures requires a time-consuming process of selecting a combination of procedures and parameters. We present an example-based framework to automatically select procedural models and estimate parameters. In our framework, we consider textures categorized by commonly used high level classes. For each high level class we build a data-driven inverse modeling system based on an extensive collection of real-world textures and procedural texture models in the form of node graphs. We use unsupervised learning on collected real-world images in a texture class to learn sub-classes. We then classify the output of each of the collected procedural models into these sub-classes. For each of the collected models we train a convolutional neural network (CNN) to learn the parameters to produce a specific output texture. To use our framework, a user provides an exemplar texture image within a high level class. The system first classifies the texture into a sub-class, and selects the procedural models that produce output in that sub-class. The pre-trained CNNs of the selected models are used to estimate the parameters of the texture example. With the predicted parameters, the system can generate appropriate procedural textures for the user. The user can easily edit the textures by adjusting the node graph parameters. In a last optional step, style transfer augmentation can be applied to the fitted procedural textures to recover details lost in the procedural modeling process. We demonstrate our framework for four high level classes and show that our inverse modeling system can produce high-quality procedural textures for both structural and non-structural textures.

Local spot noise for procedural surface details synthesis

To deal with the increasing demand for complex visual details in virtual worlds, procedural methods for content authoring are an expanding field in Computer Graphics. Focusing on-the-fly texture generation, we present in this paper a content authoring process based on Locally Controlled Spot Noise. Through the control of both the impulses distribution and the spatially-defined kernel, this process can cover a wide range of appearances. In this context, we introduce a new kernel formulation that provides an efficient anisotropic filtering of the generated texture. Furthermore, our method allows users to interactively create the desired appearance by controlling both albedo and meso-geometry of the underlying surface, tackling on-the-fly normal map generation. Our method can be used as an artist friendly tool to model high-quality surface details with direct control over the final appearance in real-time.

An Automatic Texture Generation Algorithm for 3D Shapes Based on cGAN

Texturing 3D shapes is of great importance in computer graphics with applications ranging from game design to augmented reality. However, the processes of texture generation are usually tedious, time-consuming and labor-intensive. In this paper, we propose an automatic texture generation algorithm for 3D shapes based on conditional Generative Adversarial Networks (cGAN). The core of our algorithm includes sampling the model outline and building a cGAN in order to generate model textures automatically. In particular, we propose a novel edge detection method using 3D model information which can accurately find the outline of the model to improve the quality of the generated texture. Due to the adaptability of the algorithm, our approach is suitable for texture generation for most 3D models. Experimental results show the efficiency of our algorithm which can easily generate high quality model textures.

The structural properties of chemically derived graphene nanosheets/mesophase pitch-based composite carbon fibers with high conductivities

A series of mesophase pitch-based composite carbon fibers with various contents of chemically derived graphene nanosheets (CGNSs) were prepared through melt-spinning, stabilization, carbonization, and graphitization. By means of scanning electron microscopy, X-ray diffraction, single-filament testing, and resistivity testing, the effect of CGNSs content on the structures and properties of the composite carbon fibers were investigated. And the thermal conductivity of the composite carbon fibers was predicted by indirect measurement. The results show that the presence of CGNSs can effectively avoid the generation of unwanted radial transverse texture and wedge cleavages of the carbon fibers prepared from pure mesophase pitch. Although the crystalline structure of CGNSs/mesophase pitch-based composite carbon fibers is not apparently improved, appropriate CGNSs content can enhance the tensile strength as well as the conductivity of the composite carbon fibers. When the CGNSs content is 0.1 wt%, the tensile strength, breaking elongation, electrical resistivity and thermal conductivity of the composite carbon fibers can attain 2.12 GPa, 0.35%, 0.95 μΩ m, and 1322 W m−1 K−1, respectively. And the successful preparation of the composite carbon fibers provides a new solution for the cost reduction of mesophase pitch-based carbon fibers with high thermal conductivity.

Improved generative adversarial networks using the total gradient loss for the resolution enhancement of fluorescence images.

Because of the optical properties of medical fluorescence images (FIs) and hardware limitations, light scattering and diffraction constrain the image quality and resolution. In contrast to device-based approaches, we developed a post-processing method for FI resolution enhancement by employing improved generative adversarial networks. To overcome the drawback of fake texture generation, we proposed total gradient loss for network training. Fine-tuning training procedure was applied to further improve the network architecture. Finally, a more agreeable network for resolution enhancement was applied to actual FIs to produce sharper and clearer boundaries than in the original images.

Unsupervised Single-Image Super-Resolution with Multi-Gram Loss

Recently, supervised deep super-resolution (SR) networks have achieved great success in both accuracy and texture generation. However, most methods train in the dataset with a fixed kernel (such as bicubic) between high-resolution images and their low-resolution counterparts. In real-life applications, pictures are always disturbed with additional artifacts, e.g., non-ideal point-spread function in old film photos, and compression loss in cellphone photos. How to generate a satisfactory SR image from the specific prior single low-resolution (LR) image is still a challenging issue. In this paper, we propose a novel unsupervised method named unsupervised single-image SR with multi-gram loss (UMGSR) to overcome the dilemma. There are two significant contributions in this paper: (a) we design a new architecture for extracting more information from limited inputs by combining the local residual block and two-step global residual learning; (b) we introduce the multi-gram loss for SR task to effectively generate better image details. Experimental comparison shows that our unsupervised method in normal conditions can attain better visual results than other supervised SR methods.

Realtime Generation of Audible Textures Inspired by a Video Stream

We showcase a model to generate a soundscape from a camera stream in real time. The approach relies on a training video with an associated meaningful audio track; a granular synthesizer generates a novel sound by randomly sampling and mixing audio data from such video, favoring timestamps whose frame is similar to the current camera frame; the semantic similarity between frames is computed by a pretrained neural network. The demo is interactive: a user points a mobile phone to different objects and hears how the generated sound changes.

Exploring novel deformation mechanisms in aluminum–copper alloys using in situ 4D nanomechanical testing

Even after nearly a century of extensive use of aluminum alloys in structural applications, our understanding of such precipitation-strengthened materials is far from complete. With the advent of next generation advanced characterization techniques, our ability to probe materials in unique ways and at different length scales has established a new paradigm for devising new pathways to alloy design by engineering materials and tailoring specific properties at the nanoscale. Here, we perform in situ nanomechanical testing in conjunction with synchrotron-based hard X-ray nanotomography to capture initiation and evolution of damage in 3D in Al–Cu alloys. Precipitates in these alloys are seen to exhibit unprecedented localized deformation in compression, which is attributed to novel observations of kinking in these brittle second-phase particles, accompanied with the generation of a fine polycrystalline texture in the adjacent matrix. We observe a size-dependent transition in precipitate deformation behavior that has been thoroughly investigated using a comprehensive correlative approach.

Fourier at the heart of computer music: From harmonic sounds to texture

Beyond the scope of thermal conduction, Joseph Fourier's treatise on the Analytical Theory of Heat (1822) profoundly altered our understanding of acoustic waves. It posits that any function of unit period can be decomposed into a sum of sinusoids, whose respective contributions represent some essential property of the underlying periodic phenomenon. In acoustics, such a decomposition reveals the resonant modes of a freely vibrating string. The introduction of Fourier series thus opened new research avenues on the modeling of musical timbre—a topic that was to become of crucial importance in the 1960s with the advent of computer-generated sounds. This article proposes to revisit the scientific legacy of Joseph Fourier through the lens of computer music research. We first discuss how the Fourier series marked a paradigm shift in our understanding of acoustics, supplanting the theory of consonance of harmonics in the Pythagorean monochord. Then, we highlight the utility of Fourier's paradigm via three practical problems in analysis–synthesis: the imitation of musical instruments, frequency transposition, and the generation of audio textures. Interestingly, each of these problems involves a different perspective on time–frequency duality, and stimulates a multidisciplinary interplay between research and creation that is still ongoing. Au-delà de son apport théorique dans le domaine de la conduction thermique, le mémoire de Joseph Fourier sur la Théorie analytique de la chaleur (1822) a révolutionné notre conception des ondes sonores. Ce mémoire affirme que toute fonction de période unitaire se décompose en une série de sinusoïdes, chacune représentant une propriété essentielle du phénomène périodique étudié. En acoustique, cette décomposition révèle les modes de résonance d'une corde vibrante. Ainsi, l'introduction des séries de Fourier a ouvert de nouveaux horizons en matière de modélisation du timbre musical, un sujet qui prendra une importance cruciale à partir des années 1960, avec les débuts de la musique par ordinateur. Cet article propose de thématiser l'œuvre de Joseph Fourier à la lumière de ses implications en recherche musicale. Nous retraçons d'abord le changement de paradigme que les séries de Fourier ont opéré en acoustique, supplantant un mode de pensée fondé sur les consonances du monocorde pythagoricien. Par la suite, nous soulignons l'intérêt du paradigme de Fourier à travers trois problèmes pratiques en analyse-synthèse : l'imitation d'instruments de musique, la transposition fréquentielle, et la génération de textures sonores. Chacun de ses trois problèmes convoque une perspective différente sur la dualité temps–fréquence, et suscite un dialogue multidisciplinaire entre recherche et création qui est toujours d'actualité.

SKAITMENINIO KŪRYBIŠKUMO TAIKYMAS GENERUOJANT ŽAIDIMO OBJEKTUS / COMPUTATION CREATIVITY USING PROCEDURAL GENERATION FOR GAME OBJECT GENERATION

One of the hardest task for a machine is creativity. Computational creativity defines creative task completion for a machine. Three main creative content generation methods are: exploratory, combinatorial and transformational. Video game content can be generated using procedural generation. Computational creativity, procedural generation, and application are explained in this paper. Procedural level generator is used as a base and additional features are built on top of it. The main goal of this research and modification is to increase application creative value, variety and expression. Additional functionality consists of tree and texture generation.

SYNTHETIC THERMAL BACKGROUND AND OBJECT TEXTURE GENERATION USING GEOMETRIC INFORMATION AND GAN

Abstract. Nowadays methods based on deep neural networks show the best performance among image recognition and object detection algorithms. Nevertheless, such methods require to have large databases of multispectral images of various objects to achieve state-of-the-art results. Therefore the dataset generation is one of the major challenges for the successful training of a deep neural network. However, infrared image datasets that are large enough for successful training of a deep neural network are not available in the public domain. Generation of synthetic datasets using 3D models of various scenes is a time-consuming method that requires long computation time and is not very realistic. This paper is focused on the development of the method for thermal image synthesis using a GAN (generative adversarial network). The aim of the presented work is to expand and complement the existing datasets of real thermal images. Today, deep convolutional networks are increasingly used for the goal of synthesizing various images. Recently a new generation of such algorithms commonly called GAN has become a promising tool for synthesizing images of various spectral ranges. These networks show effective results for image-to-image translations. While it is possible to generate a thermal texture for a single object, generation of environment textures is extremely difficult due to the presence of a large number of objects with different emission sources. The proposed method is based on a joint approach that uses 3D modeling and deep learning. Synthesis of background textures and objects textures is performed using a generative-adversarial neural network and semantic and geometric information about objects generated using 3D modeling. The developed approach significantly improves the realism of the synthetic images, especially in terms of the quality of background textures.

Generating Stochastic Wall Patterns On‐the‐fly with Wang Tiles

AbstractThe game and movie industries always face the challenge of reproducing materials. This problem is tackled by combining illumination models and various textures (painted or procedural patterns). Generating stochastic wall patterns is crucial in the creation of a wide range of backgrounds (castles, temples, ruins…). A specific Wang tile set was introduced previously to tackle this problem, in an iterative fashion. However, long lines may appear as visual artifacts. We use this tile set in a new on‐the‐fly procedure to generate stochastic wall patterns. For this purpose, we introduce specific hash functions implementing a constrained Wang tiling. This technique makes possible the generation of boundless textures while giving control over the maximum line length. The algorithm is simple and easy to implement, and the wall structure we get from the tiles allows to achieve visuals that reproduce all the small details of artist painted walls.

Accurate Synthesis of Multi‐Class Disk Distributions

AbstractWhile analysing and synthesising 2D distributions of points has been applied both to the generation of textures with discrete elements and for populating virtual worlds with 3D objects, the results are often inaccurate since the spatial extent of objects cannot be expressed. We introduce three improvements enabling the synthesis of more general distributions of elements. First, we extend continuous pair correlation function (PCF) algorithms to multi‐class distributions using a dependency graph, thereby capturing interrelationships between distinct categories of objects. Second, we introduce a new normalised metric for disks, which makes the method applicable to both point and possibly overlapping disk distributions. The metric is specifically designed to distinguish perceptually salient features, such as disjoint, tangent, overlapping, or nested disks. Finally, we pay particular attention to convergence of the mean PCF as well as the validity of individual PCFs, by taking into consideration the variance of the input. Our results demonstrate that this framework can capture and reproduce real‐life distributions of elements representing a variety of complex semi‐structured patterns, from the interaction between trees and the understorey in a forest to droplets of water. More generally, it applies to any category of 2D object whose shape is better represented by bounding circles than points.

Integrated modelling and analysis of micro-cutting mechanics with the precision surface generation in abrasive flow machining

Abrasive flow machining (AFM) technology is attracting more and more attention and keeps expanding into more areas by the industry and research community particularly in the context of increasing demands for post-processing of the complex aerofoil structures and additively manufactured components. It is fundamentally vital to develop an industrial feasible approach to controlling and improving the surface roughness of the structure and component, and even the profile accuracy and surface texture. In this paper, a multiscale multiphysics approach combining with micro-cutting mechanics is presented for modelling and analysis of the surface roughness and topography profile generation in the AFM process. The analysis is developed and implemented by using MATLAB programming integrated with the COMSOL multiphysics computational environment. Micro-cutting mechanics modelling and the Monte Carlo (MC) algorithms are integrated to develop simulations on the AFM generation of surface texture and topography through abrasive micro-machining with thousands of grains under complex multiscale and multiphysics working conditions. Well-designed AFM experiment trials on machining aerofoil structures are carried out to further evaluate and validate the modelling and analysis. The work presented is fundamental but essential as a part of the project for developing the simulation-based AFM virtual machining system.

Joint reconstruction and classification of tumor cells and cell interactions in melanoma tissue sections with synthesized training data

PurposeCancers are almost always diagnosed by morphologic features in tissue sections. In this context, machine learning tools provide new opportunities to describe tumor immune cell interactions within the tumor microenvironment and thus provide phenotypic information that might be predictive for the response to immunotherapy.MethodsWe develop a machine learning approach using variational networks for joint image denoising and classification of tissue sections for melanoma, which is an established model tumor for immuno-oncology research. The manual annotation of real training data would require substantial user interaction of experienced pathologists for each single training image, and the training of larger networks would rely on a very large number of such data sets with ground truth annotation. To overcome this bottleneck, we synthesize training data together with a proper tissue structure classification. To this end, a stochastic data generation process is used to mimic cell morphology, cell distribution and tissue architecture in the tumor microenvironment. Particular components of this tool are random placement and rotation of a large number of patches for presegmented cell nuclei, a stochastic fast marching approach to mimic the geometry of cells and texture generation based on a color covariance analysis of real data. Here, the generated training data reflect a large range of interaction patterns.ResultsIn several applications to histological tissue sections, we analyze the efficiency and accuracy of the proposed approach. As a result, depending on the scenario considered, almost all cells and nuclei which ought to be detected are actually marked as classified and hardly any misclassifications occur.ConclusionsThe proposed method allows for a computer-aided screening of histological tissue sections utilizing variational networks with a particular emphasis on tumor immune cell interactions and on the robust cell nuclei classification.

Domain wall dynamics in easy-cone magnets

We theoretically and numerically investigate magnetic domain wall dynamics in a nanowire of easy-cone magnet. The easy-cone domain wall exhibits several distinguishing dynamic features in comparison to the easy-axis domain wall. The features of easy-cone domain wall are related to the generation of additional chiral spin textures due to the domain wall precession, which is common for various driving sources such as magnetic fields and spin-transfer torques. The unique easy-cone domain wall dynamics could enrich magnetic domain wall study and find use in device applications based on easy-cone domain walls.

Efeito do processo de recozimento primário e secundário em aços elétricos de grão orientado

Os aços siliciosos são aqueles compostos de aproximadamente 3% de silício e são utilizados para fabricação de geradores, transformadores e motores industriais. A pesquisa foi feita com o intuito de gerar conhecimento metalúrgico acerca das propriedades magnéticas dos aços elétricos e também conciliar essas propriedades aos parâmetros de processamento do material. Uma amostra padrão foi retirada do lote selecionado para teste, decapada utilizando ácido clorídrico e submetida ao processo de recozimento. Cálculos de proporcionalidade foram feitos utilizando o diagrama ferro silício, levando em consideração o peso e as dimensões da amostra. Após o recozimento, a amostra foi submetida aos testes adequados para evidenciar o tamanho dos grãos e sua influência nas propriedades magnéticas. Seus resultados mostraram um crescimento significativo no tamanho dos grãos após o recozimento secundário, e geração da textura goss, que é desejada ao material. Com as análises foi possível concluir que após o recozimento, os grãos apresentaram tamanhos padronizados e bem ordenados, e a padronização das perdas e permeabilidade magnética que respectivamente resultaram em 1,5 W/Kg e 1,99T nas condições de 1,7T a uma frequência de 60 Hz (frequência padrão brasileira).