Scene Generation Research Articles

Active and Passive Infrared Emittance Tuning Using Optically Transparent Unpatterned ITO Thin Films

Tailoring of thermal emittance has applications in smart radiators, camouflage, infrared (IR) scene generation, stealth, and thermal management. Various studies have explored avenues to achieve both passive and active emittance tuning using methods such as meta-surfaces, phase change materials, composites, nanostructure arrays and multi-layer stacks. Most of these methods often involve complex fabrication processes like multilayer depositions and micro/nano patterning, limiting their scalability for large-scale applications. This study presents a straightforward and scalable approach for passive and active emittance tuning using Indium Tin Oxide (ITO) films deposited on a high emissivity flexible polyethylene terephthalate (PET) substrate. By changing the deposition time, the thickness of the ITO film is controlled which changes the observed emissivity of the ITO/PET composite structure over a wide range (0.2-0.94) while maintaining optical transparency. Using this approach for passive emittance tuning, we demonstrate an IR scene generation application. As the ITO film is conducting, by applying voltage to the ITO/PET sheet, its temperature is changed using which active emittance tuning is also achieved. Compared to existing methods of emittance tuning, the presented approach is simpler, gives flexible films with optical transparency, and has the potential for large-scale integration, enabling cost-effective and scalable manufacturing of emittance-tuned surfaces.

3D scene generation for zero-shot learning using ChatGPT guided language prompts

3D scene generation for zero-shot learning using ChatGPT guided language prompts

Coupling geospatial suitability simulation and life cycle carbon emissions towards uncertain optimization planning for wind-photovoltaic-hydrogen multi-energy complementary system

Building clean and modern multi-energy complementary systems is a direct path to decarbonization in energy sector. The introduction of hydrogen energy with high operational flexibility enables the multi-energy complementary system to fully accommodate renewable energy. The key issues of the optimization planning of the wind-photovoltaic-hydrogen multi-energy complementary system that need to be solved are how to finely characterize the carbon reduction potential of the system, consider the uncertainty of supply and demand in the modeling process, and incorporate the energy supply business of electric vehicles into the planning. This study constructs optimization planning model with the following three progressive stages: geospatial suitability simulation based on geographic information system, uncertainty analysis based on probabilistic scene generation technology, and capacity configuration optimization. Unlike previous studies, this model measures and optimizes the carbon emissions level of the system from a lifecycle perspective. The empirical study results indicate that carbon emissions optimization from the perspective of the entire life cycle would directly affect the system configuration results. From a lifecycle perspective, the carbon emissions accounting result is about 9 times higher than that obtained only during the operation phase. Compared to the traditional distribution system, the carbon emissions of the system constructed in this study decreased by 82.75 %. In addition, considering that the system provides charging services for electric vehicles, which can reduce carbon emissions by about 93.6 % but only increase costs by about 29.4 %. The computational results justify the proposed optimization framework. This study provides optimization models for the system, and the implementation framework can support technical reference for practice.

Retraction Note: Light image enhancement and virtual reality application in automatic generation of basketball game scenes and training data simulation

Retraction Note: Light image enhancement and virtual reality application in automatic generation of basketball game scenes and training data simulation

From inactive prehispanic archaeological monument to urban scene generator of self-sustainable communities

The pre-Hispanic archaeological monuments on the Peruvian coast, known as huacas, have pyramidal geomorphological characteristics and represent a valuable monumental heritage. The presentation focuses on thehuacaof thePuebloJoven SantaRosa inthecityofLambayeque, Peru, with thepurposeof (re)activatingit as a way of protection from urban pressure, for which they propose mechanisms and strategies for integration intothecity.Thedevelopmentisdividedintothreephases:first,thehuacainitscurrentsocialandurbancontext. Then,contemporaryurbanstrategiesforactioninthehuacaarepresentedandfinally,prospectivescenariosare proposed. Four interconnected elements were considered: individual, society andcommunity; disciplinarycorpus; projectproductionandevaluation.Theresultsrevealthepotentialofthesearchaeologicalspacesintheterritory andtheirimpactontheenvironment.Opportunitiesarealsoidentifiedtotransformthesurroundingenvironment, promoting the reactivation of these cultural vestiges and the development of sustainable communities in Lambayeque, Peru.

A pantograph-catenary arcing detection model for high-speed railway based on semantic segmentation and generative adversarial network

ABSTRACT Pantograph-catenary arcing refers to an abnormal phenomenon occurring in pantograph-catenary system due to poor contact or other factors, which significantly impacts the normal operation of high-speed railway. Therefore, the detection of arcing occurrences holds significant importance for the intelligent maintenance of pantograph-catenary systems. However, the scarcity of arcing data in pantograph-catenary datasets limits the efficacy of supervised learning methods for arcing detection. To address this issue, we propose a novel pantograph-catenary arcing detection model that integrates semantic segmentation with generative adversarial networks. The model first modifies the loss function of the U 2 -Net network to tailor it specifically for pantograph-catenary semantic segmentation. To generate finer normal pantograph-catenary images, attention mechanism is incorporated into the SPADE-based pantograph-catenary scene generation model. Finally, an improved differencing method is employed to compute the arcing image by subtracting the generated normal pantograph-catenary image from the actual pantograph-catenary image. The experimental results validate the effectiveness of the method for pantograph-catenary arcing detection in the absence of prior arcing knowledge, with a recall rate of 75.3% and an F1-Score of 69.63%. Compared to other advanced pantograph-catenary arcing methods, this method exhibits superior performance.

Economic benefit analysis of scene transformation of business model-taking Suning.cn as an example

With the rapid development of the new economy promoted by the new generation of technological revolution, new scenes, new formats and new models have become the direction and path of China's economic transformation and upgrading. For the retail industry, which is significantly affected by the impact of the Internet, scene transformation is the hottest and most powerful straw to reverse the current difficulties. This paper takes Suning.cn as the target case company to study the economic benefits brought by its scene transformation. Firstly, it points out the dilemma faced by the retail industry at present, and analyzes and explains the concept of scene with other experts' theories. Secondly, the scene layout of Suning is sorted out and compared with that of Sichuan Changhong in terms of profitability, operation ability and development ability. Finally, from the financial perspective, it summarizes the economic benefits brought by the scene transformation to Suning.

BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolation

We present BlockFusion, a diffusion-based model that generates 3D scenes as unit blocks and seamlessly incorporates new blocks to extend the scene. BlockFusion is trained using datasets of 3D blocks that are randomly cropped from complete 3D scene meshes. Through per-block fitting, all training blocks are converted into the hybrid neural fields: with a tri-plane containing the geometry features, followed by a Multi-layer Perceptron (MLP) for decoding the signed distance values. A variational auto-encoder is employed to compress the tri-planes into the latent tri-plane space, on which the denoising diffusion process is performed. Diffusion applied to the latent representations allows for high-quality and diverse 3D scene generation. To expand a scene during generation, one needs only to append empty blocks to overlap with the current scene and extrapolate existing latent tri-planes to populate new blocks. The extrapolation is done by conditioning the generation process with the feature samples from the overlapping tri-planes during the denoising iterations. Latent tri-plane extrapolation produces semantically and geometrically meaningful transitions that harmoniously blend with the existing scene. A 2D layout conditioning mechanism is used to control the placement and arrangement of scene elements. Experimental results indicate that BlockFusion is capable of generating diverse, geometrically consistent and unbounded large 3D scenes with unprecedented high-quality shapes in both indoor and outdoor scenarios.

FairScene: Learning unbiased object interactions for indoor scene synthesis

In this paper, we propose an unbiased graph neural network learning method called FairScene for indoor scene synthesis. Conventional methods directly apply graphical models to represent the correlation of objects for subsequent furniture insertion. However, due to the object category imbalance in dataset collection and complex object entanglement with implicit confounders, these methods usually generate significantly biased scenes. Moreover, the performance of these methods varies greatly for different indoor scenes. To address this, we propose a framework named FairScene which can fully exploit unbiased object interactions through causal reasoning, so that fair scene synthesis is achieved by calibrating the long-tailed category distribution and mitigating the confounder effects. Specifically, we remove the long-tailed object priors subtract the counterfactual prediction obtained from default input, and intervene in the input feature by cutting off the causal link to confounders based on the causal graph. Extensive experiments on the 3D-FRONT dataset show that our proposed method outperforms the state-of-the-art indoor scene generation methods and enhances vanilla models on a wide variety of vision tasks including scene completion and object recognition.

Conditional room layout generation based on graph neural networks

In this paper, we present a novel end-to-end variational generative model that utilizes graph-based latent representations for indoor scene synthesis at one time. In contrast to prior research, our method deviates from the practice of gradually introducing and arranging furniture in an empty room using autoregression. Instead, it focuses on acquiring a comprehensive implicit representation of the room’s original architectural structure and the placement of furniture. We initially transform the 3D room scene into a dense scene graph, where nodes correspond to the objects present in the room while edges reflect the spatial location links and functional correlations between objects. Then, a neural network is trained to acquire the graph-based latent representation of the room scene through iterative message passing, ultimately resulting in the acquisition of the data distribution on the latent space of the room layout. Given the architectural structure of an empty room as a prerequisite for scene synthesis, the generative model has the ability to sample from the prior distribution of the room’s latent representation. This allows the model to then decode and generate a variety of room layouts. We evaluate our method with the state-of-the-art 3D indoor scene dataset and generation methods. The experimental results demonstrate that our method achieves more rational and diverse outcomes in the context of generating scenes under specific conditions.

Research on the analysis strategy of electric vehicle charging demand based on multi-correlation daily scenario generation

Abstract With the continuous increase of the penetration rate of OFELECTRIC vehicles, their high random charging loads exert a more significant effect on the stability of the electrical system. Simultaneously, the past charging patterns of electric vehicles indirectly influence the possible future charging activities of these vehicles. Therefore, in order to characterize the correlation between the predicted daily charging behavior of electric vehicles and its historical daily charging behavior, this article uses scene generation theory to fully mine the original multi-correlation Internal characteristics of electric vehicle charging behavior among the original multi-correlation day charging scenario set (OMCDCSS), generating massively generating multi-correlation day charging scenarios (GMCDCS) similar to the probability distribution of OMCDCSS. Secondly, the related scene set (RSS) is screened in the generation of multi-correlation day billing scenario (GMCDCSS) on the basis of weighted Spearman rank (SR). Get the forecast result of electric vehicle load interval.

State of the Art on Diffusion Models for Visual Computing

AbstractThe field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. In these domains, diffusion models are the generative AI architecture of choice. Within the last year alone, the literature on diffusion‐based tools and applications has seen exponential growth and relevant papers are published across the computer graphics, computer vision, and AI communities with new works appearing daily on arXiv. This rapid growth of the field makes it difficult to keep up with all recent developments. The goal of this state‐of‐the‐art report (STAR) is to introduce the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model, as well as overview important aspects of these generative AI tools, including personalization, conditioning, inversion, among others. Moreover, we give a comprehensive overview of the rapidly growing literature on diffusion‐based generation and editing, categorized by the type of generated medium, including 2D images, videos, 3D objects, locomotion, and 4D scenes. Finally, we discuss available datasets, metrics, open challenges, and social implications. This STAR provides an intuitive starting point to explore this exciting topic for researchers, artists, and practitioners alike.

Comparison of Latest 3D Content Generation Models with Minimal Input Images

Three-dimensional (3D) content generation has become a popular topic in recent years. It can be widely used in movie scene generation, video games 3D modeling, industrial design, and even pharmaceutical 3D structure characterization. Before artificial intelligence (AI), it can be difficult. People need to be trained to use various industrial 3D model applications and spend plenty of time building and refining a model. With the development of virtual reality and artificial reality, the demand for 3D content is rising rapidly. Traditional 3D content production cycles cannot fit the needs. Recently, the Text-to-Image technology has got great success. With the help of artificial intelligence, people can use a limited set of descriptive words to generate images. Typically, the model generates multiple images of the same category for users to choose from. Some fundamental techniques like Neural Radiance Fields (NeRF) and Diffusion Model can generate 3D scenes, avatars, and other 3D content using a couple of images. This progress marks the possibility of creating 3D content using text. Based on the technologies available today, there will be more applications for generating 3D content in the future. Selecting the core technologies will be a crucial issue in this regard. This essay mainly talks about three of the most popular models or technologies that use a minimum number of images to produce 3D content. The goal is to find the most suitable technology based on criteria such as quality, applicability, and other indicators.

Impact of AI in the Animation Industry

Abstract: This manuscript develops into the transformative role of artificial intelligence (AI) in the field of animation. Over the years, AI has revolutionized various industries including healthcare, banking, e-commerce, agriculture, manufacturing, and others., and animation is no exception. This paper provides an overview of the applications of AI in animation, including character creation, motion synthesis, scene generation, generating facial expressions and body language for characters, planning out scene layouts and camera movements, and generating backgrounds and environments and more. It discusses the challenges and ethical considerations associated with AI-driven animation and presents insights into the future directions of AI in the animation industry

LLM-Powered Synthetic Environments for Self-Driving Scenarios

This paper outlines a proposal exploring the potential use of Large Language Models (LLMs), particularly GPT-4, in crafting realistic synthetic environments for self-driving scenarios. The envisioned approach involves dynamic scene generation within game engines, leveraging LLMs to introduce challenging elements for autonomous vehicles. The proposed evaluation process outlines assessments such as realistic testing, safety metrics, and user interaction, aiming to set the stage for potential improvements in self-driving system performance. The paper aims to contribute to the AI field by discussing how LLMs could be utilized to create valuable testing grounds for autonomous vehicles, potentially fostering the development of more robust self-driving technology. The envisioned impact is the eventual enhancement of road safety and the possible acceleration of the adoption of autonomous vehicles, paving the way for a future with safer and more efficient transportation.

Real3D: The Curious Case of Neural Scene Degeneration

Despite significant progress in utilizing pre-trained text-to-image diffusion models to guide the creation of 3D scenes, these methods often struggle to generate scenes that are sufficiently realistic, leading to "neural scene degeneration". In this work, we propose a new 3D scene generation model called Real3D. Specifically, Real3D designs a pipeline from a NeRF-like implicit renderer to a tetrahedrons-based explicit renderer, greatly improving the neural network's ability to generate various neural scenes. Moreover, Real3D introduces an additional discriminator to prevent neural scenes from falling into undesirable local optima, thus avoiding the degeneration phenomenon. Our experimental results demonstrate that Real3D outperforms all existing state-of-the-art text-to-3D generation methods, providing valuable insights to facilitate the development of learning-based 3D scene generation approaches.

Dynamic Scene Generation and Animation Rendering integrating CAD Modeling and Reinforcement Learning

Dynamic Scene Generation and Animation Rendering integrating CAD Modeling and Reinforcement Learning

Quaternary pulse width modulation based ultra-high frame rate scene projector used for hard-ware-in-the-loop testing.

The scene projector (SP) can provide simulated scene images with same optical characteristics as the real scenes to evaluate imaging systems in hard-ware-in-the-loop (HWIL) simulation testing. The single scene generation device (SGD) based SP typically projects 8-bit images at 220 fps, which is insufficient to fulfill the requirements of ultra-high frame rate imaging systems, such as star trackers and space debris detectors. In this paper, an innovative quaternary pulse width modulation (PWM) based SP is developed and implemented to realize the ultra-high frame rate projection. By optically overlapping modulation layers of two digital micro-mirror devices (DMDs) in parallel, and illuminating them with light intensities, a quaternary SGD is built up to modulate quaternary digit-planes (QDs) with four grayscale levels. And the quaternary digit-plane de-composition (QDD) is adopted to decompose an 8-bit image into 4 QDs. In addition, the exposure time of each QD is controlled by quaternary PWM, and the base time is optimized to 8 µs. The experimental results prove that the total exposure time of all QDs sequentially modulated by quaternary PWM is approximately 760 µs, namely projecting 8-bit images at 1300 fps. The quaternary PWM using two DMDs in parallel dramatically improves the grayscale modulation efficiency compared to the existing projection technologies, which provides a new approach for the SP design with ultra-high frame rate.

Toward Improving the Generation Quality of Autoregressive Slot VAEs.

Unconditional scene inference and generation are challenging to learn jointly with a single compositional model. Despite encouraging progress on models that extract object-centric representations ("slots") from images, unconditional generation of scenes from slots has received less attention. This is primarily because learning the multiobject relations necessary to imagine coherent scenes is difficult. We hypothesize that most existing slot-based models have a limited ability to learn object correlations. We propose two improvements that strengthen object correlation learning. The first is to condition the slots on a global, scene-level variable that captures higher-order correlations between slots. Second, we address the fundamental lack of a canonical order for objects in images by proposing to learn a consistent order to use for the autoregressive generation of scene objects. Specifically, we train an autoregressive slot prior to sequentially generate scene objects following a learned order. Ordered slot inference entails first estimating a randomly ordered set of slots using existing approaches for extracting slots from images, then aligning those slots to ordered slots generated autoregressively with the slot prior. Our experiments across three multiobject environments demonstrate clear gains in unconditional scene generation quality. Detailed ablation studies are also provided that validate the two proposed improvements.

Orthoscopic elemental image synthesis for 3D light field display using lens design software and real-world captured neural radiance field.

The elemental images (EIs) generation of complex real-world scenes can be challenging for conventional integral imaging (InIm) capture techniques since the pseudoscopic effect, characterized by a depth inversion of the reconstructed 3D scene, occurs in this process. To address this problem, we present in this paper a new approach using a custom neural radiance field (NeRF) model to form real and/or virtual 3D image reconstruction from a complex real-world scene while avoiding distortion and depth inversion. One of the advantages of using a NeRF is that the 3D information of a complex scene (including transparency and reflection) is not stored by meshes or voxel grid but by a neural network that can be queried to extract desired data. The Nerfstudio API was used to generate a custom NeRF-related model while avoiding the need for a bulky acquisition system. A general workflow that includes the use of ray-tracing-based lens design software is proposed to facilitate the different processing steps involved in managing NeRF data. Through this workflow, we introduced a new mapping method for extracting desired data from the custom-trained NeRF-related model, enabling the generation of undistorted orthoscopic EIs. An experimental 3D reconstruction was conducted using an InIm-based 3D light field display (LFD) prototype to validate the effectiveness of the proposed method. A qualitative comparison with the actual real-world scene showed that the 3D reconstructed scene is accurately rendered. The proposed work can be used to manage and render undistorted orthoscopic 3D images from custom-trained NeRF-related models for various InIm applications.