Photorealistic Graphics Research Articles

Development and validation of graphics‐based digital twin framework for UAV‐aided post‐earthquake inspection of high‐rise buildings

SummaryTraditional post‐earthquake inspection of civil infrastructure is conducted manually, taking a considerable amount of time and often putting inspectors in harm's way. This problem is exacerbated in modern cities, where millions of people can be left homeless until their residences are deemed safe to reinhabit. Image collection enabled by commercial unmanned aerial vehicles (UAVs) combined with computer vision‐based techniques has provided an alternative with high potential for rapid post‐earthquake inspection. However, the extracted images of the damage alone are inadequate to evaluate the system‐level safety condition of a structure. The quality of the visual information also heavily relies on the effectiveness of the UAV inspection scheme which is susceptible to environmental uncertainties. To this end, a graphics‐based digital twin (GBDT) framework is developed for UAV‐aided post‐earthquake inspection of high‐rise buildings and validated using a high‐rise building in Guangzhou, China. The GBDT is comprised of a finite element (FE) model and a photorealistic computer graphics (CG) model, with the latter being informed by the former, jointly providing as a comprehensive virtual representation of the structure so that every step of the post‐earthquake inspection procedure can be planned and evaluated virtually. First, to avoid the cumbersome nature of constructing the graphical representation of the numerous components in high‐rise buildings, the CG model in the GBDT is created by automatically importing structural components from the FE model and adding nonstructural components according to the dimensions of the as‐built structure. This fast modeling process as well as the accuracy of the virtual presentation are validated by point cloud comparisons between the CG model and the as‐built structure. Subsequently, the GBDT is used to showcase the evaluation of UAV flight schemes for post‐earthquake inspection of high‐rise buildings. To shorten flight time and place more emphasis on potential damage, FE analysis is conducted to determine the earthquake‐induced damage locations. Consistent damage hotspots are then marked on the CG model, along with restrictions from the real environment such as obstacles, weak satellite signal, wind speed, and lighting conditions considered in the synthetic environment. Finally, applying the synthetic environment as the testbed, three UAV‐aided inspection schemes are implemented virtually and the best UAV flight scheme is determined for the assumed field inspection. This example demonstrates the flexibility of the GBDT in representing the real‐world structure and environmental conditions and its efficacy in assisting decision making for rapid and effective structural inspection in the aftermath of an earthquake.

Usability Analysis of a Virtual Reality Exposure Therapy Serious Game for Blood Phobia Treatment: Phobos

Phobias are characterized as the excessive or irrational fear of an object or situation, and specific phobias affect about 10% of the world population. Blood-injection-injury phobia is a specific phobia that has a unique physical response to phobic stimuli, that is, a vasovagal syncope that causes the person to faint. Phobos is a serious game intended for blood phobia treatment that was created to be played in virtual reality with an HTC Vive that has photorealistic graphics to provide a greater immersion. We also developed a console application in C# for electrocardiography sensor connectivity and data acquisition, which gathers a 1 min baseline reading and then has continuous data acquisition during gameplay. Usability tests were conducted with self-reported questionnaires and with a case study population of 10 testers, which gave insight into the previous game experience of the tester for both digital games and virtual reality games, evaluating the discomfort for hardware on both the sensor and the virtual reality headset, as well as the game regarding usability, user experience, level of immersion, and the existence of motion sickness and its source. The results corroborate that the immersion of the game is good, which suggests that it will help with triggering the phobia.

Zurich Appearance Score for Hands: Development and Validation of an Instrument for Assessing Hand Appearance in Congenital Upper Limb Differences

The appearance of the hand is relevant to individual activity and participation. Improving appearance is often one of the essential goals of hand surgery. The aim of this study was to describe and validate an instrument for quantitatively assessing hand appearance in congenital upper limb differences (CULD). The Zurich Appearance Score for Hands (ZASH) was developed as a summed score of 6 items: 1 for overall appearance and 5 specific items for the skin, proportions of the hand, the number of digits, the shape of fingers and the thumb, and the position of the thumb. Each item can be rated on an 11-point Likert scale. Here, 448 participants (age, 14-83 years) were asked to rate the images of 17 hands, including standardized 3-dimensional photorealistic computer graphics and photographs of children's hands with or without CULDs, some after surgical correction. The sociodemographic characteristics of the participants were measured using a short questionnaire. The ZASH score for all CULDs was significantly lower than the ZASH score for normal hands. Correlations for overall appearance and the ZASH score were high (r= 0.77-0.87). The internal consistency of all ZASH scores was good to excellent (Cronbach α= 0.82-0.94). The test-retest reliability in a subgroup of 54 participants was good (r= 0.53-0.79). The interrater reliability of the ZASH score was moderate (intraclass correlation= 0.47). The ZASH is a valid and moderately reliable instrument for assessing hand appearance in children with CULDs. Achieving the best possible appearance is the one of the most relevant goals of hand surgery. Assessment with validated instruments provides evidence on how to approach this goal. Further studies may clarify whether the ZASH can be recommended for all hand conditions and to what extent observers' sociocultural and professional backgrounds affect perception.

Stopping Criterion during Rendering of Computer-Generated Images Based on SVD-Entropy.

The estimation of image quality and noise perception still remains an important issue in various image processing applications. It has also become a hot topic in the field of photo-realistic computer graphics where noise is inherent in the calculation process. Unlike natural-scene images, however, a reference image is not available for computer-generated images. Thus, classic methods to assess noise quantity and stopping criterion during the rendering process are not usable. This is particularly important in the case of global illumination methods based on stochastic techniques: They provide photo-realistic images which are, however, corrupted by stochastic noise. This noise can be reduced by increasing the number of paths, as proved by Monte Carlo theory, but the problem of finding the right number of paths that are required in order to ensure that human observers cannot perceive any noise is still open. Until now, the features taking part in the human evaluation of image quality and the remaining perceived noise are not precisely known. Synthetic image generation tends to be very expensive and the produced datasets are high-dimensional datasets. In that case, finding a stopping criterion using a learning framework is a challenging task. In this paper, a new method for characterizing computational noise for computer generated images is presented. The noise is represented by the entropy of the singular value decomposition of each block composing an image. These Singular Value Decomposition (SVD)-entropy values are then used as input to a recurrent neural network architecture model in order to extract image noise and in predicting a visual convergence threshold of different parts of any image. Thus a new no-reference image quality assessment is proposed using the relation between SVD-Entropy and perceptual quality, based on a sequence of distorted images. Experiments show that the proposed method, compared with experimental psycho-visual scores, demonstrates a good consistency between these scores and stopping criterion measures that we obtain.

ГИПЕРРЕАЛИЗМ В ЦИФРОВОМ КИНЕМАТОГРАФЕ (НА ПРИМЕРЕ СЕРИАЛА-АНТОЛОГИИ «ЛЮБОВЬ, СМЕРТЬ И РОБОТЫ»)

The article is devoted to the aesthetic features of digital cinematography. Due to the widespread use of computer graphics in films, traditional methods of analyzing cinema are no longer enough. The fusion of graphic and photographic elements and the use of photorealistic graphics enable researchers to take a fresh look at the nature of film. Digital cinema, based on contemporary technologies, no longer builds on the photographic principles of filmmaking and presents a new form of realism and representation of reality on the screen. Objects created with the help of computer graphics can transform and go beyond the simulated real objects, while remaining photorealistic, unable to exist outside the framework of the screen and the particular film. Using classical film studies and philosophical works of Jean Baudrillard, Siegfried Krakauer, André Bazin, Christian Metz and the works of modern researchers of digital cinema such as Lev Manovich, Thomas Elsaesser and Malthe Hagener, the author analyzes the process of transition of digital objects from photorealistic to hyperrealistic ones, which today can be identified as real, while not existing in reality. This phenomenon is of undoubted interest for the academic study of new forms of realism in digital cinema. Based on the analysis of the Snow in the Desert episode from the Love, Death & Robots anthology series, the author traces the development of hyperrealism in digital cinema. Through state-of-the-art digital filmmaking technologies combining both graphic and photographic techniques—motion capture, digital cloning of actors, real-time fluid and environment modelling—we have a new kind of cinematography that is not limited to classical methods of filmmaking, but goes beyond cinema itself, merging with animation and video games. All these processes affect the film aesthetics and make it more malleable, free and less restricted by the classical understanding of motion picture.

Detection of Computer Graphics Using Attention-Based Dual-Branch Convolutional Neural Network from Fused Color Components.

With the development of 3D rendering techniques, people can create photorealistic computer graphics (CG) easily with the advanced software, which is of great benefit to the video game and film industries. On the other hand, the abuse of CGs has threatened the integrity and authenticity of digital images. In the last decade, several detection methods of CGs have been proposed successfully. However, existing methods cannot provide reliable detection results for CGs with the small patch size and post-processing operations. To overcome the above-mentioned limitation, we proposed an attention-based dual-branch convolutional neural network (AD-CNN) to extract robust representations from fused color components. In pre-processing, raw RGB components and their blurred version with Gaussian low-pass filter are stacked together in channel-wise as the input for the AD-CNN, which aims to help the network learn more generalized patterns. The proposed AD-CNN starts with a dual-branch structure where two branches work in parallel and have the identical shallow CNN architecture, except that the first convolutional layer in each branch has various kernel sizes to exploit low-level forensics traces in multi-scale. The output features from each branch are jointly optimized by the attention-based fusion module which can assign the asymmetric weights to different branches automatically. Finally, the fused feature is fed into the following fully-connected layers to obtain final detection results. Comparative and self-analysis experiments have demonstrated the better detection capability and robustness of the proposed detection compared with other state-of-the-art methods under various experimental settings, especially for image patch with the small size and post-processing operations.

CAN IMMERSIVE VIRTUAL REALITY TRAVEL EXPERIENCE REPLACE REAL TRAVEL?

Purpose. The study attempts to define virtual reality in tourism as well as to characterise and present chosen applications created for the purposes of virtual sightseeing. Based on the survey conducted among tourism and recreation students of the Jerzy Kukuczka Academy of Physical Education in Katowice, the study attempts to answer the provocative question: “can virtual tourism replace real-world travel?”, and to analyse participants’ opinions on the topic of virtual trips. Methods. Analysis of scientific studies as well as information and data on the possibility of using immersive virtual reality in human life with particular consideration given to tourism. An overview of applications enabling taking trips through immersive exploration of virtual environments. The survey was conducted among AWF students. Participants expressed their opinions about virtual sightseeing. Findings. Virtual reality is increasingly used in various fields of human life. It is entering the world of tourism, in which it is used to create presentations of hotels and resorts, and to promote towns and tourist facilities. Appropriate software and IT equipment allow to take deliberate, planned and very realistic virtual trips. The authors of the work defined such tourism as all activities carried out by persons who immerse themselves in virtual reality for learning and entertainment purposes in order to experience the illusion of change of their everyday, real surroundings in time and space. Tourism understood in such a way allows us not only to go to almost any place without the necessity of leaving home. It also allows for visiting areas and objects which cannot be explored in real life. It enables a visitor, among others, to travel in space and visit historical sites which no longer exist in their original form, but have been recreated in computer applications. Virtual tourism also allows for exploring fictional locations created by designers of photorealistic graphics as well as valuable and sensitive monuments, and taking trips to places which are dangerous or prohibited. In the conducted survey, it has been concluded that even though tourism and recreation students found experience with virtual reality to be positive, the majority is not convinced that this form of tourism can replace real-world travel. Research and conclusions limitations. There are only a few publications concerning immersive virtual reality travel experience. Access to the software was quite early. Practical implications. Understanding and reorganisation of issues related to immersive virtual reality travel experience. The study may constitute an original introduction and encouragement to carry out qualitative and quantitative research on newly created virtual tourism. Originality. An original concept of understanding virtual tourism was presented. Type of paper. Empirical research and review.

Distinguishing Computer-Generated Graphics from Natural Images Based on Sensor Pattern Noise and Deep Learning.

Computer-generated graphics (CGs) are images generated by computer software. The rapid development of computer graphics technologies has made it easier to generate photorealistic computer graphics, and these graphics are quite difficult to distinguish from natural images (NIs) with the naked eye. In this paper, we propose a method based on sensor pattern noise (SPN) and deep learning to distinguish CGs from NIs. Before being fed into our convolutional neural network (CNN)-based model, these images—CGs and NIs—are clipped into image patches. Furthermore, three high-pass filters (HPFs) are used to remove low-frequency signals, which represent the image content. These filters are also used to reveal the residual signal as well as SPN introduced by the digital camera device. Different from the traditional methods of distinguishing CGs from NIs, the proposed method utilizes a five-layer CNN to classify the input image patches. Based on the classification results of the image patches, we deploy a majority vote scheme to obtain the classification results for the full-size images. The experiments have demonstrated that (1) the proposed method with three HPFs can achieve better results than that with only one HPF or no HPF and that (2) the proposed method with three HPFs achieves 100% accuracy, although the NIs undergo a JPEG compression with a quality factor of 75.

Photorealistic Computer Graphics Identifying Algorithm Based on Improvement Local Binary Pattern

Photorealistic Computer Graphics Identifying Algorithm Based on Improvement Local Binary Pattern

Training my car to see using virtual worlds

Computer vision technologies are at the core of different advanced driver assistance systems (ADAS) and will play a key role in oncoming autonomous vehicles too. One of the main challenges for such technologies is to perceive the driving environment, i.e. to detect and track relevant driving information in a reliable manner (e.g. pedestrians in the vehicle route, free space to drive through). Nowadays it is clear that machine learning techniques are essential for developing such a visual perception for driving. In particular, the standard working pipeline consists of collecting data (i.e. on-board images), manually annotating the data (e.g. drawing bounding boxes around pedestrians), learning a discriminative data representation taking advantage of such annotations (e.g. a deformable part-based model, a deep convolutional neural network), and then assessing the reliability of such representation with the acquired data. In the last two decades most of the research efforts focused on representation learning (first, designing descriptors and learning classifiers; later doing it end-to-end). Hence, collecting data and, especially, annotating it, is essential for learning good representations. While this has been the case from the very beginning, only after the disruptive appearance of deep convolutional neural networks that it became a serious issue due to their data hungry nature. In this context, the problem is that manual data annotation is a tiresome work prone to errors. Accordingly, in the late 00’s we initiated a research line consisting of training visual models using photo-realistic computer graphics, especially focusing on assisted and autonomous driving. In this paper, we summarize such a work and show how it has become a new tendency with increasing acceptance.

Detecting computer generated images based on local ternary count

Local binary patterns was used to distinguish the Photorealistic Computer Graphics and Photographic Images, however the dimension of the extracted features is too high. Accordingly, the Local Ternary Count based on the Local Ternary Patterns and the Local Binary Count was developed in this study. Furthermore, a novel algorithm is presented based on the Local Ternary Count to classify photorealistic Computer Graphics and Photographic images. The experiment results show that the proposed algorithm effectively reduces the dimension of the classification features and maintains a good classification performance.

A statistical feature based approach to distinguish PRCG from photographs

We present a passive forensics method to distinguish photorealistic computer graphics (PRCG) from natural images (photographs). The goals of our work are to improve the detection accuracy and the robustness to content-preserving image manipulations. In the proposed method, Homomorphic filtering is used to highlight the detail information of image. We find that the texture changes are different between photographs and PRCG images under same Homomorphic filtering transformation, and then we use the difference matrixes to describe the differences of texture changes. We define a customized statistical feature, named texture similarity, and combine it with the statistical features extracted from the co-occurrence matrixes of differential matrixes to construct forensics features. Then we develop a statistical model and use SVM as classifier to distinguish PRCG from photographs. Experimental results show that the proposed method enjoys following advantages: (1) Proposed method reaches higher detection accuracy, synchronously, it is robust to tolerate content-preserving manipulations such as JPEG compression, adding noise, histogram equalization, and filtering. (2) Proposed method is provided with satisfactory generalization capability, it will be available when the training samples and the testing samples come from different sources.

A passive multi-purpose scheme based on periodicity analysis of CFA artifacts for image forensics

We propose a passive multi-purpose scheme for photographic image (PIM) detection and a device class identification method. The motivation for the scheme is the periodicity phenomenon caused by color filter arrays (CFAs) and the demosaicing process. The phenomenon only occurs in the Fourier spectrum in PIMs. The proposed scheme exploits prediction error statistics, local peak detection, and a PIM classifier to analyze the phenomenon for PIM detection. We also develop a hierarchical classifier for device class identification based on the analysis of local peaks in the Fourier spectrum.To evaluate the scheme’s performance, we compile a test dataset of PIMs and PRCG (photorealistic computer graphics) images, and analyze the impact of leak peak detection, JPEG lossy compression, and cropping operations on PIM detection. The accuracy rate of the scheme on 5805 test images is 95.56%, which is higher than that of the methods proposed in Sutthiwan et al. (2009) [1] and Gallagher and Chen (2008) [2]. In addition, for device class identification, the precision rate of the proposed method is at least 93% on Canon, Sony, and Nikon images. The experiment results demonstrate the efficacy of the proposed multi-purpose scheme.

Distinguishing Computer Graphics from Natural Images Based on Statistical Characteristics

In this paper, we propose a new discrimination method using image statistical characteristics is proposed which is designed to distinguish natural images from photorealistic computer graphics. Using Benford model as statistical basis, we conclude statistical properties of the MSD (most significant digit) of AC (Alternating Current) coefficients in DCT (Discrete Cosine Transform) domain of natural images and computer graphics, and then we constructed the detection model of the proposed algorithm. Experimental results show that this method can identify natural images and computer graphics effectively, compared with the existing algorithms this method has a higher recognition rate, which comes to 95.22%.

Concepts for Cartography-Oriented Visualization of Virtual 3D City Models

Virtual 3D city models serve as an effective medium with manifold applications in geoinformation systems and services. To date, most 3D city models are visualized using photorealistic graphics. But an effective communication of geoinformation significantly depends on how important information is designed and cognitively processed in the given application context. One possibility to visually emphasize important information is based on non-photorealistic rendering, which comprehends artistic depiction styles and is characterized by its expressiveness and communication aspects. However, a direct application of non-photorealistic rendering techniques primarily results in monotonic visualization that lacks cartographic design aspects. In this work, we present concepts for cartography-oriented visualization of virtual 3D city models. These are based on coupling non-photorealistic rendering techniques and semantics-based information for a user, context, and media-dependent representation of thematic information. This work highlights challenges for cartography-oriented visualization of 3D geovirtual environments, presents stylization techniques and discusses their applications and ideas for a standardized visualization. In particular, the presented concepts enable a real-time and dynamic visualization of thematic geoinformation.

A Simulation Method of Three-Dimensional Shape Measurement System Based on Texture Mapping and Ray-Tracing Algorithm

Based on the theories of texture mapping in the Photorealistic Graphics and ray-tracing algorithm, this paper proposes a novel method to simulate the real three-dimensional shape measurement system by projecting a sinusoidal stripe texture on computer 3D models, which are generated by the ray-tracing algorithm. The sizes of the main error sources of the real measurement system can be determined by comparing the real measurement result and the simulation one when measuring a standard block, higher accuracy of the simulation system are achieved by adding these error sources, then the source of the errors in the measured object can be achieved in the same way.

Photorealistic computer graphics forensics based on leading digit law

As the advent and growing popularity of image rendering software, photorealistic computer graphics are becoming more and more perceptually indistinguishable from photographic images. If the faked images are abused, it may lead to potential social, legal or private consequences. To this end, it is very necessary and also challenging to find effective methods to differentiate between them. In this paper, a novel leading digit law, also called Benford’s law, based method to identify computer graphics is proposed. More specifically, statistics of the most significant digits are extracted from image’s Discrete Cosine Transform (DCT) coefficients and magnitudes of image’s gradient, and then the Support Vector Machine (SVM) based classifiers are built. Results of experiments on the image datasets indicate that the proposed method is comparable to prior works. Besides, it possesses low dimensional features and low computational complexity.

Identifying and prefiltering images

Given the ability of photorealistic computer graphics (photorealistic CG) to emulate photographic images, as seen in movies and the print media today, there is little doubt that uninformed viewers can easily mistake photorealistic computer- generated graphics for photographic images. In fact, there was already evidence 20 years ago that to the naked eye certain computer graphics were visually indistinguishable from photographic images. Such convincing photorealism qualifies computer graphics as a form of image forgery that can be unscrupulously exploited. Some popular Web sites even highlight examples of computer generated photorealism that human eyes find indistinguishable from photographic images.

Discriminating between photorealistic computer graphics and natural images using fractal geometry

Rendering technology in computer graphics (CG) is now capable of producing highly photorealistic images, giving rise to the problem of how to identify CG images from natural images. Some methods were proposed to solve this problem. In this paper, we give a novel method from a new point of view of image perception. Although the photorealistic CG images are very similar to natural images, they are surrealistic and smoother than natural images, thus leading to the difference in perception. A part of features are derived from fractal dimension to capture the difference in color perception between CG images and natural images, and several generalized dimensions are used as the rest features to capture difference in coarseness. The effect of these features is verified by experiments. The average accuracy is over 91.2%.

Rendering Deformable Surface Reflectance Fields

Animation of photorealistic computer graphics models is an important goal for many applications. Image-based modeling has emerged as a promising approach to capture and visualize real-world objects. Animating image-based models, however, is still a largely unsolved problem. In this paper, we extend a popular image-based representation called surface reflectance field to animate and render deformable real-world objects under arbitrary illumination. Deforming the surface reflectance field is achieved by modifying the underlying impostor geometry. We augment the impostor by a local parameterization that allows the correct evaluation of acquired reflectance images, preserving the original light model on the deformed surface. We present a deferred shading scheme to handle the increased amount of data involved in shading the deformable surface reflectance field. We show animations of various objects that were acquired with 3D photography.