Graphics Research Research Articles

Light Distribution Models for Tree Growth Simulation

AbstractThe simulation and modelling of tree growth is a complex subject with a long history and an important area of research in both computer graphics and botany. For more than 50 years, new approaches to this topic have been presented frequently, including several aspects to increase realism. To further improve these achievements, we present a compact and robust functional‐structural plant model (FSPM) that is consistent with botanical rules. While we show several extensions to typical approaches, we focus mainly on the distribution of light as a resource in three‐dimensional space. We therefore present four different light distribution models based on ray tracing, space colonization, voxel‐based approaches and bounding volumes. By simulating individual light sources, we were able to create a more specified scene setup for plant simulation than it has been presented in the past. By taking into account such a more accurate distribution of light in the environment, this technique is capable of modelling realistic and diverse tree models.

A Survey on Cage‐based Deformation of 3D Models

AbstractInteractive deformation via control handles is essential in computer graphics for the modeling of 3D geometry. Deformation control structures include lattices for free‐form deformation and skeletons for character articulation, but this report focuses on cage‐based deformation. Cages for deformation control are coarse polygonal meshes that encase the to‐be‐deformed geometry, enabling high‐resolution deformation. Cage‐based deformation enables users to quickly manipulate 3D geometry by deforming the cage. Due to their utility, cage‐based deformation techniques increasingly appear in many geometry modeling applications. For this reason, the computer graphics community has invested a great deal of effort in the past decade and beyond into improving automatic cage generation and cage‐based deformation. Recent advances have significantly extended the practical capabilities of cage‐based deformation methods. As a result, there is a large body of research on cage‐based deformation. In this report, we provide a comprehensive overview of the current state of the art in cage‐based deformation of 3D geometry. We discuss current methods in terms of deformation quality, practicality, and precomputation demands. In addition, we highlight potential future research directions that overcome current issues and extend the set of practical applications. In conjunction with this survey, we publish an application to unify the most relevant deformation methods. Our report is intended for computer graphics researchers, developers of interactive geometry modeling applications, and 3D modeling and character animation artists.

Evaluating the Effect of Outfit on Personality Perception in Virtual Characters

Designing virtual characters that are capable of reflecting a sense of personality is a key goal in research and applications in virtual reality and computer graphics. More and more research efforts are dedicated to investigating approaches to construct a diverse, equitable, and inclusive metaverse by infusing expressive personalities and styles into virtual avatars. While most previous work focused on exploring variations in virtual characters’ dynamic behaviors, characters’ visual appearance plays a crucial role in affecting their perceived personalities. This paper presents a series of experiments evaluating the effect of virtual characters’ outfits on their perceived personality. Based on the related psychology research conducted in the real world, we determined a set of outfit factors likely to reflect personality in virtual characters: color, design, and type. As a framework for our study, we used the “Big Five” personality model for evaluating personality traits. To test our hypothesis, we conducted three perceptual experiments to evaluate the outfit parameters’ contributions to the characters’ personality. In our first experiment, we studied the color factor by varying color hue, saturation, and value; in the second experiment, we evaluated the impact of different neckline, waistline, and sleeve designs; and in our third experiment, we examined the personality perception of five outfit types: professional, casual, fashionable, outdoor, and indoor. Significant results offer guidance to avatar designers on how to create virtual characters with specific personality profiles. We further conducted a verification test to extend the application of our findings to animated virtual characters in augmented reality (AR) and virtual reality (VR) settings. Results confirmed that our findings can be broadly applied to both static and animated virtual characters in VR and AR environments that are commonly used in games, entertainment, and social networking scenarios.

From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans

Great progress has been made in estimating 3D human pose and shape from images and video by training neural networks to directly regress the parameters of parametric human models like SMPL. However, existing body models have simplified kinematic structures that do not correspond to the true joint locations and articulations in the human skeletal system, limiting their potential use in biomechanics. On the other hand, methods for estimating biomechanically accurate skeletal motion typically rely on complex motion capture systems and expensive optimization methods. What is needed is a parametric 3D human model with a biomechanically accurate skeletal structure that can be easily posed. To that end, we develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton. To enable this, we need training data of skeletons inside SMPL meshes in diverse poses. We build such a dataset by optimizing biomechanically accurate skeletons inside SMPL meshes from AMASS sequences. We then learn a regressor from SMPL mesh vertices to the optimized joint locations and bone rotations. Finally, we re-parametrize the SMPL mesh with the new kinematic parameters. The resulting SKEL model is animatable like SMPL but with fewer, and biomechanically-realistic, degrees of freedom. We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods. By fitting SKEL to SMPL meshes we are able to "upgrade" existing human pose and shape datasets to include biomechanical parameters. SKEL provides a new tool to enable biomechanics in the wild, while also providing vision and graphics researchers with a better constrained and more realistic model of human articulation. The model, code, and data are available for research at https://skel.is.tue.mpg.de.

Fundamental ray tracing project based on OpenGL with acceleration

In 3D computer graphics, ray tracing is an essential technique for modeling light transport for use in a wide variety of rendering algorithms for generating digital images. And it is also an important subject for computer science students who want to discover more in Computer Graphics and image rendering. In this project the author implemented a fundamental light tracing system based on C++ in visual studio. It is able to render a delicate image with an input of a nnf model file, and output a ppm file, as a final assignment of a student computer graphics research program. This paper is divided into two chapters. The first chapter is about some fundamental theories that are critical to light tracing. The second chapter is about detailed design and implementation process of the program. And the final chapter is about some actual problems and effective solutions.

Cuttlefish: Pushing the Limits of Graphical 3-D Printing.

This article presents Cuttlefish, a 3-D printer driver developed at the Fraunhofer Institute for Computer Graphics Research IGD. Cuttlefish maximizes the reproduction quality of shape and appearance in multimaterial 3-D printing systems. It controls various printing systems and material sets, including color, transparent, and flexible materials. The article provides an overview of Cuttlefish's graphical 3-D printing pipeline and focuses on its unique features: Displaced signed distance fields for robust voxelization of objects not designed for 3-D printing, joint color and translucency reproduction, accurate color and translucency characterization, shape dithering to eliminate quantization-based staircase artifacts, and support for displacement maps and constructive solid geometry operations. The article concludes by showcasing applications of Cuttlefish in printing replacement faces for stop-motion animation, large-scale figurine production, and prosthetic eye printing.

High Speed and Accuracy of Animation 3D Pose Recognition Based on an Improved Deep Convolution Neural Network

Pose recognition in character animations is an important avenue of research in computer graphics. However, the current use of traditional artificial intelligence algorithms to recognize animation gestures faces hurdles such as low accuracy and speed. Therefore, to overcome the above problems, this paper proposes a real-time 3D pose recognition system, which includes both facial and body poses, based on deep convolutional neural networks and further designs a single-purpose 3D pose estimation system. First, we transformed the human pose extracted from the input image to an abstract pose data structure. Subsequently, we generated the required character animation at runtime based on the transformed dataset. This challenges the conventional concept of monocular 3D pose estimation, which is extremely difficult to achieve. It can also achieve real-time running speed at a resolution of 384 fps. The proposed method was used to identify multiple-character animation using multiple datasets (Microsoft COCO 2014, CMU Panoptic, Human3.6M, and JTA). The results indicated that the improved algorithm improved the recognition accuracy and performance by approximately 3.5% and 8–10 times, respectively, which is significantly superior to other classic algorithms. Furthermore, we tested the proposed system on multiple pose-recognition datasets. The 3D attitude estimation system speed can reach 24 fps with an error of 100 mm, which is considerably less than that of the 2D attitude estimation system with a speed of 60 fps. The pose recognition based on deep learning proposed in this study yielded surprisingly superior performance, proving that the use of deep-learning technology for image recognition has great potential.

The Origins of Computer Graphics in Europe: Part 2-The Early Spreading of Computer Graphics in Europe.

"The Origins of Computer Graphics in Europe," is being published in two parts: Part 1, published in the March/April issue of IEEE Computer Graphics and Applications (IEEE CG&A), is subtitled "The Beginnings in Germany"; Part 2, published in this May/June 2023 issue, is subtitled "The Early Spreading of Computer Graphics in Europe." I was a participant, contributor, and witness to the events reported here and I relate my personal story along with the broader history. Part 1 describes the origins and successful evolution of computer graphics in Germany, starting in 1965, and includes details of the people and subject matter of the earliest research groups. It describes the efforts undertaken to establish computer graphics as a proper academic discipline, including the founding of EUROGRAPHICS, and creation of institutes for both basic and applied research in computer graphics. Part 2 continues the story with a focus on activities contributing to the growth of the academic and industrial computer graphics communities across Europe and documents the two IFIP workshops held at Seillac and the development of the GKS Graphics Standard. Over the years, computer graphics gained respect and importance as a component of the computer science curricula and became an important tool and enabling technology for applications for industry and for the IT market in Europe.

A survey of Optimal Transport for Computer Graphics and Computer Vision

AbstractOptimal transport is a long‐standing theory that has been studied in depth from both theoretical and numerical point of views. Starting from the 50s this theory has also found a lot of applications in operational research. Over the last 30 years it has spread to computer vision and computer graphics and is now becoming hard to ignore. Still, its mathematical complexity can make it difficult to comprehend, and as such, computer vision and computer graphics researchers may find it hard to follow recent developments in their field related to optimal transport. This survey first briefly introduces the theory of optimal transport in layman's terms as well as most common numerical techniques to solve it. More importantly, it presents applications of these numerical techniques to solve various computer graphics and vision related problems. This involves applications ranging from image processing, geometry processing, rendering, fluid simulation, to computational optics, and many more. It is aimed at computer graphics researchers desiring to follow optimal transport research in their field as well as optimal transport researchers willing to find applications for their numerical algorithms.

Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

AbstractAiming at obtaining structural information and 3D motion of dynamic scenes, scene flow estimation has been an interest of research in computer vision and computer graphics for a long time. It is also a fundamental task for various applications such as autonomous driving. Compared to previous methods that utilize image representations, many recent researches build upon the power of deep analysis and focus on point clouds representation to conduct 3D flow estimation. This paper comprehensively reviews the pioneering literature in scene flow estimation based on point clouds. Meanwhile, it delves into detail in learning paradigms and presents insightful comparisons between the state‐of‐the‐art methods using deep learning for scene flow estimation. Furthermore, this paper investigates various higher‐level scene understanding tasks, including object tracking, motion segmentation, etc. and concludes with an overview of foreseeable research trends for scene flow estimation.

The Origins of Computer Graphics in Europe: Part 1-The Beginnings in Germany.

"The Origins of Computer Graphics in Europe," is being published in two parts: Part 1, in this issue of IEEE Computer Graphics and Applications, is subtitled "The Beginnings in Germany"; Part 2, to be published in the May/June issue, is subtitled "The Spreading of Computer Graphics in Europe." I was a participant, contributor, and witness to the events reported here and I relate my personal story along with the broader history. Part 1 describes the origins and successful evolution of computer graphics in Germany, starting in 1965, and includes details of the people and subject matter of the earliest research groups. It describes the efforts undertaken to establish computer graphics as a proper academic discipline, including the founding of EUROGRAPHICS, and creation of institutes for both basic and applied research in computer graphics. Part 2 continues the story with a focus on activities contributing to the growth of the academic and industrial computer graphics communities across Europe and documents the two IFIP workshops at Seillac and the development of the GKS Graphics Standard. Over these years, computer graphics gained respect and importance as a component of the computer science curricula and became an important tool and enabling technology for applications for industry and for the IT market in Europe.

Rain Removal from Light Field Images with 4D Convolution and Multi-scale Gaussian Process.

Existing deraining methods mainly focus on a single input image. However, with just a single input image, it is extremely difficult to accurately detect and remove rain streaks, in order to restore a rain-free image. In contrast, a light field image (LFI) embeds abundant 3D structure and texture information of the target scene by recording the direction and position of each incident ray via a plenoptic camera, which has emerged as a popular device in the computer vision and graphics research communities. However, making full use of the abundant information available from LFIs, such as 2D array of sub-views and the disparity map of each sub-view, for effective rain removal is still a challenging problem. In this paper, we propose a novel network, 4D-MGP-SRRNet, for rain streak removal from LFIs. Our method takes as input all sub-views of a rainy LFI. In order to make full use of the LFI, we adopt 4D convolutional layers to build the proposed rain steak removal network to simultaneously process all sub-views of the LFI. In the proposed network, the rain detection model, MGPDNet, with a novel Multi-scale Self-guided Gaussian Process (MSGP) module is proposed to detect high-resolution rain streaks from all sub-views of the input LFI at multi-scales. Semi-supervised learning is introduced for MSGP to accurately detect rain streaks by training on both virtual-world rainy LFIs and real-world rainy LFIs at multi-scales via calculating pseudo ground truths for real-world rain streaks. We then feed all sub-views subtracting the predicted rain streaks into a 4D convolution-based Depth Estimation Residual Network (DERNet) to estimate the depth maps, which are later converted into fog maps. Finally, all sub-views concatenated with the corresponding rain streaks and fog maps are fed into a powerful rainy LFI restoring model based on the adversarial recurrent neural network to progressively eliminate rain streaks and recover the rain-free LFI. Extensive quantitative and qualitative evaluations conducted on both synthetic LFIs and real-world LFIs demonstrate the effectiveness of our proposed method.

Interview of Ivan Sutherland

Ivan E. Sutherland was born in Hastings, Nebraska, in 1938. He received a BS from the Carnegie Institute of Technology (1959) in electrical engineering, an MS from Caltech (1960) and a PhD in electrical engineering from MIT (1963) <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sutherland's dissertation, Sketchpad: A Man Machine Graphical Communication System, described a groundbreaking interactive computer-aided design system, one whose graceful interaction and functionality inspire admiration among computer graphics professionals even today <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">From 1964 to 1966, Sutherland was director of the Information Processing Techniques Office of the Advanced Research Projects Agency (ARPA), where he guided computer research across the United States. In 1968, he joined colleague David Evans to build a center of computer graphics research at the University of Utah and to found with him the Evans and Sutherland Computer Corporation <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">From 1976 to 1980, Sutherland served as chairman of computer science at Caltech, and from 1980 onwards, was a vice president of the consulting firm Sutherland, Sproull and Associates, acquired by Sun Microsystems in 1990 to form the basis of its corporate research laboratory <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sutherland has won many awards for his work, including the ACM Turing Award (1988) the IEEE John von Neumann Medal (1998). He is a Fellow of the Computer History Museum.

A fully implicit method for robust frictional contact handling in elastic rods

Accurate frictional contact is critical in simulating the assembly of rod-like structures in the practical world, such as knots, hairs, flagella, and more. Due to their high geometric nonlinearity and elasticity, rod-on-rod contact remains a challenging problem tackled by researchers in both computational mechanics and computer graphics. Typically, frictional contact is regarded as constraints for the equations of motions of a system. Such constraints are often computed independently at every time step in a dynamic simulation, thus slowing down the simulation and possibly introducing numerical convergence issues. This paper proposes a fully implicit penalty-based frictional contact method, Implicit Contact Model (IMC), that efficiently and robustly captures accurate frictional contact responses. We showcase our algorithm’s performance in achieving visually realistic results for the challenging and novel contact scenario of flagella bundling in fluid medium, a significant phenomenon in biology that motivates novel engineering applications in soft robotics. In addition to this, we offer a side-by-side comparison with Incremental Potential Contact (IPC), a state-of-the-art contact handling algorithm. We show that IMC possesses comparable performance to IPC while converging at a faster rate.

(Retracted) Computer-aided three-dimensional animation design and enhancement based on spatial variation and convolution algorithm

Computer-aided technology is a system of personal confidential interaction between computer and users under the guidance of people. Due to the continuous development of computer technology, this technology has been used in many fields. Computer-aided design is the use of computer systems to assist designers in engineering or product design. It is a technology to achieve the best design effect. Three-dimensional (3D) animation is not limited by time and space. It uses various forms of expression to express complex and abstract program content, scientific principles, and abstract concepts in centralized and simplified forms. The creative feature of animation is that the author does not use the advanced 3D technology to present the real world to the audience. Here, we aim to study the 3D animation image design and image enhancement based on computer-aided technology, hoping to make the 3D animation more realistic and make the film and television picture clearer with the help of computer-aided technology. We expound the current situation of 3D graphics research. We analyze the current development of the 3D animation industry. We propose an extreme value constraint based on spatial variation and a convolution algorithm based on learning a ringing images. The experimental results show that after Retinex method processes the image, the matching feature points of the target image in the scale-invariant feature transformation algorithm are 55. The correct matching rate of feature points after homomorphic filtering is 0.85. The number of correct matching points of the image processed by Retinex method changes greatly, and the change of homomorphic filtering is small.

The Art and Science of Displaying Visual Space

This paper considers the problem of how to display visual space naturalistically in image media. A long-standing solution is linear perspective projection, which is currently used in imaging technologies from cameras to 3D graphics renderers. Linear perspective has many strengths but also some significant weaknesses and over the centuries alternative techniques have been developed for creating more naturalistic images. Here we discuss the problem, its scientific background, and some of the approaches taken by artists and computer graphics researchers to find solutions. We briefly introduce our own approach, which is a form of nonlinear 3D geometry modelled on the perceptual structure of visual space and designed to work on standard displays. We conclude that perceptually modelled nonlinear approaches can make 3D imaging technology more naturalistic than methods based on linear perspective.

Application of Line Integral Convolution Techniques in the Generation of Fine Art Painting Effects

Since the 1980s, nonrealistic drawing techniques have become one of the main focuses of research in computer graphics. In recent years, with the rapid development of computer graphics, nonrealistic drawing techniques have become capable of rendering any digital image into an image with an alternative artistic style. As a result, many scholars have focused on the pencil sketch style. Against this backdrop, research into how to create more realistic art drawings has become a hot topic. Although the rapid development of computer technology has led to advances in art drawing techniques, traditional fine art painting methods are not yet able to simulate realistic pencil sketches in a quite realistic way. In other words, the following issues have been noted. For example, they fail to simulate the gradation of shades and densities of real sketches, the direction of the shadows does not conform to the general rules of sketching, and the results produced are not artistic and realistic. After all, unlike realistic graphics, which seeks a photo-like realism, nonrealistic graphics wants to highlight the message of the image with the help of computer simulations. As a result, nonrealistic graphics often ignores unimportant details in the image. On the other hand, nonrealistic graphics can also simulate an artist’s painting to a certain extent, thus producing an artistically effective image. Many nonrealistic drawing methods have been devised to simulate the styles of oil, watercolour, and ink, but not many algorithms have been developed to generate colour sketches. In order to address this topic, this study improves a fine art drawing generation approach based on the line integral convolution techniques. To be specific, this investigation begins by segmenting the colour images through cluster analysis. Two basic colours are then assigned to each region by calculating the colour variability, and the density of each colour is calculated using a two-tone mapping technique. After that, the line integral convolution technique is applied to generate sketch textures for each of the two basic colour layers, and the two layers are fused to produce the coloured textures. At the same time, a neon transform is used to generate the sketch contour lines. Eventually, the contours are blended with the coloured textures to obtain the coloured sketch effect. The experimental results show that the art painting generation method designed in this paper is able to automatically transform from colour images to coloured pencil drawings, thus improving the efficiency of the process as well as meeting the real-time requirements.

A Microfacet‐based Hair Scattering Model

AbstractThe development of scattering models and rendering algorithms for human hair remains an important area of research in computer graphics. Virtually all available models for scattering off hair or fur fibers are based on separable lobes, which bring practical advantages in importance sampling, but do not represent physically‐plausible microgeometry. In this paper, we contribute the first microfacet‐based hair scattering model. Based on a rough cylinder geometry with tilted cuticle scales, our far‐field model is non‐separable by nature, yet allows accurate importance sampling. Additional benefits include support for elliptical hair cross‐sections and an analytical solution for the reflected lobe using the GGX distribution. We show that our model captures glint‐like forward scattering features in the R lobe that have been observed before but not properly explained.

A Real-Time Distance Prediction via Deep Learning and Microsoft Kinect

3D(Dimension) understanding has become the herald of computer vision and graphics research in the era of technology. It benefits many applications such as autonomous cars, robotics, and medical image processing. The pros and cons of 3D detection bring convenience to the human community instead of 2D detection. The 3D detection consists of RGB (Red, Green and Blue) colour images and depth images which are able to perform better than 2D in real. The current technology is relying on the high costing light detection and ranging (LiDAR). However, the use of Microsoft Kinect has replaced the LiDAR systems for 3D detection gradually. In this project, a Kinect camera is used to extract the depth of image information. From the depth images, the distance can be defined easily. As in the colour scale, the red colour is the nearest and the blue colour is the farthest. The depth image will turn black when reaching the limitation of the Kinect camera measuring range. The depth information collected will be trained with deep learning architecture to perform a real-time distance prediction.

18‐1: Invited Paper: Modeling and Optimizing Human‐in‐the‐Loop Visual Perception Using Immersive Displays: A Review

New and rapidly‐evolving classes of display devices bridge the gap between us and the immersive experiences of the future. The most intimate of these displays are the Virtual‐ and Augmented‐Reality (VR and AR) ones, because they are capable of presenting synthetic environments that rival those in the real world. This ecosystem of personal and highly‐immersive displays offers new challenges for research in computer graphics, display technologies, and human visual perception. While the extensive advancements in the areas of display and computer graphics technologies traditionally end at the on‐screen “image,” there are several untapped opportunities for advances that exploit the interplay between the display characteristics and how our visual system perceives them.In this article, we review recent progress in understanding and modeling the perception of immersive displays, as well as perceptually optimizing display technologies for immersive experiences. We present this review in the form of a taxonomy that maps the various properties of modern displays with the perceptual phenomenon that most closely interacts with them. From this taxonomy, we deduce several unsolved challenges in understanding human perception of displays, as well as perceptually‐optimal characteristics of future displays.