Abstract
In this paper, we present RayTracer.jl, a renderer in Julia that is fully differentiable using source-to-source Automatic Differentiation (AD). This means that RayTracer not only renders 2D images from 3D scene parameters, but it can be used to optimize for model parameters that generate a target image in a Differentiable Programming (DP) pipeline. We interface our renderer with the deep learning library Flux for use in combination with neural networks. We demonstrate the use of this differentiable renderer in rendering tasks and in solving inverse graphics problems.
Highlights
Rendering is a technique of generating photo-realistic or nonphoto-realistic 2D projections from 3D objects
We explore the idea of differentiability through a renderer, by leveraging the Automatic Differentiation (AD) in Julia [3]
Our renderer contains very little code for integration with Zygote, and in theory, we can plug in any other AD software written in Julia
Summary
Rendering is a technique of generating photo-realistic or nonphoto-realistic 2D projections from 3D objects. The ray can undergo reflection and refraction due to interactions with the objects in its path This technique, is very computationally expensive and difficult to do in real-time. Since ray tracing leverages the properties of the materials of the objects in the scene, a natural extension to the rendering problem would be to extract the exact properties of the materials, lighting, and so on, given an image of a scene. Calculating analytic gradients for every single parameter of the scene is a very tedious process and prone to errors This has made it a difficult task to present a general gradient-based inverse rendering method. Rendering is a computationally expensive technique, and so it is generally done in static languages like C++ Developing software in such languages are incredibly time-consuming. Our renderer contains very little code for integration with Zygote, and in theory, we can plug in any other AD software written in Julia
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
