Partitioning Terabyte-scale Faceted Geometry Models for Efficient Parallel Ray Tracing Using Out-of-core Memory

Abstract

Generating high-fidelity, synthetic, infrared images of authentic landscapes is an important step in training cutting edge IED recognition and target identification algorithms. This process requires radiative transfer modeling using ray tracing on memory-intensive meshes, potentially petabytes in size. The current software architectures for performing these simulations lack the capability to handle large, complex environments efficiently. To address this scalability problem, a state-of-the-art architecture is being developed to generate realistic, virtual images using ray tracing methods on geometry for large, complex environments (e.g. forested landscapes). This research presents a parallel architecture for distributing geometry and creating structures to support efficient ray tracing on these memory-intensive models. The proposed architecture uses an out-of-core memory framework to efficiently handle models larger than a single computing element's available RAM without employing constant disk I/O. Geometry data is partitioned to fit the architecture using spatial sorting methods, collective communication, and parallel I/O. MPI is used for the communication and I/O layer of the application, providing a portable, scalable method for distributing the spatially sorted geometry data for the system and facilitating ray casting. The approach used for spatial sorting will be discussed, and the collective communication protocol and parallel I/O system will be explained. Applying this methodology to build a ray tracing engine has resulted in an architecture that is able to efficiently trace rays in faceted geometry data over a terabyte in size. Timings gathered of the process running on a supercomputer have shown scalability as the size of the problem has increased. This provides the capability to analyze landscapes far larger than was previously possible.