Optimising Cloud Operation Cost Utilising External Memory

Abstract

Handling large volumes of data on computers incur operational costs when physical hardware is considered, especially RAM, creating a need for intelligent solutions that both maintain an acceptable level of performance and enable cheaper scaling. The authors extend their previous work converting their existing point cloud processing and analysis tool to use external memory via the STXXL C++ library, replacing the entire dataset storage layer with STXXL's intelligent caching system. A rationale for adopting this technique, and methodology for testing previous and modified versions of the software is put forth, and the authors investigate the behaviour of their software tool to establish trade-offs. Competing versions of their software are fed sample datasets in E57 and IFC formats; the results of which are captured and analysed. The authors find that while execution speed is lowered, reduced memory consumption contributes to a higher throughput, enabling greater efficiency and real hardware cost savings.