WORF (Write Once, Read Forever) Next Generation Archival Big Data Storage

Abstract

In this paper we discuss a next generation archival architecture for ultra rapid, high-density, parallel writing and retrieval of Big Data. We term this WORF (Write Once, Read Forever)-repurposing a proven long-life optical media technology for digital data, using no power for storage. The data is stored as nano-scale standing waves, optimized for a high SNR, embedding its resultant optical pattern in a monochromatic, silver halide (AgX), super-high-resolution photosensitive emulsion. Applying M-ary arithmetic (instead of binary), multiple numerical states are stored in these standing waves, enabling multiple bytes to be stored at one data location. The WORF storage system retrieves data in parallel arrays, providing true random data access, and consequentially transferring data faster than current processors can absorb the stream. Legacy storage devices must continually replicate data during their lifetime in order to mitigate hardware and media failure, and corruption from malware, bit-rot read/write errors, and spaceborne radiation. In contrast, WORF applies a proven archival media which can simply be put in a drawer, under normal ambient environmental conditions, and remain stable for centuries. This would be ideal for astronomical observatories, such as the telescope due to go online in Chile in 2023 that will collect 20 Tbytes per day of data-a streaming and long-term sustainability challenge for cloud servers. WORF media is immutable, hack-free, resistant to counterfeiting, anti-microbial, and has been tested by NASA on the International Space Station during 2019 demonstrating that it is impervious to space hazards.