BioTIFF: A New BigTIFF File Structure For Organizing Large Image Datasets And Their Associated Metadata

Abstract

The Tagged Image File Format data structure (TIFF) is a well known and widely used digital image format. The standard TIFF uses 32 bit offsets giving a 4G maximum size for a multi-page TIFF. However, recently a Big TIFF project has been initiated (http://www.awaresystems.be/imaging/tiff/bigtiff.html) allowing for 64bit offsets so that one file can contain 16 ExaBytes (16 Million TeraBytes). Although that size may at first glance appear to be essentially infinite, mega TeraByte data sets are not inconceivable when using high performance/resolution colour cameras to explore biophysical phenomena in multiple equivalent samples under multiple conditions before averaging. Cloud computing and storage services allows for ubiquitous ad hoc access to large data dataset processing environments opening up the possibility of comprehensive and distributed “whole dataset” analysis of the fit between theory and experiment. The BioTIFF format enables a non-relational presentation of large datasets for such distributed analysis. BioTIFF 1.0 is being released as a free and opens source TIFF writer reader that can annotate and locate large image-based datasets (see www.biotiff.org). It includes encryption tags as well as means for remotely accessioning, curating and reusing original data and interpretations, assuring their provenance and enabling distributed usage. The potential of using the BioTIFF for parallel high-throughput high-content screening will be illustrated. A BioTIFF reader-writer running under the LTS 64 bit Ubuntu 8.0 operating system will demonstrated.