The LOFAR long-term archive: e-infrastructure on petabyte scale

Abstract

The Low Frequency Array (LOFAR) is a large distributed radio telescope that observes at frequencies from 10 MHz to 240 MHz. LOFAR combines phased array antenna stations in Germany, the UK, France, and Sweden with forty stations in the Netherlands. The archive of science data products is expected to grow by five petabyte per year. The LOFAR long-term archive (LTA) provides an e-Infrastructure for the storage, distribution, and analysis of the science data produced by LOFAR. It builds on national and international e-Infrastructure on a European scale pioneering the international cyber-infrastructures of the future such as needed for e.g. the Square Kilometer Array. For astronomers, the LOFAR LTA is the principal interface not only to LOFAR data retrieval and data mining but also to processing facilities for this data. Each site involved in the LTA provides storage capacity and optionally processing capabilities. To allow collaboration with a variety of institutes and projects, the LOFAR LTA merges different technologies (EGI, global file systems, Astro-WISE dataservers). For its connectivity it utilizes the national research networks and explores new technologies for high bandwidth on demand and long distance data streaming. A centrally operated catalogue provides a searchable database for scientific products stored in the LTA from raw visibilities to calibrated images and derived source lists. The user administration synchronizes accounts across the LTA and is designed to connect to federated infrastructures. For its data analysis capabilities it builds on the data processing frameworks provided by EGI and Astro-WISE.