Astronomical Data Preprocessing Implementation Based on FPGA and Data Transformation Strategy for the FAST Telescope as a Giant CPS

Abstract

The emergence of cyber-physical-social systems (CPSS) as a novel paradigm has revolutionized the relationship between humans, computers and the physical environment. CPSS extend cyber-physical systems (CPS) to include the social domain, which introduces a challenge of massive data processing. As a typically giant CPS, the Five-hundred-meter Aperture Spherical radio Telescope (FAST), the world’s largest filled-aperture radio telescope, generates massive volume of data which poses a huge storage problem that CPSS face likewise and requires real-time data compressing to reduce data storage and movement overhead. The recently introduced Bitshuffle preprocessing algorithm is a novel approach towards exploiting spatial redundancy incorporation to improve the compression ratio with a specific compressor. However, the existing high-performance CPU-based solutions cannot satisfy the performance requirement and power budget requirement simultaneously. In the paper, we propose the implementation of this algorithm on Field Programmable Gate Array (FPGA) and present an unique data transformation strategy to turn raw FAST data in classic FITS format into another format to support huge file sizes, i.e. Hierarchical Data Format (HDF5). Evaluation results show that our implementation can achieve 3.2Gbyte/s throughput which can be equipped with LZ4 compressor to be high performance compressor. This makes Bitshuffle on FPGAs a candidate for meeting the computational and energy efficiency constraints of radio telescopes and provide reference for CPSS facing the same situation.