Near lossless data compression onboard a hyperspectral satellite

Abstract

To deal with the large volume of data produced by hyperspectral sensors, the Canadian Space Agency (CSA) has developed and patented two near lossless data compression algorithms for use onboard a hyperspectral satellite: successive approximation multi-stage vector quantization (SAMVQ) and hierarchical self-organizing cluster vector quantization (HSOCVQ). This paper describes the two compression algorithms and demonstrates their near lossless feature. The compression error introduced by the two compression algorithms was compared with the intrinsic noise of the original data that is caused by the instrument noise and other noise sources such as calibration and atmospheric correction errors. The experimental results showed that the compression error was not larger than the intrinsic noise of the original data when a test data set was compressed at a compression ratio of 20:1. The overall noise in the reconstructed data that contains both the intrinsic noise and the compression error is even smaller than the intrinsic noise when the data is compressed using SAMVQ. A multi-disciplinary user acceptability study has been carried out in order to evaluate the impact of the two compression algorithms on hyperspectral data applications. This paper briefly summarizes the evaluation results of the user acceptability study. A prototype hardware compressor that implements the two compression algorithms has been built using field programmable gate arrays (FPGAs) and benchmarked. The compression ratio and fidelity achieved by the hardware compressor are similar to those obtained by software simulation