Compression Of Ultraspectral Sounder Data Research Articles

Adaptive quantized PCA with 3D prediction and positive packing for lossless compression of ultraspectral sounder data

Given the unprecedented size of three-dimensional ultraspectral sounder data with high spectral resolution, lossless compression is preferable to avoid substantial degradation of the geophysical retrieval. A lossless compression method for ultraspectral sounder data is therefore developed. A quantized-principal-component-analysis-based scheme is presented by combining 3D prediction, positive mapping, and histogram packing using binary indexing vectors (positive packing) followed by a range coder. In order to achieve the optimal trade-off between residual errors and side information, an algorithm is proposed to determine adaptively the number of selected PCs and quantization parameters. Numerical experiments show that the proposed method outperforms the state-of-the-art methods (i.e. linear prediction with constant coefficients (LP-CC) and linear prediction with optimal granule ordering (LP-OGO)) by 1.77% in terms of compression ratio.

GPU Acceleration of Predictive Partitioned Vector Quantization for Ultraspectral Sounder Data Compression

For the large-volume ultraspectral sounder data, compression is desirable to save storage space and transmission time. To retrieve the geophysical paramters without losing precision the ultraspectral sounder data compression has to be lossless. Recently there is a boom on the use of graphic processor units (GPU) for speedup of scientific computations. By identifying the time dominant portions of the code that can be executed in parallel, significant speedup can be achieved by using GPU. Predictive partitioned vector quantization (PPVQ) has been proven to be an effective lossless compression scheme for ultraspectral sounder data. It consists of linear prediction, bit depth partitioning, vector quantization, and entropy coding. Two most time consuming stages of linear prediction and vector quantization are chosen for GPU-based implementation. By exploiting the data parallel characteristics of these two stages, a spatial division design shows a speedup of 72x in our four-GPU-based implementation of the PPVQ compression scheme.

Digital signal processor-based three-dimensional wavelet error-resilient lossless compression of high-resolution spectrometer data

Ultraspectral sounders revolutionize the way remote sensing data are collected, retrieved, and assimilated to provide better weather, climate, and environmental prediction and monitoring. This unprecedented amount of ultraspectral data increases the burden of data bandwidth and the chance of transmission noise and error contamination. It is desired that source coding of ultraspectral data also have some error-resilient capability, in addition to the error-correcting channel coding. Earlier, we developed three-dimensional wavelet reversible variable-length coding (3DWT-RVLC) for lossless compression of ultraspectral sounder data, which has significantly better error resilience than JPEG2000 Part 2 at only a small reduction in compression gain. The reversible variable-length codes allow instantaneous decoding in both directions, which affords better detection of bit errors due to synchronization losses over a noisy channel. To explore the feasibility of 3DWT-RVLC for real-time ultraspectral data processing, we implement a memory-limited digital signal processor (DSP) version of 3DWT-RVLC. Compression experiments on 10 ultraspectral test granules obtained from the NASA Atmospheric Infrared Sounder show that the memory-limited DSP-based 3DWT-RVLC is able to perform high-speed data processing at only a small reduction in compression ratio as compared to the original 3DWT-RVLC.

Constant coefficients linear prediction for lossless compression of ultraspectral sounder data using a graphics processing unit

The amount of data generated by ultraspectral sounders is so large that considerable savings in data storage and transmission bandwidth can be achieved using data compression. Due to this large amount of data, the data compression time is of utmost importance. Increasing the programmability of the commodity Graphics Processing Units (GPUs) offer potential for considerable increases in computation speeds in applications that are data parallel. In our experiments, we implemented a spectral image data compression method called Linear Prediction with Constant Coefficients (LP-CC) using NVIDIA's CUDA parallel computing architecture. LP-CC compression method represents a current state-of-the-art technique in lossless compression of ultraspectral sounder data. The method showed an average compression ratio of 3.39 when applied to publicly available NASA AIRS data. We achieved a speed-up of 86 compared to a single threaded CPU version. Thus, the commodity GPU was able to significantly decrease the computational time of a compression algorithm based on a constant coefficient linear prediction.

Ultraspectral sounder data compression review

Ultraspectral sounders provide an enormous amount of measurements to advance our knowledge of weather and climate applications. The use of robust data compression techniques will be beneficial for ultraspectral data transfer and archiving. This paper reviews the progress in lossless compression of ultraspectral sounder data. Various transform-based, prediction-based, and clustering-based compression methods are covered. Also studied is a preprocessing scheme for data reordering to improve compression gains. All the coding experiments are performed on the ultraspectral compression benchmark dataset collected from the NASA Atmospheric Infrared Sounder (AIRS) observations.