АЛГОРИТМ ПОТОКОВОЙ КОМПРЕССИИ ДАННЫХ С ПЛАВАЮЩЕЙ ЗАПЯТОЙ В ИНФОРМАЦИОННЫХ СИСТЕМАХ ОБЕСПЕЧЕНИЯ НАУЧНЫХ ЭКСПЕРИМЕНТОВ

Abstract

The paper presents an original algorithm and its implementation for single pass real-timecompression of streams of numeric floating-point data. The purpose of the research is to developand formalize a single-pass algorithm of stream floating-point data compression in order to increaseperformance of both encoding and decoding, because a use of existing implementationsprovides insufficient speed of compression, are too restrictive on hardware resources and limitedin applicability to real-time stream data compression when it comes to floating-point data.For that, the following issues have been addressed. The developed mathematical model and thealgorithm for compression of scalar floating-point data are described together with results of experimentalresearch of the compression method applied to single-dimensional and twodimensionalscientific data. The model is based upon the commonly-used binary_64 representation,of the IEEE-754 standard, onto which extended real-line values are mapped. The algorithmcan be implemented as part of high-performance distributed systems in which performance ofinput-output operations, as well as internetwork communication, are critical to overall efficiency.The performance and applicability of the algorithm in data stream compression result from itssingle-pass behaviour, relatively low requirements to a priori known and statically defined size ofmemory required to implement history of compression, which the predictor, used in compressionand decompression, is based on. Indeed, the measured compression ratios are comparable to oneswhich result from using more resource-intensive universal coders but providing significantly lowerlatency. Provided synchronization of parameters of both compressor and decompressor applied toa stream of vector values and assuming a correlation between absolute values of scalars of thesame dimension within the vectors, further improvement of the predictor performance can be attainedby means of SIMD-class parallelism which, in turn, is beneficial for overall performance ofcompression and decompression, provided that the underlying hardware is capable of addressingrandom-access memory based on offsets in a vector register, such as by employment of theVGATHER class instructions of Intel processors. In order to reduce the bottlenecks associatedwith input-output, an implementation of the algorithm is employed by the authors as part of acomputing system used for parallel simulation of wave fields which is distributed via a network.The experiments described in the paper demonstrate significant performance increase of the proposedcoder compared to well-known universal compressors, RAR, ZIP and 7Z, while the achievedcompression factors remain comparable.