An effective compression algorithm for real-time transmission data using predictive coding with mixed models of LSTM and XGBoost

Abstract

In this paper, a compression and decompression algorithm is proposed for real-time transmission data by combining discrete wavelet decomposition and reconstruction, hybrid model prediction of long short-term memory (LSTM) and extreme gradient boosting (XGBoost), and the development of quantization sequences. The forward compression and reverse decompression are achieved by continuously acquiring the prediction error and data series with quantization error through the iterative shifting approach. To accurately forecast data series with quantization errors, three classes of data with specific distribution characteristics collected from the measurement-while-drilling (MWD) operation field are decomposed into approximate data and detailed data, which are then predicted by autoregressive integrated moving average (ARIMA), support vector regression (SVR), XGBoost, backpropagation neural network (BPNN), radial basis function neural network (RBFNN), and LSTM models. Particularly, the LSTM model and the XGBoost model are proven to be suitable strategies for working with approximate data and detailed data separately by comparing with conventional approaches in terms of the mean absolute error (MAE), root mean square error (RMSE), and coefficient of determination (R2) predictive indexes. Besides, experiment studies show that almost 50% compression performance can be reached under one in ten thousand distortions. It is verified that the proposed algorithm is appropriate and efficient for real-time data transmission applications with high performance.