Lossless Data Compression Algorithm Research Articles

Aerosense: A Self-Sustainable and Long-Range Bluetooth Wireless Sensor Node for Aerodynamic and Aeroacoustic Monitoring on Wind Turbines

This paper presents a low-power, self-sustainable, and modular wireless sensor node for aerodynamic and acoustic measurements on wind turbines and other industrial structures. It includes 40 high-accuracy barometers, 10 microphones, 5 differential pressure sensors, and implements a lossy and a lossless on-board data compression algorithm to decrease the transmission energy cost. The wireless transmitter is based on Bluetooth Low Energy 5.1 tuned for long-range and high throughput while maintaining adequate per bit energy efficiency (80 nJ). Moreover, we fieldassessed the node capability to collect precise and accurate aerodynamic data. Outdoor experimental tests revealed that the system can acquire and sustain a data rate of 850 kbps over 438 m. The power consumption while collecting and streaming all measured data is 120 mW, enabling self-sustainability and long-term <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-situ</i> monitoring with a 111 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> photovoltaic panel.

Low-Power Multi-Lead Wearable ECG System With Sensor Data Compression

This study proposes a low-power wearable electrocardiogram (ECG) acquisition system for monitoring human health in daily life. A flexible sensing electrode was designed to replace traditional silver–silver chloride (Ag/AgCl) electrodes for measuring ECG signals from leads I, II, and V1. The traditional multi-lead-wired ECG detector limits the movements and activities of the patient and requires medical staff to assist in collecting ECG signals. The portable ECG acquisition device proposed in this article is not only comfortable to wear but also convenient for patients to use anytime, anywhere. To increase the sampling rate of the device to obtain more ECG signal information, this study proposes an innovative Huffman lossless ECG data compression algorithm based on variable-word length coding (VLC). This algorithm can efficiently realize data compression and reduce power consumption. It makes the whole system operate continuously for a long period of time and acquire abundant ECG information, which is helpful for clinical diagnosis. Experimental results show that when the sampling rate is 400 Hz, the working current is only 3.64 mA, and the compression ratio (CR) of the device can reach 2.38.

A new real-time lossless data compression algorithm for ECG and PPG signals

ObjectiveData compression is a useful process in tele-monitoring applications, in which lesser number of bits are needed to represent the same data. In this work, a run-time lossless compression of single-channel Electrocardiogram (ECG) and Photoplethysmogram (PPG) signals is proposed, maintaining all dominant features. MethodsThe single-channel data are first quantized using optimal quantization level, so that fewer number of bits are needed to represent it, maintaining low quantization error. Then, second order delta encoding and run-length encoding (RLE) based data compression are proposed in this work. A new approach of using ‘buffer array’ along with RLE is also introduced, so that minimum bits are needed to store. ResultsThis algorithm was tested on various single-lead ECG and PPG signals available in Physionet. An average compression ratio (CR) was achieved of 6.52, 3.82, and 2.49 for 547 PTBDB ECG records, 48 MITDB ECG records, and 53 MIMIC-II PPG records, respectively. This algorithm was also performed on single-channel ECG, collected from 10 healthy volunteers using AD8232 ECG module, with 125 Hz sampling frequency and 10-bit data resolution, which resulted in average CR of 2.34. ConclusionThis algorithm was also performed on a smartphone device that provided user-friendly operation. The low computational complications and standalone operation of data collection, compression, and transmission encouraged its implementation for run-time operation. SignificanceA comparative study of the proposed work with previously published works proved this fact that this algorithm provided better performance in the area of run-time patient health monitoring applications.

Design of a Compact Wireless EEG Recorder With Extra-High Sampling Rate and Precise Time Synchronization for Auditory Brainstem Response

This paper proposed a compact wearable Electroencephalography (EEG) acquisition and analysis system targeting for event-related, wide-bandwidth biopotentials including Auditory Brainstem Response (ABR). The proposed system enables a 2-channel high EEG sampling mode with a sampling rate up to 16kHz, and an 8-channel low EEG sampling mode with a sampling rate of 4kHz. An audio codec is integrated generating synchronized audio stimulation and potential audio recording. A Bluetooth 5 based in-the-air protocol is designed featuring lossless neural data compression algorithm and lost-packet-retransmission strategy. Hydrogels are used as the tissue interface on the electrodes to improve the conductivity and stability. A precise time synchronization method with a standard deviation of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.07\mu \text{s}$ </tex-math></inline-formula> is implemented for timestamping when external stimulus is requested. A Graphic User Interface (GUI) is developed for remote control of the proposed device as well as for real-time display of the acquired neural signal and analysis results. Both bench test and in-vivo click-ABRs test on human subject were performed on the proposed device. The experimental results proved the ability of the proposed EEG recorder to acquire high-time-resolution ABRs while giving periodic auditory stimulation.

Telemetry Data Compression Algorithm Using Balanced Recurrent Neural Network and Deep Learning.

Telemetric information is great in size, requiring extra room and transmission time. There is a significant obstruction of storing or sending telemetric information. Lossless data compression (LDC) algorithms have evolved to process telemetric data effectively and efficiently with a high compression ratio and a short processing time. Telemetric information can be packed to control the extra room and association data transmission. In spite of the fact that different examinations on the pressure of telemetric information have been conducted, the idea of telemetric information makes pressure incredibly troublesome. The purpose of this study is to offer a subsampled and balanced recurrent neural lossless data compression (SB-RNLDC) approach for increasing the compression rate while decreasing the compression time. This is accomplished through the development of two models: one for subsampled averaged telemetry data preprocessing and another for BRN-LDC. Subsampling and averaging are conducted at the preprocessing stage using an adjustable sampling factor. A balanced compression interval (BCI) is used to encode the data depending on the probability measurement during the LDC stage. The aim of this research work is to compare differential compression techniques directly. The final output demonstrates that the balancing-based LDC can reduce compression time and finally improve dependability. The final experimental results show that the model proposed can enhance the computing capabilities in data compression compared to the existing methodologies.

Analysis for Lossless Data Compression Algorithms for Low Bandwidth Networks

Analysis for loss less data compression delivers the relevant data about variations of them as well as to describe the possible causes for each algorithm and best performing data types. It describes the basic lossless techniques of data compression Huffman encodes, Arithmetic Encoding, and Lempel Ziv Encodings then briefly with their effectiveness under varying data types of Latin text, audio and video. These properties give the solution of which lossless compression algorithm more suitable compared to other from the Saving Percentage, compression ratio, time of compression and time of decompression with Low Bandwidth Network. Moreover here Lossless Data Compression Algorithms (LDCA) being implemented and tested Huffman compression, Arithmetic compression, and Lempel Ziv algorithms, the implemented result shows that LZW algorithm saves more size than that of the others two with text file, Huffman compression algorithm saves more file sizes and the time takes to compressed decompress is higher than that of other two for audio file type and finally Huffman performs greater on very huge data compressions that is due to much compressing capability.

FPGA implementation of Lempel-Ziv data compression

When transmitting the data in digital communication, it is well desired that the transmitting data bits should be as minimal as possible, so many techniques are used to compress the data. In this paper, a Lempel-Ziv algorithm for data compression was implemented through VHDL coding. One of the most lossless data compression algorithms commonly used is Lempel-Ziv. The work in this paper is devoted to improve the compression rate, space-saving, and utilization of the Lempel-Ziv algorithm using a systolic array approach. The developed design is validated with VHDL simulations using Xilinx ISE 14.5 and synthesized on Virtex-6 FPGA chip. The results show that our design is efficient in providing high compression rates and space-saving percentage as well as improved utilization. The Throughput is increased by 50% and the design area is decreased by more than 23% with a high compression ratio compared to comparable previous designs.

Design and Implementation of Low Energy Wireless Network Nodes Based on Hardware Compression Acceleration

Background: With the fast development of information, the information data is increasing in geometric multiples, and the speed of information transmission and storage space are required to be higher. Objective: In order to reduce the use of storage space and further improve the transmission efficiency of data, data need to be compressed. In the process of data compression, it is very important to ensure the lossless nature of data, and lossless data compression algorithms appear. The gradual optimization design of the algorithm can often achieve the energy-saving optimization of data compression. Similarly, the effect of energy saving can also be obtained by improving the hardware structure of node. Methods: In this paper, a new structure is designed for sensor node, which adopts hardware acceleration, and the data compression module is separated from the node microprocessor. Result: On the basis of the ASIC design of the algorithm, by introducing hardware acceleration, the energy consumption of the compressed data was successfully reduced, and the proportion of energy consumption and compression time saved by the general-purpose processor was as high as 98.4 % and 95.8 %, respectively. It greatly reduces the compression time and energy consumption.

ΔRLE

Lossless data compression algorithms can use statistical redundancy to represent data using a fewer number of bits in comparison to the original uncompressed data. Run-Length Encoding (RLE) is one of the simplest lossless compression algorithms in terms of understanding its principles and software implementation, as well as in terms of temporal and spatial complexity. If this principle is applied to individual bits of original uncompressed data without respecting the byte boundaries, this approach is referred to as bit-level Run-Length Encoding. Lightweight algorithm for lossless data compression proposed in this paper optimizes bit-level RLE data compression, uses special encoding of repeating data blocks, and, if necessary, combines it with delta data transformation or representation of data in its original form intending to increase compression efficiency compared to a conventional bit-level RLE approach. The advantage of the algorithm proposed in this paper is in its low time and memory consumption which are basic features of RLE, along with the simultaneous increase of compression ratio, compared to the classical bit-level RLE approach.

Data Compression Algorithms for Wireless Sensor Networks: A Review and Comparison

Energy consumption has risen to be a bottleneck in wireless sensor networks. This is caused by the challenges faced by these networks due to their tiny sensor nodes that have limited memory storage, small battery capacity, limited processing capability, and bandwidth. Data compression has been used to reduce energy consumption and improve network lifetime, as it reduces data size before it can be forwarded from the sensing node to the sink node in the network. In this paper, a survey and comparison of currently available data compression techniques in wireless sensor networks are conducted. Suitable sets of criteria are defined to classify existing data compression algorithms. An adaptive lossless data compression algorithm (ALDC) is analyzed through MATLAB coding and simulation from the reviewed data compression techniques. The analysis aims to discover strategies that can be used to reduce the amount of data further before it is transmitted. From this analysis, it was discovered that encoding residue samples, rather than raw data samples, reduced the bitstream from 112 bits to a range of 30 to 36 bits depending on the sample block sizes. The average length of data samples to be passed to the encoder was minimized from the original 14 bits per symbol to 1.125 bits per symbol. This demonstrated a 0.875 code efficiency or redundancy. It resulted in an energy saving of 67.8% to 73.2%. This work further proposes a data compression algorithm that encodes the residue samples with fewer bits than the ALDC algorithm. The algorithm reduced the bitstream to 26 bits. The average length of the code is equal to the entropy of the data samples, demonstrating zero redundancy and an improved energy saving of 76.8% compared to ALDC . The proposed algorithm, therefore, shows improved energy efficiency through data compression.

Secure and Efficient Data Collection and Storage of IoT in Smart Ocean

Due to the abundant marine resources, smart ocean has attracted much attention of the government, industry, and academy. The Internet-of-Things (IoT) architectures for smart ocean have been proposed to collect various of data from the ocean, thereby assisting environmental protection, military reconnaissance, and so on. However, few researchers have paid attention to the security and privacy issues of data collection and transmission. In this article, for the unreliable underwater environment, we present a secure, efficient, and complete data collection, and transmission and storage scheme for IoT in smart ocean. Especially, to prolong the lifetime of the underwater node, two novel data compression algorithms [lossy data compression algorithm (LCA) and lossless data compression algorithm (NLCA)] are also proposed. Moreover, due to the vulnerability of underwater nodes, we also propose a corresponding IoT framework and data collection pattern to resist the single point failure attack. Besides, to guarantee the confidentiality, reliability, and integrity of transmitting data, Elliptic Curve-ElGamal (EC-ElGamal) and elliptic curve digital signature algorithm (ECDSA) are employed. The consensus algorithm and blacklisting mechanism are also employed to detect and address failure or malicious nodes. Finally, the security analysis demonstrates that our scheme is able to resist many typical attacks for underwater nodes, such as manipulation attacks, Distributed Denial-of-Service (DDoS) attacks, malicious node injection attacks, and so on. Additionally, relevant experimental results show that the scheme is feasibility and efficiency.

Increasing the input data length of RSA cryptosystem by applying a hybrid lossless data compression algorithm

RSA is a well-known and widely used cryptosystem. One major limitation of RSA is its maximum input length depends on the chosen key size and the selected padding. This paper proposed a hybrid data compression algorithm to increase the input data length to be encrypted by RSA. Firstly, a plain text is compressed by the LZW algorithm. The compressed data is padded. After that, the size of the compressed data is further reduced by using continued fraction. Double-compressed data is, then, used as an input of RSA. When using a combination of continued fraction and LZW algorithm, the results showed that, on average, the input data length is increased by 18.91 per cent. In most cases, the processing time is much better than not doing any compression. In the case of decryption, a cipher text is decrypted by RSA first. The padding is removed. Then, the Euclidean algorithm and LZW are used respectively to restore the original data. The hybrid data compression is not only relieved the limitation of RSA but also enhances the security level.

Data Compression mechanisms in an intelligent E-Health gateway for medical monitoring applications

Abstract In this article, we present data Compression mechanisms in an intelligent E-Health gateway for medical monitoring applications, to improve the collection and processing of data in the context of medical surveillance. These mechanisms, will not only optimize the data storage space required by the gateway, but also to accelerate the processing of the data flow for the fast and lossless transfer of data on the WSN (Wireless Sensor Network) and WLAN (Wireless Local Area Network) networks. In addition, all of the modern data compression mechanisms presented in this article are the following: Forecast compression, Sampling compression, Scheduling compression, distributed compression. We apply some formats of lossless data compression algorithms to these mechanisms (LZ4, LZO and Bzip2), to establish a comparative study between mechanisms using metric parameters. The implementation of these mechanisms in the smart gateway highlights an advanced Internet of Things data processing system for health and medical surveillance, based on a comprehensive and efficient study of advanced data compression techniques.

Experimental comparison of data comprassion algorithms

The amount of data that is stored and transferred grows regularly and rapidly. When it comes to transferring large data volumes, a data compression algorithm can be useful. A well-chosen data compression algorithm can reduce the size of the data to up to 60%. The problem of creating new and modifying or optimizing old algorithms is up to date. This article discloses some of the most widespread algorithms of data comparison; more exactly, four of them. Shannon–Fano coding, named after Claude Shannon and Robert Fano, is a technique for constructing a prefix code based on a set of symbols and their probabilities. It is suboptimal in the sense that it does not achieve the lowest possible expected code word length like Huffman coding. Huffman code is a particular type of the optimal prefix code that is commonly used for lossless data compression. The process of finding or using such a code proceeds by means of Huffman coding; an algorithm developed by David A. Huffman. Lempel–Ziv–Storer–Szymanski (LZSS) is a lossless data compression algorithm, a derivative of LZ77 that was created in 1982 by James Storer and Thomas Szymanski. The main difference between LZ77 and LZSS is that in LZ77 the dictionary reference could actually be longer than the string it was replacing. In LZSS, such references are omitted if the length is less than the “break even” point. Furthermore, LZSS uses one-bit flags to indicate whether the next chunk of data is a literal (byte) or a reference to an offset/length pair. Lempel–Ziv–Welch (LZW) is a lossless data compression algorithm created by Abraham Lempel, Jacob Ziv, and Terry Welch. It was published as an improved implementation of the LZ78 algorithm. As part of the work on the article, these algorithms were implemented and an experimental analysis of their quality and speed was carried out. Those experiments gave the conclusion that the best compression speed results were shown by LZSS and the best compression ratio was reached by LZW. The work can be useful for researchers in the field of data compression.

An Intelligent Multimedia Data Encryption and Compression and Secure Data Transmission of Public Cloud

Data compression is a method of reducing the size of the data file so that the file should take less disk space for storage. Compression of a file depends upon encoding of file. In lossless data compression algorithm there is no data loss while compressing a file, therefore confidential data can be reproduce if it is compressed using lossless data compression. Compression reduces the redundancy and if a compressed file is encrypted it is having a better security and faster transfer rate across the network than encrypting and transferring uncompressed file. Most of the computer applications related to health are not secure and these applications exchange lot of confidential health data having different file formats like HL7, DICOM images and other audio, image, textual and video data formats etc. These types of confidential data need to be transmitted securely and stored efficiently. Therefore this paper proposes a learning compression- encryption model for identifying the files that should be compressed before encrypting and the files that should be encrypted without compressing them.

Using cantor sets for error detection.

Error detection is a fundamental need in most computer networks and communication systems in order to combat the effect of noise. Error detection techniques have also been incorporated with lossless data compression algorithms for transmission across communication networks. In this paper, we propose to incorporate a novel error detection scheme into a Shannon optimal lossless data compression algorithm known as Generalized Luröth Series (GLS) coding. GLS-coding is a generalization of the popular Arithmetic Coding which is an integral part of the JPEG2000 standard for still image compression. GLS-coding encodes the input message as a symbolic sequence on an appropriate 1D chaotic map Generalized Luröth Series (GLS) and the compressed file is obtained as the initial value by iterating backwards on the map. However, in the presence of noise, even small errors in the compressed file leads to catastrophic decoding errors owing to sensitive dependence on initial values, the hallmark of deterministic chaos. In this paper, we first show that repetition codes, the oldest and the most basic error correction and detection codes in literature, actually lie on a Cantor set with a fractal dimension of n}{} frac{1}{n} , which is also the rate of the code. Inspired by this, we incorporate error detection capability to GLS-coding by ensuring that the compressed file (initial value on the chaotic map) lies on a Cantor set. Even a 1-bit error in the initial value will throw it outside the Cantor set, which can be detected while decoding. The rate of the code can be adjusted by the fractal dimension of the Cantor set, thereby controlling the error detection performance.

Repair and Restoration of Corrupted LZSS Files

Data compression and decompression have been widely used in modern communication and data transmission. But how to decompress the corrupted lossless compressed files remains a challenge. Aiming at the Lempel-Ziv-Storer-Szymanski (LZSS), a lossless data compression algorithm widely used in the field of general coding, this paper proposes an effective method to repair the errors and decompress and restore the corrupted LZSS files, and provides the theoretical basis for the method. By using the residual redundancy left by the LZSS encoder to carry the check information, the method can repair the errors in LZSS compressed data without any loss of compression performance. The proposed method neither requires additional bits nor changes coding rules or data formats. It is fully compatible with standard algorithms. That is, the data compressed by LZSS with error repair capability can still be decompressed by the standard LZSS decoder. The experimental results verify the validity and practicability of the proposed method.

High efficiency referential genome compression algorithm.

With the development and the gradually popularized application of next-generation sequencing technologies (NGS), genome sequencing has been becoming faster and cheaper, creating a massive amount of genome sequence data which still grows at an explosive rate. The time and cost of transmission, storage, processing and analysis of these genetic data have become bottlenecks that hinder the development of genetics and biomedicine. Although there are many common data compression algorithms, they are not effective for genome sequences due to their inability to consider and exploit the inherent characteristics of genome sequence data. Therefore, the development of a fast and efficient compression algorithm specific to genome data is an important and pressing issue. We have developed a referential lossless genome data compression algorithm with better performance than previous algorithms. According to a carefully designed matching strategy selection mechanism, the advantages of local matching and global matching are reasonably combined together to improve the description efficiency of the matched sub-strings. The effects of the length and the position of matched sub-strings to the compression efficiency are jointly taken into consideration. The proposed algorithm can compress the FASTA data of complete human genomes, each of which is about 3 GB, in about 18 min. The compressed file sizes are ranging from a few megabytes to about forty megabytes. The averaged compression ratio is higher than that of the state-of-the-art genome compression algorithms, the time complexity is at the same order of the best-known algorithms. https://github.com/jhchen5/SCCG. Supplementary data are available at Bioinformatics online.

Efficient point cloud lossless data compression method based on an embedded Gray code structured light pattern sequence.

Despite the benefits derived from the development of 3D techniques to improve the acceleration and accuracy of 3D scanning operations over the past two decades and its wide range applications in various industries (e.g., quality control and inspection, reverse engineering, robotics), there are restrictions on data transfer, data storage, and even the development of real-time scanning methods due to the enlarging data size (point cloud). According to the importance of maintaining all the output data of the scanner in instrumentation engineering, the need to apply minimum loss or lossless data compression algorithms is more than ever evident. In this regard, this paper presents a novel method in point cloud lossless data compression, using a Gray code structured light pattern sequence and image-based compression. The empirical evaluation and results of the proposed method demonstrate that this idea is reliable and practical to achieve a distinct compression ratio among other lossless point cloud compression methods.

Performance analysis of data compression algorithms for heterogeneous architecture through parallel approach

Today, there is a huge demand for data compression due to the need to reduce the transmission time and increase the capacity of data storage. Data compression is a technique which represents an information, images, video files in a compressed or in a compact format. There are various data compression techniques which keep information as accurately as possible with the fewest number of bits and send it through communication channel. Arithmetic algorithm, Lempel–Ziv 77 (LZ77) and run length encoding with a K-precision (K-RLE) algorithms are lossless data compression algorithms which have lower performance rate because of their processing complexity as well as execution time. This paper presents an efficient parallel approach to reduce execution time for compression algorithms. The proposed OpenMP is an efficient tool for programming within parallel shared-memory environments. Finally, it shows that performance parallel model experimented using Open Multi-Processing (OpenMP) Application Programming Interface through Intel Parallel studio on multicore architecture platform with spec of Core 2 duo—2.4 GHz, 1 Gb RAM machine of parallel approach for compression algorithms has been improved remarkably against sequential approach. The improvement in compression ratio through an efficient parallel approach leads to reduction on transmission cost, reduction in storage space and bandwidth without additional hardware infrastructure. An overall performance evaluation shows arithmetic data compression algorithm with 46% which is better than LZ77 of 44% as well as K-RLE of 37% data compression algorithms.