Lossless Research Articles

Lossless compression of wireless capsule image using the Tetrolet transform coupled with modified capsule SPIHT

ABSTRACT The diagnostic technique known as wireless capsule endoscopy (WCE) is helpful in determining gastrointestinal issues. The capsule’s passive motion produces redundant data, and sending large amounts of data uncompressed costs more money and energy. Due to the intrinsic properties of the capsule, it is particularly desirable to have a low-complexity, low-power design that can compress the lossless compression module while still complying with capsule endoscopy regulations. This work’s primary goal is to propose a lossless compression with minimal complexity and power technique for WCE image compression. WCE images are first transformed to YUV colour space in order to remove unnecessary information from chroma components. The Tetrolet transform is then used to break down the YUV image. A Tetrolet transform is a shape made up of four tetrominoes, or squares of equal size. In order to reduce latency and raise threshold, the decomposed picture is then compressed using a Modified Capsule SPIHT (MCSPIHT) encoder. The resulting coefficients are encoded using the low complexity MCSPIHT encoder. Our proposed method results in an average compression ratio of 90.447.

Signalling and social learning in swarms of robots.

This paper investigates the role of communication in improving coordination within robot swarms, focusing on a paradigm where learning and execution occur simultaneously in a decentralized manner. We highlight the role communication can play in addressing the credit assignment problem (individual contribution to the overall performance), and how it can be influenced by it. We propose a taxonomy of existing and future works on communication, focusing on information selection and physical abstraction as principal axes for classification: from low-level lossless compression with raw signal extraction and processing to high-level lossy compression with structured communication models. The paper reviews current research from evolutionary robotics, multi-agent (deep) reinforcement learning, language models and biophysics models to outline the challenges and opportunities of communication in a collective of robots that continuously learn from one another through local message exchanges, illustrating a form of social learning.This article is part of the theme issue 'The road forward with swarm systems'.

A lossless reference-free sequence compression algorithm leveraging grammatical, statistical, and substitution rules.

Deoxyribonucleic acid (DNA) or ribonucleic acid (RNA) sequence compressors for novel species frequently face challenges when processing wide-scale raw, FASTA, or multi-FASTA structured data. For years, molecular sequence databases have favored the widely used general-purpose Gzip and Zstd compressors. The absence of sequence-specific characteristics in these encoders results in subpar performance, and their use depends on time-consuming parameter adjustments. To address these limitations, in this article, we propose a reference-free, lossless sequence compressor called GraSS (Grammatical, Statistical, and Substitution Rule-Based). GraSS compresses sequences more effectively by taking advantage of certain characteristics seen in DNA and RNA sequences. It supports various formats, including raw, FASTA, and multi-FASTA, commonly found in GenBank DNA and RNA files. We evaluate GraSS's performance using ten benchmark DNA sequences with reduced number of repeats, two highly repetitive RNA sequences, and fifteen raw DNA sequences. Test results indicate that the weighted average compression ratios (WACR) for DNA and RNA sequences are 4.5 and 19.6, respectively. Additionally, the entire DNA sequence corpus has a total compression time (TCT) of 246.8 seconds (s). These results demonstrate that the proposed compression method performs better than several advanced algorithms specifically designed to handle various levels of sequence redundancy. The decompression times, memory usage, and CPU usage are also very competitive. Contact: anirban@klyuniv.ac.in.

Lossless and reference-free compression of FASTQ/A files using GeneSqueeze

As sequencing becomes more accessible, there is an acute need for novel compression methods to efficiently store sequencing files. Omics analytics can leverage sequencing technologies to enhance biomedical research and individualize patient care, but sequencing files demand immense storage capabilities, particularly when sequencing is utilized for longitudinal studies. Addressing the storage challenges posed by these technologies is crucial for omics analytics to achieve their full potential. We present a novel lossless, reference-free compression algorithm, GeneSqueeze, that leverages the patterns inherent in the underlying components of FASTQ files to solve this need. GeneSqueeze’s benefits include an auto-tuning compression protocol based on each file’s distribution, lossless preservation of IUPAC nucleotides and read identifiers, and unrestricted FASTQ/A file attributes (i.e., read length, number of reads, or read identifier format). We compared GeneSqueeze to the general-purpose compressor, gzip, and to a domain-specific compressor, SPRING, to assess performance. Due to GeneSqueeze’s current Python implementation, GeneSqueeze underperformed as compared to gzip and SPRING in the time domain. GeneSqueeze and gzip achieved 100% lossless compression across all elements of the FASTQ files (i.e. the read identifier, sequence, quality score and ‘ + ’ lines). GeneSqueeze and gzip compressed all files losslessly, while both SPRING’s traditional and lossless modes exhibited data loss of non-ACGTN IUPAC nucleotides and of metadata following the ‘ + ’ on the separator line. GeneSqueeze showed up to three times higher compression ratios as compared to gzip, regardless of read length, number of reads, or file size, and had comparable compression ratios to SPRING across a variety of factors. Overall, GeneSqueeze represents a competitive and specialized compression method for FASTQ/A files containing nucleotide sequences. As such, GeneSqueeze has the potential to significantly reduce the storage and transmission costs associated with large omics datasets without sacrificing data integrity.

Upgrade of the CMS Electromagnetic Calorimeter for High-Luminosity LHC

The CMS Electromagnetic Calorimeter (ECAL) is a homogeneous PbWO4-based detector. The upcoming High-Luminosity upgrade of the CERN LHC will provide detectors with unprecedented instantaneous luminosity, entailing an increase of the average number of proton-proton collisions per bunch crossing up to a value of 200. To cope with such busy environment and with the trigger latency and rate, the ECAL readout and trigger electronics system will be replaced. The front-end electronics will deploy custom ASICs optimised for precise timing, improved signal discrimination, lossless data compression, and fast transmission. Off-detector FPGA processor boards will form trigger primitives and perform basic signal reconstruction. The upgraded system has been validated over several test beam campaigns. In terms of physics performance, it will match the outstanding energy resolution of the current ECAL detector, while significantly enhancing the time resolution of electrons and photons above 50 GeV.

LogGzip: Towards log Parsing with lossless compression

LogGzip: Towards log Parsing with lossless compression

Improved intensity rounding and division near lossless image compression algorithm using delta encoding

This paper presents RIFD-DLT, an advanced near-lossless image compression algorithm that combines Delta Encoding with the original rounding the intensity followed by division (RIFD) method. The RIFD method first minimizes the image intensities, which makes the next compression stages more efficient. Subsequently, Delta Encoding subtracts neighboring rows in each of the image's three-color matrices, using the proximity of pixel values in adjacent rows to further reduce the image intensity. Extensive investigations show that RIFD-DLT outperforms the state-of-the-art algorithms and benchmarks with respect to compression ratio and processing time. More specifically, RIFD-DLT compresses data from 11520 KBs to 2131 KBs with an 81.93% reduction and a 43.7% improvement over original RIFD-Huffman when compressing Kodak Image set. When comparing the RIFD-DLT with LICA algorithm, the total file size is reduced by 71.2%, representing a 10.73% improvement for RIFD-DLT. Also, RIFD-DLT shows notable speed gains over RIFD-Huffman, requiring only 34.62 seconds to compress and decompress all images from three datasets (Waterloo, Kodak and EPFL), as compared to 50.99 seconds for RIFD-Huffman. As for the image quality, the proposed algorithm achieved an average PSNR values of 58.51 dB, 51.3 dB, and 52.22 dB for the EPFL, Kodak, and Waterloo image sets, respectively, demonstrate the excellent image quality that persists after decompression, with a minimal distortion that is imperceptible to the human visual system and identically to the RIFD-Huffman PSNR. These findings show that, while preserving excellent image quality, RIFD-DLT provides an incredibly efficient and fast method of image compression.

Learning Lossless Compression for High Bit-Depth Volumetric Medical Image

Learning Lossless Compression for High Bit-Depth Volumetric Medical Image

Convolutional auto-encoder and discrete wavelet transform for lossy region-based medical image compression

Medical imaging is a vital and ever-evolving discipline that plays a critical role in detecting, diagnosing, and planning surgeries for diseases. In medical imaging, image compression systems are employed to reduce storage and bandwidth requirements. This paper presents a region of interest (ROI) based image compression method for brain MRI images by employing a convolutional auto-encoder (CAE) and discrete wavelet transform (DWT) to achieve both efficient compression and high-quality reconstruction in terms of lossy compression. Our compression approach involves separating image regions, employing a lossy compression technique based on CAE and DWT, with a lossless compression technique based arithmetic encoding. To achieve optimal compression of non-ROI regions, the encoding network of the CAE generates a compacted feature map that preserves structural information, which aids the DWT based codec and the decoding network of the CAE in reconstructing the output. To maintain diagnostically significant information, the ROI portion of the image is losslessly compressed using arithmetic encoding technique. To assess the effectiveness of our proposed method, we compared our results to standard lossy compression algorithms and recent approaches. Our method achieved a peak signal-to-noise ratio (PSNR) of 37.91 dB and a mean structural similarity index measure (MS-SSIM) of 98.62% at a high compression ratio of 30.

A DNA Data Storage Method Using Spatial Encoding Based Lossless Compression.

With the rapid increase in global data and rapid development of information technology, DNA sequences have been collected and manipulated on computers. This has yielded a new and attractive field of bioinformatics, DNA storage, where DNA has been considered as a great potential storage medium. It is known that one gram of DNA can store 215 GB of data, and the data stored in the DNA can be preserved for tens of thousands of years. In this study, a lossless and reversible DNA data storage method was proposed. The proposed approach employs a vector representation of each DNA base in a two-dimensional (2D) spatial domain for both encoding and decoding. The structure of the proposed method is reversible, rendering the decompression procedure possible. Experiments were performed to investigate the capacity, compression ratio, stability, and reliability. The obtained results show that the proposed method is much more efficient in terms of capacity than other known algorithms in the literature.

Lossless Image Compression Using Context-Dependent Linear Prediction Based on Mean Absolute Error Minimization.

This paper presents a method for lossless compression of images with fast decoding time and the option to select encoder parameters for individual image characteristics to increase compression efficiency. The data modeling stage was based on linear and nonlinear prediction, which was complemented by a simple block for removing the context-dependent constant component. The prediction was based on the Iterative Reweighted Least Squares (IRLS) method which allowed the minimization of mean absolute error. Two-stage compression was used to encode prediction errors: an adaptive Golomb and a binary arithmetic coding. High compression efficiency was achieved by using an author's context-switching algorithm, which allows several prediction models tailored to the individual characteristics of each image area. In addition, an analysis of the impact of individual encoder parameters on efficiency and encoding time was conducted, and the efficiency of the proposed solution was shown against competing solutions, showing a 9.1% improvement in the bit average of files for the entire test base compared to JPEG-LS.

Design and Implementation of a High-Performance VLSI Architecture for Canonical Huffman Encoding

A key component of contemporary computer systems, data compression makes it possible for information to be stored and transmitted efficiently. A popular technique for lossless data compression, Huffman coding can drastically cut down on data size without sacrificing intensity. But conventional Huffman encoding techniques can have scalability and speed issues, especially when used in hardware. A high-throughput Very Large-Scale Integration (VLSI) design for a Canonical Huffman Encoder is shown in this study. To accomplish quick and effective encoding, the suggested approach makes use of parallel processing and improved algorithms. The system's use of the canonical Huffman algorithm lowers computational complexity and streamlines hardware implementation without sacrificing compression performance. The architecture incorporates real-time operation-optimized modules for frequency analysis, code generation, and symbol encoding. Results from simulations show that the suggested design achieves significantly higher throughput compared to conventional approaches, making it suitable for applications in data storage, multimedia, and communication systems. This study contributes to advancing VLSI designs for high-performance hardware Key Words: Data Compression, Huffman Coding, Lossless Compression, Very Large-Scale Integration (VLSI), Canonical Huffman Encoder, High-Throughput Encoding, Parallel Processing, Efficient Encoding, Hardware Implementation, Computational Complexity, Code Generation, Symbol Encoding, Frequency Analysis, Real-Time Operation, Throughput Optimization, Multimedia Systems, Communication Systems, Data Storage, Hardware Architecture, VLSI Design, Performance Optimization, Simulations, Compression Performance, Hardware Design, Throughput Enhancement, System-Level Optimization, Power Efficiency, Resource Management, Data Transfer Efficiency, Low-Cost Implementation.

Improved fractal coding and hyperchaotic system for lossless image compression and encryption

Improved fractal coding and hyperchaotic system for lossless image compression and encryption

A case study on entropy-aware block-based linear transforms for lossless image compression

Data compression algorithms tend to reduce information entropy, which is crucial, especially in the case of images, as they are data intensive. In this regard, lossless image data compression is especially challenging. Many popular lossless compression methods incorporate predictions and various types of pixel transformations, in order to reduce the information entropy of an image. In this paper, a block optimisation programming framework Φ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Phi$$\\end{document} is introduced to support various experiments on raster images, divided into blocks of pixels. Eleven methods were implemented within Φ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Phi$$\\end{document}, including prediction methods, string transformation methods, and inverse distance weighting, as a representative of interpolation methods. Thirty-two different greyscale raster images with varying resolutions and contents were used in the experiments. It was shown that Φ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Phi$$\\end{document} reduces information entropy better than the popular JPEG LS and CALIC predictors. The additional information associated with each block in Φ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Phi$$\\end{document} is then evaluated. It was confirmed that, despite this additional cost, the estimated size in bytes is smaller in comparison to the sizes achieved by the JPEG LS and CALIC predictors.

Enhancing Data Security through Digital Image Steganography: An Implementation of the Two Least Significant Bits (2LSB) Method

The rapid advancement of digital communication technologies has heightened the demand for robust data security methods, particularly for safeguarding sensitive information against cyber threats. Steganography, the art of concealing data within digital media, offers an effective means of securing communication by masking the very existence of hidden messages. This study evaluates the Two Least Significant Bits (2LSB) method as an enhancement to the conventional Least Significant Bit (LSB) technique, focusing on improving storage capacity and resistance to detection. The research employs a quantitative methodology, conducting experiments to analyze the impact of data embedding on image quality using metrics such as Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR). Data embedding was tested on images of varying sizes and resolutions, with results compared before and after embedding. The study also evaluates the system’s tolerance to compression and robustness against steganalysis attacks, such as histogram analysis and noise detection. The results demonstrate that the 2LSB method significantly enhances storage capacity while maintaining high image quality. For instance, PSNR values remained above 40 dB across all tested images, indicating minimal visual distortion. The method also showed resilience to lossless compression formats such as PNG, with minimal impact on data integrity. However, lossy compression formats like JPEG posed challenges in maintaining the embedded data. The MSE values before and after embedding were minor, ensuring that the hidden data remained undetectable under standard analysis. This research contributes to developing more efficient and secure steganography techniques, providing practical applications in protecting sensitive data during transmission. The findings underline the 2LSB method's potential as a viable solution for digital security, balancing enhanced storage capacity with robust resistance to detection. Future work should further explore integrating advanced encryption with 2LSB to strengthen data protection in diverse digital environments

Application of difference color models to RGB image fragments before progressive hierarchical lossless compression

The method and corresponding algorithm for hierarchical bypass of minimum code units (MCU - blocks of 8 x 8 pixels) in the process of preprocessing of lossless image compression is proposed. The algorithm for selecting for each MCU an effective difference color model with integer coefficients from the list of basic alternative models, which predictable minimizes the image compression ratio based on entropy analysis is given. An algorithm for compact hierarchical storage of numbers of selected difference color models for each MCU, which takes into account the high probability of repetition of these numbers for adjacent MCUs is described and implemented. It is shown that both color models for individual MCUs and a single color model for the entire image can be effective for difference images. To evaluate the effectiveness of preprocessing algorithms during hierarchical compression, the feasibility of using the weighted average entropy by passes instead of the total entropy, which is calculated only by pixels processed by a context-independent algorithm is proposed. It is noted that the entropy by 4 passes is always less than the total entropy, because it takes into account the different unevenness of the probability distribution of elements on different passes, that is why it is advisable to store the data of different passes in different compressed blocks. On the well-known Archive Comparison Test suite, it has been demonstrated that applying integer difference color models to MCU image fragments with the option of switching to a single difference color model for the entire image can reduce compression ratios by an average of 0.02 bpb but at the same time slows down the encoding by a factor of 2.88 and decoding by 3.39%, so such preprocessing is not recommended to be used in graphic formats. It is emphasized that difference color models should be applied to individual MCUs in archivers in order to ensure maximum lossless image compression.

A Coverless Image Steganography Based on Huffman Encoding and Robust Image Wavelet Hashing

Data hiding approaches are crucial in insecure communication to protect sensi- tive information from illegal access. The robustness of existing coverless image steganogra- phy methods that utilize mapping rules is derived from their ability to extract feature patterns within the spatial domain. In order to protect sensitive data more securely and to increase its durability against attacks, a new coverless steganography is introduced in this study. The me- thod used is based on the frequency domain. A coverless image steganography based on ro- bust image wavelet hashing and Huffman encoding. Initially, the confidential data is encoded through lossless compression using Huffman Encoding, which reduces the payload and en- hances the capacity for embedding. Second, segments of an 8-bit size are created from the compressed secret data. The efficient DWT hashing technique has been employed to generate a hash sequence for an image. An inverted index structure is established, allowing for the se- lection of the image whose hash corresponds to the secret data segment. Subsequently, all chosen images along with supplementary information are transmitted to the recipient. Testing has shown that the proposed method is robust against various image processing attacks, in- cluding scaling, low pass filtering, JPEG compression, rotation, noise introduction, and me- dian/mean filtering. Research findings and analysis indicate that this method enhances the latest coverless steganography algorithms in terms of capacity, resilience, and security.

ACTF: An efficient lossless compression algorithm for time series floating point data

The volume of time series data across various fields is steadily increasing. However, this unprocessed massive data challenges transmission efficiency, computational arithmetic, and storage capacity. Therefore, the compression of time series data is essential for improving transmission, computation, and storage. Currently, improving time series floating-point coding rules is the primary method for enhancing compression algorithms efficiency and ratio. This paper presents an efficient lossless compression algorithm for time series floating point data, designed based on existing compression algorithms. We employ three optimization strategies data preprocessing, coding category expansion, and feature refinement representation to enhance the compression ratio and efficiency of compressing time-series floating-point numbers. Through experimental comparisons and validations, we demonstrate that our algorithm outperforms Chimp, Chimp128, Gorilla, and other compression algorithms across multiple datasets. The experimental results on 30 datasets show that our algorithm improves the compression ratio of time series algorithms by an average of 12.25% and compression and decompression efficiencies by an average of 27.21%. Notably, it achieves a 24.06% compression ratio improvement on the IOT1 dataset and a 42.96% compression and decompression efficiency improvement on the IOT4 dataset.

A novel lossless image compression algorithm using multiset permutation encoding

A novel lossless image compression algorithm using multiset permutation encoding

File Compression System Using Huffman Coding

Abstract—File compression, which minimizes file sizes without significantly compromising the quality of information contained in the file, is among the most critical aspects of data management since it facilitates fast flow and storage of data. Storage manage- ment has become more challenging due to the exponential growth of digital data; hence, efficient file compression has been an issue of great importance to help in decreasing storage expenses and increasing data transfer rates. This paper presents a web interface for the file compression system that is based on lossless data compression known as Huffman coding to reduce the size of the files without changing their content. A standard internet portal allows the users to upload their files to the system and get back resized files within the shortest time possible. The technique employed in the compression is astute in that it uses Huffman codes which give characters codes depending with their frequencies in the file. It then follows that since most of the file will have short codes; the file will take up less space. To safeguard the compressed information, the system generates a data structure called a Huffman tree in which no code is a sub- string of any other. This application does seize to be usable from a few selected devices such as desktops and laptop computers but can also be accessed from projectors and cell phones because of the effective design. The file compression system is also versatile and accommodates various file categories such text, image and binary among others providing users with convenience in a number of sectors. Index Terms—Data Compression, Hoffman Coding, HTML( Hypertext Markup Language), CSS ( Cascading Styling Sheet ), JavaScript.