Compression Algorithm Research Articles

Hybrid image compression algorithm based on wavelet transform and number theoretic transformation

Hybrid image compression algorithm based on wavelet transform and number theoretic transformation

Adaptive region algorithm based on a ternary optical computer

The adaptive region algorithm is an improved compression algorithm based on Huffman coding. Because of the large number of rules for dividing regions in the algorithm, there are problems such as high computing costs, slow speed, and low compression efficiency. To address these problems, this paper investigates the adaptive region algorithm based on a ternary optical computer (TOC) combined with the characteristics of a ternary optical computer, such as many data bits, high parallelism, and three-valued coding. According to the characteristics of TOC three-valued coding, this paper designs a three-valued character coding scheme that can effectively shorten the coding length of characters by changing the original coding rules and further improve the compression efficiency of the adaptive region algorithm. Furthermore, in conjunction with the advantages of TOC enabling parallel computation, this paper presents an efficient computational scheme capable of effectively improving computational efficiency during the process of region partitioning. Through case studies, the compression efficiency and computational efficiency of the adaptive region algorithm implemented on TOC and an electronic computer were analyzed, respectively. It was found that the compression efficiency of the TOC-based algorithm is 50.4%, while that of the electronic-computer-based algorithm is only 36%. In the comparison of computational efficiency, the computational time complexity of TOC is O(n), whereas that of the electronic computer (EC) is O(n2). Finally, it is concluded through experimental validation that the TOC-based adaptive region compression algorithm performs well in terms of computational performance and compression efficiency, giving full play to the three-valued coding characteristics of TOC-based as well as the advantages of being able to realize parallel computation.

A deep learning-based compression and classification technique for whole slide histopathology images

This paper presents an autoencoder-based neural network architecture to compress histopathological images while retaining the denser and more meaningful representation of the original images. Current research into improving compression algorithms is focused on methods allowing lower compression rates for Regions of Interest (ROI-based approaches). Neural networks are great at extracting meaningful semantic representations from images and, therefore can select the regions to be considered of interest for the compression process. In this work, we focus on the compression of whole slide histopathology images. The objective is to build an ensemble of neural networks that enables a compressive autoencoder in a supervised fashion to retain a denser and more meaningful representation of the input histology images. Our proposed system is a simple and novel method to supervise compressive neural networks. We test the compressed images using transfer learning-based classifiers and show that they provide promising accuracy and classification performance.

SHM data compression and reconstruction based on IGWO-OMP algorithm

The long-term operating structural health monitoring (SHM) system generates massive monitoring data, whose transmission and storage are challenging tasks. To address this issue, this study proposed an improved grey wolf optimizer-orthogonal matching pursuit (IGWO-OMP) algorithm for SHM data compression and reconstruction. The effectiveness of the proposed algorithm was validated using four SHM datasets including ultrasonic guided wave (UGW), acceleration, and audio signals. Firstly, the optimization results of IGWO were compared with those of other classical algorithms to verify its superiority in reconstruction accuracy. Secondly, the reconstruction accuracy under various compression factors was investigated and the optimal compression factor was determined. Finally, the relationship between reconstruction accuracy and signal sparsity as well as the performance of IGWO-OMP in dealing with noise-containing signals were discussed. The results demonstrate the generalizability and excellent noise robustness of IGWO-OMP in SHM data compression and reconstruction. Compared with other optimization algorithms, IGWO exhibits the best global optimal solution searching ability and obtains the reconstructed signal closest to the original one. Comprehensively considering the compressed signal length, reconstruction accuracy and computational efficiency, the optimal compression factor is suggested to be 0.2. In IGWO-OMP, the signal reconstruction accuracy is related to its sparsity, the variation of the CCD index for the reconstructed signal with relative sparsity follows an exponential growth relationship.

Investigation of fast and efficient lossless compression algorithms for macromolecular crystallography experiments.

Structural biology experiments benefit significantly from state-of-the-art synchrotron data collection. One can acquire macromolecular crystallography (MX) diffraction data on large-area photon-counting pixel-array detectors at framing rates exceeding 1000 frames per second, using 200 Gbps network connectivity, or higher when available. In extreme cases this represents a raw data throughput of about 25 GB s-1, which is nearly impossible to deliver at reasonable cost without compression. Our field has used lossless compression for decades to make such data collection manageable. Many MX beamlines are now fitted with DECTRIS Eiger detectors, all of which are delivered with optimized compression algorithms by default, and they perform well with current framing rates and typical diffraction data. However, better lossless compression algorithms have been developed and are now available to the research community. Here one of the latest and most promising lossless compression algorithms is investigated on a variety of diffraction data like those routinely acquired at state-of-the-art MX beamlines.

Mutual information as a measure of mixing efficiency in viscous fluids

Because of the kinematic reversibility of the Stokes equation, fluid mixing at the microscale requires an interplay between advection and diffusion. Here, we introduce mutual information between particle positions before and after mixing as a measure of mixing efficiency. We demonstrate its application in a Couette flow in an annulus and show that nonuniform rotation sequences can lead to more efficient mixing. We also determine mutual information from Brownian dynamics simulations using data compression algorithms. Our results show that mutual information provides a universal and assumption-free measure of mixing efficiency in microscale flows. Published by the American Physical Society 2024

Compressing spectral kernels in Gaussian Process: Enhanced generalization and interpretability

The modeling capabilities of a Gaussian Process (GP), such as generalization, nonlinearity, and smoothness, are largely determined by the choice of its kernel. A popular family of kernels for GPs, the spectral mixture (SM) kernels, have the desirable property that with a large number of spectral components they can approximate any stationary kernel. However, using a large number of SM components increases the risk of overfitting and hinders interpretability. To overcome these challenges, we propose a compression algorithm incorporating component pruning and component merging for GPs. Here SM components with small signal variance are removed, and a moment-matching merge method is proposed to further reduce the number of SM components. The main novelty of the proposed method is a similarity measure between SM components based on their normalized cross-correlation, which is related to the Bhattacharyya coefficient. We derive a greedy GP compression algorithm and perform a comparative evaluation over various learning tasks in terms of forecasting performance and compression capability. Results substantiate the beneficial effect of the method, both in terms of generalization and interpretability.11Source code:https://github.com/ck2019ML/Gaussian-Process-Model-Compression.

P‐57: A Novel Compression Algorithm using Machine Learning for Mura Compensation of OLED Panel

In the modern display panel manufacturing industry, the Mura problem that occurs during initial shipment has a significant impact on improving image quality. The purpose of this paper is to effectively solve these Mura defects while reducing additional production costs. We propose an innovative method of storing and compressing the De‐Mura compensation data of the display panel in external memory. Two‐dimensional compensation data compression based on machine learning using surface fitting and vector quantization is presented as an effective response to the Mura problem. This approach, which achieves up to 8:1 compression ratio while minimizing image quality degradation through vector quantization, is expected to provide economic benefits to manufacturers. The paper also introduces a new surface prediction method that replaces the existing DC‐shift and uses residual vector quantization, which can reduce the implementation complexity of vector quantization. Through experimental results, this study demonstrates that the compensated data compression technique yields excellent results in post‐compression restoration performance.

Enhanced Sampling in Molecular Dynamics Simulations: How Many MD Snapshots can be Needed to Reproduce the Biological Behavior?

Since its early days in the 19th century, medicinal chemistry has concentrated its efforts on the treatment of diseases, using tools from areas such as chemistry, pharmacology, and molecular biology. The understanding of biological mechanisms and signaling pathways is crucial information for the development of potential agents for the treatment of diseases mainly because they are such complex processes. Given the limitations that the experimental approach presents, computational chemistry is a valuable alternative for the study of these systems and their behavior. Thus, classical molecular dynamics, based on Newton's laws, is considered a technique of great accuracy, when appropriated force fields are used, and provides satisfactory contributions to the scientific community. However, as many configurations are generated in a large MD simulation, methods such as Statistical Inefficiency and Optimal Wavelet Signal Compression Algorithm are great tools that can reduce the number of subsequent QM calculations. Accordingly, this review aims to briefly discuss the importance and relevance of medicinal chemistry allied to computational chemistry as well as to present a case study where, through a molecular dynamics simulation of AMPK protein (50 ns) and explicit solvent (TIP3P model), a minimum number of snapshots necessary to describe the oscillation profile of the protein behavior was proposed. For this purpose, the RMSD calculation, together with the sophisticated OWSCA method was used to propose the minimum number of snapshots.

Selected Aspects of Research on Railway Transport Safety

This article showcases the findings of research conducted at the Faculty of Transport, Electrotechnics, and Informatics of the Technological-Humanistic University in Radom. Initially, it delves into a statistical analysis of measurements procured during automatic ultrasonic testing. By integrating these measurements with distinct features of the diagnostic method and fusing differential coding with traditional lossless compression techniques (like Huffman and dictionary coding), a potent measurement compression algorithm was developed. This algorithm subsequently found application in railway track diagnostics. The primary objective of this article is to illustrate this algorithm's efficiency and benchmark its advantages against other compression techniques. Subsequently, the paper delineates the design of a system tailored for collating operational data and evaluating the reliability and safety of railway automation systems. This concept was actualized under the auspices of a project sponsored by the National Center for Research and Development, PBS3/A6/29/2015. A spotlight will be thrown on the software that acts as an integrative tool, assimilating data from various systems. In its final segments, the article discusses the initiative to harness open systems, premised on the PN-EN 50159:2011 standard, to bolster the safety standards of railway transportation. The first one resulted from the maximum train speed, the second one is the variety of rolling stock used on this line (compartmental and non-compartmental wagons). The part of research concerning collection of measurement material was performed within the framework of completed research work PBS3/A6/29/2015. As a result of conducted in-depth literature analysis as well as performed measurements and calculations, it allowed to develop a model and software for simulating system operation in real conditions. A prototype of this system, accompanied by select research outcomes from railway routes, will be laid out. Ultimately, the article seeks to emphasize the significance of diverse research endeavors that hold the potential to directly enhance the safety and operational efficiency of railway transport.

47‐4: A Novel Demura Compensation Data Compression Algorithm based on JPEG‐LS

JPEG‐LS is a lossless/near‐lossless image compression algorithm based on context modeling, which has the advantages of easy implementation, low resource consumption and high compression rate. It is widely used in the compression field of continuous‐tone still images. However, single image compression has significant latency and high hardware resource consumption issues, as the JPEG‐LS algorithm requires pixel by pixel prediction and real‐time context updates during the compression process, which is not conducive to algorithm IP implementation. Based on Demura compensation data compression requirements, this article has made changes to the pixel prediction method of the JPEG‐LS algorithm and delayed the update of the context to solve the compression latency and resource consumption issues. After algorithm optimization (NJPEGLS), the linebuffer resource occupation was reduced by 8/9, the clock frequency reached above 140 MHz, and the comprehensive compression rate loss was within 4.5%, meeting the demand for Demura compensation data compression/decompression. In this paper, we also conducted statistics and analysis on the distribution of values, and found that the distribution of values was very regular, and we propose an adaptive value scheme (KNJPEG‐LS) to further simplify the hardware circuit.

Leveraging Seed Generation for Efficient Hardware Acceleration of Lossless Compression of Remotely Sensed Hyperspectral Images

In the field of satellite imaging, effectively managing the enormous volumes of data from remotely sensed hyperspectral images presents significant challenges due to the limited bandwidth and power available in spaceborne systems. In this paper, we describe the hardware acceleration of a highly efficient lossless compression algorithm, specifically designed for real-time hyperspectral image processing on FPGA platforms. The algorithm utilizes an innovative seed generation method for square root calculations to significantly boost data throughput and reduce energy consumption, both of which represent key factors in satellite operations. When implemented on the Cyclone V FPGA, our method achieves a notable operational throughput of 1598.67 Mega Samples per second (MSps) and maintains a power requirement of under 1 Watt, leading to an efficiency rate of 1829.1 MSps/Watt. A comparative analysis with existing and related state-of-the-art implementations confirms that our system surpasses conventional performance standards, thus facilitating the efficient processing of large-scale hyperspectral datasets, especially in environments where throughput and low energy consumption are prioritized.

Efficient Data Management in Agricultural IoT: Compression, Security, and MQTT Protocol Analysis.

The integration of Internet of Things (IoT) technology into agriculture has revolutionized farming practices by using connected devices and sensors to optimize processes and facilitate sustainable execution. Because most IoT devices have limited resources, the vital requirement to efficiently manage data traffic while ensuring data security in agricultural IoT solutions creates several challenges. Therefore, it is important to study the data amount that IoT protocols generate for resource-constrained devices, as it has a direct impact on the device performance and overall usability of the IoT solution. In this paper, we present a comprehensive study that focuses on optimizing data transmission in agricultural IoT solutions with the use of compression algorithms and secure technologies. Through experimentation and analysis, we evaluate different approaches to minimize data traffic while protecting sensitive agricultural data. Our results highlight the effectiveness of compression algorithms, especially Huffman coding, in reducing data size and optimizing resource usage. In addition, the integration of encryption techniques, such as AES, provides the security of the transmitted data without incurring significant overhead. By assessing different communication scenarios, we identify the most efficient approach, a combination of Huffman encoding and AES encryption, to strike a balance between data security and transmission efficiency.

Approximation in the extended functional tensor train format

This work proposes the extended functional tensor train (EFTT) format for compressing and working with multivariate functions on tensor product domains. Our compression algorithm combines tensorized Chebyshev interpolation with a low-rank approximation algorithm that is entirely based on function evaluations. Compared to existing methods based on the functional tensor train format, the adaptivity of our approach often results in reducing the required storage, sometimes considerably, while achieving the same accuracy. In particular, we reduce the number of function evaluations required to achieve a prescribed accuracy by up to over 96%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$96\\%$$\\end{document} compared to the algorithm from Gorodetsky et al. (Comput. Methods Appl. Mech. Eng. 347, 59–84 2019).

Compression algorithm for colored de Bruijn graphs

A colored de Bruijn graph (also called a set of k-mer sets), is a set of k-mers with every k-mer assigned a set of colors. Colored de Bruijn graphs are used in a variety of applications, including variant calling, genome assembly, and database search. However, their size has posed a scalability challenge to algorithm developers and users. There have been numerous indexing data structures proposed that allow to store the graph compactly while supporting fast query operations. However, disk compression algorithms, which do not need to support queries on the compressed data and can thus be more space-efficient, have received little attention. The dearth of specialized compression tools has been a detriment to tool developers, tool users, and reproducibility efforts. In this paper, we develop a new tool that compresses colored de Bruijn graphs to disk, building on previous ideas for compression of k-mer sets and indexing colored de Bruijn graphs. We test our tool, called ESS-color, on various datasets, including both sequencing data and whole genomes. ESS-color achieves better compression than all evaluated tools and all datasets, with no other tool able to consistently achieve less than 44% space overhead. The software is available at http://github.com/medvedevgroup/ESSColor.

BiSAMNet: A Novel Approach in Maritime Data Completion Using Deep Learning and NLP Techniques

In the extensive monitoring of maritime traffic, maritime management frequently encounters incomplete automatic identification system (AIS) data. This deficiency poses significant challenges to safety management, requiring effective methods to infer corresponding ship information. We tackle this issue using a classification approach. Due to the absence of a fixed road network at sea unlike on land, raw trajectories are difficult to convert and cannot be directly fed into neural networks. We devised a latitude–longitude gridding encoding strategy capable of transforming continuous latitude–longitude data into discrete grid points. Simultaneously, we employed a compression algorithm to further extract significant grid points, thereby shortening the encoding sequence. Utilizing natural language processing techniques, we integrate the Word2vec word embedding approach with our novel biLSTM self-attention chunk-max pooling net (biSAMNet) model, enhancing the classification of vessel trajectories. This method classifies targets into ship types and ship lengths within static information. Employing the Taiwan Strait as a case study and benchmarking against CNN, RNN, and methods based on the attention mechanism, our findings underscore our model’s superiority. The biSAMNet achieves an impressive trajectory classification F1 score of 0.94 in the ship category dataset using only five-dimensional word embeddings. Additionally, through ablation experiments, the effectiveness of the Word2vec pre-trained embedding layer is highlighted. This study introduces a novel method for handling ship trajectory data, addressing the challenge of obtaining ship static information when AIS data are unreliable.

Pebble-depth

In this paper we introduce a new feasible notion of Bennett's logical depth based on pebble transducers. This notion is defined based on the difference between the minimal length descriptional complexity of prefixes of infinite sequences from the perspective of finite-state transducers and pebble transducers. Our notion of pebble-depth satisfies the four fundamental properties of depth: i.e. deep sequences exist, trivial sequences are not deep, random sequences are not deep, and the existence of a slow growth law type result. We also compare pebble-depth to other depth notions based on finite-state transducers, pushdown compressors, and the Lempel-Ziv 78 compression algorithm. We first demonstrate how there exists a normal pebble-deep sequence even though there is no normal finite-state-deep sequence. We next build a sequence that has a pebble-depth level of roughly 1, a pushdown-depth level of roughly 1/2 and a finite-state-depth level of roughly 0. We then build a sequence that has a pebble-depth level of roughly 1/2 and a Lempel-Ziv-depth level of roughly 0.

Fast Linde-Buzo-Gray (FLBG) Algorithm for Image Compression through Rescaling Using Bilinear Interpolation.

Vector quantization (VQ) is a block coding method that is famous for its high compression ratio and simple encoder and decoder implementation. Linde-Buzo-Gray (LBG) is a renowned technique for VQ that uses a clustering-based approach for finding the optimum codebook. Numerous algorithms, such as Particle Swarm Optimization (PSO), the Cuckoo search algorithm (CS), bat algorithm, and firefly algorithm (FA), are used for codebook design. These algorithms are primarily focused on improving the image quality in terms of the PSNR and SSIM but use exhaustive searching to find the optimum codebook, which causes the computational time to be very high. In our study, our algorithm enhances LBG by minimizing the computational complexity by reducing the total number of comparisons among the codebook and training vectors using a match function. The input image is taken as a training vector at the encoder side, which is initialized with the random selection of the vectors from the input image. Rescaling using bilinear interpolation through the nearest neighborhood method is performed to reduce the comparison of the codebook with the training vector. The compressed image is first downsized by the encoder, which is then upscaled at the decoder side during decompression. Based on the results, it is demonstrated that the proposed method reduces the computational complexity by 50.2% compared to LBG and above 97% compared to the other LBG-based algorithms. Moreover, a 20% reduction in the memory size is also obtained, with no significant loss in the image quality compared to the LBG algorithm.

Circuit-Level Realization of an Adaptive Trending Prediction Algorithm for Lossless ECG Compression

This paper introduces an advanced approach to the hardware implementation of lossless compression for electrocardiogram (ECG) signals, combining Content Adaptive Golomb Rice (CAGR) coding with Adaptive Trending Prediction (ATP). The CAGR technique dynamically adjusts its parameters based on local signal characteristics, effectively optimizing compression efficiency. Simultaneously, Adaptive Trending Prediction analyzes and predicts signal trends to further minimize redundancy before encoding, enhancing overall compression ratios. The hardware architecture presented in this study is meticulously designed to leverage these techniques, ensuring efficient utilization of resources while maintaining real time performance suitable for medical applications. Design considerations include algorithmic complexity, throughput, and resource utilization metrics tailored to meet the stringent requirements of medical monitoring systems. Experimental evaluations validate the efficacy of the proposed method, demonstrating substantial reductions in data size while preserving clinical fidelity crucial for accurate diagnosis and monitoring. The findings underscore the feasibility and practicality of integrating advanced compression algorithms into dedicated hardware platforms, thereby facilitating more efficient data transmission and storage in medical environments.

A feature enhanced EEG compression model using asymmetric encoding–decoding network * *This work was supported by the National Natural Science Foundation of China (No. NSFC62227801) and the Lingang Laboratory, Grant No. LG-TKN-202205-01.

Objective. Recently, the demand for wearable devices using electroencephalography (EEG) has increased rapidly in many fields. Due to its volume and computation constraints, wearable devices usually compress and transmit EEG to external devices for analysis. However, current EEG compression algorithms are not tailor-made for wearable devices with limited computing and storage. Firstly, the huge amount of parameters makes it difficult to apply in wearable devices; secondly, it is tricky to learn EEG signals’ distribution law due to the low signal-to-noise ratio, which leads to excessive reconstruction error and suboptimal compression performance. Approach. Here, a feature enhanced asymmetric encoding–decoding network is proposed. EEG is encoded with a lightweight model, and subsequently decoded with a multi-level feature fusion network by extracting the encoded features deeply and reconstructing the signal through a two-branch structure. Main results. On public EEG datasets, motor imagery and event-related potentials, experimental results show that the proposed method has achieved the state of the art compression performance. In addition, the neural representation analysis and the classification performance of the reconstructed EEG signals also show that our method tends to retain more task-related information as the compression ratio increases and retains reliable discriminative information after EEG compression. Significance. This paper tailors an asymmetric EEG compression method for wearable devices that achieves state-of-the-art compression performance in a lightweight manner, paving the way for the application of EEG-based wearable devices.