Compression Algorithm Research Articles

Assembly Theory is an approximation to algorithmic complexity based on LZ compression that does not explain selection or evolution

We formally prove the equivalence between Assembly Theory (AT) and Shannon Entropy via a method based upon the principles of statistical compression that belongs to the LZ family of popular compression algorithms. Such popular lossless compression algorithms behind file formats such as ZIP and PNG have been shown to empirically reproduce the results that AT considers its cornerstone. The same results have also been reported before AT in successful application of other complexity measures in the areas covered by AT such as separating organic from non-organic molecules and in the context of the study of selection and evolution. We demonstrate that the assembly index is equivalent to the size of a minimal context-free grammar. The statistical compressibility of such a method is bounded by Shannon Entropy and other equivalent traditional LZ compression schemes, such as LZ77 and LZW. We also demonstrate that AT, and the algorithms supporting its pathway complexity, assembly index, and assembly number, define compression schemes and methods that are subsumed into algorithmic information theory. We conclude that the assembly index and the assembly number do not lead to an explanation or quantification of biases in generative (physical or biological) processes, including those brought about by (abiotic or biotic) selection and evolution, that could not have been arrived at using Shannon Entropy, or that have not been already reported before using classical information theory or algorithmic complexity.

Compressed Point Cloud Quality Index by Combining Global Appearance and Local Details

In recent years, many standardized algorithms for point cloud compression (PCC) has been developed and achieved remarkable compression ratios. To provide guidance for rate-distortion optimization and codec evaluation, point cloud quality assessment (PCQA) has become a critical problem for PCC. Therefore, in order to achieve a more consistent correlation with human visual perception of a compressed point cloud, we propose a full-reference PCQA algorithm tailored for static point clouds in this article, which can jointly measure geometry and attribute deformations. Specifically, we assume that the quality decision of compressed point clouds is determined by both global appearance (e.g., density, contrast, and complexity) and local details (e.g., gradient and hole). Motivated by the nature of compression distortions and the properties of the human visual system, we derive perceptually effective features for the above two categories, such as content complexity, luminance/geometry gradient, and hole probability. Through systematically incorporating measurements of variations in the local and global characteristics, we derive an effective quality index for the input compressed point clouds. Extensive experiments and analyses conducted on popular PCQA databases show the superiority of the proposed method in evaluating compression distortions. Subsequent investigations validate the efficacy of different components within the model design.

Multilevel approximation of Gaussian random fields: Covariance compression, estimation, and spatial prediction

The distribution of centered Gaussian random fields (GRFs) indexed by compacta such as smooth, bounded Euclidean domains or smooth, compact and orientable manifolds is determined by their covariance operators. We consider centered GRFs given as variational solutions to coloring operator equations driven by spatial white noise, with an elliptic self-adjoint pseudodifferential coloring operator from the Hörmander class. This includes the Matérn class of GRFs as a special case. Using biorthogonal multiresolution analyses on the manifold, we prove that the precision and covariance operators, respectively, may be identified with bi-infinite matrices and finite sections may be diagonally preconditioned rendering the condition number independent of the dimension p of this section. We prove that a tapering strategy by thresholding applied on finite sections of the bi-infinite precision and covariance matrices results in optimally numerically sparse approximations. That is, asymptotically only linearly many nonzero matrix entries are sufficient to approximate the original section of the bi-infinite covariance or precision matrix using this tapering strategy to arbitrary precision. The locations of these nonzero matrix entries can be determined a priori. The tapered covariance or precision matrices may also be optimally diagonally preconditioned. Analysis of the relative size of the entries of the tapered covariance matrices motivates novel, multilevel Monte Carlo (MLMC) oracles for covariance estimation, in sample complexity that scales log-linearly with respect to the number p of parameters. In addition, we propose and analyze novel compressive algorithms for simulating and kriging of GRFs. The complexity (work and memory vs. accuracy) of these three algorithms scales near-optimally in terms of the number of parameters p of the sample-wise approximation of the GRF in Sobolev scales.

A method for compressing AIS trajectory based on the adaptive core threshold difference Douglas–Peucker algorithm

Traditional trajectory compression algorithms, such as the siliding window (SW) algorithm and the Douglas–Peucker (DP) algorithm, typically use static thresholds based on fixed parameters like ship dimensions or predetermined distances, which limits their adaptive capabilities. In this paper, the adaptive core threshold difference-DP (ACTD-DP) algorithm is proposed based on traditional DP algorithm. Firstly, according to the course value of automatic identification system (AIS) data, the original trajectory data is preprocessed and some redundant points are discarded. Then the number of compressed trajectory points corresponding to different thresholds is quantified. The function relationship between them is established by curve fitting method. The characteristics of the function curve are analyzed, and the core threshold and core threshold difference are solved. Finally, the compression factor is introduced to determine the optimal core threshold difference, which is the key parameter to control the accuracy and efficiency of the algorithm. Five different algorithms are used to compress the all ship trajectories in the experimental water area. The average compression ratio (ACR) of the ACTD-DP algorithm is 87.53%, the average length loss ratio (ALLR) is 23.20%, the AMSED (mean synchronous Euclidean distance of all trajectories) is 68.9747 mx, and the TIME is 25.6869 s. Compared with the other four algorithms, the ACTD-DP algorithm shows that the algorithm can not only achieve high compression ratio, but also maintain the integrity of trajectory shape. At the same time, the compression results of four different trajectories show that ACTD-DP algorithm has good robustness and applicability. Therefore, ACTD-DP algorithm has the best compression effect.

Massive Compression for High Data Rate Macromolecular Crystallography (HDRMX): Impact on Diffraction Data and Subsequent Structural Analysis.

New higher-count-rate, integrating, large area X-ray detectors with framing rates as high as 17,400 images per second are beginning to be available. These will soon be used for specialized MX experiments but will require optimal lossy compression algorithms to enable systems to keep up with data throughput. Some information may be lost. Can we minimize this loss with acceptable impact on structural information?

Robust Mixed-Rate Region-of-Interest-Aware Video Compressive Sensing for Transmission Line Surveillance Video

Classic video compression methods usually suffer from long encode time and requires large memories, making it hard to deploy on edge devices; thus, video compressive sensing, which requires less resources during encoding, is receiving more attention. We propose a robust mixed-rate ROI-aware video compressive sensing algorithm for transmission line surveillance video compression. The proposed method compresses foreground targets and background frames separately and uses reversible neural network to reconstruct original frames. The result on transmission line surveillance video data shows that the proposed compressive sensing method can achieve 26.47, 34.71 PSNR and 0.6839, 0.9320 SSIM higher than existing methods on 1.5% and 15% measurement rates, and the proposed ROI extraction net can precisely retrieve regions under high noise levels. This research not only demonstrates the potential for a more efficient video compression technique in resource-constrained environments, but also lays a foundation for future advancements in video compressive sensing techniques and their applications in various real-time surveillance systems.

Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments.

To deploy machine learning models on-device, practitioners use compression algorithms to shrink and speed up models while maintaining their high-quality output. A critical aspect of compression in practice is model comparison, including tracking many compression experiments, identifying subtle changes in model behavior, and negotiating complex accuracy-efficiency trade-offs. However, existing compression tools poorly support comparison, leading to tedious and, sometimes, incomplete analyses spread across disjoint tools. To support real-world comparative workflows, we develop an interactive visual system called COMPRESS AND COMPARE. Within a single interface, COMPRESS AND COMPARE surfaces promising compression strategies by visualizing provenance relationships between compressed models and reveals compression-induced behavior changes by comparing models' predictions, weights, and activations. We demonstrate how COMPRESS AND COMPARE supports common compression analysis tasks through two case studies, debugging failed compression on generative language models and identifying compression artifacts in image classification models. We further evaluate COMPRESS AND COMPARE in a user study with eight compression experts, illustrating its potential to provide structure to compression workflows, help practitioners build intuition about compression, and encourage thorough analysis of compression's effect on model behavior. Through these evaluations, we identify compression-specific challenges that future visual analytics tools should consider and COMPRESS AND COMPARE visualizations that may generalize to broader model comparison tasks..

An adaptive trajectory compression and feature preservation method for maritime traffic analysis

Ship trajectory data extracted from Automatic Identification System (AIS) has been extensively used for maritime traffic analysis. Yet the enormous volume of AIS data has come with substantial challenges related to storing, processing, analyzing, transmitting, and transferring. Trajectory compression techniques have been widely investigated to remedy the challenge. However, conventional compression techniques such as Douglas-Peucker (DP) algorithm mainly depend on line simplification algorithms, falling short in accurately identifying and preserving crucial information within trajectories. Moreover, using kinematic information from AIS data has posed difficulties associated with compression threshold determination. Hence, an adaptive method capable of considering multiple information from AIS is required. In this paper, a Top-Down Kinematic Compression (TDKC) algorithm aimed at adaptive trajectory compression and feature preservation is proposed. By incorporating time, position, speed, and course attributes from AIS data, TDKC exploits a Compression Binary Tree (CBT) method to address the recursion termination problem and determine the threshold automatically. A case study was conducted to evaluate the performance of TDKC using AIS data from Gulf of Finland, where a comparison with conventional algorithms and their improved versions based on specific performance evaluation metrics was involved. The results demonstrate TDKC's superiority in facilitating maritime traffic analysis.

A hybrid improved compressed particle swarm optimization WSN node location algorithm

The improvement of positioning accuracy in Wireless Sensor Networks (hereafter, WSN) is crucial to develop advanced Internet of Things (IOT, for short) applications. However, the conventional distance vector-hop (DV-Hop) localization algorithm has shortcomings such as low accuracy and weak stability. To overcome these shortcomings, this paper proposes a hybrid improved compressed particle swarm optimization algorithm (HICPSO), which consists of a scheme of linearly decreasing inertia weights, compressed velocity vectors, population Gaussian variants and optimal boundary selection. Then, HICPSO is integrated with DV-Hop to gradually reduce the distance error of least squares method (LSM) estimated with the efficient search advantage of HICPSO. Our simulation results show that the HICPSO algorithm possesses better computational accuracy and search performance on the 22 benchmark test functions compared with the algorithms such as the Improved Adaptive Genetic Algorithm (IAGA) and Adaptive Weighted Particle Swarm Optimizer (AWPSO). Meanwhile, compared with IAGA and AWPSO, the positioning accuracy of HICPSO-based positioning algorithm is improved by 4.28% and 4.76% respectively, and the stability is improved by one order of magnitude.

Electrocardiogram Signal Compression Using Deep Convolutional Autoencoder with Constant Error and Flexible Compression Rate

ObjectivesElectrocardiogram (ECG) signals are beneficial for diagnosing cardiac diseases. The cardiac patients' life quality likely increases with continuous or long-period recording and monitoring of ECG signals, leading to better and early diagnosis of disease and heart attacks. However, continuous ECG recording necessitates high data rates and storage, which means high costs. Therefore, ECG compression is a handy concept that facilitates continuous monitoring of ECG signals. Deep neural networks open up new horizons for compression and also for ECG compression by providing high compression rates and quality. Although they bring constant compression ratios with better average quality, the compression quality of individual samples is not guaranteed, which may lead to misdiagnoses. This study aims to investigate the effect of compression quality on the diagnoses and to develop a deep neural network-based compression strategy that guarantees a quality-bound in return for varying compression ratios.Materials and methodsThe effect of the compression quality on the arrhythmia diagnoses is tested by comparing the performance of the deep learning-based ECG classifier on the original ECG recordings and the distorted recordings using a lossy compression algorithm with different compression error levels. Then, a compression error upper limit is calculated in terms of normalized percent root mean square difference (PRDN) error, which also coincides with the findings of the previous studies in the literature. Lastly, to enable deep learning in ECG compression, a single encoder-multi-decoder convolutional autoencoder architecture, and multiple quantization levels are proposed to guarantee a desired upper limit on the error rate.ResultsThe efficiency of the proposed method is demonstrated on a popular benchmark data set for ECG compression methods using a transfer learning approach. The PRDN error is fixed to various values, and the average compression rates are reported. An average of 13.019:1 compression is achieved for a 10% PRDN error rate, assessed as a fair quality threshold for reconstruction error. It has also been shown that the compression model has a runtime that can be run in real-time on wearable devices such as commercial smartwatches.ConclusionThis study proposes a deep learning-based ECG compression algorithm that guarantees a desired upper limit on the compression error. This model may facilitate an eHealth solution for continuous monitoring of ECG signals of individuals, especially cardiac patients.

The Noise Reduction Algorithm May Not Compensate for the Degradation in Output Signal-to-Noise Ratio Caused by Wide Dynamic Range Compression.

Most modern hearing aids (HAs) employ wide dynamic range compression (WDRC) and noise reduction (NR) algorithms. It is known that the nonlinear effects of WDRC and NR cause changes to the output signal-to-noise ratio (SNR) of an HA. However, the relative contributions of WDRC and NR to the nonlinear effects are not fully understood. The current study investigated (a) whether WDRC or NR dominates the nonlinear effects measured at the output of a digital HA and (b) whether the electroacoustic effectiveness of NR depends on WDRC parameters while input SNR and background noise are systematically varied. Test stimuli were Connected Speech Test sentences in multitalker babble noise (2- or 20-talker), presented at input SNRs ranging from -10 to +10 dB. The HA was programmed using multiband WDRC set according to the National Acoustic Laboratories for Nonlinear HA fitting formula 2 prescriptive fits for four standard audiograms and two compression speeds. The NR algorithm of the HA was switched on or off in separate conditions. Nonlinear electroacoustic effects from the WDRC and NR algorithms were assessed by measuring the output SNR of the HA using a phase-inversion technique. To investigate whether there are other factors that may be important besides the output SNR, the Hearing Aid Speech Intelligibility Index and the Hearing Aid Speech Quality Index were applied to the recordings to generate inferences on aided speech intelligibility and perceived speech quality. Results showed that WDRC dominated the net nonlinear effect at low-input SNRs, and the net nonlinear effect of WDRC and NR was reduced at high-input SNRs. Results also showed that the effectiveness of NR depended on compression parameters. The effectiveness of NR was partially explained by the trend of Hearing Aid Speech Intelligibility Index and Hearing Aid Speech Quality Index scores, potentially indicating that the Hearing Aid Speech Intelligibility Index and Hearing Aid Speech Quality Index scores may capture factors that cannot be captured by the output SNR metric. Results suggest that the individual signal-processing stages in an HA should not be considered as independent. Electroacoustic evaluation of WDRC and NR algorithms in isolation is not sufficient to capture the combined nonlinear effect of the two algorithms. https://doi.org/10.23641/asha.25962541.

An optimized compression algorithm for distributed photovoltaic data

An optimized compression algorithm for distributed photovoltaic data

Dynamic performance-guaranteed adaptive event-triggered trajectory tracking control for underactuated surface vehicles

This paper investigates the trajectory tracking control problem of underactuated surface vehicles in the presence of model uncertainties and external disturbances. A dynamic performance-guaranteed adaptive event-triggered control scheme is proposed to transform the original constrained control problem into an equivalent unconstrained problem and ensure that the trajectory tracking is achieved with prescribed performance. By developing a dynamic event-triggered mechanism with adjustable thresholds that can be updated in an adaptive way, the communication burden and actuator wear can be effectively alleviated. To avoid differential explosions that may increase system complexities, an improved transformation-adapted dynamic surface control (DSC) is designed to eliminate adverse effects induced by performance functions and make the DSC applicable to performance-transformed systems. By utilizing the parameter compression algorithm, model uncertainties and external disturbances can be effectively addressed, and the adaptive laws can be integrated into a concise form, with only two online updating parameters required. Comprehensive analysis is provided to verify that all error signals are semi-globally uniformly ultimately bounded (SGUUB). Compared to existing approaches, the proposed control scheme exhibits lower update frequency.

Smart industrial information integration: a lightweight privacy protection model in an intelligent manufacturing architecture

In the industrial ecosystem, both the manufacturing and edge clouds collaborate via virtualisation technology, playing a crucial role in the efficient aggregation and processing of vast data from endpoints, offering computational and analytical support for enhanced decision-making in all industrial operations. Consequently, ensuring the reliability of virtualised manufacturing environments is paramount. Remote attestation offers a viable solution, however, current schemes are plagued by inefficiencies and privacy leakage issues. To address these concerns, we propose an efficient remote attestation approach. The approach actively conceals sensitive industrial information in measurement logs by using a hash algorithm, thereby preventing privacy breaches. Furthermore, it streamlines log entries with a log compression algorithm, enhancing remote verification efficiency in the industrial information architecture. Extensive experiments were conducted in an authentic cloud environment by applying this approach across both mesh and hierarchical remote verification structures, reducing time overhead by approximately 81.32%. Additionally, it marginally impacts server performance with only a 1.49% reduction. The results affirm the proposed approach's exceptional efficacy in decreasing verification durations and mitigating remote verification attacks, demonstrating its promising utility and feasibility in the intelligent industrial platform.

Segmentation of coronary artery based on discriminative frequency learning and coronary-geometric refinement

Coronary artery segmentation is crucial for physicians to identify and locate plaques and stenosis using coronary computed tomography angiography (CCTA). However, the low contrast of CCTA images and the intricate structures of coronary arteries make this task challenging. To address these difficulties, we propose a novel model, the DFS–PDS network. This network comprises two subnetworks: a discriminative frequency segment subnetwork (DFS) and a position domain scales subnetwork (PDS). DFS introduced a gated mechanism within the feed-forward network, leveraging the Joint Photographic Experts Group (JPEG) compression algorithm, to discriminatively determine which low- and high-frequency information of the features should be preserved for latent image segmentation. The PDS aims to learn the shape prototype by predicting the radius. Additionally, our model has the consistent ability to guarantee region and boundary features through boundary consistency loss. During training, both subnetworks are optimized jointly, and in the testing stage, the coarse segmentation and radius prediction are produced. A coronary-geometric refinement method refines the segmentation masks by leveraging the shape prior to being reconstructed from the radius map, reducing the difficulty of segmenting coronary artery structures from complex surrounding structures. The DFS–PDS network is compared with state-of-the-art (SOTA) methods on two coronary artery datasets to evaluate its performance. The experimental results demonstrate that the DFS–PDS network performs better than the SOTA models, including Vnet, nnUnet, DDT, CS2-Net, Unetr, and CAS-Net, in terms of Dice or connectivity evaluation metrics.

An Adaptive Characteristic Model-Based Event-Triggered Sigmoid Prescribed Performance Control Approach for Tracking the Trajectory of Autonomous Underwater Vehicles

This paper introduces an event-triggered sigmoid prescribed performance control method, enhanced by an adaptive characteristic model, for tracking the trajectory of autonomous underwater vehicles (AUVs). The AUV model is simplified into a function reliant solely on second-order parameter information through the use of characteristic modeling and a compression algorithm, which is then approximated by a neural network. We propose integrating prescribed performance control into event-triggered sliding mode control to accelerate convergence in AUV trajectory tracking. A novel prescribed performance function is employed in this integration, creating an event-triggered, non-singular terminal sliding mode control strategy. The stability of this controller is rigorously proven. This control strategy is not only robust against model uncertainties but also mitigates the jitter commonly associated with sliding mode control and the singularities from preset performance control due to sudden random disturbances. Comparative simulation experiments demonstrate that the proposed control method achieves superior control accuracy and a quicker response.

Research on image compression technology based on improved SPIHT compression algorithm for power grid data

Abstract At present, the amount of information on power grid operation and maintenance monitoring image data is increasing, and the requirements for data compression are higher and higher. Based on the improved SPIHT image compression algorithm, this study presents the research of power grid data compression. First, the basic theory of image compression, and the principle of one-dimensional wavelet transform and two-dimensional wavelet transform are introduced. The development process, characteristics, and advantages of image coding are discussed. Then, the shortcomings of the SPIHT algorithm are analyzed, and the SPIHT coding is improved by parallel computation. The parallel wavelet transform algorithm based on the block idea and the parallel SPIHT coding algorithm based on the code tree are proposed in the data parallelism of the compression algorithm. At the same time, the data dependence between tasks in the process of SPIHT image compression coding is analyzed, and the task parallelism in the compression algorithm is realized by using the relative independence of tasks in different threshold coding. Finally, the application and simulation analysis of power grid data based on the SPIHT compression algorithm, the construction of power grid data model simulation, and the composition of two-dimensional power grid data images are carried out. Secondly, the obtained 2D power grid data image is compressed by the SPIHT algorithm and improved SPIHT algorithm, respectively, and the compression effect of the two algorithms on the power grid data image is compared. When the bit rate is 0.5, the compression effect of the improved SPIHT algorithm is 13.6506. When the bit rate is 1, the compression effect of the improved SPIHT algorithm is 18.9287. The results show that the improved SPIHT algorithm can compress the grid data to obtain better grid image quality.

Image Compression Based on DWT Using EZW/SPIHT Encoders

this paper reviews recent research on image compression techniques utilizing the Embedded Zero-Tree Wavelet (EZW) and Set Partitioning in Hierarchical Trees (SPIHT) algorithms. The study addresses the question of which algorithm, EZW or SPIHT, provides superior compression performance for various image types. The research method involves a comprehensive analysis of published research papers that employed these algorithms for image compression. The findings indicate that SPIHT generally outperforms EZW in terms of compression ratio, computational efficiency, and image quality, making it a more versatile and effective choice for various image compression applications.

Complexity Scaling of Liquid Dynamics.

According to excess-entropy scaling, dynamic properties of liquids like viscosity and diffusion coefficient are determined by the entropy. This link between dynamics and thermodynamics is increasingly studied and of interest also for industrial applications, but hampered by the challenge of calculating entropy efficiently. Utilizing the fact that entropy is basically the Kolmogorov complexity, which can be estimated from optimal compression algorithms [Avinery etal., Phys. Rev. Lett. 123, 178102 (2019)PRLTAO0031-900710.1103/PhysRevLett.123.178102; Martiniani etal., Phys. Rev. X 9, 011031 (2019)PRXHAE2160-330810.1103/PhysRevX.9.011031], we here demonstrate that the diffusion coefficients of four simple liquids follow a quasiuniversal exponential function of the optimal compression length of a single equilibrium configuration. We conclude that "complexity scaling" has the potential to become a useful tool for estimating dynamic properties of any liquid from a single configuration.

Extremely-Compressed SSDs with I/O Behavior Prediction

As the data volume continues to grow exponentially, there is an increasing demand for large storage system capacity. Data compression techniques effectively reduce the volume of written data, enhancing space efficiency. As a result, many modern SSDs have already incorporated data compression capabilities. However, data compression introduces additional processing overhead in critical I/O paths, potentially affecting system performance. Currently, most compression solutions in flash-based storage systems employ fixed compression algorithms for all incoming data without leveraging differences among various data access patterns. This leads to sub-optimal compression efficiency. This article proposes a data-type-aware Flash Translation Layer (DAFTL) scheme to maximize space efficiency without compromising system performance. First, we propose an I/O behavior prediction method to forecast future access on specific data. Then, DAFTL matches data types with distinct I/O behaviors to compression algorithms of varying intensities, achieving an optimal balance between performance and space efficiency. Specifically, it employs higher-intensity compression algorithms for less frequently accessed data to maximize space efficiency. For frequently accessed data, it utilizes lower-intensity but faster compression algorithms to maintain system performance. Finally, an improved compact compression method is proposed to effectively eliminate page fragmentation and further enhance space efficiency. Extensive evaluations using a variety of real-world workloads, as well as the workloads with real data we collected on our platforms, demonstrate that DAFTL achieves more data reductions than other approaches. When compared to the state-of-the-art compression schemes, DAFTL reduces the total number of pages written to the SSD by an average of 8%, 21.3%, and 25.6% for data with high, medium, and low compressibility, respectively. In the case of workloads with real data, DAFTL achieves an average reduction of 10.4% in the total number of pages written to SSD. Furthermore, DAFTL exhibits comparable or even improved read and write performance compared to other solutions.