Astro-L3C: boosting lossless solar image compression with Kolmogorov–Arnold-guided learning

Abstract The petabyte-scale data generated by High Energy Physics (HEP) experiments presents a significant storage challenge. We present the Bytewise Online Autoregressive (BOA) Constrictor, a new pseudo-streaming lossless neural compressor built upon the Mamba state space model. BOA achieves competitive compression ratios across diverse structured HEP datasets, matching or exceeding LZMA, ZSTD and ZLIB at maximum compression, among other tested algorithms. With a 2.21 MB model, BOA achieves an effective compression ratio (defined as the ratio of original to compressed file size, inclusive of model size) of 7.23× on ATLAS Open Data (HDF5) and 9.13× on simulated particle collision records (HepMC v3), outperforming the next-best traditional algorithm (6.79× and 5.33×, respectively on each dataset). BOA also demonstrates robust cross-file and cross-condition generalisation on CMS Open Data (NanoAOD format), where it obtains comparable or improved effective compression ratios (within 5%) with respect to the next-best traditional algorithm. Ablation studies show that transitioning to half-precision (FP16) weights reduces the model footprint without degrading predictive accuracy, and data-type analyses reveal BOA performs best on high-entropy float32 payloads. The model has also been tested in other kinds of scientific data, yielding 1.61× (vs. 1.14× for next-best algorithm) in computational fluid dynamics and up to 1.53× (vs. 1.27×) in cosmology (CAMELS) datasets. BOA is supported by a deterministic reference C++ implementation which ensures bit-exact reproducibility across different CUDA architectures. In this proof-of-principle implementation, BOA delivers a ∼3.5 to 45 MB/s compression and ∼1.5 to 25 MB/s decompression throughput that is not yet competitive with optimised algorithms such as ZSTD or LZMA, but still provides a first step towards data compression improvements for next-generation scientific data.

Objective.The complexity of neural data changes as the brain processes information during events. Universal lossless compression algorithms, which are broadly applicable and grounded in information theory, identify and exploit redundancies in data in order to compress it to essentially-optimal sizes regardless of underlying statistics. These algorithms may be used to efficiently estimate a signal's Shannon entropy rate, a biologically relevant measure of the complexity of a signal. It is therefore natural to explore their effectiveness in the analysis of spiking neural data.Approach.This work uses the inverse compression ratio (ICR) to analyze recordings (Utah arrays) taken from motor cortex of animals performing reaching tasks three days before and three days after administering electrolytic lesions (SubjectU: 4 lesions,H: 3). We calculate ICR with temporally-independent lossless compression (gzip) and temporally-dependent lossy compression (H.264, MPEG-2). Compression-based ICR was compared to single-neuron measures used to understand spiking data (average firing rates and Fano factor), as well as common dimensionality reduction techniques (principal component analysis and factor analysis).Main Results.ICR is able to significantly (Mann-Whitney U test,p<0.01) detect lesions with higher accuracy than single-neuron metrics, but not dimensionality reduction (ICR methods: 85.7%, single-neuron methods: 78.6%, dimensionality reduction: 100%). Additionally, statistical results on the same data show that ICR metrics remain more stable than single-neuron methods after lesion. The bitrate parameter of lossy compression algorithms is swept to better understand the effect of information rates and 'optimal' compression on lesion detection performance. Simulated data shows that ICR is computationally advantageous.Significance.These results suggest that compression algorithms may be a useful tool to detect and better understand perturbations to the underlying structure of neural data. Information-theoretic analyses may complement techniques like dimensionality reduction and firing rate tuning as a convenient and useful tool to characterize neural data.

Large genetic datasets are terabytes in size, presenting a computational challenge that will intensify as sequencing efforts scale. We present a lossless compression algorithm, kodama , which supports matrix multiplication and is suitable for large-scale statistical analyses. Kodama leverages genealogical relatedness among nominally unrelated individuals and infers a novel data structure similar to the ancestral recombination graph (ARG), called the linear ARG. We applied kodama to whole genome sequencing data from UK Biobank and All of Us. Inferred linear ARGs were 17-89 times smaller on disk compared to the input data; the entire UK Biobank N=200k dataset can be loaded into memory (58GB). Compared with the recently proposed genotype representation graph (GRG), the linear ARG is 2.5 times smaller. Genotype matrix multiplications, which are the bottleneck in most statistical applications, are extremely fast with the linear ARG; we performed a GWAS on the UK Biobank 200k cohort across 89 traits with 42 covariates in 100 seconds, representing a 4,700-fold speedup over PLINK 2.0. We expect that the linear ARG will enable genetic analyses to scale to millions of samples.

We propose Next Bit Prediction (NBP), a unified framework that simultaneously addresses lossless compression and lossy reconstruction of 3D point cloud geometry through a next-bit probability estimation paradigm. Our key insight is that both lossless compression and lossy reconstruction fundamentally rely on accurate probability estimation of geometric symbols, though targeting different metrics. Lossless compression minimizes bitrate via precise symbol distribution prediction, while lossy reconstruction enhances reconstruction fidelity through probability-guided geometry refinement. Recognizing that point clouds become sparser with increasing bit depth, NBP introduces two key technical innovations. For more significant bits, where the point density is higher, we develop a multi-stage Occupancy Probability Estimation (OPE) mechanism to estimate the probability distribution of occupancy status across multiple iteration stages, with each stage supporting either lossless or lossy mode. For less significant bits that focus on point placement, a Disentangled Probability Estimation (DPE) module is proposed to handle density information and binary residuals, simultaneously enabling lossless compression and facilitating probability-driven coordinate refinement for high-quality lossy reconstruction. Extensive experiments demonstrate the advantages of NBP, including low complexity, progressive coding, and superior coding efficiency, achieving state-of-the-art results both quantitatively and qualitatively.

Lossless image compression encryption algorithm based on multi-scroll chaotic systems and quadtree coding

Transferring large volumes of high-resolution images during wind turbine inspections introduces a bottleneck in assessing and detecting severe defects. Efficient coding must preserve high fidelity in blade regions while aggressively compressing the background. In this work, we propose an end-to-end deep learning framework that jointly performs segmentation and dual-mode (lossy and lossless) compression. The segmentation module accurately identifies the blade region, after which our region-of-interest (ROI) compressor encodes it at superior quality compared to the rest of the image. Unlike conventional ROI schemes that merely allocate more bits to salient areas, our framework integrates: 1) a robust segmentation network (BU-Netv2+P) with a CRF-regularized loss for precise blade localization; 2) a hyperprior-based autoencoder optimized for lossy compression; and 3) an extended bits-back coder with hierarchical models for fully lossless blade reconstruction. Furthermore, our ROI framework removes the sequential dependency in bits-back coding by reusing background-coded bits, enabling parallelized and efficient dual-mode compression. To the best of our knowledge, this is the first fully integrated learning-based ROI codec combining segmentation, lossy, and lossless compression, ensuring that subsequent defect detection is not compromised. Experiments on a large-scale wind turbine dataset demonstrate superior compression performance and efficiency, offering a practical solution for automated inspections.

Vectorcardiogram (VCG) signal compression is very much in demand in the present-day scenario due to the increasing number of cardiac patients. Hence, in this paper, a new technique is proposed that compresses VCG signal by optimizing the tunable quality wavelet transform (TQWT) parameters. The noise in VCG signal is firstly removed by applying a Savitzky-Golay filter, and then passing noise-free signal to an optimization algorithm that optimizes the TQWT parameters, and obtains the frequency domain signal. This signal is then quantized through dead-zone quantization and processed by a lossless compression mechanism: run-length encoding (RLE) to improve the compression ratio & encode the signal. This compressed signal is reconstructed by Inverse RLE to obtain the decoded signal. Inverse of TQWT is applied to get the reconstructed signal back from the transformed frequency domain to time domain. The parameters of TQWT, especially theQandR, are optimized to get the highestCRat lowest percent root-mean-square-difference(PRD)with best reconstruction quality and least distortions, along with acceptable values of signal-to-noise-ratio(SNR), quality score(QS), andSimilaritywith lowest mean-square-error(MSE). The comparative analysis of different optimization methods indicates that the sparse-particle swarm optimization is best among all the approaches for the tuning of parameters in TQWT for VCG signal compression and reconstruction achieving aCRof 48.18 at aPRDof 3.68,SNRof 29.39,QSof 15.71, similarity of 0.99845,MSEof 0.00016, withQvalue of 2.04307 andRvalue of 1.20568 withcomputational timeof 4.48508 s.

In this article we propose a novel algorithm for lossless image compression based on 8 8 px block division and vector quantization.Each block is assigned to one of k classes and encoded using linear predictor assigned to a class (calculated using the Iterative Reweighted Least Squares (IRLS) algorithm).Several methods for initializing classes based on binary division of a set are proposed.Their advantages and disadvantages are shown, and the best performing ones are selected in order to develop an original dictionary initialization algorithm, which is used in the vector quantization method adapted to the lossless image compression.The hierarchical, binary tree-based initialization method is a combination of these algorithms, in which class initialization procedure follows the pattern of a complete binary tree with k leaves.The proposed initialization method significantly reduced time required for the main vector quantization process.Proposed codec belongs to the time-asymmetric compression methods with a short decoding time and is characterized by high compression efficiency, offering on average a 7.22% lower bit average compared to JPEG-LS.

Astro-SReC: Attention-enhanced Neural Networks for Lossless Compression of Super-resolution Solar Observations

Lossless Compression of Modern Astronomical Data Using a Novel Learned Predictor

This paper proposes a novel fault diagnosis method that integrates a Relative Position Matrix (RPM), a Downsampling Attention Module (DAM), an Improved Residual Network (IResNet), and transfer learning to address the challenges of scarce fault data and poor generalization under variable working conditions. The RPM converts 1D vibration signals into 2D images to enhance feature representation. The DAM achieves lossless feature compression and selection via Haar wavelet downsampling and convolutional attention. An IResNet then performs deep feature learning and classification. A transfer learning strategy further enables effective knowledge adaptation from data-rich source domains to data-scarce target domains, significantly improving performance in cross-condition and small-sample scenarios. Experiments on multiple bearing and gear datasets demonstrate that the proposed method achieves over 99.5% accuracy, with 100% in key transfer tasks, outperforming existing state-of-the-art approaches. The main contributions of this work include the unified RPM-DAM-IResNet framework, a targeted small-sample transfer strategy, and comprehensive validation of its superior accuracy and robustness.

AI-Enhanced Lossless Compression Methods for Efficient Storage of Medical Imaging Data

Social media platforms, surveillance systems, and medical imaging systems produce large image collections that require significant storage space. Traditional JPEG compression processes each image independently, preventing the exploitation of redundancy among similar images. This paper proposes a lossless inter-image predictive compression framework operating in the Discrete Cosine Transform (DCT) domain to improve compression efficiency for JPEG collections. A graph-based prediction structure is constructed using similarity features between images. Residual modeling is then applied relative to selected reference images, followed by shared entropy coding across the collection. Experimental results demonstrate that the proposed framework achieves 86.62% compression efficiency compared with 78.76% efficiency obtained from independent JPEG compression, providing approximately 8% additional bit savings while maintaining lossless reconstruction.

This paper investigates phase transitions in complexity within mobile automata governed by non-local rules. Unlike traditional cellular automata, mobile automata involve a single active cell navigating and updating a one-dimensional array of binary-state cells based on a rule set. By varying the number and symmetry of dependent cells in non-local rules, we observe abrupt changes in system behavior that we identify as computational phase transitions. Using active cell growth and three complementary metrics (Shannon block entropy and estimates of Kolmogorov–Chaitin complexity based on block decomposition and lossless compression), we quantitatively analyze the complexity of automata across a range of dependent cell configurations. Our results reveal that certain increases in non-locality trigger dramatic shifts in entropy and compressibility, while other expansions produce negligible or even simplifying effects. We categorize the observed transitions into categories based on their entropy and growth patterns and demonstrate that complexity does not scale linearly with non-locality. This paper provides a formal foundation for understanding structural complexity in mobile automata and contributes to the broader theory of emergent computation in simple rule-based systems.

Scientific data in structural biology are being produced faster and in larger volumes than can be comfortably stored, processed, or shared. To address this challenge, we introduced the next generation TERSE/PROLIX (TRPX) algorithm for efficient, fast, and lossless compression of integer greyscale data, implemented in C++20. Here, we report a multithreaded extension with additional options for compressing low-intensity integer images and for lossless or lossy compression of greyscale float data. This new implementation is accessible through a dedicated, multithreaded Python library (pyterse) and as an HDF5 filter (terse), allowing seamless integration into existing scientific workflows. Benchmarks show that TRPXv2.0 is at least 2.5 times faster than existing compression schemes for diffraction data, without increasing file sizes, and often with better compression ratios. By combining speed, flexibility, and interoperability, TRPXv2.0 provides a practical and scalable solution for high-throughput data handling in modern structural biology.

Learned lossless medical image compression via dual transform and subimage-wise auto-regression

Tele-monitoring is a useful platform for remote monitoring of cardiac patients, where compression plays a significant role in reducing the link burden and memory utilization of the source device. This paper describes a new approach for lossless ECG compression based on a deep-learning method via an adaptive autoregressive integrated moving average (ARIMA) model. Raw ECG signals were denoised and preprocessed to generate beat-cells for further processing. The ARIMA model uses the individual cardiac cycles to generate model parameters, which are then compressed. In this research, the optimal model hyperparameters were predicted by a deep autoencoder followed by a multilayer perceptron neural network (MLPNN) regressor combination. The predictor was tuned offline via particle swarm optimization (PSO), which produced the reference data for MLPNN tuning. The technique uses 46 records of mitdb under PhysioNet, including 10 major abnormal beats: H, A, V, P, L, R, a, f, F and j. Because of the adaptive nature, compression quality is high with negligible loss. No deviations in the clinical features of the reconstructed beats are found. The mean CR and PRD% values were 41.51 and 0.209%, respectively, which are superior to those reported in published research on ECG compression. The proposed adaptive ECG compression model can be useful for real-time telemonitoring applications, efficient storage and transmission of streamlined data of critical patients under continuous monitoring.

The growing number of Internet of Things (IoT) devices has driven the need for energy-efficient communication in long-range, low-power networks like LoRa. LoRa offers wide coverage with minimal transmission power. However, radio communication remains the main energy consumer in end devices. Data compression can mitigate this issue by reducing packet size and transmission frequency. This work presents a comprehensive evaluation of classical and cutting-edge lossless compression algorithms applied to LoRa networks. Evaluated algorithms include Huffman, LZW, BSC, CMIX, PAQ8PX, GMIX, and LSTM-compress. Experiments were conducted using a Raspberry Pi 5 integrated with an RFM95W LoRa module and INA219 sensors to measure real-time power consumption, CPU load, and memory usage. Results show that classical methods, particularly LZW, achieve the best energy efficiency and reduce LoRa transmission energy by up to 7.41%. In contrast, cutting-edge machine learning (ML)-based algorithms, such as CMIX and PAQ8PX, achieve higher compression ratios but exhibit excessive computational and memory overhead, resulting in negative energy gains. Metadata overheads, including dynamic Huffman tables (28-128 bytes), also affect payload efficiency for small packets. These findings indicate that LZW is the most practical choice for energy-constrained LoRa nodes. At the same time, modern compressors, including ML-based ones, are better suited for gateways or edge servers with higher computational capacity. An open-source implementation of the experimental framework and scripts used in this study is available in the project's public GitHub repository.

This paper proposes an image compression and encryption algorithm based on 3D Logistic-Sine-Memristor coupled map (3D-LSMCM), compressed sensing (CS) and the Game of Life scrambling with Semi-Tensor Product (STP-GoL) to solve the problems of computational redundancy and security vulnerability in image transmission. First, the 3D-LSMCM is initialized by using a hash function to generate a random chaotic sequence iteratively and it improves key unpredictability. Second, The Semi-Tensor Product (STP) measurement matrix constructed from these sequences significantly reduces storage requirements in CS whilst supporting lossless compression through arithmetic coding. STP technology enables flexible matrix dimension design, effectively lowering computational costs. Finally, STP is used to convert logical operations into matrix operations in the entire encryption process. The cryptographic technique consists of the STP-GoL scrambling, chaotic global scrambling with STP (STP-CG) and the chain XOR diffusion based on STP (STP-CXOR). Furthermore, theoretical analysis indicates that the computational complexity of this algorithm is expected to be reduced by approximately 28% compared with recent studies, while the average information entropy of the ciphertext reaches 7.9987. Overall, this study proposes a novel, secure and efficient image compression and encryption method, with its source code publicly available at https://github.com/Ssshou7/STP-GoL.