Optimal Compression Scheme Research Articles

HocoPG: A Database System with Homomorphic Compression for Text Processing

Databases employ out-of-line storage and compression strategies to manage extensive text data. However, the growth in both the size of individual data items and overall data volume has significantly increased the burden of decompression, adversely affecting query performance. To address this challenge, we develop HocoPG, an innovative system that incorporates homomorphic compression theory within RDBMS, enabling direct computation on compressed data to enhance query efficiency and system usability. HocoPG performs homomorphic evaluations across a suite of basic text operations, enabling the execution of intricate text queries by combining these operations flexibly. In this demonstration, we showcase the deployment and usage of HocoPG through a database terminal. Additionally, we introduce the HocoPG Admin, a tool that provides insights to DBAs and general users for choosing the optimal compression scheme based on their individual needs.

Unlabeled sample compression schemes for oriented matroids

A long-standing sample compression conjecture asks to linearly bound the size of the optimal sample compression schemes by the Vapnik-Chervonenkis (VC) dimension of an arbitrary class. In this paper, we explore the rich metric and combinatorial structure of oriented matroids (OMs) to construct proper unlabeled sample compression schemes for the classes of topes of OMs bounded by their VC-dimension. The result extends to the topes of affine OMs, as well as to the topes of the complexes of OMs that possess a corner peeling. The main tool that we use are the solutions of certain oriented matroid programs.

Comprehensive evaluation of lossless compression algorithms in a real use case for smart grid applications

The so-called “energy digitalization” is pervading the power and energy industry by providing many state-of-the-art digital technologies to collect, store, and analyze the very heterogeneous information managed in modern power systems. Requirements in terms of sampling frequency, temporal and user aggregation, measured quantities, measurement aggregations, involved players, and applications can be very different. Compression strategies, removing redundancy and over-specification in collected samples, can play a relevant role in efficiently move and store such heterogeneous data. In this work, a reference dataset from a real-world use case has been collected for comparing the performance of 62 lossless compression algorithms derived from the LZ77/LZSS/LZMA strategies. Compression of the six-day long acquisition, including active power measurements of a prosumer’s system equipped with a Photovoltaic (PV) and a Battery Energy Storage System, highlights the different obtainable performance when the aggregation interval is changed from 15 min to 24 h. In particular, once the top performing algorithms have been selected, it has been possible to stress the impact of night–day cycle, mainly due to the different sparsity of PV-related data. The obtained results demonstrate that, globally, the compression ratio increases by increasing the aggregation interval, by reaching values close to 9.7. In particular, when offline operation is tolerated, optimal compression schemes can be easily applied, leading to consistent improvement of the compression ratio (up to 24%, depending on the actual algorithm and aggregation interval), which can be very significant when large number of data sources is considered.

A data compression and optimal galaxy weights scheme for Dark Energy Spectroscopic Instrument and weak lensing data sets

ABSTRACT Combining different observational probes, such as galaxy clustering and weak lensing, is a promising technique for unveiling the physics of the Universe with upcoming dark energy experiments. The galaxy redshift sample from the Dark Energy Spectroscopic Instrument (DESI) will have a significant overlap with major ongoing imaging surveys specifically designed for weak lensing measurements: the Kilo-Degree Survey (KiDS), the Dark Energy Survey (DES), and the Hyper Suprime-Cam (HSC) survey. In this work, we analyse simulated redshift and lensing catalogues to establish a new strategy for combining high-quality cosmological imaging and spectroscopic data, in view of the first-year data assembly analysis of DESI. In a test case fitting for a reduced parameter set, we employ an optimal data compression scheme able to identify those aspects of the data that are most sensitive to cosmological information and amplify them with respect to other aspects of the data. We find this optimal compression approach is able to preserve all the information related to the growth of structures.

RETRACTED: Compressing project to minimize the increased risk and cost

In a highly competitive market environment, organizations must improve their productivity, reduce production costs and improve management methods, in order to maintain a favorable competitive position. Project factors, such as time, cost and risk are closely linked. If one of those factors is out of control, the progress of the whole project will inevitably be delayed. When the progress of the project is delayed, duration compression measures such as crashing or fast tracking are usually employed to ensure the backward schedule returns to the original plan. However, the implementation of duration compression measures increases the cost and risk of the project. In most previous study of the critical path, only activity time is involved, and cost and risk are not considered. In the present study, a mathematical model is built to solve the optimal duration compression scheme objectives of the minimization of risk and cost. The model is verified by two cases, and the best solution is obtained by using LINGO software and Excel solver. The mathematical model is found to provide the best duration compression scheme for the project, with the least increases in cost and risk.

Compressed Secret Key Agreement:Maximizing Multivariate Mutual Information per Bit

The multiterminal secret key agreement problem by public discussion is formulated with an additional source compression step where, prior to the public discussion phase, users independently compress their private sources to filter out strongly correlated components in order to generate a common secret key. The objective is to maximize the achievable key rate as a function of the joint entropy of the compressed sources. Since the maximum achievable key rate captures the total amount of information mutual to the compressed sources, an optimal compression scheme essentially maximizes the multivariate mutual information per bit of randomness of the private sources, and can therefore be viewed more generally as a dimension reduction technique. Single-letter lower and upper bounds on the maximum achievable key rate are derived for the general source model, and an explicit polynomial-time computable formula is obtained for the pairwise independent network model. In particular, the converse results and the upper bounds are obtained from those of the related secret key agreement problem with rate-limited discussion. A precise duality is shown for the two-user case with one-way discussion, and such duality is extended to obtain the desired converse results in the multi-user case. In addition to posing new challenges in information processing and dimension reduction, the compressed secret key agreement problem helps shed new light on resolving the difficult problem of secret key agreement with rate-limited discussion by offering a more structured achieving scheme and some simpler conjectures to prove.

Objective Quality Assessment and Perceptual Compression of Screen Content Images.

Screen content image (SCI) has recently emerged as an active topic due to the rapidly increasing demand in many graphically rich services such as wireless displays and virtual desktops. SCIs are often composed of pictorial regions and computer generated textual/graphical content, which exhibit different statistical properties that often lead to different viewer behaviors. Inspired by this, we propose an objective quality assessment approach for SCIs that incorporates both visual field adaptation and information content weighting into structural similarity based local quality assessment. Furthermore, we develop a perceptual screen content coding scheme based on the newly proposed quality assessment measure, targeting at further improving the SCI compression performance. Experimental results show that the proposed quality assessment method not only better predicts the perceptual quality of SCIs, but also demonstrates great potentials in the design of perceptually optimal SCI compression schemes.

Linear-Time Compression of Bounded-Genus Graphs into Information-Theoretically Optimal Number of Bits

A $\textit{compression scheme}$ $A$ for a class $\mathbb{G}$ of graphs consists of an encoding algorithm $\textit{Encode}_A$ that computes a binary string $\textit{Code}_A(G)$ for any given graph $G$ in $\mathbb{G}$ and a decoding algorithm $\textit{Decode}_A$ that recovers $G$ from $\textit{Code}_A(G)$. A compression scheme $A$ for $\mathbb{G}$ is $\textit{optimal}$ if both $\textit{Encode}_A$ and $\textit{Decode}_A$ run in linear time and the number of bits of $\textit{Code}_A(G)$ for any $n$-node graph $G$ in $\mathbb{G}$ is information-theoretically optimal to within lower-order terms. Trees and plane triangulations were the only known nontrivial graph classes that admit optimal compression schemes. Based upon Goodrich's separator decomposition for planar graphs and Djidjev and Venkatesan's planarizers for bounded-genus graphs, we give an optimal compression scheme for any hereditary (i.e., closed under taking subgraphs) class $\mathbb{G}$ under the premise that any $n$-node graph of $\mathbb{G}$ to be encoded comes with a genus-$o(\frac{n}{\log^2 n})$ embedding. By Mohar's linear-time algorithm that embeds a bounded-genus graph on a genus-$O(1)$ surface, our result implies that any hereditary class of genus-$O(1)$ graphs admits an optimal compression scheme. For instance, our result yields the first-known optimal compression schemes for planar graphs, plane graphs, graphs embedded on genus-$1$ surfaces, graphs with genus $2$ or less, $3$-colorable directed plane graphs, $4$-outerplanar graphs, and forests with degree at most $5$. For non-hereditary graph classes, we also give a methodology for obtaining optimal compression schemes. From this methodology, we give the first known optimal compression schemes for triangulations of genus-$O(1)$ surfaces and floorplans.

Optimal compression in synchronised time‐lens source for CRS imaging

An optimal pulse compression scheme is proposed for the synchronised time-lens source for coherent Raman scattering (CRS) imaging, using CRS signals as the criterion. The simulation results show that instead of compressing the pulse width of the time-lens source to its minimum, peak power should be optimised to maximise CRS signals. In terms of experiment, this can be done by optimising the second-harmonic signal or two-photon current of the time-lens output.

An Adaptive Lossless Data Compression Scheme for Wireless Sensor Networks

Energy is an important consideration in the design and deployment of wireless sensor networks (WSNs) since sensor nodes are typically powered by batteries with limited capacity. Since the communication unit on a wireless sensor node is the major power consumer, data compression is one of possible techniques that can help reduce the amount of data exchanged between wireless sensor nodes resulting in power saving. However, wireless sensor networks possess significant limitations in communication, processing, storage, bandwidth, and power. Thus, any data compression scheme proposed for WSNs must be lightweight. In this paper, we present an adaptive lossless data compression (ALDC) algorithm for wireless sensor networks. Our proposed ALDC scheme performs compression losslessly using multiple code options. Adaptive compression schemes allow compression to dynamically adjust to a changing source. The data sequence to be compressed is partitioned into blocks, and the optimal compression scheme is applied for each block. Using various real-world sensor datasets we demonstrate the merits of our proposed compression algorithm in comparison with other recently proposed lossless compression algorithms for WSNs.

The Data Compression Theorem for Ergodic Quantum Information Sources

We extend the data compression theorem to the case of ergodic quantum information sources. Moreover, we provide an asymptotically optimal compression scheme which is based on the concept of high probability subspaces. The rate of this compression scheme is equal to the von Neumann entropy rate.

Thouless-Anderson-Palmer approach for lossy compression.

We study an ill-posed linear inverse problem, where a binary sequence will be reproduced using a sparse matrix. According to the previous study, this model can theoretically provide an optimal compression scheme for an arbitrary distortion level, though the encoding procedure remains an NP-complete problem. In this paper, we focus on the consistency condition for a dynamics model of Markov-type to derive an iterative algorithm, following the steps of Thouless-Anderson-Palmer approach. Numerical results show that the algorithm can empirically saturate the theoretical limit for the sparse construction of our codes, which is also very close to the rate-distortion function.

The asymptotics of string matching probabilities for Gaussian random sequences

AbstractWyner and Ziv (1989) studied the asymptotic properties of recurrence times of stationary ergodic processes, and applied the results to obtain optimal data compression schemes in information transmission. Since then many data compression algorithms based upon string matching of a sequence from an information source with a database have been proposed and studied. In this paper we consider Gaussian stationary processes representing an information source and a database, and study problems of string matching with distortion. We prove theorems concerning the asymptotic behavior of the probability of string matching with distortion and the waiting time for the string matching.