Lossy Compression Algorithm Research Articles

Reduced basis decomposition: A certified and fast lossy data compression algorithm

Dimension reduction is often needed in the area of data mining. The goal of these methods is to map the given high-dimensional data into a low-dimensional space preserving certain properties of the initial data. There are two kinds of techniques for this purpose. The first, projective methods, builds an explicit linear projection from the high-dimensional space to the low-dimensional one. On the other hand, the nonlinear methods utilizes nonlinear and implicit mapping between the two spaces. In both cases, the methods considered in literature have usually relied on computationally very intensive matrix factorizations, frequently the Singular Value Decomposition (SVD). The computational burden of SVD quickly renders these dimension reduction methods infeasible thanks to the ever-increasing sizes of the practical datasets.In this paper, we present a new decomposition strategy, Reduced Basis Decomposition (RBD), which is inspired by the Reduced Basis Method (RBM). Given X the high-dimensional data, the method approximates it by YT(≈X) with Y being the low-dimensional surrogate and T the transformation matrix. Y is obtained through a greedy algorithm thus extremely efficient. In fact, it is significantly faster than SVD with comparable accuracy. T can be computed on the fly. Moreover, unlike many compression algorithms, it easily finds the mapping for an arbitrary “out-of-sample” vector and it comes with an “error indicator” certifying the accuracy of the compression. Numerical results are shown validating these claims.

Embedded Zero Tree Wavelet based Artificial Neural Network Image Classification Algorithm - A Study

In this work, an urban area land cover is proposed to classify the large resolution image. It aims to extract the features like texture, shape, size and spectral information in the feature extraction process. Embedded Zero tree Wavelet transform is a lossy image compression algorithm. Most of the coefficients at low bit rates bent through a sub band transform will be zero, or very close to zero. These features data are used for the classification process. Here, we used various classification algorithms namely, Radial Basis Function, SMO, Multilayer Perceptron and Random Forest are implemented. The classification accuracy constantly depends on the efficiency of the extracted features and classification algorithms. The result of the proposed classification algorithms are merged with EZW. Experimental results illustrate that the better accuracy performance is obtained by the Multilayer Perceptron algorithm than other classification algorithms.

An improved medical image compression technique with lossless region of interest

Hospitals and medical centers produce an enormous amount of digital medical images every day, which are used for different purposes such as surgical and diagnostic plans. The ease of storing and transmission of digital medical images is a boon to patients and medical professionals. Due to the large volume of images, image compression is required to reduce the redundancies in image and represents it in shorter manner for efficient archiving and transmission of images. However, compressing digital medical images as the region of interest for diagnosis is generally small when compared to the whole image. Lossless compression techniques compress without loss of any information but have low compression rate, and lossy compression techniques can compress at high compression ratio but with a slight loss of data. Using lossless techniques in medical image does not give enough advantage in transmission and storage and lossy techniques may lose crucial data required for diagnosis. In this paper, an improved medical image compression technique based on region of interest (ROI) is proposed to maximize compression. The image is firstly divided into two parts: ROI regions and non-ROI regions. Lossless compression algorithm is then applied to the marked area of ROI, and image restoration technique and the wavelet-based lossy compression algorithm are utilized to the other area of the image. Finally, a set of experiments is designed to assess the effectiveness of the proposed compression method.

Using off-the-shelf lossy compression for wireless home sleep staging

BackgroundRecently, there has been increasing interest in the development of wireless home sleep staging systems that allow the patient to be monitored remotely while remaining in the comfort of their home. However, transmitting large amount of Polysomnography (PSG) data over the Internet is an important issue needed to be considered. In this work, we aim to reduce the amount of PSG data which has to be transmitted or stored, while having as little impact as possible on the information in the signal relevant to classify sleep stages. New methodWe examine the effects of off-the-shelf lossy compression on an all-night PSG dataset from 20 healthy subjects, in the context of automated sleep staging. The popular compression method Set Partitioning in Hierarchical Trees (SPIHT) was used, and a range of compression levels was selected in order to compress the signals with various degrees of loss. In addition, a rule-based automatic sleep staging method was used to automatically classify the sleep stages. ResultsConsidering the criteria of clinical usefulness, the experimental results show that the system can achieve more than 60% energy saving with a high accuracy (>84%) in classifying sleep stages by using a lossy compression algorithm like SPIHT. Comparison with existing method(s)As far as we know, our study is the first that focuses how much loss can be tolerated in compressing complex multi-channel PSG data for sleep analysis. ConclusionsWe demonstrate the feasibility of using lossy SPIHT compression for wireless home sleep staging.

SimpleTrack: Adaptive Trajectory Compression With Deterministic Projection Matrix for Mobile Sensor Networks

Some mobile sensor network applications require the sensor nodes to transfer their trajectories to a data sink. This paper proposes an adaptive trajectory (lossy) compression algorithm based on compressive sensing. The algorithm has two innovative elements. First, we propose a method to compute a deterministic projection matrix from a learnt dictionary. Second, we propose a method for the mobile nodes to adaptively predict the number of projections needed based on the speed of the mobile nodes. Extensive evaluation of the proposed algorithm using 6 datasets shows that our proposed algorithm can achieve sub-metre accuracy. In addition, our method of computing projection matrices outperforms two existing methods. Finally, comparison of our algorithm against a state-of-the-art trajectory compression algorithm show that our algorithm can reduce the error by 10-60 cm for the same compression ratio.

Building a better mousetrap: Compressing mouse cursor activity for web analytics

Websites can learn what their users do on their pages to provide better content and services to those users. A website can easily find out where a user has been, but in order to find out what content is consumed and how it was consumed at a sub-page level, prior work has proposed client-side tracking to record cursor activity, which is useful for computing the relevance for search results or determining user attention on a page. While recording cursor interactions can be done without disturbing the user, the overhead of recording the cursor trail and transmitting this data over the network can be substantial. In our work, we investigate methods to compress cursor data, taking advantage of the fact that not every cursor coordinate has equal value to the website developer. We evaluate 5 lossless and 5 lossy compression algorithms over two datasets, reporting results about client-side performance, space savings, and how well a lossy algorithm can replicate the original cursor trail. The results show that different compression techniques may be suitable for different goals: LZW offers reasonable lossless compression, but lossy algorithms such as piecewise linear interpolation and distance-thresholding offer better client-side performance and bandwidth reduction.

The impact of JPEG2000 lossy compression on the scientific quality of radio astronomy imagery

The sheer volume of data anticipated to be captured by future radio telescopes, such as, the Square Kilometer Array (SKA) and its precursors present new data challenges, including the cost and technical feasibility of data transport and storage. Image and data compression are going to be important techniques to reduce the data size. We provide a quantitative analysis of the effects of JPEG2000’s lossy wavelet image compression algorithm on the quality of the radio astronomy imagery data. This analysis is completed by evaluating the completeness, soundness and source parameterisation of the Duchamp source finder using compressed data. Here we found the JPEG2000 image compression has the potential to denoise image cubes, however this effect is only significant at high compression rates where the accuracy of source parameterisation is decreased.

Data Compression for the Exascale Computing Era - Survey

While periodic checkpointing has been an important mechanism for tolerating faults in high performance computing HPC systems, it is cost-prohibitive as the HPC system approaches exascale. Applying compression techniques is one common way to mitigate such burdens by reducing the data size, but they are often found to be less effective for scientific datasets. Traditional lossless compression techniques that look for repeated patterns are ineffective for scientific data in which high-precision data is used and hence common patterns are rare to find. In this paper, we present a comparison of several lossless and lossy data compression algorithms and discuss their methodology under the exascale environment. As data volume increases, we discover an increasing trend of new domain-driven algorithms that exploit the inherent characteristics exhibited in many scientific dataset, such as relatively small changes in data values from one simulation iteration to the next or among neighboring data. In particular, significant data reduction has been observed in lossy compression. This paper also discusses how the errors introduced by lossy compressions are controlled and the tradeoffs with the compression ratio.

Multidimensional color image storage, retrieval, and compression based on quantum amplitudes and phases

In this study, we propose a new representation method for multidimensional color images, called an n-qubit normal arbitrary superposition state (NASS), where n qubits represent the colors and coordinates of 2n pixels (e.g., a three-dimensional color image of 1024×1024×1024 using only 30 qubits). Based on NASS, we present an (n+1)-qubit normal arbitrary superposition state with relative phases (NASSRP) and an (n+2)-qubit normal arbitrary superposition state with three components (NASSTC) for lossless and lossy quantum compression, respectively. We also design three general quantum circuits to generate NASS, NASSRP, and NASSTC states, where we retrieve an image from a quantum system using different projection measurement operators. Finally, we define the quantum compression ratio and analyze lossless and lossy quantum compression algorithms of multidimensional quantum images. For the first time, we implemented the compression of multidimensional color images on a quantum computer. Thus, we address the theoretical and practical aspects of image processing on a quantum computer.

Speech Compression Based on Discrete Walsh Hadamard Transform

This paper presents a new lossy compression algorithm for stationary signal based on Discrete Walsh Hadamard Transform (DWHT). The principle of compression algorithm consists in framing the original speech signal into stationary frames and applying the DWHT. Then, the obtained coefficients are thresholded in order to truncate all coefficients below a given thresholds values. Compression is achieved by efficient encoding of the string values of zeros. A comparative study of performance between the algorithms based on DWHT and Discrete Wavelet Transform (DWT) is performed in terms of some objective criteria: compression ratio (CR), signal to noise ratio, peak signal to noise ratio (SNR), normalized root mean square error (NRMSE) and CPU time. The simulation results show that the algorithm based on DWHT is characterized by a very low complexity implementation and improved CR, SNR, PSNR and NRMSE compared to the DWT algorithm and this for stationary frame.

QualComp: a new lossy compressor for quality scores based on rate distortion theory

BackgroundNext Generation Sequencing technologies have revolutionized many fields in biology by reducing the time and cost required for sequencing. As a result, large amounts of sequencing data are being generated. A typical sequencing data file may occupy tens or even hundreds of gigabytes of disk space, prohibitively large for many users. This data consists of both the nucleotide sequences and per-base quality scores that indicate the level of confidence in the readout of these sequences. Quality scores account for about half of the required disk space in the commonly used FASTQ format (before compression), and therefore the compression of the quality scores can significantly reduce storage requirements and speed up analysis and transmission of sequencing data.ResultsIn this paper, we present a new scheme for the lossy compression of the quality scores, to address the problem of storage. Our framework allows the user to specify the rate (bits per quality score) prior to compression, independent of the data to be compressed. Our algorithm can work at any rate, unlike other lossy compression algorithms. We envisage our algorithm as being part of a more general compression scheme that works with the entire FASTQ file. Numerical experiments show that we can achieve a better mean squared error (MSE) for small rates (bits per quality score) than other lossy compression schemes. For the organism PhiX, whose assembled genome is known and assumed to be correct, we show that it is possible to achieve a significant reduction in size with little compromise in performance on downstream applications (e.g., alignment).ConclusionsQualComp is an open source software package, written in C and freely available for download at https://sourceforge.net/projects/qualcomp.

Remote technologies in Education. Compression algorithms and data formats for transmission of text, sound and video information

The article describes the features of the application of compression algorithms of information in multimedia learning technologies. There are given the examples of lossy compression algorithms and lossless compression and basic formats of compression of text, graphics and audiovisual information. The analysis of the effectiveness of various algorithms and recommendations for their use in different types of information are presented. The example of Huffman coding is given.

WDCT-based Image Coding Algorithm with Down-Sampling and APDCT Interpolation

Abstract The DCT-based JPEG standard is the most commonly used lossy compression algorithm for still continuous-tone images due to its high effectiveness and low computational complexity. However, the visually disturbing blocking artifacts generate when image is compressed at low bit rates. With the frequency warping, the warped discrete cosine transform (WDCT) makes the signal energy distribution more suitable for image coding. In order to improve the coding performance at low bit rates, this paper presents an image coding frame based on down-sampling and all phase DCT (APDCT) interpolation. Experimental results show that, no matter for PSNR or subjective result, the proposed method obviously improves the performance of image coding at low bit rates, and reduces the blocking artifacts significantly.

Induced Redundancy based Lossy Data Compression Algorithm

A Wireless Sensor Network (WSN) is an increasingly important mechanism for enabling continuous monitoring and sensing of physical variables like temperature, humidity etc. The tiny sensor nodes are powered by low capacity batteries. As the WSNs are usually deployed in remote areas, battery replacement becomes difficult. To minimize the power consumption in WSN, the data compression schemes play a vital role. If applied appropriately, these data compression schemes can result in drastic increase in the lifetime of the network. In this paper, we present a scheme called Induced Redundancy based Lossy Data Compression Algorithm, (IRLDCA), which is best suited for WSNs that sense data with higher correlation. Our algorithm induces certain amount of redundancy into the data set to achieve more effective data compression and also gives the user a flexibility to control the compression ratio and loss of data. Simulation results prove the effectiveness of the proposed scheme over the existing ones. General Terms Data Compression, Optimization, Redundancy.

Lossy data compression of vibrotactile material-like textures.

Tactile content will be delivered over the Internet in the near future. Vibrotactile material-like textures that resemble the surfaces of wood, leather, etc., are representative of such content. We performed lossy compression of texture data for reducing the data size. We confirmed the effectiveness of two compression strategies: quantization and truncation of data beneath a shifted perceptual threshold curve. In the quantization strategy, the amplitude spectra of vibrotactile textures could be quantized in 14 steps. This reduced the data size to approximately one quarter without any noticeable quality deterioration. The method for truncating frequency components with amplitudes smaller than a shifted perceptual threshold curve was also effective, and it was preferable to the automatic deletion of subthreshold amplitudes. We reduced the data size of vibrotactile material textures to 10-20 percent of their original size by combining the lossy data compression strategy with Huffman coding, which is a lossless data compression method. Lossy compression algorithms will enhance the online delivery of vibrotactile material-like textures by decreasing their data size without significant loss of quality.

Complexity-compression tradeoffs in lossy compression via efficient random codebooks and databases

The compression-complexity trade-off of lossy compression algorithms that are based on a random codebook or a random database is examined. Motivated, in part, by recent results of Gupta-Verdu-Weissman (GVW) and their underlying connections with the pattern-matching scheme of Kontoyiannis' lossy Lempel-Ziv algorithm, we introduce a nonuniversal version of the lossy Lempel-Ziv method (termed LLZ). The optimality of LLZ for memory-less sources is established, and its performance is compared to that of the GVW divide-and-conquer approach. Experimental results indicate that the GVW approach often yields better compression than LLZ, but at the price of much higher memory requirements. To combine the advantages of both, we introduce a hybrid algorithm (HYB) that utilizes both the divide-and-conquer idea of GVW and the single-database structure of LLZ. It is proved that HYB shares with GVW the exact same rate-distortion performance and implementation complexity, while, like LLZ, requiring less memory, by a factor which may become unbounded, depending on the choice of the relevant design parameters. Experimental results are also presented, illustrating the performance of all three methods on data generated by simple discrete memory-less sources. In particular, the HYB algorithm is shown to outperform existing schemes for the compression of some simple discrete sources with respect to the Hamming distortion criterion.

Denoising via MCMC-Based Lossy Compression

It has been established in the literature, in various theoretical and asymptotic senses, that universal lossy compression followed by some simple postprocessing results in universal denoising, for the setting of a stationary ergodic source corrupted by additive white noise. However, this interesting theoretical result has not yet been tested in practice in denoising simulated or real data. In this paper, we employ a recently developed MCMC-based universal lossy compressor to build a universal compression-based denoising algorithm. We show that applying this iterative lossy compression algorithm with appropriately chosen distortion measure and distortion level, followed by a simple derandomization operation, results in a family of denoisers that compares favorably (both theoretically and in practice) with other MCMC-based schemes, and with the discrete universal denoiser DUDE.

An Enhanced Resolution Three-Dimensional Transformation Method Based on Discrete Wavelet Transform

With the development of interactive multimedia technologies, image and video compression algorithms necessitated a number of better performance and functionality. Wavelet transform based embedded image coding method is the basis of JPEG2000. Lossy image compression algorithms sacrifice perfect image reconstruction in favor of decreased storage requirements. JPEG2000 algorithm has been developed based on the discrete wavelet transform (DWT) techniques, which have shown how the results achieved in different areas in information technology can be applied to enhance the performance.

Enhanced Three-Dimensional Color Conversion Based on Discrete Wavelet Transform

JPEG2000 algorithm has been developed based on the DWT techniques, which have shown how the results achieved in different areas in information technology can be applied to enhance the performance. Lossy image compression algorithms sacrifice perfect image reconstruction in favor of decreased storage requirements. Wavelets have become a popular technology for information redistribution for high-performance image compression algorithms. Lossy compression algorithms sacrifice perfect image reconstruction in favor of improved compression rates while minimizing image quality lossy.

State Trajectory Compression for Optimal Control with Parabolic PDEs

In optimal control problems with nonlinear time-dependent three-dimensional (3D) PDEs, full four-dimensional (4D) discretizations are usually prohibitive due to the storage requirement. For this reason gradient- and Newton-type methods working on the reduced functional are often employed. The computation of the reduced gradient requires one solve of the state equation forward in time and one backward solve of the adjoint equation. The state enters into the adjoint equation, again requiring the storage of a full 4D data set. We propose to use lossy compression algorithms, using an inexact but cheap predictor for the state data, with additional entropy coding of prediction errors. As the data is used inside a discretized, iterative algorithm, lossy compression maintaining a certain error bound turns out to be sufficient. We provide an error analysis, derive expected compression rates, and present numerical examples validating the results.