Compression Tools Research Articles

IMPROVED COMPRESSION OF XML FILES FOR FAST IMAGE TRANSMISSION

ICTACT Journals are peer reviewed International Journals published quarterly. With the objective of creating Indian-based Journals of International quality and to build up local research content, ICTACT Journals exists for the purpose of providing a quality peer-reviewed platform for Scientists, Researchers, Academicians and Engineers who carry out their original research. The Journals will highlight the continued growth and new challenges in both basic research and application development.

A novel compression tool for efficient storage of genome resequencing data

With the advent of DNA sequencing technologies, more and more reference genome sequences are available for many organisms. Analyzing sequence variation and understanding its biological importance are becoming a major research aim. However, how to store and process the huge amount of eukaryotic genome data, such as those of the human, mouse and rice, has become a challenge to biologists. Currently available bioinformatics tools used to compress genome sequence data have some limitations, such as the requirement of the reference single nucleotide polymorphisms (SNPs) map and information on deletions and insertions. Here, we present a novel compression tool for storing and analyzing Genome ReSequencing data, named GRS. GRS is able to process the genome sequence data without the use of the reference SNPs and other sequence variation information and automatically rebuild the individual genome sequence data using the reference genome sequence. When its performance was tested on the first Korean personal genome sequence data set, GRS was able to achieve ∼159-fold compression, reducing the size of the data from 2986.8 to 18.8 MB. While being tested against the sequencing data from rice and Arabidopsis thaliana, GRS compressed the 361.0 MB rice genome data to 4.4 MB, and the A. thaliana genome data from 115.1 MB to 6.5 KB. This de novo compression tool is available at http://gmdd.shgmo.org/Computational-Biology/GRS.

A NOVEL APPROACH FOR IMAGE COMPRESSION USING LIFTING TECHNIQUE AND TSVQ

The objective of image compression is to reduce the redundant amount of data and to achieve low bit rate without any apparent loss of image quality. In this paper, the compression process is achieved by using wavelet transform with lifting technique for decomposition and tree-structured vector scheme for quantization. Huffman coding method is used for encoding stage. Wavelet technique provides the most promising tool for high-quality image compression. Lifting transform provides a flexible tool for the construction of sub-band decomposition and perfect reconstruction. TSVQ has its advantages over conventional vector quantization by minimizing the computational complexity. Huffman coding is used for efficient coding of images at lower bit rate with minimal loss of information. The performance of the proposed work has been assessed with parameters like bits per pixel (bpp), compression ratio (CR), peak signal to noise ratio (PSNR), codebook size (CS), and mean squared error (MSE). The experimental results show that the proposed work yields a higher compression with better quality reconstructed image than the other conventional compression methods.

A HIGH PERFORMANCE 3D EXACT EUCLIDEAN DISTANCE TRANSFORM ALGORITHM FOR DISTRIBUTED COMPUTING

The Euclidean distance transform (EDT) is used in various methods in pattern recognition, computer vision, image analysis, physics, applied mathematics and robotics. Until now, several sequential EDT algorithms have been described in the literature, however they are time- and memory-consuming for images with large resolutions. Therefore, parallel implementations of the EDT are required specially for 3D images. This paper presents a parallel implementation based on domain decomposition of a well-known 3D Euclidean distance transform algorithm, and analyzes its performance on a cluster of workstations. The use of a data compression tool to reduce communication time is investigated and discussed. Among the obtained performance results, this work shows that data compression is an essential tool for clusters with low-bandwidth networks.

GENBIT COMPRESS TOOL(GBC): A JAVA-BASED TOOL TO COMPRESS DNA SEQUENCES AND COMPUTE COMPRESSION RATIO(BITS/BASE) OF GENOMES

We present a Compression Tool, "GenBit Compress", for genetic sequences based on our new proposed "GenBit Compress Algorithm". Our Tool achieves the best compression ratios for Entire Genome (DNA sequences) . Significantly better compression results show that GenBit compress algorithm is the best among the remaining Genome compression algorithms for non-repetitive DNA sequences in Genomes. The standard Compression algorithms such as gzip or compress cannot compress DNA sequences but only expand them in size. In this paper we consider the problem of DNA compression. It is well known that one of the main features of DNA Sequences is that they contain substrings which are duplicated except for a few random Mutations. For this reason most DNA compressors work by searching and encoding approximate repeats. We depart from this strategy by searching and encoding only exact repeats. our proposed algorithm achieves the best compression ratio for DNA sequences for larger genome. As long as 8 lakh characters can be given as input While achieving the best compression ratios for DNA sequences, our new GenBit Compress program significantly improves the running time of all previous DNA compressors. Assigning binary bits for fragments of DNA sequence is also a unique concept introduced in this program for the first time in DNA compression.

Fiber hartley transform and optical indirect computation of discrete cosine transform

We present a recursive approach to implement the Discrete Hartley Transform in the optical domain by using a passive fiber network of asymmetric couplers, without phase shifters. The optical implementation of Discrete Cosine Transform is also derived, resulting in effective and flexible tools for optical data compression and signal processing.

Parallel 3D fast wavelet transform on manycore GPUs and multicore CPUs

Abstract GPUs have recently attracted our attention as accelerators on a wide variety of algorithms, including assorted examples within the image analysis field. Among them, wavelets are gaining popularity as solid tools for data mining and video compression, though this comes at the expense of a high computational cost. After proving the effectiveness of the GPU for accelerating the 2D Fast Wavelet Transform [1] , we present in this paper a novel implementation on manycore GPUs and multicore CPUs for a high performance computation of the 3D Fast Wavelet Transform (3DFWT). This algorithm poses a challenging access pattern on matrix operators demanding high sustainable bandwidth, as well as mathematical functions with remarkable arithmetic intensity on ALUs and FPUs. On the GPU side, we focus on CUDA programming to develop methods for an efficient mapping on manycores and to fully exploit the memory hierarchy, whose management is explicit by the programmer. On multicore CPUs, OpenMP and Pthreads are used as counterparts to maximize parallelism, and renowned techniques like tiling and blocking are exploited to optimize the use of memory. Experimental results on an Nvidia Tesla C870 GPU and an Intel Core 2 Quad Q6700 CPU indicate that our implementation runs three times faster on the Tesla and up to fifteen times faster when communications are neglected, which enables the GPU for processing real-time videos in many applications where the 3D-FWT is involved.

Determination of an acceptable level of spectral data compression by Discrete Wavelet Transforms

Today, due to the ever increasing amount of data generated by analytical instruments, good compression methods are desired to keep computation time acceptable. The lower the volume and noise content of data, the easier it becomes to investigate and interpret the modeling results. Discrete Wavelet Transform (DWT) is an effective data compression and noise suppression tool. Compression can be performed at different levels, in each, the size of signal part of the data reduces to half the size. This work includes an approach for determining an acceptable level of compression of data where the aim is to achieve minimal loss of information and no significant change in the structure of data, which could mean, e.g. no loss in predictive ability or the effective rank of the data-set. The method is based on estimation of the Singular Values (SVs) from a data matrix and the Singular Values at each level of compression followed by the application of Median Absolute Deviation (MAD) of the correlation between original SVs and compression SVs as a simple statistical test for the determination of the optimum level of compression. We illustrate the method using FT-Raman data from aqueous solutions of three sugars (glucose, trehalose and sucrose) and NMR data from mixtures of three alcohols. A sudden change in prediction error sum of square plots from Partial Least Squares (PLS) modeling confirms the results from MAD statistics.

Lossless compression of ATLAS Tile Calorimeter raw data

Recording and storing the Tile Calorimeter data at 100 KHz frequency is an important task in ATLAS experiment processing. At this moment Amplitude, Time and Quality Factor (QF) parameters are calculated using Optimal Filtering Reconstruction method. If QF is considered good enough, these three parameters are only stored, otherwise the data quality is considered bad and it is proposed to store raw data for further offline analysis. Without any compression, bandwidth limitation allows to send up to 9 channels of additional raw data. Simple considerations show that when QF is bad due to the shape differences between standard pulse shape and current signal (e.g. when several signals overlap), all channels are likely to report bad QF while the contained data may still be valuable. So, the possibility to save just 9 samples is insufficient and we have to compress the data. Experiments show that standard compression tools such as RAR, ZIP, etc. cannot successfully deal with this problem because they cannot take benefit of smooth curved shape of the raw data and correlations between the channels. In the present paper a lossless data compressing algorithm is proposed which is likely to better meet existing challenges. This method has been checked on SPLASH events (run 87851, contains 26 SPLASH events) and proved to be sufficient to save ALL channels data using the existing bandwidth. Unlike the common purpose compressing tools the proposed method exploits heavily the geometry-dependent correlations between different channels. It is important to note that the method relies on the only assumption that the registered signal shape is smooth enough and it does not require exact information about the standard pulse shape function to compress the data. Thus this method can be applied for recording pilled-up or unexpected signals as well.

A bit-level text compression scheme based on the ACW algorithm

This paper presents a description and performance evaluation of a new bit-level, lossless, adaptive, and asymmetric data compression scheme that is based on the adaptive character wordlength (ACW(n)) algorithm. The proposed scheme enhances the compression ratio of the ACW(n) algorithm by dividing the binary sequence into a number of subsequences (s), each of them satisfying the condition that the number of decimal values (d) of the n-bit length characters is equal to or less than 256. Therefore, the new scheme is referred to as ACW(n, s), where n is the adaptive character wordlength and s is the number of subsequences. The new scheme was used to compress a number of text files from standard corpora. The obtained results demonstrate that the ACW(n, s) scheme achieves higher compression ratio than many widely used compression algorithms and it achieves a competitive performance compared to state-of-the-art compression tools.

BeadDatapackR: A Tool to Facilitate the Sharing of Raw Data from Illumina BeadArray Studies

Microarray technologies have been an increasingly important tool in cancer research in the last decade, and a number of initiatives have sought to stress the importance of the provision and sharing of raw microarray data. Illumina BeadArrays provide a particular problem in this regard, as their random construction simultaneously adds value to analysis of the raw data and obstructs the sharing of those data. We present a compression scheme for raw Illumina BeadArray data, designed to ease the burdens of sharing and storing such data, that is implemented in the BeadDataPackR BioConductor package ( http://bioconductor.org/packages/release/bioc/html/BeadDataPackR.html ). It offers two key advantages over off-the-peg compression tools. First it uses knowledge of the data formats to achieve greater compression than other approaches, and second it does not need to be decompressed for analysis, but rather the values held within can be directly accessed.

Strain Imaging of the Breast by Compression Microwave Imaging

A method that uses microwave pulses to achieve strain imaging of the breast is presented. In the proposed method, the breast is inserted in an enclosure that defines the boundary conditions for the breast deformation under the influence of external pressure. The upper plate of the enclosure, which also includes an antenna array, is attached to a compression tool, whereas the lower and the front plates are fixed. The breast is allowed to extend in the lateral direction when pressed by the top plate. Each of the antennas at the top plate is used to send an ultrawideband pulse to penetrate the breast and measure the backscattered pulse. Two sets of measurements are taken: one pre- and another post-compression of the breast. A sliding window of cross correlation is then performed on the two sets of data to establish the time delay of each segment of the scattered pulse due to compression. That time delay is then employed to get a three-dimensional strain image of the breast. As lesion tissue is typically much stiffer than normal breast tissue, then regions of low or zero strain indicate areas in need of further diagnostic checks. Full-wave simulations are used to validate the presented imaging method.

Empirical Orthogonal Functions: The Medium is the Message

Abstract Empirical orthogonal function (EOF) analysis is a powerful tool for data compression and dimensionality reduction used broadly in meteorology and oceanography. Often in the literature, EOF modes are interpreted individually, independent of other modes. In fact, it can be shown that no such attribution can generally be made. This review demonstrates that in general individual EOF modes (i) will not correspond to individual dynamical modes, (ii) will not correspond to individual kinematic degrees of freedom, (iii) will not be statistically independent of other EOF modes, and (iv) will be strongly influenced by the nonlocal requirement that modes maximize variance over the entire domain. The goal of this review is not to argue against the use of EOF analysis in meteorology and oceanography; rather, it is to demonstrate the care that must be taken in the interpretation of individual modes in order to distinguish the medium from the message.

3D Model compression using Connectivity-Guided Adaptive Wavelet Transform built into 2D SPIHT

Connectivity-Guided Adaptive Wavelet Transform based mesh compression framework is proposed. The transformation uses the connectivity information of the 3D model to exploit the inter-pixel correlations. Orthographic projection is used for converting the 3D mesh into a 2D image-like representation. The proposed conversion method does not change the connectivity among the vertices of the 3D model. There is a correlation between the pixels of the composed image due to the connectivity of the 3D mesh. The proposed wavelet transform uses an adaptive predictor that exploits the connectivity information of the 3D model. Known image compression tools cannot take advantage of the correlations between the samples. The wavelet transformed data is then encoded using a zero-tree wavelet based method. Since the encoder creates a hierarchical bitstream, the proposed technique is a progressive mesh compression technique. Experimental results show that the proposed method has a better rate distortion performance than MPEG-3DGC/MPEG-4 mesh coder.

Motion-compensated temporal filtering with optimized motion estimation

The lifting-based discrete wavelet transforms applied temporally along the motion trajectory is a powerful tool in scalable video compression. The longer filter is used for motion-compensated temporal filtering (MCTF) for better de-correlation. The bidirectional motion estimation (ME) used for MCTF is the major speed bottleneck. This paper propose two stages motion predictive technique to optimize the ME. In the first stage, zero motion block prediction is carried out using the content of high frequency frame, obtained after TF without motion compensation. In the second stage, the inter-frame and inter-layer prediction technique is applied to skip the coarse search of any fast ME, except for the first pair of frames in a group of pictures. The proposed algorithm is implemented in MC-EZBC. The variations observed in objective or subjective quality of the decoded video is negligible compared to the speed gained.

An Alternate Way to Develop Lossless Graphical Data Compression Package using Non-Linear Single Cycle Multiple Attractor Cellular Automata

Compression is one of the major topics in the field of research as it is used for storing data in a lesser storage space of the repository. Compression is furthermore required in case of sending data from source to destination through network (internet or intranet). The time required is less in case of sending fewer amounts of data. So, compression is a necessity for transmission of original huge data as a modified small amount of data with a low overhead. Image compression is the application of data compression on digital images. This research work takes into consideration an image as the source file (.bmp) and converts it to the standard PNM format and further in plain PNM format (raw file) to utilize the binary format specification used in PNM. Single Cycle Multiple Attractor Cellular Automata (SMACA) is used as a compression tool. The results achieved through experimentation are based on lossless image compression. The first pass compresses the PNM image using SMACA based techniques followed by the second pass which utilizes Huffman variable length encoding scheme. Initially, 31% (approx.) & 95% (approx.) compression ratio are achieved using SMACA structure on the PNM & raw files respectively. Final compressed data of the image obtained achieves a further compression by 6% over the previous compressed data using the Huffman algorithm.

Reduced complexity intra mode decision for resolution reduction on h.264/AVC transcoders

Intra frame prediction is one of the most compression efficient tools of the H.264/AVC codec over its predecessor. In video transcoding for picture size reduction, pattern directionality of the output frames may differ from the input, affecting the H.264/AVC intra prediction modes. This paper proposes two simple algorithms for efficient mode decision by exploiting the input modes. To further improve the conversion performance two refinement methods are also proposed. One is to look at the neighboring of the selected modes and the other to look at horizontal, vertical and DC modes, as the most frequently used modes. Compared with the full rate-distortion mode decision, the proposed algorithms achieve more than 50% complexity reduction while showing a negligible PSNR penalty of only 0.1 dB and an excess bitrate of 1-6%.

XML compression techniques: A survey and comparison

XML has been acknowledged as the defacto standard for data representation and exchange over the World Wide Web. Being self describing grants XML its great flexibility and wide acceptance but on the other hand it is the cause of its main drawback that of being huge in size. The huge document size means that the amount of information that has to be transmitted, processed, stored, and queried is often larger than that of other data formats. Several XML compression techniques has been introduced to deal with these problems. In this paper, we provide a complete survey over the state-of-the-art of XML compression techniques. In addition, we present an extensive experimental study of the available implementations of these techniques. We report the behavior of nine XML compressors using a large corpus of XML documents which covers the different natures and scales of XML documents. In addition to assessing and comparing the performance characteristics of the evaluated XML compression tools, the study also tries to assess the effectiveness and practicality of using these tools in the real world. Finally, we provide some guidelines and recommendations which are useful for helping developers and users for making an effective decision towards selecting the most suitable XML compression tool for their needs.

Fitting solid meshes to animated surfaces using linear elasticity

Computing correspondence between time frames of a time-dependent 3D surface is essential for the understanding of its motion and deformation. In particular, it can be a useful tool in compression, editing, texturing, or analysis of the physical or structural properties of deforming objects. However, correspondence information is not trivial to obtain for experimentally acquired 3D animations, such as time-dependent visual hulls (typically represented as either a binary occupancy grid or as a sequence of meshes of varying connectivity). In this article we present a new nonrigid fitting method that can compute such correspondence information for objects that do not undergo large volume or topological changes, such as living creatures. Experimental results show that it is robust enough to handle visual hull data, allowing to convert it into a constant connectivity mesh with vertices moving in time. Our procedure first creates a rest-state mesh from one of the input frames. This rest-state mesh is then fitted to the consecutive frames. We do this by iteratively displacing its vertices so that a combination of surface distance and elastic potential energy is minimized. A novel rotation compensation method enables us to obtain high-quality results with linear elasticity, even in presence of significant bending.

Modeling of Pattern-Based Block Motion Estimation and Its Application

Pattern-based block motion estimation (PBME) is one of the most widely adopted compression tools in the contemporary video coding systems. However, despite that many researches have studied PBME, few have yet attempted to construct an analytical model that can explain the underneath principle and mechanism of various PBME algorithms. In this paper, we propose a statistical PBME model that consists of two components: 1) a statistical probability distribution for motion vectors and 2) the minimal number of search points (so-called weighting function) achieved by a search algorithm. We first verify the accuracy of the proposed model by checking the experimental data. Then, an application example using this model is shown. Starting from an ideal weighting function, we devise a novel genetic rhombus pattern search (GRPS) to match the design target. Simulations show that, comparing to the other popular search algorithms, GRPS reduces the average search points for more than 20% and, in the meanwhile, it maintains a similar level of coded image quality.