Parallel Decompression Research Articles

OpenCL을 사용한 Intel FPGA SoC 기반 실시간 병렬 다상분해 주파수 대역압축 및 복원 알고리즘

OpenCL을 사용한 Intel FPGA SoC 기반 실시간 병렬 다상분해 주파수 대역압축 및 복원 알고리즘

Parallel compression/decompression-based datapath architecture for multibeam mask writers

Multibeam electron beam systems will be used in the future for mask writing and for complementary lithography. The major challenges of the multibeam systems are in meeting throughput requirements and in handling the large data volumes associated with writing grayscale data on the wafer. In terms of future communications and computational requirements, Amdahl’s law suggests that a simple increase of computation power and parallelism may not be a sustainable solution. We propose a parallel data compression algorithm to exploit the sparsity of mask data and a grayscale video-like representation of data. To improve the communication and computational efficiency of these systems at the write time, we propose an alternate datapath architecture partly motivated by multibeam direct-write lithography and partly motivated by the circuit testing literature, where parallel decompression reduces clock cycles. We explain a deflection plate architecture inspired by NuFlare Technology’s multibeam mask writing system and how our datapath architecture can be easily added to it to improve performance.

2D-RBUC for efficient parallel compression of residuals

In this paper, we present a method for lossless compression of residuals with an efficient SIMD parallel decompression. The residuals originate from lossy or near lossless compression of height fields, which are commonly used to represent models of terrains. The algorithm is founded on the existing RBUC method for compression of non-uniform data sources. We have adapted the method to capture 2D spatial locality of height fields, and developed the data decompression algorithm for modern GPU architectures already present even in home computers. In combination with the point-level SIMD-parallel lossless/lossy high field compression method HFPaC, characterized by fast progressive decompression and seamlessly reconstructed surface, the newly proposed method trades off small efficiency degradation for a non negligible compression ratio (measured up to 91%) benefit.

ViraPipe: scalable parallel pipeline for viral metagenome analysis from next generation sequencing reads.

Next Generation Sequencing (NGS) technology enables identification of microbial genomes from massive amount of human microbiomes more rapidly and cheaper than ever before. However, the traditional sequential genome analysis algorithms, tools, and platforms are inefficient for performing large-scale metagenomic studies on ever-growing sample data volumes. Currently, there is an urgent need for scalable analysis pipelines that enable harnessing all the power of parallel computation in computing clusters and in cloud computing environments. We propose ViraPipe, a scalable metagenome analysis pipeline that is able to analyze thousands of human microbiomes in parallel in tolerable time. The pipeline is tuned for analyzing viral metagenomes and the software is applicable for other metagenomic analyses as well. ViraPipe integrates parallel BWA-MEM read aligner, MegaHit De novo assembler, and BLAST and HMMER3 sequence search tools. We show the scalability of ViraPipe by running experiments on mining virus related genomes from NGS datasets in a distributed Spark computing cluster. ViraPipe analyses 768 human samples in 210 minutes on a Spark computing cluster comprising 23 nodes and 1288 cores in total. The speedup of ViraPipe executed on 23 nodes was 11x compared to the sequential analysis pipeline executed on a single node. The whole process includes parallel decompression, read interleaving, BWA-MEM read alignment, filtering and normalizing of non-human reads, De novo contigs assembling, and searching of sequences with BLAST and HMMER3 tools. ilari.maarala@aalto.fi. https://github.com/NGSeq/ViraPipe.

A speculative parallel decompression algorithm on Apache Spark

Data decompression is one of the most important techniques in data processing and has been widely used in multimedia information transmission and processing. However, the existing decompression algorithms on multicore platforms are time-consuming and do not support large data well. In order to expand parallelism and enhance decompression efficiency on large-scale datasets, based on the software thread-level speculation technique, this paper raises a speculative parallel decompression algorithm on Apache Spark. By analyzing the data structure of the compressed data, the algorithm firstly hires a function to divide compressed data into blocks which can be decompressed independently and then spawns a number of threads to speculatively decompress data blocks in parallel. At last, the speculative results are merged to form the final outcome. Comparing with the conventional parallel approach on multicore platform, the proposed algorithm is very efficiency and obtains a high parallelism degree by making the best of the resources of the cluster. Experiments show that the proposed approach could achieve 2.6\(\times \) speedup when comparing with the traditional approach in average. In addition, with the growing number of working nodes, the execution time cost decreases gradually, and the speedup scales linearly. The results indicate that the decompression efficiency can be significantly enhanced by adopting this speculative parallel algorithm.

Parallel Compression and Decompression of DNA Sequence Reads in FASTQ Format

Large volumes of short reads of genomic data are generated by high-throughput sequencing instruments. The FASTQ format is widely accepted as the input format of genomic reads and has presented challenges in data storage, management, and transfer. The performance of this type of serial algorithms such as G-SQZ and DSRC is limited by the single processor and the memory in a single computer. Utilizing data parallelism, the circular dual queues of buffers, memory mapping integrated with superblocks, pipeline parallelism with multi threads, and so on, we present the parallel compression and decompression methods for DNA sequence reads in FASTQ format based on the parallel computer architectures of the cluster and the SMP. Experimental results for the parallel DSRC algorithm clearly show the efficiency of using the powerful computing resources from multi computing nodes and multi cores of each node. The speedups vary from 46 to 62 for parallel compression and vary from 40 to 58 for parallel decompression by using 10 nodes of the cluster in Tianhe-1A super computer. Test results on the SMP machine are also pleasant. The methods could be applied to any serial compressing algorithms of DNA sequence reads in FASTQ format only if they have the traits of index and superblocks.

Practical speculative parallelization of variable-length decompression algorithms

Variable-length coding is widely used for efficient data compression. Typically, the compressor splits the original data into blocks and compresses each block with variable-length codes, hence producing variable-length compressed blocks. Although the compressor can easily exploit ample block-level parallelism, it is much more difficult to extract such coarse-grain parallelism from the decompressor because a block boundary cannot be located until decompression of the previous block is completed. This paper presents novel algorithms to efficiently predict block boundaries and a runtime system that enables efficient block-level parallel decompression, called SDM . The SDM execution model features speculative pipelining with three stages: Scanner, Decompressor, and Merger. The scanner stage employs a high-confidence prediction algorithm that finds compressed block boundaries without fully decompressing individual blocks. This information is communicated to the parallel decompressor stage in which multiple blocks are decompressed in parallel. The decompressed blocks are merged in order by the merger stage to produce the final output. The SDM runtime is specialized to execute this pipeline correctly and efficiently on resource-constrained embedded platforms. With SDM we effectively parallelize three production-grade variable-length decompression algorithms?zlib, bzip2, and H.264?with maximum speedups of 2.50× and 8.53× (and geometric mean speedups of 1.96× and 4.04×) on 4-core and 36-core embedded platforms, respectively.

A Universal Placement Technique of Compressed Instructions for Efficient Parallel Decompression

Instruction compression is important in embedded system design since it reduces the code size (memory requirement) and thereby improves the overall area, power, and performance. Existing research in this field has explored two directions: efficient compression with slow decompression, or fast decompression at the cost of compression efficiency. This paper combines the advantages of both approaches by introducing a novel bitstream placement method. Our contribution in this paper is a novel compressed bitstream placement technique to support parallel decompression without sacrificing the compression efficiency. The proposed technique enables splitting a single bitstream (instruction binary) fetched from memory into multiple bitstreams, which are then fed into different decoders. As a result, multiple slow decoders can simultaneously work to produce the effect of high decode bandwidth. We prove that our approach is a close approximation of the optimal placement scheme. Our experimental results demonstrate that our approach can improve the decode bandwidth up to four times with minor impact (less than 3%) on the compression efficiency.

Multilevel-Huffman Test-Data Compression for IP Cores With Multiple Scan Chains

Various compression methods have been proposed for tackling the problem of increasing test-data volume of contemporary, core-based systems. Despite their effectiveness, most of the approaches that are based on classical codes (e.g., run-lengths, Huffman) cannot exploit the test-application-time advantage of multiple-scan-chain cores, since they are not able to perform parallel decompression of the encoded data. In this paper, we take advantage of the inherent parallelism of Huffman decoding and we present a generalized multilevel Huffman-based compression approach that is suitable for cores with multiple scan chains. The size of the encoded data blocks is independent of the slice size (i.e., the number of scan chains), and thus it can be adjusted so as to maximize the compression ratio. At the same time, the parallel data-block decoding ensures the exploitation of most of the scan chains' parallelism. The proposed decompression architecture can be easily modified to suit any Huffman-based compression scheme.

Fast vector quantization for efficient rendering of compressed point-clouds

We present a point-cloud compression algorithm that allows fast parallel decompression on the GPU suitable for interactive applications. The algorithm is based on vector quantization of an atlas of height-fields that have been sampled over primitive shapes which approximate the geometry. We introduce novel vector quantization acceleration techniques to facilitate fast compression as well. We achieve bitrates of less than four bits per normal-equipped point. Our method enables hole-free level-of-detail point rendering. We also show that using only up to two bits per point, high-quality renderings can still be obtained if normals are estimated in image-space. Even lower bitrates are obtained for storage on disk if arithmetic coding is used.

Code Compression for VLIW Embedded Systems Using a Self-Generating Table

We propose a new class of methods for VLIW code compression using variable-sized branch blocks with self-generating tables. Code compression traditionally works on fixed-sized blocks with its efficiency limited by their small size. A branch block, a series of instructions between two consecutive possible branch targets, provides larger blocks for code compression. We compare three methods for compressing branch blocks: table-based, Lempel-Ziv-Welch (LZW)-based and selective code compression. Our approaches are fully adaptive and generate the coding table on-the-fly during compression and decompression. When encountering a branch target, the coding table is cleared to ensure correctness. Decompression requires a simple table lookup and updates the coding table when necessary. When decoding sequentially, the table-based method produces 4 bytes per iteration while the LZW-based methods provide 8 bytes peak and 1.82 bytes average decompression bandwidth. Compared to Huffman's 1 byte and variable-to-fixed (V2F)'s 13-bit peak performance, our methods have higher decoding bandwidth and a comparable compression ratio. Parallel decompression could also be applied to our methods, which is more suitable for VLIW architectures.

An on-chip cache compression technique to reduce decompression overhead and design complexity

This research explores a compressed memory hierarchy model which can increase both the effective memory space and bandwidth of each level of memory hierarchy. It is well known that decompression time causes a critical effect to the memory access time and variable-sized compressed blocks tend to increase the design complexity of the compressed memory systems. This paper proposes a selective compressed memory system (SCMS) incorporating the compressed cache architecture and its management method. To reduce or hide decompression overhead, this SCMS employs several effective techniques, including selective compression, parallel decompression and the use of a decompression buffer. In addition, fixed memory space allocation method is used to achieve efficient management of the compressed blocks. Trace-driven simulation shows that the SCMS approach can not only reduce the on-chip cache miss ratio and data traffic by about 35% and 53%, respectively, but also achieve a 20% reduction in average memory access time (AMAT) over conventional memory systems (CMS). Moreover, this approach can provide both lower memory traffic at a lower cost than CMS with some architectural enhancement. Most importantly, the SCMS is a more attractive approach for future computer systems because it offers high performance in cases of long DRAM latency and limited bus bandwidth.