Compressed File Size Research Articles

Improvement of Lossless Text Compression Methods using a Hybrid Method by the Integration of RLE, LZW and Huffman Coding Algorithms

In the field of computer science, data compression is essential in the process of data transfer because it reduces file size without losing information. Issues that depend on the characteristics of the text can greatly affect the effectiveness of any compression algorithm. Consequently, each traditional compression algorithm has its own strengths and limitations. Therefore, a highly efficient compression method that achieves higher compression ratio is needed. In this paper, based on the sequential implementation of Huffman, RLE and LZW algorithms, a hybrid data compression method has been proposed. The algorithms have been individually evaluated and compared in terms of compression ratio and compressed file size. We conclude that in terms of compression ratio and compressed file size, LZW is better than Huffman and RLE. The results show that the proposed hybrid algorithm achieves the highest average compression ratio is 2.42 and the lowest average compressed file size is 1138.5 or less compared to the individual implementations of the algorithms. Thus, it can be concluded that the proposed hybrid method enhances lossless text compression in terms of compression ratio and file size according to compression metrics.

Prediction-based coding with rate control for lossless region of interest in pathology imaging

Online collaborative tools for medical diagnosis produced from digital pathology images have experimented an increase in demand in recent years. Due to the large sizes of pathology images, rate control (RC) techniques that allow an accurate control of compressed file sizes are critical to meet existing bandwidth restrictions while maximizing retrieved image quality. Recently, some RC contributions to Region of Interest (RoI) coding for pathology imaging have been presented. These encode the RoI without loss and the background with some loss, and focus on providing high RC accuracy for the background area. However, none of these RC contributions deal efficiently with arbitrary RoI shapes, which hinders the accuracy of background definition and rate control. This manuscript presents a novel coding system based on prediction with a novel RC algorithm for RoI coding that allows arbitrary RoIs shapes. Compared to other methods of the state of the art, our proposed algorithm significantly improves upon their RC accuracy, while reducing the compressed data rate for the RoI by 30%. Furthermore, it offers higher quality in the reconstructed background areas, which has been linked to better clinical performance by expert pathologists. Finally, the proposed method also allows lossless compression of both the RoI and the background, producing data volumes 14% lower than coding techniques included in DICOM, such as HEVC and JPEG-LS.

Compression strategies for large-scale electrophysiology data.

Objective.With the rapid adoption of high-density electrode arrays for recording neural activity, electrophysiology data volumes within labs and across the field are growing at unprecedented rates. For example, a one-hour recording with a 384-channel Neuropixels probe generates over 80 GB of raw data. These large data volumes carry a high cost, especially if researchers plan to store and analyze their data in the cloud. Thus, there is a pressing need for strategies that can reduce the data footprint of each experiment.Approach.Here, we establish a set of benchmarks for comparing the performance of various compression algorithms on experimental and simulated recordings from Neuropixels 1.0 (NP1) and 2.0 (NP2) probes.Main results.For lossless compression, audio codecs (FLACandWavPack) achieve compression ratios (CRs) 6% higher for NP1 and 10% higher for NP2 than the best general-purpose codecs, at the expense of decompression speed. For lossy compression, theWavPackalgorithm in 'hybrid mode' increases the CR from 3.59 to 7.08 for NP1 and from 2.27 to 7.04 for NP2 (compressed file size of ∼14% for both types of probes), without adverse effects on spike sorting accuracy or spike waveforms.Significance.Along with the tools we have developed to make compression easier to deploy, these results should encourage all electrophysiologists to apply compression as part of their standard analysis workflows.

ANALYSIS OF THE POSSIBILITY OF USING THE SINGULAR VALUE DECOMPOSITION IN IMAGE COMPRESSION

In today’s highly computerized world, data compression is a key issue to minimize the costs associated with data storage and transfer. In 2019, more than 70% of the data sent over the network were images. This paper analyses the feasibility of using the SVD algorithm in image compression and shows that it improves the efficiency of JPEG and JPEG2000 compression. Image matrices were decomposed using the SVD algorithm before compression. It has also been shown that as the image dimensions increase, the fraction of eigenvalues that must be used to reconstruct the image in good quality decreases. The study was carried out on a large and diverse set of images, more than 2500 images were examined. The results were analyzed based on criteria typical for the evaluation of numerical algorithms operating on matrices and image compression: compression ratio, size of compressed file, MSE, number of bad pixels, complexity, numerical stability, easiness of implementation.

Foveated 3D range geometry compression via loss-tolerant variable precision depth encoding.

The capacity of three-dimensional (3D) range geometry acquisition methods to capture high-precision scans at high frame rates increases every year. These improvements have influenced a broadening range of disciplines to implement 3D range geometry capture systems, including telepresence, medicine, the visual arts, and many others. However, its increased popularity, precision, and capture rates have caused mounting pressure on the storage and transmission of 3D range geometry, thus straining their capacities. Compression techniques seek to alleviate this pressure by offering reduced file sizes, while maintaining the levels of precision needed for particular applications. Several such compression methods use sinusoidal modulation approaches to encode floating-point 3D data into conventional 2D red, green, and blue (RGB) images. In some applications, such as telepresence, high precision may only be required in a particular region within a depth scan, thus allowing less important data to be compressed more aggressively. This paper proposes a feature-driven compression method that provides a way to encode regions of interest at higher levels of precision while encoding the remaining data less precisely to reduce file sizes. This method supports both lossless and lossy compression, enabling even greater file-size savings. For example, in the case of a depth scan of a bust, an algorithmically extracted bounding box of the face was used to create a foveated encoding distribution so that the facial region was encoded at higher precisions. When using JPEG 80, the RMS reconstruction error of this novel, to the best of our knowledge, encoding was 0.56mm in the region of interest, compared to a globally fixed higher precision encoding where the error was 0.54mm in the same region. However, the proposed encoding achieved a 26% reduction in overall compressed file size compared to the fixed, higher-precision encoding.

Construction of a high-precision general geographical location words dataset

Geographical location words (GLWs) are words associated with geographical locations. GLWs are the significant foundation for text data processing and social network location inference. In this paper, we propose a framework for constructing high-precision general GLWs datasets, and a Chinese GLWs dataset (named GeoCN) is constructed. To some extent, GeoCN solves the problem of lacking a Chinese GLWs lexicon with diverse categories, high accuracy, and robust versatility. GeoCN consists of three parts: a) points of interest (POI) data collected based on the electronic map API, b) administrative division data constructed based on the national information platform, and c) GLWs data expanded and filtered by automated procedures and manual processing. We establish a GLWs glossary for each administrative region and map each GLW to its location. GeoCN covers 34 provincial-level administrative regions, 392 prefecture-level administrative regions, and 3,160 county-level administrative regions in China. The number of GLWs in GeoCN reaches 1,763,476, and the compressed file size is 117 MB.

Implementasi Algoritma Arithmetic Coding dan Sannon-Fano Pada Kompresi Citra PNG

The rapid development of technology plays an important role in the rapid exchange of information. In sending information in the form of images, there are still problems, including because of the large size of the image so that the solution to this problem is to perform compression. In this thesis, we will implement and compare the performance of the Arithmetic Coding and Shannon-Fano algorithms by calculating the compression ratio, compressed file size, compression and decompression process speed. Based on all test results, that the Arithmetic Coding algorithm produces an average compression ratio of 62.88% and a Shannon-Fano compression ratio of 61.73%, then Arithmetic Coding the average speed in image compression is 0.072449 seconds and Shannon-Fano 0.077838 second. Then the Shannon-Fano algorithm has an average speed for decompression of 0.028946 seconds and the Arithmetic Coding algorithm 0.034169 seconds. The decompressed image on the Arithmetic Coding and Shannon-Fano algorithm is in accordance with the original image. It can be concluded from the test results that the Arithmetic Coding algorithm is more efficient in compressing *.png images than the Shannon-Fano algorithm, although in terms of decompression Shannon-Fanose is a little faster compared to Arithmetic Coding.

Feature-driven 3D range geometry compression via spatially-aware depth encoding

The ever-growing variety of capture methods and applications for 3D range geometry continually increases the need for efficient methods of data storage and transmission. Compression techniques address this need by offering reduced file sizes while maintaining the precision needed for a particular application. Several such compression methods use phase-shifting principles to encode the 3D data into a 2D RGB image. In some applications, such as telepresence, high precision may only be required in a particular region within a scan. This paper proposes a feature-driven compression method that provides a way to encode regions of interest at higher precision while encoding the remaining data at lower precision to reduce file sizes. This method supports both lossless and lossy compression, enabling even greater file size savings and a wider range of applications. In the case of a depth scan of a bust, an extracted bounding box of the face was used to create an encoding distribution such that the facial region was encoded at higher precisions. When using JPEG 80, the global RMS reconstruction accuracy of this novel encoding was 99.72%; however, in the region of interest, the accuracy was 99.88%. This feature-driven encoding achieved a 26% reduction in compressed file size compared to a fixed, high precision encoding.

Depth range reduction for 3D range geometry compression

Three-dimensional (3D) shape measurement devices and techniques are being rapidly adopted within a variety of industries and applications. As acquiring 3D range data becomes faster and more accurate it becomes more challenging to efficiently store, transmit, or stream this data. One prevailing approach to compressing 3D range data is to encode it within the color channels of regular 2D images. This paper presents a novel method for reducing the depth range of a 3D geometry such that it can be stored within a 2D image using lower encoding frequencies (or a fewer number of encoding periods). This allows for smaller compressed file sizes to be achieved without a proportional increase in reconstruction errors. Further, as the proposed method occurs prior to encoding, it is readily compatible with a variety of existing image-based 3D range geometry compression methods.

Forward Support—Cloud Based Mosaic Imagery for Emergency Operations

The Institute for Creative Technologies (ICT) has pursued the creation of One World Terrain (OWT), which aims to provide a set of 3D global terrain capabilities and services that can replicate the coverage and complexities of the operational environment. Research was conducted in support of One World Terrain through development of best practices for the delivery of a raster mosaic via cloud hosting service, created using OptimizeRasters Geoprocoessing Toolbox and the Mosaic Dataset Configuration Script. Though ultimately successful in developing the raster mosaic and hosting it online; JPEG compression lossiness was a key issue with the larger Rose Bowl dataset. Additionally, hosting the imagery via ArcGIS Online was found to increase the compressed file size; making it comparable to the original file size of the data. Future testing should consider usage of an enterprise server to avoid this issue. MRF_LERC compression was identified as the ideal file configuration; and ArcGIS Online was identified as a poor enterprise hosting medium. We have also identified a variety of ways to improve the MDCS script in order to automate the whole process more efficiently.

High efficiency referential genome compression algorithm.

With the development and the gradually popularized application of next-generation sequencing technologies (NGS), genome sequencing has been becoming faster and cheaper, creating a massive amount of genome sequence data which still grows at an explosive rate. The time and cost of transmission, storage, processing and analysis of these genetic data have become bottlenecks that hinder the development of genetics and biomedicine. Although there are many common data compression algorithms, they are not effective for genome sequences due to their inability to consider and exploit the inherent characteristics of genome sequence data. Therefore, the development of a fast and efficient compression algorithm specific to genome data is an important and pressing issue. We have developed a referential lossless genome data compression algorithm with better performance than previous algorithms. According to a carefully designed matching strategy selection mechanism, the advantages of local matching and global matching are reasonably combined together to improve the description efficiency of the matched sub-strings. The effects of the length and the position of matched sub-strings to the compression efficiency are jointly taken into consideration. The proposed algorithm can compress the FASTA data of complete human genomes, each of which is about 3 GB, in about 18 min. The compressed file sizes are ranging from a few megabytes to about forty megabytes. The averaged compression ratio is higher than that of the state-of-the-art genome compression algorithms, the time complexity is at the same order of the best-known algorithms. https://github.com/jhchen5/SCCG. Supplementary data are available at Bioinformatics online.

SBPG: Secure Better Portable Graphics for Trustworthy Media Communications in the IoT

Smart Healthcare is envisioned as the combination of traditional healthcare augmented by smart bio-sensors, wearable devices, and a plethora of on-body sensors that communicate with smart hospitals, smart emergency response systems, and ambulances, through advanced information and communication technologies. The vision of smart healthcare as part of a smart city relies on the framework of the Internet of Things (IoT) as the underlying core technology that enables the design and operation of a city, whereby smart technology, energy grids, transportation, buildings, communication, and information technology, are all interconnected. This paper addresses some of the challenges faced in the IoT infrastructure, specifically secure communication and user authentication in the context of automated analysis of biomedical images and communication of the analysis results and related metadata in a smart healthcare framework. A hardware architecture for a secure digital camera integrated with the secure better portable graphics (SBPG) compression algorithm, suitable for applications in the IoT, is proposed in this paper. The focus of this paper is on patient data protection and authentication. The proposed SBPG architecture offers two layers of protection, concurrent encryption and watermarking, which address all issues related to security, privacy, and digital rights management. The experimental results demonstrate that the new compression technique BPG outperforms JPEG in terms of compression quality and compressed file size while providing increased image quality. High performance requirements of BPG have been met by employing two techniques: 1) insertion of an encrypted signature in the center portion of the image and 2) frequency-domain watermarking using blockwise DCT of size $8\times 8$ pixels. These approaches optimize the proposed architecture by decreasing computational complexity while maintaining strong protection, with concomitant increase of the speed of the watermarking and compression processes. A Simulink® prototype for the proposed architecture has been built and tested. To the best of our knowledge, the hardware architecture for BPG compression with built-in image authentication capability for integration with a secure digital camera is the first one ever proposed .

A joint image lossless compression and encryption method based on chaotic map

Nowadays poor security, low transmission and storage efficiency of images have become serious concerns. In order to improve the situation, this paper put forward a new image lossless compression joint encryption algorithm based on chaotic map with all original information intact. The lossless compression uses SPIHT(Set Partitioning in Hierarchical Trees) encoding method based on integer wavelet transform, and encrypt multiple rounds in the process of wavelet coefficients and SPIHT coding applying many kinds of chaotic maps. Experimental results show that the compressed file size is about 50 % of the original file size, which achieves relatively good lossless compression ratio. Besides, the encryption method passes many security tests, such as sensitivity test, entropy test, autocorrelation test, NIST SP800---22 test. There is a high application value in the medical field and the national security department whose image files require a relatively high quality.

MetaCRAM: an integrated pipeline for metagenomic taxonomy identification and compression.

BackgroundMetagenomics is a genomics research discipline devoted to the study of microbial communities in environmental samples and human and animal organs and tissues. Sequenced metagenomic samples usually comprise reads from a large number of different bacterial communities and hence tend to result in large file sizes, typically ranging between 1–10 GB. This leads to challenges in analyzing, transferring and storing metagenomic data. In order to overcome these data processing issues, we introduce MetaCRAM, the first de novo, parallelized software suite specialized for FASTA and FASTQ format metagenomic read processing and lossless compression.ResultsMetaCRAM integrates algorithms for taxonomy identification and assembly, and introduces parallel execution methods; furthermore, it enables genome reference selection and CRAM based compression. MetaCRAM also uses novel reference-based compression methods designed through extensive studies of integer compression techniques and through fitting of empirical distributions of metagenomic read-reference positions. MetaCRAM is a lossless method compatible with standard CRAM formats, and it allows for fast selection of relevant files in the compressed domain via maintenance of taxonomy information. The performance of MetaCRAM as a stand-alone compression platform was evaluated on various metagenomic samples from the NCBI Sequence Read Archive, suggesting 2- to 4-fold compression ratio improvements compared to gzip. On average, the compressed file sizes were 2-13 percent of the original raw metagenomic file sizes.ConclusionsWe described the first architecture for reference-based, lossless compression of metagenomic data. The compression scheme proposed offers significantly improved compression ratios as compared to off-the-shelf methods such as zip programs. Furthermore, it enables running different components in parallel and it provides the user with taxonomic and assembly information generated during execution of the compression pipeline.AvailabilityThe MetaCRAM software is freely available at http://web.engr.illinois.edu/~mkim158/metacram.html. The website also contains a README file and other relevant instructions for running the code. Note that to run the code one needs a minimum of 16 GB of RAM. In addition, virtual box is set up on a 4GB RAM machine for users to run a simple demonstration.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-016-0932-x) contains supplementary material, which is available to authorized users.

Achieving Better Compression Applying Index-based Byte-Pair Transformation before Arithmetic Coding

Arithmetic coding is used in many compression techniques during the entropy encoding stage. Further compression is not possible without changing the data model and increasing redundancy in the data set. To increase the redundancy, we have applied index based byte-pair transformation (BPT-I) as a pre-processing to arithmetic coding. BPT-I transforms most frequent byte-pairs (2-byte integers). Here, most frequent byte-pairs are sorted in the order of their frequency and groups consisting of 256 byte-pairs are formed. Each bytepair in a group is then encoded using two tokens: group number and the location in a group. Group number is denoted using variable length prefix codeword; whereas location within a group is denoted using 8-bit index. BPT-I is designed to be applied on any type of source; not necessarily text. More the number of groups considered during transformation, better is the compression. Experimental results have shown around 4.30% additional reduction in compressed file size when arithmetic coding is applied after byte-pair data transformation BPT-I. General Terms Data Compression, Algorithms

Examining complexity across domains: relating subjective and objective measures of affective environmental scenes, paintings and music.

Subjective complexity has been found to be related to hedonic measures of preference, pleasantness and beauty, but there is no consensus about the nature of this relationship in the visual and musical domains. Moreover, the affective content of stimuli has been largely neglected so far in the study of complexity but is crucial in many everyday contexts and in aesthetic experiences. We thus propose a cross-domain approach that acknowledges the multidimensional nature of complexity and that uses a wide range of objective complexity measures combined with subjective ratings. In four experiments, we employed pictures of affective environmental scenes, representational paintings, and Romantic solo and chamber music excerpts. Stimuli were pre-selected to vary in emotional content (pleasantness and arousal) and complexity (low versus high number of elements). For each set of stimuli, in a between-subjects design, ratings of familiarity, complexity, pleasantness and arousal were obtained for a presentation time of 25 s from 152 participants. In line with Berlyne’s collative-motivation model, statistical analyses controlling for familiarity revealed a positive relationship between subjective complexity and arousal, and the highest correlations were observed for musical stimuli. Evidence for a mediating role of arousal in the complexity-pleasantness relationship was demonstrated in all experiments, but was only significant for females with regard to music. The direction and strength of the linear relationship between complexity and pleasantness depended on the stimulus type and gender. For environmental scenes, the root mean square contrast measures and measures of compressed file size correlated best with subjective complexity, whereas only edge detection based on phase congruency yielded equivalent results for representational paintings. Measures of compressed file size and event density also showed positive correlations with complexity and arousal in music, which is relevant for the discussion on which aspects of complexity are domain-specific and which are domain-general.

Fractal Analysis of Periapical Bone from Lossy Compressed Radiographs: A Comparison of Two Lossy Compression Methods

The aim of the study was to evaluate the effect of two lossy image compression methods on fractal dimension (FD) calculation. Ten periapical images of the posterior teeth with no restorations or previous root canal therapy were obtained using storage phosphor plates and were saved in TIF format. Then, all images were compressed with lossy JPEG and JPEG2000 compression methods at five compression levels, i.e., 90, 70, 50, 30, and 10. Compressed file sizes from all images and compression ratios were calculated. On each image, two regions of interest (ROIs) containing healthy trabecular bone in the posterior periapical area were selected. The FD of each ROI on the original and compressed images was calculated using differential box counting method. Both image compression and analysis were performed by a public domain software. Altogether, the FD of 220 ROIs was calculated. FDs were compared using ANOVA and Dunnett tests. The FD decreased gradually with compression level. A statistically significant decrease of the FD values was found for JPEG 10, JPEG2000 10, and JPEG2000 30 compression levels (p < 0.05). At comparable file sizes, the JPEG induced a smaller FD difference. In conclusion, lossy compressed images with appropriate compression level may be used for FD calculation.

BINARY TREE IMAGE CODING ALGORITHM BASED ON NON-SEPARABLE WAVELET TRANSFORM VIA LIFTING SCHEME

In recent years, image coding based on wavelet transform has made rapid progress. In this paper, quincunx lifting scheme in wavelet transform is introduced and all phase interpolation filter banks which can be used in the lifting scheme for prediction and update are designed. Based on the basic idea of set partitioning in hierarchical trees (SPIHT) algorithm, the binary tree image coding algorithm is proposed. Just like SPIHT, the encoding algorithms can be stopped at any compressed file size or let run until the compressed file is a representation of a nearly lossless image. The experimental results on test images show that compared with SPIHT algorithm, the PSNRs of the proposed algorithm are superior by about 0.5 dB at the same bit rates and the subjective quality of reconstructed images is also better.

High-grade Video Compression of Echocardiographic Studies: A Multicenter Validation Study of Selected Motion Pictures Expert Groups (MPEG)-4 Algorithms

Large files produced by standard compression algorithms slow down spread of digital and tele-echocardiography. We validated echocardiographic video high-grade compression with the new Motion Pictures Expert Groups (MPEG)-4 algorithms with a multicenter study. Seven expert cardiologists blindly scored (5-point scale) 165 uncompressed and compressed 2-dimensional and color Doppler video clips, based on combined diagnostic content and image quality (uncompressed files as references). One digital video and 3 MPEG-4 algorithms (WM9, MV2, and DivX) were used, the latter at 3 compression levels (0%, 35%, and 60%). Compressed file sizes decreased from 12 to 83 MB to 0.03 to 2.3 MB (1:1051-1:26 reduction ratios). Mean SD of differences was 0.81 for intraobserver variability (uncompressed and digital video files). Compared with uncompressed files, only the DivX mean score at 35% (P = .04) and 60% (P = .001) compression was significantly reduced. At subcategory analysis, these differences were still significant for gray-scale and fundamental imaging but not for color or second harmonic tissue imaging. Original image quality, session sequence, compression grade, and bitrate were all independent determinants of mean score. Our study supports use of MPEG-4 algorithms to greatly reduce echocardiographic file sizes, thus facilitating archiving and transmission. Quality evaluation studies should account for the many independent variables that affect image quality grading.

An Information Theory Analysis of Visual Complexity and Dissimilarity

The subjective complexity of a computer-generated bitmap image can be measured by magnitude estimation scaling, and its objective complexity can be measured by its compressed file size. There is a high correlation between these measures of subjective and objective complexity over a large set of marine electronic chart and radar images. The subjective dissimilarity of a pair of bitmap images can be predicted from subjective and objective measures of the complexity of each image, and from the subjective and objective complexity of the image produced by overlaying the two simple images. In addition, the subjective complexity of the image produced by overlaying two simple images can be predicted from the subjective complexity of the simple images and the subjective dissimilarity of the image pair. The results of the experiments that generated these complexity and dissimilarity judgments are consistent with a theory, outlined here, that treats objective and subjective measures of image complexity and dissimilarity as vectors in Euclidean space.