Data Compression Processing Research Articles

Quantum variational distance-based centroid classifier

Quantum machine learning is a rapidly growing field with the potential to surpass classical methods and offer novel solutions to learning problems. Supervised classification, a prominent pattern recognition task, has attracted significant attention for improving existing methods using quantum techniques. Although previous quantum algorithms for distance-based classification have shown promise, they are not compatible with current Noisy Intermediate-Scale Quantum (NISQ) devices and cannot handle multiclass problems. To address these limitations, we present the quantum Variational Distance-based Centroid Classifier (qVDCC), a hybrid approach that combines the Quantum One-Class Classifier (QOCC) and Parameterized Quantum Circuits (PQCs). This approach greatly improves the feasibility of multiclass classification on NISQ devices, as it eliminates the need for a label qubit and uses a single-qubit measurement, regardless of data size. Our simulations on seven benchmark datasets using both noisy and noise-free simulations show that qVDCC is competitive with the k-nearest neighbors algorithm, while providing exponential data compression and efficient quantum data processing.

Compression of electrical code violation recognition data using the improved swinging door trending algorithm

Abstract Aiming at the challenge of storing massive power grid data, this paper proposes an improved swing gate trend algorithm to effectively compress 5G data. The algorithm first performs least squares smoothing on the original data to reduce noise interference on the SDT algorithm, which enables the data compression process to more accurately determine the data trend. Further, the shortcomings of the original SDT algorithm are improved, including adaptive frequency conversion data processing, dynamic threshold adjustment, and anomaly recording strategy, to enhance the practicality and efficiency of the algorithm. Through simulation analysis and example data validation, the study shows that the data compression ratio can be stabilized at about 23.98 when the data compression time reaches 1.6 minutes, and the actual error is very close to the desired error. The time overhead of the improved SDT algorithm is only 0.225 seconds, indicating that the algorithm is efficient and reliable. Combined with different data compression storage strategies, the algorithm can further reduce the data compression time. This study provides an adequate data compression method for electric code violation identification, which offers a practical solution for processing and storing large-scale grid data.

Batch correction of single-cell sequencing data via an autoencoder architecture.

Technical differences between gene expression sequencing experiments can cause variations in the data in the form of batch effect biases. These do not represent true biological variations between samples and can lead to false conclusions or hinder the ability to integrate multiple datasets. Since there is a growing need for the joint analysis of single-cell sequencing datasets from different sources, there is also a need to correct the resulting batch effects while maintaining the true biological variations in the data. We developed a semi-supervised deep learning architecture called Autoencoder-based Batch Correction (ABC) for integrating single-cell sequencing datasets. Our method removes batch effects through a guided process of data compression using supervised cell type classifier branches for biological signal retention. It aligns the different batches using an adversarial training approach. We comprehensively evaluate the performance of our method using four single-cell sequencing datasets and multiple measures for batch effect removal and biological variation conservation. ABC outperforms 10 state-of-the-art methods for this task including Seurat, scGen, ComBat, scanorama, scVI, scANVI, AutoClass, Harmony, scDREAMER, and CLEAR, correcting various types of batch effects while preserving intricate biological variations.

Architectural Design of Representational State Transfer Application Programming Interface with Application-Level Base64-Encoding and Zlib Data Compression

Representational State Transfer (REST) is an architectural style that underlies the protocol of HyperText Transfer Protocol (HTTP)and is used by web applications. The implementation of REST principles in web services is often known as RESTful API. The standard communication used in RESTful APIs uses HTTP without involving data compression. There are quite a lot of RESTful API clients in the form of mobile applications that use metered networks. Thus, reducing the required bandwidth during transmission is suggested to be beneficial. The data compression technique is widely known to reduce data size, but this additional step may yield side effects such as increased memory usage and processing time. This study aims to investigate how the data compression process, which specifically uses Zlib and Base64 encoding, may put additional load on the whole process of delivering content to a RESTful API. The performance and characteristics in terms of bandwidth saved when distributing JSON data from a RESTful API in compressed format are also be investigated. According to the experiment result, it is suggested that the compression process can reduce network bandwidth by up to 66% with negligible additional memory usage for the compression and decompression processes.

Uncertainty analysis of UAM TMI-1 benchmark by STREAM/RAST-K

This study rigorously examined uncertainty in the TMI-1 benchmark within the Uncertainty Analysis in Modeling (UAM) benchmark suite using the STREAM/RAST-K two-step method. It presents two pivotal advancements in computational techniques: (1) Development of an uncertainty quantification (UQ) module and a specialized library for the pin-based pointwise energy slowing-down method (PSM), and (2) Application of Principal Component Analysis (PCA) for UQ. To evaluate the new computational framework, we conducted verification tests using SCALE 6.2.2. Results demonstrated that STREAM's performance closely matched SCALE 6.2.2, with a negligible uncertainty discrepancy of ±0.0078% in TMI-1 pin cell calculations. To assess the reliability of the PSM covariance library, we performed verification tests, comparing calculations with Calvik's two-term rational approximation (EQ 2-term) covariance library. These calculations included both pin-based and fuel assembly (FA-wise) computations, encompassing hot zero-power and hot full-power operational conditions. The uncertainties calculated using both the EQ 2-term and PSM resonance treatments were consistent, showing a deviation within ±0.054%. Additionally, the data compression process yielded compression ratios of 88.210% and 92.926% for on-the-fly and data-saving approaches, respectively, in TMI fuel assembly calculations. In summary, this study provides a comprehensive explanation of the PCA process used for UQ calculations and offers valuable insights into the robustness and reliability of newly developed computational methods, supported by rigorous verification tests.

Distributed Compressive Sensing for Wireless Signal Transmission in Structural Health Monitoring: An Adaptive Hierarchical Bayesian Model-Based Approach.

Signal transmission plays an important role in the daily operation of structural health monitoring (SHM) systems. In wireless sensor networks, transmission loss often occurs and threatens reliable data delivery. The massive amount of data monitoring also leads to a high signal transmission and storage cost throughout the system's service life. Compressive Sensing (CS) provides a novel perspective on alleviating these problems. Based on the sparsity of vibration signals in the frequency domain, CS can reconstruct a nearly complete signal from just a few measurements. This can improve the robustness of data loss while facilitating data compression to reduce transmission demands. Extended from CS methods, distributed compressive sensing (DCS) can exploit the correlation across multiple measurement vectors (MMV) to jointly recover the multi-channel signals with similar sparse patterns, which can effectively enhance the reconstruction quality. In this paper, a comprehensive DCS framework for wireless signal transmission in SHM is constructed, incorporating the process of data compression and transmission loss together. Unlike the basic DCS formulation, the proposed framework not only activates the inter-correlation among channels but also provides flexibility and independence to single-channel transmission. To promote signal sparsity, a hierarchical Bayesian model using Laplace priors is built and further improved as the fast iterative DCS-Laplace algorithm for large-scale reconstruction tasks. Vibration signals (e.g., dynamic displacement and accelerations) acquired from real-life SHM systems are used to simulate the whole process of wireless transmission and test the algorithm's performance. The results demonstrate that (1) DCS-Laplace is an adaptative algorithm that can actively adapt to signals with various sparsity by adjusting the penalty term to achieve optimal performance; (2) compared with CS methods, DCS methods can effectively improve the reconstruction quality of multi-channel signals; (3) the Laplace method has advantages over the OMP method in terms of reconstruction performance and applicability, which is a better choice in SHM wireless signal transmission.

Huffman Algorithm Valuation Using Residue Number System

The Huffman algorithm is a widely used method for lossless data compression, which assigns variable-length codes to characters based on their frequency of occurrence in the input data. However, the traditional implementation of Huffman coding using binary arithmetic can be computationally intensive, particularly for large data sets. In recent years, the Residue Number System (RNS) has emerged as a promising alternative to binary arithmetic for certain types of computations, due to its potential for parallel processing and reduced hardware complexity. This paper evaluates the use of RNS as a basis for implementing the Huffman algorithm, comparing its performance with the traditional binary approach. The results demonstrate that RNS-based Huffman coding can achieve comparable or superior compression ratios, while reducing the computational requirements and potentially enabling faster compression and decompression. The study also highlights the importance of choosing appropriate RNS moduli and operands to optimize performance. Overall, the evaluation suggests that RNS can be a viable and efficient alternative to binary arithmetic for implementing the Huffman algorithm, particularly in applications with high computational demands or limited hardware resources. However, further research is needed to explore the potential benefits and limitations of RNS in other areas of data compression and signal processing.

Intelligent compression then encryption scheme for resource constrained sustainable and smart healthcare environment

With the latest and emerging developments of artificial technologies, ubiquitous computing techniques become necessary in day to day lives, particularly in healthcare. Smart healthcare systems involve electronic health records (EHR) sharing system that considerably shares the medical data and delivers proper aid to the patients and research communities. At the same time, the smart healthcare system is a resource constraint environment where large amount of healthcare data needs to be stored on the cloud server. Therefore, data compression can significantly reduce a large amount of data transmission and storage process, thereby achieving effective memory and resource utilization. Compression then encryption (CTE) is an effective way to achieve effective resource utilization and security in the smart healthcare environment. Encryption transforms the EHR data to an illegible format offering considerable authentication of the digital form of medicinal data. With this motivation, this paper designs a novel compression then encryption model for secure healthcare data storage and access management (CTE-SHDSA) technique in cloud server. The proposed model involves two main stages namely compression and encryption. Initially, neighboring indexing sequence (NIS) is employed for the data compression process. The NIS-BWT technique makes use of the correlation among the neighboring bits for reducing the amount of redundant data transmission. Besides, an enhanced artificial butterfly optimization with signcryption (EABO-SC) technique is applied to proficient encrypt the compressed data. The EABO algorithm is applied to optimally choose the encryption keys of the SC technique. The extensive set of simulations was performed on benchmark medical datasets and the outcomes are inspected under several evaluation parameters. The experimental values highlighted the superior performance of the presented method on the recent state of art methods.

СПЕЦІАЛІЗОВАНИЙ ПРОЦЕСОР ДЛЯ УЩІЛЬНЕННЯ ДАНИХ

One of the effective approaches to data compression is the approach based on the use of optimizing properties of Fibonacci numbers. The essence of the approach is that in the process of compaction the block of digital data is considered as a large positive integer, given in the form of a linear Fibonacci form. The implementation of data compression methods based on the linear form of Fibonacci software requires a lot of time, which is associated with calculations over large numbers (up to 8000 binary digits). For some applications, such time is unacceptable, so there is a need to create a specialized processor that will speed up the process of data compression. The development of mathematical and structural models of a specialized processor and its components is carried out using a functional-structural approach to the design of digital devices. Based on the generalized model of the process of adaptive data compression based on the linear Fibonacci form, the main functions to be implemented by a specialized processor are identified. This processor is part of a computer system and is in some way connected to the computer's CPU. Because the files to be compressed and the compressed files are stored in computer memory, the CPU is expected to read and write the file, generate P and P* sequences, and implement a sequence-level optimization function. The specialized processor is responsible for calculations over large numbers. To implement a set of all functions, it is proposed to build more than one operating machine, and to decompose it into machines, each of which implements the corresponding function. Mathematical models and structures of such modules of the specialized processor are considered: modeling of a data source, coding, decoding, optimization at the level of blocks, formation of structure of sequence P*. Hardware implementation of calculations over large numbers and the ability to implement basic functional transformations of individual modules in the pipeline mode provides acceleration of the data compression process compared to software implementation.

Multi-Dimensional Data Compression and Query Processing in Array Databases

In recent times, the production of multidimensional data in various domains and their storage in array databases has witnessed a sharp increase; this rapid growth in data volumes necessitates compression in array databases. However, existing compression schemes used in array databases are general-purpose and not designed specifically for the databases. They could degrade query performance with complex analytical tasks, which incur huge computing costs. Thus, a compression scheme that considers the workflow of array databases is required. This study presents a compression scheme, SEACOW, for storing and querying multidimensional array data. The scheme is specially designed to be efficient for both dimension-based and value-based exploration. It considers data access patterns for exploration queries and embeds a synopsis, which can be utilized as an index, in the compressed array. In addition, we implement an array storage system, namely MSDB, to perform experiments. We evaluate query performance on real scientific datasets and compared it with those of existing compression schemes. Finally, our experiments demonstrate that SEACOW provides high compression rates compared to existing compression schemes, and the synopsis improves analytical query processing performance.

Analysis on the Application of Multimedia-Assisted Music Teaching Based on AI Technology

In order to improve the effect of modern music teaching, this paper combines AI technology to construct a multimedia-assisted music teaching system, combines music teaching data processing requirements to improve the algorithm, proposes appropriate music data filtering algorithms, and performs appropriate data compression processing. Moreover, the functional structure analysis of the intelligent music teaching system is carried out with the support of the improved algorithm, and the three-tier framework technology that is currently more widely used is used in the music multimedia teaching system. Finally, in order to realize the complex functions of the system, the system adopts a layered approach. From the experimental research results, it can be seen that the multimedia-assisted music teaching system based on AI technology proposed in this paper can effectively improve the effect of modern music teaching.

A Sliding Window Data Compression Method for Spatial-Time DOA Estimation

This paper presents a sliding window data compression method for spatial-time direction-of-arrival (DOA) estimation using coprime array. The signal model is firstly formulated by jointly using the temporal and spatial information of the impinging sources. Then, a sliding window data compression processing is performed on the array output matrix to realize fast calculation of time average function, and the computational burden has been reduced accordingly. Based on the concept of sum and difference co-array (SDCA), the vectorized conjugate augmented MUSIC is adopted, with which more sources than twice of the physical sensors can be resolved. Additionally, the sparse array robustness to sensor failure has been evaluated by introducing the concept of essential sensors. The theoretical analysis and numerical simulations are provided to confirm the effectiveness performance of the proposed method.

Text File Compression Using Hybrid Run Length Encoding (Rle) Algorithm With Even Rodeh Code (Erc) And Variable Length Binary Encoding (Vlbe) To Save Storage Space

The increase of data usage causes problems in data storage, indirectly making the need for data storage also to increase. One alternative solution that can be done is to compress the file so that the file becomes smaller in size so it saves storage space. The algorithm used in this research is the Run Length Encoding algorithm, the Even Rodeh Code algorithm, and the Variable Length Binary Encoding algorithm which are the types of lossless compression. The algorithm will calculate its performance based on Compression Ratio, Ratio of Compression, Redundancy, Compression Time, and Decompression Time. The file that will be used in the data compression process is the file extension *.txt. This study used homogeneous strings (strings that have the same character) and heterogeneous strings (strings that have different characters) in testing the algorithm. In the compression process with a homogeneous string, the combination of the Run Length Encoding algorithm with the Variable Length Binary Encoding algorithm is better than the combination of the Run Length Encoding algorithm and the Even Rodeh Code algorithm with a Compression Ratio of 18.84% and a decompression time of 0.01295 ms. While the compression process on heterogeneous strings from the combination of the Run Length Encoding algorithm with the Even Rodeh Code algorithm is better than the combination of the Run Length Encoding algorithm with Variable Length Binary Encoding algorithms with Compression Ratio of an average of 52.45% and fewer decompression times of 4.93002 ms

Exception Handling Method Based on Event from Look-Up Table Applying Stream-Based Lossless Data Compression

It is getting popular to implement an environment where communications are performed remotely among IoT edge devices, such as sensory devices and the cloud servers due to applying, for example, artificial intelligence algorithms to the system. In such situations that handle big data, lossless data compression is one of the solutions to reduce the big data. In particular, the stream-based data compression technology is focused on such systems to compress infinitely continuous data stream with very small delay. However, during the continuous data compression process, it is not able to insert an exception code among the compressed data without any additional mechanisms, such as data framing and the packeting technique, as used in networking technologies. The exception code indicates configurations for the compressor/decompressor and/or its peripheral logics. Then, it is used in real time for the configuration of parameters against those components. To implement the exception code, data compression algorithm must include a mechanism to distinguish original data before compression and the exception code clearly. However, the conventional algorithms do not include such mechanism. This paper proposes novel methods to implement the exception code in data compression that uses look-up table, called the exception symbol. Additionally, we describe implementation details of the method by applying it to algorithms of stream-based data compression. Because some of the proposed mechanisms need to reserve entries in the table, we also discuss the effect against data compression performance according to experimental evaluations.

Novel method of data compression for the online detection signal of coal mine wire rope

Coal mine wire rope detection is related to personnel and production safety. With the Chinese coal mining trend tending towards deep mining, a considerable amount of data is critical for the online detection of deep well lifting wire rope. To improve the sampling rate, decrease the analysis processing time and realise real-time online detection, this paper proposes an online detection data compression processing method. The study focuses on the distortion compression method for the online detection signal of deep well hoisting wire rope. The set partitioning in hierarchical trees (SPIHT) algorithm is one of the most advanced methods in the field of image transformation coding. Compared with other coding algorithms, the SPIHT algorithm demonstrates desired characteristics such as a high signal-to-noise ratio, lower complexity and decreased computational load, among others. This paper discusses how, in combination with the image processing method, a compression coding method for the one-dimensional signal of the magnetic leakage detection of the mining wire rope is developed. Furthermore, the set partitioning sorting algorithm is investigated and analysed, the temporal orientation tree structure of the one-dimensional signal of the wavelet coefficient is defined for wire rope magnetic leakage detection and the SPIHT algorithm is presented, in addition to an example of the one-dimensional signal from the magnetic leakage detection of the wire rope. The results reveal that under the condition of the normalised mean square error (NMSE; NMSE < 0.01) of distortion, the compression ratio improved by 30%. The online detection signal lossy compression method proposed in this study has a considerable influence on the recovery of the original signal, in addition to a higher compression ratio and a reduced computation time, compared to the existing method.

Hybrid compression of biomedical ECG and EEG signals based on differential clustering and encoding techniques

AbstractSignal processing techniques incorporated with data compression processes enrich the signals and boost up storage efficiency and transmission reliability. Transmitting uncompressed original data consume wide bandwidth, which increases transmission time and leads to data hammering. These limitations enforce to look for strategic data compression techniques. Lossless compression techniques are requisite where it is important that the original and the decompressed data should be identical or where deviations from the original data would lead to catastrophic consequences, especially in biomedical signal analysis and diagnostics. For which, the input signal preprocessed with differential pulse code modulation (DPCM) reduces the interchannel dependencies to get the desired output. A whole array of unique compression techniques are being utilized in the compression process. The combination of (K‐means clustering, arithmetic encoding [AE], Huffman encoding [HE]) clustering and coding compression techniques are analyzed using electro cardiogram (ECG) and electroencephalogram (EEG) signals. The proposed method employs k‐means clustering combined with Huffman encoding (DiKHE) and k‐means clustering combined with arithmetic encoding (DiKAE) individually. Compression ratio (CR) is analyzed with these combinations of compression techniques for various cluster size K (K = 2,3,4,5,6). A maximum CR of 6.03144 and 4.54126 is obtained for ECG and EEG signals respectively. The compressions based on these techniques are efficient since the compressed signal is reconstructed perfectly as it matches exactly with the original signal.

Систематизація технологій стиснення зображень у системах поліграфічного репродукування

This work investigate the popular data compression techniques, which can be used to reduce the size of graphic image formats in printing reproduction systems. The modern printing production process is characterized by a high degree of automation and computerization, which is characterized by a large dependence on the productivity and efficiency of the desktop publishing system and the capacity of information exchange networks. Therefore, the use of data compression methods for digital images can reduce the total amount of data for illustrative original and increase the efficiency of the printing reproduction system.The analysis of previous studies on compression technologies for various areas: prepress printing processes, digital photography, video processing, various multimedia and telecommunication technologies. The main generalizations concern the consideration of lossless and lossy methods of data compression, which differ in coding algorithms.Based on the conducted research the systematization of basic image compression technologies in printing reproduction systems has been developed. The present systematization different in the main features: data compression methods, coding algorithms, image classes, compression value and image quality after coding.The basic encoding algorithms for compressing digital images RLE, JBIG, CCITT Group, LZW, Huffman, Wavelet (recursive), fractal and JPEG. The features of the data compression process according to Lossy JPEG (lossy) and Lossless JPEG2000 (lossless) algorithms are analyzed. Possible criteria for determining the efficiency and quality of image compression using various coding algorithms, which can be used for practical comparison of various compression techniques, are defined.

The Research and Practice of a Long Pulse Synchronous Data Measurement Method

The research and implementation of a long pulse synchronous data measurement method is mainly presented in this paper from the aspects of requirement analysis, hardware platform support, software design and testing result. The NI multi-function data acquisition board card is adopted as the hardware platform. The software program platform is based on the LabVIEW virtual instrument platform. The long-pulse data acquisition technology, LabVIEW multithread data processing technology, computer transmission control technology and data compression processing technology as the key technologies are applied in this paper. The main functions of the system include (1) accurate sampling for the long pulse measured signal at the specified sampling frequency and sampling time; (2) a variety of acquisition control modes are provided to meet the different application requirements; (3) the data collected is easily stored and retrieved after a specified algorithm; (4) the real-time display and historical playback functions for measuring signals; (5) the data transmission and storage functions; (6) simple and friendly interface which is easy to operate. By software testing, the software can meet the functional requirements above and it can meet the data sampling at 1000s or even longer with 50KHz sampling rate.Experiments show that it can work stably and reliably, and it has a very important practical application value. This software has universality and can provide valuable reference for the applications of long pulse data measurement based on other hardware systems.

On the dynamical behaviour of linear higher-order cellular automata and its decidability

Higher-order cellular automata (HOCA) are a variant of cellular automata (CA) used in many applications (ranging, for instance, from the design of secret sharing schemes to data compression and image processing), and in which the global state of the system at time t depends not only on the state at time t−1, as in the original model, but also on the states at time t−2, …,t−n, where n is the memory size of the HOCA. We provide decidable characterizations of two important dynamical properties, namely, sensitivity to the initial conditions and equicontinuity, for linear HOCA over the alphabet Zm. These characterizations have an impact in applications since the involved linear HOCA are usually required to exhibit a chaotic or stable behaviour. Moreover, they extend the ones shown in [28] for linear CA (LCA) over the alphabet Zmn in the case n=1. We also show that linear HOCA of memory size n over Zm form a class that is indistinguishable from a specific subclass of LCA over Zmn. This enables to decide injectivity and surjectivity for linear HOCA of memory size n over Zm using the decidable characterization provided in [2] and [25] for injectivity and surjectivity of LCA over Zmn. Finally, we prove an equivalence between LCA over Zmn and an important class of non-uniform CA, another variant of CA used in many applications.

Loop Closure Detection Via Maximization of Mutual Information

An image can be described in terms of appearance frequency of visual words. This representation is implemented in bag-of-visual-words (BoVW)-based loop closure detection for its efficiency and effectiveness. However, traditional BoVW-based approaches are strongly affected by false positive loops due to scene ambiguity caused by redundant words in the vocabulary and fail to detect bidirectional loops in monocular mode. Aiming at overcoming these problems, we propose a novel vocabulary construction algorithm named hierarchical sequential information bottleneck (HsIB) by leveraging the maximization of mutual information (MMI) mechanism. First, feature descriptors are extracted from training images for visual vocabulary construction. Second, HsIB extracts discriminative yet informative visual words through the MMI mechanism in vocabulary construction, which treats feature descriptors clustering as a process of data compression. Finally, the clustering process reaches a tradeoff between compactness and discrimination and improves the performance of traditional BoVW-based loop closure detection. The proposed method is compared with state-of-the-art methods on publicly available datasets. We also create a challenging dataset to further evaluate the performance of HsIB on bidirectional loops. To the best of our knowledge, we are the first to implement information bottleneck (IB) method in visual-SLAM (vSLAM) loop closure detection, and we obtain impressive results.