Data Compression Research Articles

Clustering protocols for energy efficiency analysis in WSNS and the IOT

Throughout the development of energy efficient routing protocol for wireless sensor network clustering technique has been widely adopted approach. The Selection of cluster heads is also very important for the energy efficiency of the network. In the past, researchers have proposed multiple routing protocols; however, problems are still alive and need to be resolved because of the diversity of WSN applications. This article presents an energy-efficient routing protocol using the advanced approaches of Artificial Intelligence, the most promising field of computer science currently providing the best solutions. The proposed model uses the Deep Q-network to select the cluster head. Moreover, collected data at the cluster head is generalized as low, moderate, and high values using the fuzzy logic technique. After that, the Predictive coding theory algorithm is used for the data compression, and the lossy compression technique is applied to the data. Its compressed form also gives complete information of the data in small size and is delivered to the base station. Again, the transmitted data is reconstructed into its actual format. In the end, to justify the performance of the newly designed routing protocol, simulations are performed using the Matlab tool, and its results are evaluated in quality of service matrices and compared with well-known routing protocols.

Dual-input ultralight multi-head self-attention learning network for hyperspectral image classification

ABSTRACT In the hyperspectral image (HSI) classification tasks, various deep learning models have achieved remarkable success. However, most deep learning models are compute-intensive, requiring significant computing power, time, and other resources. It becomes a challenge to pursue better results while saving computational resources. Therefore, a novel dual-input ultralight multi-head self-attention learning network (DUMS-LN) is proposed for HSI classification. The proposed DUMS-LN consists of three main core modules, namely the high-dimensional reduced module (HDRM), lightweight multi-head self-attention (LMHSA) module, and linearized hierarchical conversion module (LHCM). HDRM is used as a pre-processing module with efficient data compression and combines spatial and spectral information extraction from the raw data to provide cleaner and more comprehensive feature data for subsequent processing. In addition, the core computational module of DUMS-LN is the LMHSA module, which is lightweight but possesses better data processing capability than the traditional multi-head self-attention module. Finally, the LHCM divides the model into two phases, reducing the dimensionality of the feature data phase by phase so that the LMHSA module can perform feature extraction at different levels. Experiments on four benchmark HSI datasets show that the proposed DUMS-LN outperforms the comparison HSI classification algorithms regarding speed and classification accuracy.

Test and performance of the LiTE-DTU ASIC for the HL-LHC upgrade of the CMS ECAL barrel

A data conversion and compression ASIC, named LiTE-DTU, has been developed for the upgrade of the CMS electromagnetic calorimeter (ECAL) for the High-Luminosity phase of LHC. The ASIC integrates two 12-bit 160 MS/s ADCs, a data processing unit for gain selection and data compression, and a 1.28 Gb/s serializer.The ASIC has been extensively tested in laboratory and in beam tests showing excellent yield and performance. The radiation tolerance has been verified with dedicated test campaigns for both total ionizing dose and single event effects. Results from these tests, showing the design readiness for mass production, will be presented.

PairwiseHist: Fast, Accurate and Space-Efficient Approximate Query Processing with Data Compression

Exponential growth in data collection is creating significant challenges for data storage and analytics latency. Approximate Query Processing (AQP) has long been touted as a solution for accelerating analytics on large datasets, however, there is still room for improvement across all key performance criteria. In this paper, we propose a novel histogram-based data synopsis called PairwiseHist that uses recursive hypothesis testing to ensure accurate histograms and can be built on top of data compressed using Generalized Deduplication (GD). We thus show that GD data compression can contribute to AQP. Compared to state-of-the-art AQP approaches, Pairwise-Hist achieves better performance across all key metrics, including 2.6× higher accuracy, 3.5× lower latency, 24× smaller synopses and 1.5--4× faster construction time.

On-beam system test of the new readout electronics for the CMS Electromagnetic Calorimeter upgrade

Starting in 2029, the High Luminosity LHC (HL-LHC) at CERN is projected to achieve unparalleled instantaneous luminosity (7.5 × 1034 cm-2s-1) for an integrated luminosities of 4500 fb-1. To cope with the extreme conditions of 200 collisions per bunch crossing (pile-up), both on-detector and off-detector electronics of the CMS Electromagnetic Calorimeter (ECAL) will be upgraded. The new system will incorporate a dual-gain trans-impedance amplifier alongside an Application-Specific Integrated Circuit (ASIC) that includes dual 160 MHz ADC channels and features for gain selection and data compression. The lead tungstate crystals and avalanche photodiodes (APDs) will remain in service due to their proven reliability. To counteract the noise amplification in the APDs, attributed to radiation-induced dark currents, the operational temperature of the ECAL will be reduced from 18°C to roughly 9°C. Preliminary tests of these new electronics have yielded promising results. During tests beams at the CERN H4 beamline using an electron beam, the new electronics met both required energy resolution and linearity and achieved a timing resolution of 30 ps for energies above 50 GeV.

A DEEP LEARNING TECHNIQUE FOR EFFICIENT MULTIMEDIA FOR DATA COMPRESSION

Medical image compression plays a pivotal role in efficient data storage and transmission, crucial for modern healthcare systems. This research proposes a convolutional transfer learning technique scheme tailored for multimedia data compression, specifically targeting medical images. In the background, the growing volume of medical imaging data and the demand for efficient storage and transmission underscore the need for innovative compression methods. Leveraging transfer learning from pre-trained convolutional neural networks (CNNs) designed for image recognition tasks, our methodology optimizes the compression process for medical images. The proposed scheme utilizes a pre-trained CNN’s feature extraction capabilities to capture relevant patterns in medical images, followed by fine-tuning on a specialized dataset. This approach capitalizes on the inherent ability of CNNs to learn hierarchical representations, enhancing the compression model’s adaptability to medical imaging nuances. The contribution of this research lies in the development of a tailored transfer learning scheme that effectively balances generic feature extraction and domain-specific adaptation for medical images. Results demonstrate significant improvements in compression efficiency, preserving diagnostic information while achieving substantial data reduction. The proposed scheme showcases promise for enhancing medical image storage, transmission, and retrieval systems, contributing to the advancement of healthcare technology.

Cosmological Inflation in N-Dimensional Gaussian Random Fields with Algorithmic Data Compression

Cosmological Inflation in N-Dimensional Gaussian Random Fields with Algorithmic Data Compression

Quantum variational distance-based centroid classifier

Quantum machine learning is a rapidly growing field with the potential to surpass classical methods and offer novel solutions to learning problems. Supervised classification, a prominent pattern recognition task, has attracted significant attention for improving existing methods using quantum techniques. Although previous quantum algorithms for distance-based classification have shown promise, they are not compatible with current Noisy Intermediate-Scale Quantum (NISQ) devices and cannot handle multiclass problems. To address these limitations, we present the quantum Variational Distance-based Centroid Classifier (qVDCC), a hybrid approach that combines the Quantum One-Class Classifier (QOCC) and Parameterized Quantum Circuits (PQCs). This approach greatly improves the feasibility of multiclass classification on NISQ devices, as it eliminates the need for a label qubit and uses a single-qubit measurement, regardless of data size. Our simulations on seven benchmark datasets using both noisy and noise-free simulations show that qVDCC is competitive with the k-nearest neighbors algorithm, while providing exponential data compression and efficient quantum data processing.

Low‐frequency seismic deghosting in a compressed domain using parabolic dictionary learning

AbstractDeghosting is an important technique in the marine seismic industry, as it plays a crucial role in mitigating the effects of ghost reflections from the sea surface, which can significantly impact the accuracy and resolution of subsurface imaging. In recent years, various acquisition‐based techniques have been developed to tackle the challenge of removing receiver–ghost reflections, which is the focus of our paper. These state‐of‐the‐art approaches, such as dual‐sensor or multicomponent towed streamer acquisitions, have demonstrated exceptional accuracy by combining pressure and particle motion data. However, such methods face limitations when dealing with low frequencies due to heavy noise contamination in the particle motion data. Consequently, ghost‐free data reconstruction at low frequencies typically relies on processing‐based approaches, which exclusively utilize recorded pressure data. This study presents a novel deghosting method for low‐frequency applications based on parabolic dictionary learning, which relies solely on recorded pressure data. The proposed method has the advantage of being applicable directly in a compressed domain, eliminating the need for data decompression prior to the deghosting process when compression is applied before the processing steps. This not only reduces costs related to data storage and transfer but also provides a cost‐effective alternative to conventional deghosting by operating directly on the compressed data format, which is smaller in size. The effectiveness of the proposed method was evaluated using both synthetic and field datasets. The results obtained from a synthetic data example indicate that the proposed method achieves similar results to an industry‐standard frequency–wavenumber method, while achieving a compression rate of over 7. Furthermore, the method was tested using a field dataset consisting of a full sail‐line of marine seismic acquisition. The comparison of 2D pre‐stack migrated images between the proposed method and the industry‐standard frequency–wavenumber method revealed insignificant differences, while achieving a compression ratio higher than 5 when our method was used.

A Multichannel, Multipulse, Multiweight Block-Adaptive Quantization (3MBAQ) Algorithm Based on Space-Borne Multichannel SAR Doppler Domain A-BAQ

Amid the conflict between the current huge data volumes and the requirement for large-compression-ratio compression in space-borne multichannel SAR, this paper proposes a data compression method that uses the adaptive bit allocation BAQ method in the Doppler domain of space-borne multichannel SAR. Specifically, 3MBAQ, which denotes multichannel, multipulse, multiweight block-adaptive quantization, performs signal reconstruction through a Krieger filter to obtain the Doppler spectrum under azimuthal multichannel and multipulse conditions and then achieves nonuniform bit allocation by using the multiple weight contributions between subband spectra. Furthermore, large-compression-ratio compression can be realized in homogeneous scenarios by using the adaptive bit allocation BAQ method together with the improved bit error control (BEC) algorithm. In this paper, the effectiveness of the 3MBAQ method is verified on GF3 measured data. Moreover, compared to the SQNR and NMSE of the BAQ, A-BAQ, A-BAQ-BEC, MCBAQ-ASQ, and MCBAQ-BEC methods under the same conditions, it is shown that there is an optimal 1.3434 dB improvement in SQNR and a 0.0241 improvement in NMSE using the 3MBAQ algorithm, and the complexity of the algorithm for 3MBAQ is O(N2log(N)), which is within acceptable limits, clearly proving the superiority of the 3MBAQ method.

An EEG monitoring method based on compressed sensing for fatigue driving

In order to reduce the amount of EEG collected by the fatigue driving monitoring system, this paper proposes a new EEG monitoring method based on compressed sensing. According to the driver’s driving duration, the new proposed method adopts different sampling rate to compressive sample the EEG of the driver in different driving stages, and then realizes the accurate reconstruction of the compressed EEG data in the computer of vehicle. The experiment verifies effectiveness of the proposed method by collecting EEG signals from multiple subjects during simulated driving. The experimental results show that, compared with traditional sampling, the new method reduces the amount of sampled data by 3 orders of magnitude. And when the compression ratio is not higher than 70%, the reconstruction error is less than 0.16.

Panoramic Radiograph Generation and Image Reconstruction from Latent Vectors Using a Generative Adversarial Network.

In this study, StyleGAN2 was trained with panoramic radiographs, and original images were projected into the latent space of StyleGAN2. The resulting latent vectors were input into StyleGAN2, and corresponding images were generated to reconstruct the original images. The original and reconstructed images were evaluated by pediatric dentists and found to be similar. Our results suggest that StyleGAN2 could be applied to the anonymization and data compression of medical images.

TinyML Algorithms for Big Data Management in Large-Scale IoT Systems

In the context of the Internet of Things (IoT), Tiny Machine Learning (TinyML) and Big Data, enhanced by Edge Artificial Intelligence, are essential for effectively managing the extensive data produced by numerous connected devices. Our study introduces a set of TinyML algorithms designed and developed to improve Big Data management in large-scale IoT systems. These algorithms, named TinyCleanEDF, EdgeClusterML, CompressEdgeML, CacheEdgeML, and TinyHybridSenseQ, operate together to enhance data processing, storage, and quality control in IoT networks, utilizing the capabilities of Edge AI. In particular, TinyCleanEDF applies federated learning for Edge-based data cleaning and anomaly detection. EdgeClusterML combines reinforcement learning with self-organizing maps for effective data clustering. CompressEdgeML uses neural networks for adaptive data compression. CacheEdgeML employs predictive analytics for smart data caching, and TinyHybridSenseQ concentrates on data quality evaluation and hybrid storage strategies. Our experimental evaluation of the proposed techniques includes executing all the algorithms in various numbers of Raspberry Pi devices ranging from one to ten. The experimental results are promising as we outperform similar methods across various evaluation metrics. Ultimately, we anticipate that the proposed algorithms offer a comprehensive and efficient approach to managing the complexities of IoT, Big Data, and Edge AI.

Load Flow Calculation For Electrical Power System Based On Run Length Encoding Algorithm

This paper presents Data Compression Simulated algorithm for load flow calculation in electrical power systems. Real time monitor of grids required less computation time in calculation of power system analysis. Load flow problem is heart of this analysis and it basically required calculate active and reactive power flow in lines connected between buses in networks. Many topology and structures for Transmission and Distribution Systems has been proposed to reduce CPU times and memory. The proposed algorithm used Data compression technique tested different systems and results shows it is efficiency. More accuracy for large systems will need more iterations calculation which mean increasing time consumption, while Run Length Encoding (RLE) algorithm is fitness to optimized calculation numbers to exact number cause it has no zero values included. Network structure was represented as one dimension vector instead of 2D Matrix and it is effectiveness results was valid, by avoid exponential increased, by utilized this algorithm. Matlab results obtained by applied this algorithm match theoretical results.

Optical cryptosystem based on computational ghost imaging and nonlinear authentication.

We propose an optical encryption system that combines computational ghost imaging (CGI) with image authentication to enhance security. In this scheme, Hadamard patterns are projected onto the secret images, while their reflected light intensities are captured using a bucket detector (BD). To further strengthen the security of the collected secret data, we encrypt it as a series of binary matrices serving as ciphertext. During the authentication key generation, these encoded binary matrices serve as illumination patterns in the CGI system for a non-secret image, which is used as a reference image for authentication. The data captured by the BD is then binarized to generate the authentication key. Upon successful authentication, the receiver obtains the decryption keys. This method achieves both data compression for secret images and enhanced security during information transmission. We validate the feasibility of this method through computer simulations and optical experiments.

Multifocal multiview imaging and data compression based on angular-focal-spatial representation.

Multifocal multiview (MFMV) is an emerging high-dimensional optical data that allows to record richer scene information but yields huge volumes of data. To unveil its imaging mechanism, we present an angular-focal-spatial representation model, which decomposes high-dimensional MFMV data into angular, spatial, and focal dimensions. To construct a comprehensive MFMV dataset, we leverage representative imaging prototypes, including digital camera imaging, emerging plenoptic refocusing, and synthesized Blender 3D creation. It is believed to be the first-of-its-kind MFMV dataset in multiple acquisition ways. To efficiently compress MFMV data, we propose the first, to our knowledge, MFMV data compression scheme based on angular-focal-spatial representation. It exploits inter-view, inter-stack, and intra-frame predictions to eliminate data redundancy in angular, focal, and spatial dimensions, respectively. Experiments demonstrate the proposed scheme outperforms the standard HEVC and MV-HEVC coding methods. As high as 3.693 dB PSNR gains and 64.22% bitrate savings can be achieved.

Prediction-based coding with rate control for lossless region of interest in pathology imaging

Online collaborative tools for medical diagnosis produced from digital pathology images have experimented an increase in demand in recent years. Due to the large sizes of pathology images, rate control (RC) techniques that allow an accurate control of compressed file sizes are critical to meet existing bandwidth restrictions while maximizing retrieved image quality. Recently, some RC contributions to Region of Interest (RoI) coding for pathology imaging have been presented. These encode the RoI without loss and the background with some loss, and focus on providing high RC accuracy for the background area. However, none of these RC contributions deal efficiently with arbitrary RoI shapes, which hinders the accuracy of background definition and rate control. This manuscript presents a novel coding system based on prediction with a novel RC algorithm for RoI coding that allows arbitrary RoIs shapes. Compared to other methods of the state of the art, our proposed algorithm significantly improves upon their RC accuracy, while reducing the compressed data rate for the RoI by 30%. Furthermore, it offers higher quality in the reconstructed background areas, which has been linked to better clinical performance by expert pathologists. Finally, the proposed method also allows lossless compression of both the RoI and the background, producing data volumes 14% lower than coding techniques included in DICOM, such as HEVC and JPEG-LS.

Scale perception data compression method with blockchain technology for UAV communication

The demand for high data throughput and security in Unmanned Aerial Vehicle (UAV) communication is rising. Efficient data transmission remains challenging due to limited wireless resources. This paper introduces a Scale Perception Compression Method that leverages compression and blockchain technology to optimize UAV communication. The method involves normalizing data into matrices, generating compression and decompression matrix codebook pairs, and selecting suitable pairs for small or large-scale matrices. Simulation results show that the error rate is associated with the distributions of the data and compression matrix. It remains relatively low when the data and compression matrix subjected to Uniform distribution compared to Gaussian distribution, regardless of the values of the number of rows m and columns r in compression matrix. Bubble method is more suitable to find out the proper matrix pair to compress and decompress transmitted data for small-scale codebook because it is stable, fast and results in lower difference between original and recovered data matrices. The same applies to Merge method for the large-scale codebook. The ER is always lower based on a small-scale codebook searching by Bubble method than that of on a large-scale codebook searching by Merge method, which indicates the former is sufficient for compression.

ПРИНЦИП СУБЪЕКТИВНОЙ ИЗБЫТОЧНОСТИ И МЕТОД СЖАТИЯ ДАННЫХ, ПОЛУЧАЕМЫХ ОТ ЦИФРОВЫХ АНТЕННЫХ РЕШЕТОК

The problem of limiting the capacity of the diagram-forming circuit of the digital antenna array is considered, due to the capabilities of existing communication lines with serial data transmission from the receiving channels to the computer. This problem limits the possibilities of implementing various methods of signal processing in digital antenna arrays. The subject of the study is the possibility of increasing the capacity of the diagram-forming circuit of the digital antenna array due to software compression of radar data. The purpose of the work is to create a scientific and methodological apparatus that will increase the capcity of the diagram-forming scheme by using the principle of subjective redundancy in relation to radar data processing tasks. The results of the work include: justification and formulation of the principle of subjective redundancy of radar data and the method of compression of radar data, as well as the results of numerical studies confirming the operability of the proposed scientific and methodological apparatus. The principle of subjective redundancy is based on the independence of the separation of signal processing processes in the spatial and temporal domains when receiving a single echo signal, and in the case of receiving several echo signals, on the principle of superposition of fields from various sources in the far antenna zone. The method of radar data compression is based on the principle of subjective redundancy of radar data and differs from the known iterative procedure for approximating the matrix of complex samples in the form of a superposition of products of one-dimensional cosine spectra. It is shown that the application of the radar data compression method makes it possible to increase the transmission capacity of the digital antenna array transmission lines several times.

Implementasi Kompresi File MP3 dengan Menerapkan Algoritma Levenstein

MP3 is an audio format that is often used because the data stored in MP3 format resembles the actual data when recorded.Storage space requirements will also rise if you keep a large number of MP3 files. Because of this, MP3 files must be compressed in order to prevent the storage space from filling up too quickly. This is because the compressed data will be lower in size. The Lossless compression method can be used to compress audio. The author uses the Levenstein Algorithm to implement audio compression in MP3 format. Levenstein's algorithm compresses data without any loss. When a file is compressed using lossless compression, the resulting file has the exact same size as the original file upon decompression. The author will be able to assess the effectiveness of MP3 file compression by using the Levenstein Algorithm to compress MP3 files. This will allow for faster transmission times and less data storage space consumption because the compressed MP3 file will shrink from its initial large size. After compression, the MP3 file's size—which was originally 128 bits—was reduced to 104 bits, yielding an 81.25% compression ratio. The purpose of this computation is to determine whether the compressed files are similar enough to be either decompressed or restored to their original size.