Data Compression Applications Research Articles

Application of optimized sparse encoding algorithm in data compression

Data transmission is crucial in the process of equipment monitoring. The compression algorithms are adopted to reduce the amount of data transmission. When sparse encoding algorithms are used for this purpose, despite extensive attempts to improve their performance, some problems still remain. The main issues pertain to: (1) Dictionary construction − As practical dictionary better matches the input signal, such a dictionary must be efficiently established; 2) The setting of sparsity value − It is difficult to determine the size of sparsity values when prior knowledge is insufficient (when the parameter settings are too large, the cost of data transmission increases considerably, while excessively small values can result in poor reconstruction accuracy); and (3) How to implement engineering applications of data compression based on sparse encoding. To address these issues, in this work, an optimized sparse encoding algorithm is proposed by combining the non-negative Online Dictionary Learning (ODL) with the optimized Orthogonal Matching Pursuit (OMP) algorithm. First, a non-negative ODL algorithm is adopted for dictionary learning based on the training dataset to obtain an effective dictionary. Next, the optimized OMP algorithm is used to obtain the sparsity and the sparse coefficient matrix. Further analyses confirm that the reconstructed signal has a small reconstruction error. Finally, A lossy compression framework is proposed for the compressing equipment condition monitoring data using sparse encoding algorithms. Compared with the compression algorithms based on compressive sensing and DCT, the average compression ratio can reach 42.7 when the reconstruction accuracy is similar. In terms of comprehensive compression performance, the quality score is also higher compared to other algorithms.

Efficient GPU implementation of randomized SVD and its applications

Matrix decompositions are ubiquitous in machine learning, including applications in dimensionality reduction, data compression and deep learning algorithms. Typical solutions for matrix decompositions have polynomial complexity which significantly increases their computational cost and time. In this work, we leverage efficient processing operations that can be run in parallel on modern Graphical Processing Units (GPUs), predominant computing architecture used e.g. in deep learning, to reduce the computational burden of computing matrix decompositions. More specifically, we reformulate the randomized decomposition problem to incorporate fast matrix multiplication operations (BLAS-3) as building blocks. We show that this formulation, combined with fast random number generators, allows to fully exploit the potential of parallel processing implemented in GPUs. Our extensive evaluation confirms the superiority of this approach over the competing methods and we release the results of this research as a part of the official CUDA implementation.11https://docs.nvidia.com/cuda/cusolver/index.html.

Methodology of the countries’ economic development data analysis

The paper examines the issue of improving the methods of identification of economic objects and their analysis using algorithms of intelligent data processing. The use of the developed methodology in the economic analysis allows for improvement in the quality of management. It can be the basis for creating decision support systems to prevent potentially dangerous changes in the economic status of the research object. In this work, an improved method of c-means data clustering with agent-oriented modification is proposed, and a radial-basis neural network and its extension are proposed to determine whether the obtained clusters are relevant and to analyze the informativeness of state variables and obtain a subset of informative variables. The effect of applying data compression using an autoencoder on the accuracy of the methods is also considered. According to the results of testing of the developed methodology, it was proved that the probability of incorrect determination of the state was reduced when identifying the states of economic systems, and a reduced value of the error of the third kind was obtained when classifying the states of objects.

Research on two-way intelligent self-service system of electric power business hall applying data compression algorithm

Abstract This paper applies a data compression algorithm to designing a two-way intelligent self-service system for an electric power business hall and proposes a multiwavelet embedded zero-tree coding method and compression of electric power data. Based on the multiwavelet transform, a multiwavelet threshold power data compression algorithm is proposed, and the decomposition reconstruction comparison of the multiwavelet transform, and the implementation process of the compression algorithm are discussed. The implementation effect of the electric power intelligent business hall is discussed through the evaluation analysis of the service before and after the intelligent business hall. The results show that the platform operation effect score of the electric power intelligent service management platform project of company H is 0.8723. This paper's platform design and implementation provide useful research references for improving the service quality and efficiency of the electric power business hall.

The Advance of Dialogue System with Neural Network

With the rapid reform of artificial intelligence, the dialogue system as an important branch of artificial intelligence has also developed greatly in recent years. However, dialogue systems also face many challenges, and accurate and well-sampled training data in dialogue systems is often not readily available. How to use the relationship extraction model based on meta-learning from a small number of samples to improve the accuracy of the relationship prediction method on few-shot learning training data. In addition, this article also explores methods of data compression, summarizing previous compression methods in different ways and related work. At the same time, the application of data compression in the dialogue chat model is also studied. The comparison before and after data compression is carried out to make the dialogue chat model maintain the original answer quality when the data part is reduced, and how to use knowledge graph and reinforcement learning to improve the answer quality of the dialogue system after data compression.

Image Splicing Compression Algorithm Based on the Extended Kalman Filter for Unmanned Aerial Vehicles Communication

The technology of unmanned aerial vehicles (UAVs) is extensively utilized across various domains due to its exceptional mobility, cost-effectiveness, rapid deployment, and flexible configuration. However, its limited storage space and the contradiction between the large amount of image data returned and the limited bandwidth in emergency scenarios necessitate the development of a suitable UAVs image compression technology. In this research, we propose an image splicing compression algorithm based on the extended Kalman filter for UAV communication (SEKF-UC). Firstly, we introduce the concept of image splicing to enhance the compression ratio while ensuring image quality, leveraging the characteristics of deep learning-based compression, which are relatively unaffected by the size of the compressed data. Secondly, to address the issue of slow processing speed of deep neural networks (DNN) caused by the substantial volume of data involved in image splicing, we incorporate an extended Kalman filter module to expedite the process. Simulation experiments show that our proposed algorithm outperforms existing methods in multiple ways, achieving a significant compression ratio improvement, from 2:1 to 25:1, with a marginal reduction of 6.5% in structural similarity (SSIM) compared to the non-spliced approach. Moreover, for deep neural networks (DNN), our method incorporating the extended Kalman filter module achieves the same error level with only 30 iterations—a significant reduction compared to the traditional BP algorithm’s requirement of over 4000 iterations—while improving the average network operation speed by an impressive 89.76%. Additionally, our algorithm excels in image quality, with peak signal-to-noise ratio (PSNR) improving by 92.7% and SSIM by 42.1%, at most, compared to other algorithms. These results establish our algorithm as a highly efficient and effective solution, suitable for various image processing and data compression applications.

Boundary-constrained interpretable image reconstruction network for deep compressive sensing

Compressive Sensing (CS) networks have received increasing attention due to their wide applications in data compression, single pixel imaging, medical imaging, etc. However, there are several difficult problems for CS reconstruction. First, many early CS methods treat deep networks as black boxes. Second, some of the newest methods combine iterative optimizers with neural networks to make their architecture explainable, but their performances need to be further improved. Third, most CS reconstruction methods based on deep learning ignore the significance of edge information, greatly limiting the representation ability of their reconstruction models. To this end, this paper proposes a boundary-constrained CS reconstruction model, and we cast a boundary-constrained CS reconstruction problem as two subproblems, which can be alternatively optimized. The alternatively-iterative optimization process can be expanded into an Edge Guided Interpretable image compressive sensing Network (EGINet) for image reconstruction. The proposed EGINet has three modules: an edge-aware feature-extraction module, an edge-guided intermediate-variable updating module and an intermediate-variable guided image reconstruction module. To resolve the problem of information loss in the iterative process, we introduce a multiple-memory enhancement mechanism to explore feature dependency across different iteration phases of EGINet. Meanwhile, we design a parallel-cross fusion module to selectively fuse boundary features and image features. A large number of experimental results have shown that, compared with both black-box and explainable CS reconstruction methods, the proposed EGINet has better reconstruction performances in terms of SSIM and PSNR, especially on recovering image boundary information, while maintaining network interpretability.

Information loss and run time from practical application of quantum data compression

We examine information loss, resource costs, and run time from practical application of quantum data compression. Compressing quantum data to fewer qubits enables efficient use of resources, as well as applications for quantum communication and denoising. In this context, we provide a description of the quantum and classical components of the hybrid quantum autoencoder algorithm, implemented using IBMs Qiskit language. Utilizing our own data sets, we encode bitmap images as quantum superposition states, which correspond to linearly independent vectors with density matrices of discrete values. We successfully compress this data with near-lossless compression using simulation, and then run our algorithm on an IBMQ quantum chip. We describe conditions and run times for training and compressing our data on quantum devices, and relate trainability to specific characteristics and performance metrics of our parametric quantum circuits.

Single planar photonic chip with tailored angular transmission for multiple-order analog spatial differentiator

Analog spatial differentiation is used to realize edge-based enhancement, which plays an important role in data compression, microscopy, and computer vision applications. Here, a planar chip made from dielectric multilayers is proposed to operate as both first- and second-order spatial differentiator without any need to change the structural parameters. Third- and fourth-order differentiations that have never been realized before, are also experimentally demonstrated with this chip. A theoretical analysis is proposed to explain the experimental results, which furtherly reveals that more differentiations can be achieved. Taking advantages of its differentiation capability, when this chip is incorporated into conventional imaging systems as a substrate, it enhances the edges of features in optical amplitude and phase images, thus expanding the functions of standard microscopes. This planar chip offers the advantages of a thin form factor and a multifunctional wave-based analogue computing ability, which will bring opportunities in optical imaging and computing.

Retraction Note: Augmented Recurrence Hopping Based Run-Length Coding for Test Data Compression Applications

Retraction Note: Augmented Recurrence Hopping Based Run-Length Coding for Test Data Compression Applications

FFT-PEEC: A Fast Tool From CAD to Power Electronics Simulations

A fast and general partial element equivalent circuit (PEEC) method based on the fast Fourier transform (FFT) is proposed for the first time. The numerical tool only requires common CAD data input files (e.g., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$.\text{stl}$</tex-math></inline-formula> format), then the discretization process is performed automatically by means of a fast voxelization technique based on ray intersection, thus, drastically reducing the human effort required to setup the model. The method allows for considering at the same time inductive and capacitive effects, and is focused on power electronics applications where propagation effects can be neglected, whereas all the other electromagnetic phenomena are considered. Specifically, the proposed method is particularly suited for problems where both electric and magnetic fields are equally important and, therefore, quasistatic approximations do not apply. An ad-hoc preconditioner which significantly speeds-up the solver is also proposed and, thanks to the FFT, both memory and computation time are significantly reduced, without the need of applying data compression. Both linear and nonlinear materials are considered by the proposed FFT-PEEC method. Sample implementation of the method is made publicly available.

Comprehensive practical evaluation of wired and wireless internet base smart grid communication

Internet-based communications is a key solution for enabling low-cost and scalable communication infrastructure for different applications of the smart grid. However, the performance of this network needs to be evaluated practically in the context of smart grid applications based on key metrics such as latency and reliability. This article is a comprehensive evaluation of the United Kingdom Internet network characteristics which will allow the smart grid systems designer to consider the essential parameters for communication applications. This article will focus not only on three smart grid applications, but also the outcome of this research which is relevant to a wider range of Internet of Things (IoT) applications. Different combinations of off-the-shelf wired and wireless last-mile communication technologies are evaluated using real-world transport protocols such as the Transport Control Protocol (TCP)and the User Datagram Protocol (UDP). The performance of TCP/UDP has been tested in a realistic client-server communication test-bed. The results from extensive evaluations show that typical latency values are between 200 and 600 ms for data packets and 50 bytes and kbytes for short control packets. Moreover by applying data compression techniques ,the results can be improved about 5%–20% for different last mile communications.

Multi-level adaptive neuro-fuzzy inference system-based reconstruction of 1D ISOMAP representations

This paper addresses the inverse problem of isometric feature mapping (ISOMAP) via multi-level adaptive neuro-fuzzy inference system (ML-ANFIS). ISOMAP is a conventional nonlinear dimensionality reduction (NLDR) method, which prospects for low dimensional interior structure embedded in high dimensional data space. The inverse problem of ISOMAP reconstructs the original high dimensional data from the related low dimensional ISOMAP representations and holds promising applications in data representations, generation, compression and visualization. Because the reconstruction of 1D ISOMAP representations is ill-posed and undetermined, ML-ANFIS is wielded to augment the recovery quality of general ISOMAP reconstruction algorithm. By linearly combing inputs with nonlinear weights as output, ML-ANFIS can efficiently achieve the latent nonlinear relationship between the low-performance result of general ISOMAP reconstruction algorithm and its original data. The membership functions and fuzzy rules of ML-ANFIS can be automatically learned by gradient descent method as deep learning. It is demonstrated by the experimental results that, in the situation of 1D representations, the proposed method is superior to the state-of-the-art methods, such as nearest neighbor (NN), discrete cosine transformation (DCT), sparse representations (SR) and classical ANFIS algorithms, in the reconstruction performance of video data.

Solving longest common subsequence problems via a transformation to the maximum clique problem

Longest common subsequence problems find various applications in bioinformatics, data compression and text editing, just to name a few. Even though numerous heuristic approaches were published in the related literature for many of the considered problem variants during the last decades, solving these problems to optimality remains an important challenge. This is particularly the case when the number and the length of the input strings grows. In this work we define a new way to transform instances of the classical longest common subsequence problem and of some of its variants into instances of the maximum clique problem. Moreover, we propose a technique to reduce the size of the resulting graphs. Finally, a comprehensive experimental evaluation using recent exact and heuristic maximum clique solvers is presented. Numerous, so-far unsolved problem instances from benchmark sets taken from the literature were solved to optimality in this way.

Least-Squares Fuzzy Transforms and Autoencoders: Some Remarks and Application

In this article, analogies and differences between a type of fuzzy transform and a type of autoencoder, both based on a least-squares optimization, will be discussed. Such schemes have been recently introduced in the literature in different contexts. In particular, in this article, the data compression application will be considered. As it will be shown, the least-squares fuzzy transform can be regarded as a kind of autoencoder with a lower computational cost, without losing accuracy. The numerical comparison against existing results for the considered application shows the good performance of the fuzzy transform based approach.

InP AAC for data compression applications

The authors propose a 2 × 2 asymmetric adiabatic coupler (AAC) indium phosphide (InP) photonic integrated circuit (PIC), suitable for tunable power applications such as all-optical data processing. The device was idealised as the key building block in Haar transform network used for image compression, allowing to perform the necessary separate addition and subtraction of incoming input signals at its output ports. The tunable behaviour of the coupler was demonstrated experimentally for addition/subtraction and splitting (with experimental coupling ratios of 77:23/14:86 and 50:50, respectively) through phase control. The use of the developed InP PIC enables a low footprint structure design together with the high-quality active components (lasers, photodetectors and phase modulators) of Fraunhofer Gesellschaft Heinrich Hertz Institute design toolkit. The proposed study provides a full experimental characterisation of the AAC, exploring its tuning properties and enabling further usage in a plethora of applications in high processing computing and data communications.

KOMPRESI DATA CITRA EFISIENSI PENYIMPANAN DAN PENGIRIMAN PADA MEDIA KOMUNIKASI MOBILE

In data communication between work networks, especially image data or image files, messages sent are often of very large size so that sending time is long, as a result the network becomes busier, it can even result in corrupted image data, as well as in image data storage or large image files take up large storage space. Both of these problems can be overcome by encoding the message or the contents of the archive as short as possible, so the message delivery time is also relatively fast, and the storage space required is also small. This method of coding is called data compression or compression. In this study the Run Length Encoding Method is used for compression or compression of image or image data. The study was conducted using the literature study method, testing the type of sample data and making comparison tables. The results obtained provide input regarding the implementation of Image Data Compression (Image) applications and contribute to computer network, internet, intranet or extranet users in sending image data, both for personal and organizational needs. The final results of this study, the size of the image file or image can be reduced to 95.23% of the actual size (can be seen in table 1). Keywords : compression, decompression, image data, Â Run Length Encoding

Analisis Performa Rasio Kompresi Pada Metode Differensiasi ASCII Dan Lempel Ziv Welch (LZW)

ASCII differentiation is a compression method that utilizes the difference value or the difference between the bytes contained in the input character. Technically, the ASCII differentiation method can be done using a coding dictionary or using windowing block instead of the coding dictionary. Previous research that has been carried out shows that the ASCII differentiation compression ratio is good enough but still needs to be analyzed on performance from the perspective of the compression ratio of the method compared to other methods that have been widely used today. In this study an analysis of the comparison of the ASCII Difference method with other compression methods such as LZW will be carried out. The selection of LZW itself is done by reason of the number of data compression applications that use the method so that it can be the right benchmark. Comparison of the compression ratio performed shows the results of ASCII differentiation have advantages compared to LZW, especially in small input characters. Whereas in large input characters, LZW can optimize the probability of pairs of characters that appear compared to ASCII differentiation which is glued to the difference values ​​in each block of input characters so that in large size characters LZW has a greater compression ratio compared to ASCII differentiation.

IMPLEMENTASI KOMPRESI CITRA DIGITAL DENGAN MENGATUR KUALITAS CITRA DIGITAL

Image is one of the most popular media used in today’s information exchange. Increased needs for the use of images must also be supported by available storage media. The imagery generated from each high-resolution camera device has a relatively large size. Image compression is a data compression application performed on digital image in order to reduce the redundancy of the data contained in the image so that it can be stored or transmitted efficiently.Image compression techniques can be grouped into two namely lossless compression and lossy compression. Lossy compression is a method to compress the image where compressed image decompression results are not the same as the original image because there is missing information, but still can be tolerated by the eye. Lossless compression is an image compression where the decompression of compressed images is the same as the original image, no information is lost. For the following image compression by reducing image quality that can still be tolerated by the eye.

Real Time Object Tracking and Protection: Hybrid Logistics Architecture in Ubiquitous Environments (HLAiUE)

Realtime object security and vehicle tracking have not been successfully implemented in commercial vehicles due to limitations in processing large amount of data and hardware capabilities. This paper aimed to improve the object security, processing power and data storage in a distributed environment. The proposed system (HLAiUE) consists of a three-layer architecture, first layer is used to collect data, second layer distributes the data processing and the last layer stores the processed data. The designed architecture improves the performance and security by applying data compression algorithms in a real time and removes the hardware dependency by processing the data in a distributed environment using Hadoop and Spark framework. The results demonstrate that the proposed architecture improves processing time by 80% compared to other algorithms and provides more reliability, security and flexibility due to ubiquity and the absence of hardware dependency in comparison with other existing architectures which are hardware dependent. Also, using an NOSQL database server in a distributed environment optimizes data storage by up to 80% and is flexible because all the infrastructure is in a distributed environment. The proposed architecture improves the object tracking by implementing compression technique in a distributed architecture. Thus, this system improves the accuracy up to 80% without depending on hardware resources.