Real-time Data Compression Research Articles

Second-Order Level-Crossing Sampling Analog to Digital Converter for Electrocardiogram Delineation and Premature Ventricular Contraction Detection.

This article presents an electrocardiogram (ECG) delineation and arrhythmia heartbeat detection system using a novel second-order level-crossing sampling analog to digital converter (ADC) for real-time data compression and feature extraction. The proposed system consists of the front-end integrated circuit of the data converter, the delineation algorithm, and the arrhythmia detection algorithm. Compared with conventional level-sampling ADCs, the proposed circuit updates tracking thresholds using linear extrapolation, which forms a second-order level-crossing sampling ADC that has sloped sampling levels. The computing is done digitally and is implemented by modifying the digital control logic of a conventional Successive-approximation-register (SAR) ADC. The system separates the sampling and quantization processes and only selects the turning points in the input waveform for quantization. The output of the proposed data converter consists of both the digital value of the selected sampling points and the timestamp between the selected sampling points. The main advantages are data savings for the data converter and the following digital signal processing or communication circuits, which are ideal for low-power sensors. The test chip was fabricated using a 180 nm CMOS process. When sensing sparse signals such as ECG signals the proposed ADC achieves a compression factor of 8.33. The delineation algorithm uses a triangle filter method to locate the fiducial points and measures the intervals, slopes, and morphology of the QRS complex and the P/T waves. Those extracted features are then used in the arrhythmia heartbeat detection algorithm to identify Premature Ventricular Contraction (PVC). The overall performance of the system is evaluated using the MIT-BIH database and the QT database, which is also compared with the recently reported systems. The accuracy, sensitivity, specificity, PPV, and F1 score are 97.3%, 89.6%, 97.8%, 73.3%, and 0.81 for detecting PVC.

Notice of Removal: A Triggering-and-Encoding Lossless Compression Scheme for Waveform Capturing Rate Enhancement of Android Bluetooth Oscilloscope

Android-based oscilloscopes have been developed for research and education purpose because they would be able to acquire, transmit, display and analyze any electrical signals and be mobile, operable easily. Due to the speed limitation of wireless transmission channel, Android Oscilloscopes have some limited performances such as bandwidth, waveform capturing rate (WCR) and so on. In this paper, we propose a real-time lossless data compression scheme to solve a bottleneck of continuous data flow from DAQ to smartphone. This compression scheme consists of triggering a signal and encoding differences between waveforms. Also, an advanced structure of Android Bluetooth Oscilloscope is proposed to implement the proposed compression algorithm with FPGA-based hardware and software. To evaluate WCR improvement, we firstly analyze relationship between WCR and compression ratio. And then the compression efficiency is verified by the MATLAB simulations using various waveforms such as simple sinusoidal, complex periodic, square, chirp waveforms. We finally study robustness of this instrument in the range of 0~15% noise amplitude and 0~2rad phase offset. All experimental results show that proposed scheme can enhance WCR and also be applied to the wireless transmission fields collecting and displaying electrical waveforms.

Real-time Synchrophasor Data Compression Technique with Phasor Interpolation and Extrapolation

Real-time Synchrophasor Data Compression Technique with Phasor Interpolation and Extrapolation

АЛГОРИТМ ПОТОКОВОЙ КОМПРЕССИИ ДАННЫХ С ПЛАВАЮЩЕЙ ЗАПЯТОЙ В ИНФОРМАЦИОННЫХ СИСТЕМАХ ОБЕСПЕЧЕНИЯ НАУЧНЫХ ЭКСПЕРИМЕНТОВ

The paper presents an original algorithm and its implementation for single pass real-timecompression of streams of numeric floating-point data. The purpose of the research is to developand formalize a single-pass algorithm of stream floating-point data compression in order to increaseperformance of both encoding and decoding, because a use of existing implementationsprovides insufficient speed of compression, are too restrictive on hardware resources and limitedin applicability to real-time stream data compression when it comes to floating-point data.For that, the following issues have been addressed. The developed mathematical model and thealgorithm for compression of scalar floating-point data are described together with results of experimentalresearch of the compression method applied to single-dimensional and twodimensionalscientific data. The model is based upon the commonly-used binary_64 representation,of the IEEE-754 standard, onto which extended real-line values are mapped. The algorithmcan be implemented as part of high-performance distributed systems in which performance ofinput-output operations, as well as internetwork communication, are critical to overall efficiency.The performance and applicability of the algorithm in data stream compression result from itssingle-pass behaviour, relatively low requirements to a priori known and statically defined size ofmemory required to implement history of compression, which the predictor, used in compressionand decompression, is based on. Indeed, the measured compression ratios are comparable to oneswhich result from using more resource-intensive universal coders but providing significantly lowerlatency. Provided synchronization of parameters of both compressor and decompressor applied toa stream of vector values and assuming a correlation between absolute values of scalars of thesame dimension within the vectors, further improvement of the predictor performance can be attainedby means of SIMD-class parallelism which, in turn, is beneficial for overall performance ofcompression and decompression, provided that the underlying hardware is capable of addressingrandom-access memory based on offsets in a vector register, such as by employment of theVGATHER class instructions of Intel processors. In order to reduce the bottlenecks associatedwith input-output, an implementation of the algorithm is employed by the authors as part of acomputing system used for parallel simulation of wave fields which is distributed via a network.The experiments described in the paper demonstrate significant performance increase of the proposedcoder compared to well-known universal compressors, RAR, ZIP and 7Z, while the achievedcompression factors remain comparable.

Dynamic Rearrangement Compression Algorithm for Intelligent Connected Vehicles

The intelligent vehicle terminal T-BOX of intelligent connected vehicles (ICVs) is directly connected to the vehicle interface and constantly exchanges real-time data inside and outside the vehicle, which significantly increases the communication pressure on the in-vehicle network. Excessive data will increase the load on the channel due to the limited bandwidth of the vehicle. Although the existing data compression algorithm can effectively reduce the communication pressure, it requires significant driving data to count the number of signal changes, which increases the preliminary labor cost. Meanwhile, it is difficult to meet the stable compression rate using a static configuration when a large amount of CAN data is transmitted with fluctuating data changes, which cannot satisfy the requirement of reducing the bus load rate, and satisfying the real-time requirements of ICV is challenging. Thus, we propose a real-time dynamic data compression algorithm for vehicles to satisfy such requirements. The experiments show that for the frequently changing signals, the proposed algorithm can improve the compression rate by about 10% compared with the existing compression algorithm. The execution time of the compression process is 4.55 μs at a clock frequency of 400 MHz and the decompression process execution time is 86.48 μs. Compared with the bus load when the original message is transmitted, the proposed algorithm can reduce the bus load by 19% on average. In the future, a stable compression rate will be crucial when loading authentication codes on the CAN to ensure its security.

A new real-time lossless data compression algorithm for ECG and PPG signals

ObjectiveData compression is a useful process in tele-monitoring applications, in which lesser number of bits are needed to represent the same data. In this work, a run-time lossless compression of single-channel Electrocardiogram (ECG) and Photoplethysmogram (PPG) signals is proposed, maintaining all dominant features. MethodsThe single-channel data are first quantized using optimal quantization level, so that fewer number of bits are needed to represent it, maintaining low quantization error. Then, second order delta encoding and run-length encoding (RLE) based data compression are proposed in this work. A new approach of using ‘buffer array’ along with RLE is also introduced, so that minimum bits are needed to store. ResultsThis algorithm was tested on various single-lead ECG and PPG signals available in Physionet. An average compression ratio (CR) was achieved of 6.52, 3.82, and 2.49 for 547 PTBDB ECG records, 48 MITDB ECG records, and 53 MIMIC-II PPG records, respectively. This algorithm was also performed on single-channel ECG, collected from 10 healthy volunteers using AD8232 ECG module, with 125 Hz sampling frequency and 10-bit data resolution, which resulted in average CR of 2.34. ConclusionThis algorithm was also performed on a smartphone device that provided user-friendly operation. The low computational complications and standalone operation of data collection, compression, and transmission encouraged its implementation for run-time operation. SignificanceA comparative study of the proposed work with previously published works proved this fact that this algorithm provided better performance in the area of run-time patient health monitoring applications.

Review on Compressive Sensing Algorithms for ECG Signal for IoT Based Deep Learning Framework

Nowadays, healthcare is becoming very modern, and the support of Internet of Things (IoT) is inevitable in a personal healthcare system. A typical personal healthcare system acquires vital parameters from human users and stores them in a cloud platform for further analysis. Acquiring fundamental biomedical signal, such as with the Electrocardiograph (ECG), is also considered for specific disease analysis in personal healthcare systems. When such systems are scaled up, there is a heavy demand for internet channel capacity to accommodate real time seamless flow of discrete samples of biomedical signals. So, there is a keen need for real time data compression of biomedical signals. Compressive Sensing (CS) has recently attracted more interest due to its compactness and its feature of the faithful reconstruction of signals from fewer linear measurements, which facilitates less than Shannon’s sampling rate by exploiting the signal sparsity. The most common biomedical signal that is to be analyzed is the ECG signal, as the prediction of heart failure at an early stage can save a human life. This review is for a vast use-case of IoT framework in which CS measurements of ECG are acquired, communicated through Internet to a server, and the arrhythmia are analyzed using Machine learning (ML). Assuming this use-case specific for ECG, in this review many technical aspects are considered regarding various research components. The key aspect is on the investigation of the best sensing method, and to address this, various sensing matrices are reviewed, analyzed and recommended. The next aspect is the selection of the optimal sparsifying method, and the review recommends unexplored ECG compression algorithms as sparsifying methods. The other aspects are optimum reconstruction algorithms, best hardware implementations, suitable ML methods and effective modality of IoT. In this review all these components are considered, and a detailed review is presented which enables us to orchestrate the use-case specified above. This review focuses on the current trends in CS algorithms for ECG signal compression and its hardware implementation. The key to successful reconstruction of the CS method is the right selection of sensing and sparsifying matrix, and there are many unexplored sparsifying methods for the ECG signal. In this review, we shed some light on new possible sparsifying techniques. A detailed comparison table of various CS algorithms, sensing matrix, sparsifying techniques with different ECG dataset is tabulated to quantify the capability of CS in terms of appropriate performance metrics. As per the use-case specified above, the CS reconstructed ECG signals are to be subjected to ML analysis, and in this review the compressive domain inference approach is discussed. The various datasets, methodologies and ML models for ECG applications are studied and their model accuracies are tabulated. Mostly, the previous research on CS had studied the performance of CS using numerical simulation, whereas there are some good attempts for hardware implementations for ECG applications, and we studied the uniqueness of each method and supported the study with a comparison table. As a consolidation, we recommend new possibilities of the research components in terms of new transforms, new sparsifying methods, suggestions for ML approaches and hardware implementation.

Spectral Variation-Based Signal Compression Technique for Gapless Power Quality Waveform Recording in Smart Grids

The current scenario of electric power systems expansion requires the insertion of new devices and technologies, contributing to the establishment of smart grids. However, the more instruments are connected to the grid, the greater the amount of data resulting from the measurements, which can cause a bottleneck in data transferring. The present article addresses this problem by proposing a system that implements a novel and efficient real-time data compression method based on the power signal spectral content variation, focused on long-term gapless oscillographic analysis. This system also provides a high-fidelity reconstruction for full oscillography power quality analysis. Comparison results with synthetic and real signals using another gapless power quality disturbance recorder present in literature and a commercial power quality disturbance recorder show that the proposed technique is capable to achieve a compression rate more than ten times greater than both while providing a better reconstruction quality, reaching compression ratio values of up to 10780:1 for synthetic signals and 9470:1 for real signals.

Simulation and Design of Circular Scanning Airborne Geiger Mode Lidar for High-Resolution Topographic Mapping

Over the last two decades, Geiger-mode lidar (GML) systems have been developing rapidly in defense and commercial applications, demonstrating high point density and great collection efficiency. We presented a circular scanning GML system simulation model for performance prediction and developed a GML system for civilian mapping. The lidar system used an eye-safe fiber laser at 1545 nm coupled with a 64 × 64 pixels photon-counting detector array. A real-time data compression algorithm was implanted to reduce half of the data transmission rate and storage space compared to the uncompressing situation. The GML system can operate at aircraft above-ground levels (AGLs) between 0.35 km and 3 km, and at speeds in excess of 220 km/h. The initial flight tests indicate that the GML system can operate day and night with an area coverage of 366 km2/h. The standard deviations of the relative altimetric accuracy and the relative planimetric accuracy are 0.131 m and 0.152 m, respectively. The findings presented in this article guide the implementation of designing an airborne GML system and the data compression method.

Agent-based beat-by-beat compression of 12-lead electrocardiogram signal using adaptive Fourier decomposition

ObjectiveThe multilead electrocardiogram (MECG) signal provides much more detailed information related to the cardiac activity of the human heart than single channel signal. Long-term recording of MECG data needs a huge amount of storage space, thus data compression becomes truly necessary. MethodsIn this work, a beat-wise MECG data compression is proposed that is based on adaptive Fourier decomposition (AFD). To reduce dimensionality, an ECG beat was treated as a multiagent, upon which principal component (PC) analysis was used in non-linear space. A new Möbius transform was introduced along with AFD, to convert the dominant PCs in complex domain using Nevanlinna factorization. An offline trained multilayer perceptron neural network was also employed to provide optimal decomposition levels in AFD to limit the PRD within 3%. ResultThe entire work was tested on 546 ptbdb MECG records available in Physionet which yielded an average compression ratio and a percent root mean squared difference (PRD) of 48.21 and 3.88, respectively. Regardless of annotation type, PRD variance within each lead was found to be nearly uniform. ConclusionWithin a single beat, the agent-based compression established a hybrid compression technique, almost completely preserving critical clinical aspects. SignificanceBeat-wise compression, therefore requires less buffer memory, allowing it to be used for real-time data compression. The low reconstruction error enhanced the acceptability of proposed work in medical applications.

Evaluation of Objective Distortion Measures for Automatic Quality Assessment of Processed PPG Signals for Real-Time Health Monitoring Devices

Real-time photoplethysmogram (PPG) denoising and data compression has become most essential requirements for accurately measuring vital parameters and efficient data transmission but that may introduce different kinds of waveform distortions due to the lossy processing techniques. Subjective quality assessment tests are the most reliable way to assess the quality, but they are time expensive and also cannot be incorporated with quality-driven compression mechanism. Thus, finding a best objective distortion measure is highly demanded for automatically evaluating quality of reconstructed PPG signal that must be subjectively meaningful and simple. In this paper, we present four types of objective distortion measures and evaluate their performance in terms of quality prediction accuracy, Pearson correlation coefficient and computational time. The performance evaluation is performed on different kinds of PPG waveform distortions introduced by the predictive coding, compressed sampling, discrete cosine transform and discrete wavelet transform. On the normal and abnormal PPG signals taken from five standard databases, evaluation results showed that different subjective quality evaluation groups (5-point, 3-point and 2-point rating scale) had different best objective distortion measures in terms of prediction accuracy and Pearson correlation coefficient. Moreover, selection of a best objective distortion measure depends upon type of PPG features that need to be preserved in the reconstructed signal.

High bandwidth data and image transmission using a scalable link model with integrated real-time data compression

Our society increasingly relies on the transmission and reception of vast amounts of data using serial connections featuring ever-increasing bit rates. In imaging systems, for example, the frame rate achievable is often limited by the serial link between a camera and host even when modern serial buses with the highest bit rates are used. This paper reports a scalable link system with a bandwidth and interface standard that can easily be adapted to suit a particular application. The scalable serial link approach has been extended with lossless data compression with the aim of further increasing dataflow at a given bit rate. The compression method is integrated into the scalable transceivers providing a comprehensive solution for optimal data transmission over a variety of different interfaces. The system is fully implemented on an FPGA using a fully hardware based system. A Terasic DE4 board was used for implementing and testing the system using Quartus-II software and tools for design and debugging. The impact of compressing the image and carrying the compressed data through parallel lines is similar to the impact of compressed the same image inside a single core with a higher compression ratio, in this system between 7.5 and 126.8.

A Wireless Electro-Optic Platform for Multimodal Electrophysiology and Optogenetics in Freely Moving Rodents.

This paper presents the design and the utilization of a wireless electro-optic platform to perform simultaneous multimodal electrophysiological recordings and optogenetic stimulation in freely moving rodents. The developed system can capture neural action potentials (AP), local field potentials (LFP) and electromyography (EMG) signals with up to 32 channels in parallel while providing four optical stimulation channels. The platform is using commercial off-the-shelf components (COTS) and a low-power digital field-programmable gate array (FPGA), to perform digital signal processing to digitally separate in real time the AP, LFP and EMG while performing signal detection and compression for mitigating wireless bandwidth and power consumption limitations. The different signal modalities collected on the 32 channels are time-multiplexed into a single data stream to decrease power consumption and optimize resource utilization. The data reduction strategy is based on signal processing and real-time data compression. Digital filtering, signal detection, and wavelet data compression are used inside the platform to separate the different electrophysiological signal modalities, namely the local field potentials (1–500 Hz), EMG (30–500 Hz), and the action potentials (300–5,000 Hz) and perform data reduction before transmitting the data. The platform achieves a measured data reduction ratio of 7.77 (for a firing rate of 50 AP/second) and weights 4.7 g with a 100-mAh battery, an on/off switch and a protective plastic enclosure. To validate the performance of the platform, we measured distinct electrophysiology signals and performed optogenetics stimulation in vivo in freely moving rondents. We recorded AP and LFP signals with the platform using a 16-microelectrode array implanted in the primary motor cortex of a Long Evans rat, both in anesthetized and freely moving conditions. EMG responses to optogenetic Channelrhodopsin-2 induced activation of motor cortex via optical fiber were also recorded in freely moving rodents.

A Data Compress Algorithm for Daisy Chain Communication System

In the era of big data, traditional data compression methods cannot complete data compression with features of large volume, rapid growth and complex structure, especially in the processing of real-time data compression. The compression algorithm proposed in this paper is improved on the basis of a lossless bit-swap compression coding algorithm. The coding algorithm is based on VAE model and uses classical ANS compression technology to code multiple potential variables, so as to realize the rapid compression and decompression of data. The experimental results show that the algorithm proposed in this paper can compress data with a high compression rate(47.65%), which is higher than LZAM algorithm and Bit-Swap algorithm.

Multidimensional spatial transformation of streaming video in satellite communication channels

The problem of compression of high-resolution streaming video for transmission over low-speed satellite communication channels of small spacecraft is considered. It has been determined that the specificity of using small spacecraft in low orbits, on the one hand, imposes a number of restrictions on weight and size characteristics and energy consumption, on the other hand, it opens up new opportunities for obtaining high-resolution visual information. Approaches to real-time video data compression are considered, which make it possible to implement a subminiature encoder in the “system-onchip” basis. A coding method based on the complex principles of AV1 coding and multidimensional spatial transformation is proposed. The practical attainability of the developed method and algorithms is shown. The ways of developing the method and creating a video data compression system are proposed.

Real-time energy data compression strategy for reducing data traffic based on smart grid AMI networks

In the future, the Internet of things (IoT) may provide huge volumes of data. Smart grids are a class of IoT electricity distribution systems that can control bidirectional energy flows between consumers and service providers (Barman et al. “IOT Based Smart Energy Meter for Efficient Energy Utilization in Smart Grid”, 978-1-5386-4769-1 1831.00, 2018 IEEE). A typical smart grid features an advanced metering infrastructure (AMI), which automatically collects meter data from widely distributed sensors. A utility company that intends to use AMI must deploy smart meters, data concentrator units, and a meter data management system (MDMS). Concentrators collect ubiquitous messages from smart meters and transmit aggregated data to their MDMS. Although ubiquitous messages may enhance the efficiency of some electricity grids, any excessive volume of messages causes data congestion. This research considers the Reference Energy Disaggregation Data Set. Our proposed algorithm can compress meter data efficiently. Our contributions are as follows: First, thus far, numerous researchers have attempted to address smart grid problems with AMI systems, and here, we provide a relatively complete review of these attempts. Second, this paper presents a strategy to analyze this problem and propose a compression algorithm to compress meter data. This proposed algorithm combines several existing compression algorithms and operates from 2 to 10% more efficiently than previously published algorithms.

Appliance Identification Based on Smart Meter Data and Event-Driven Processing in the 5G Framework

The digitization and IoT advancement is evolving the energy sector. 5G is playing an important role in connecting various smart grid modules and stockholders. In this framework, the idea of utilizing smart meters is increasing. A fine-grained metering data acquisition and processing is crucial to help the smart grid stake holders. The classical data sampling approach is of time invariant nature. Thus, it includes in the acquisition, transmission, and processing stages a large amount of redundant data. This deficit can be eliminated by employing the event-driven sampling. It provides a real-time data compression. Therefore, a novel event-driven adaptive-rate sampling approach is utilized for the data acquisition and features extraction. The relevant features related to the appliances power consumption patterns are subsequently utilized for their identification by employing the support vector machine. Results confirm a 3.7 folds compression and processing gains of the suggested approach while achieving 96% classification accuracy. Thanks to the 5G network, findings are effectively logged on the cloud for further analysis and decision support.

Event-Driven Sampling Based Li-Ion Batteries SoH Estimation in the 5G Framework

The digitization and IoT advancement are evolving the energy sector. 5G is playing an important role in connecting various smart grid modules and stockholders. Batteries are elementary parts of smart grids and are extensively employed. To assure an effective utilization and longer life the Battery Management Systems (BMSs) are employed. Recent BMSs are becoming sophisticated and consequently cause a higher consumption overhead on the battery. To enhance the BMSs power efficiency, this work employs event-driven sensing and processing. In contrast to the traditional counterparts, the battery cell parameters like voltages and currents are no more captured periodically but are acquired based on events. It results in significant real-time data compression. Afterward, a novel algorithm, for a real-time determination of the State of Health (SoH), employs this non-uniformly partitioned information. The estimated SoH is calibrated by using an original event-driven approach. The devised system comparison is made with the traditional counterparts. Results demonstrate a more than third order of magnitude outperformance in terms of compression gain and computational efficiency while assuring an analogous SoH estimation precision. Thanks to the 5G network, findings are effectively logged on the cloud for further analysis and decision making.

A Proficient Li-Ion Battery State of Charge Estimation Based on Event-Driven Processing

The lithium-ion batteries are recurrently used in a variety of applications. To assure an effective battery utilization and longer life, the battery management systems (BMSs) are employed. Recent BMSs are becoming sophisticated and causes a higher consumption overhead on the battery. To enhance the BMS power efficiency, this work exploits the input signal non-stationary nature. The idea is to employ event-driven sensing and processing. In contrast to the traditional counterparts, the battery cells parameters like voltages and currents are no more captured periodically but are acquired based on events. It results in significant real-time data compression. Afterward, this non-uniformly partitioned information is employed by a novel event-driven Coulomb counting algorithm for a real-time determination of the State of Charge (SoC). The estimated SoCis calibrated by using an original event-driven Open Circuit Voltage (OCV) to SoC curve relation. The devised system comparison is made with the traditional counterparts. Results demonstrate a more than third-order of magnitude outperformance of the proposed system in terms of compression gain and computational efficiency while assuring an analogous SoC estimation precision.

Implementation of Text Compression using Adaptive Shannon-Fano Algorithm

This study aims to implement the Shannon-fano Adaptive data compression algorithm on characters as input data. This study also investigates the data compression ratio, which is the ratio between the number of data bits before and after compression. The resulting program is tested by using black-box testing, measuring the number of character variants and the number of types of characters to the compression ratio, and testing the objective truth with the Mean Square Error (MSE) method. The description of the characteristics of the application made is done by processing data in the form of a collection of characters that have different types of characters, variants, and the number of characters. This research presents algorithm that support the steps of making adaptive Shannon-fano compression applications. The length of the character determines the variant value, compression ratio, and the number of input character types. Based on the results of test results, no error occurs according to the comparison of the original text input and the decompression results. A higher appearance frequency of a character causes a greater compression ratio of the resulting file; the analysis shows that a higher number of types of input characters causes a lower compression ratio, which proves that the proposed method in real-time data compression improves the effectiveness and efficiency of the compression process.