Data Compression Rate Research Articles

High-order second-generation wavelet upwind schemes with multiresolution self-adaptive capabilities for hyperbolic conservation laws

We introduce a novel system of high-order wavelet collocation upwind schemes utilizing a broad class of second-generation wavelets for hyperbolic conservation laws. These schemes possess multiresolution self-adaptive capabilities and are integrated into a meshfree framework. Unlike traditional high-order schemes that necessitate frequent variable transformations between physical and characteristic spaces involving numerous local projections, our schemes enable the direct discretization of relevant spatial derivatives in the physical space. The newly proposed schemes make the most use of wavelet properties, i.e., applying asymmetrical interpolating wavelets to achieve the upwind property and utilizing the symmetrical second-generation wavelets to preserve shapes of functions and obtain higher data compression rates. Numerical tests of which solutions contain strong discontinuities and different scale smooth structures are conducted to demonstrate the enhanced performance of the proposed schemes based on the second-generation wavelets. The proposed schemes can utilize only 9 % of the nodes in the WENO5 scheme to yield similar results when addressing the advection of a square wave and achieve the convergent solution for the two interacting blast waves problem without local characteristics projection. In comparison to conventional methods and wavelet upwind schemes based on the interpolating wavelets we previously introduced, the newly proposed schemes exhibit a significantly higher data compression rate and yield substantial computational savings without compromising accuracy.

A Self-Adaptive Compression Method for Ship Trajectories without Threshold Setting

Addressing the shortcomings of existing ship trajectory compression methods that rely on the empirical setting of a fixed threshold and face challenges in controlling the spatial similarity before and after compression, this paper proposes a self-adaptive compression method for ship trajectories that does not require threshold setting. Initially, a hierarchical and sequential tree structure based on the trajectory characteristics is constructed to determine the importance of a ship’s trajectory points. Subsequently, the dynamic time warping (DTW) operator is introduced to assess the spatial similarity of the trajectory before and after compression, exploring the laws governing similarity variations in the step changes during the compression process from lower to higher levels of the hierarchical and sequential sequence. Finally, the trajectory point that causes the largest step change in similarity within the hierarchical and sequential sequence is identified, and points at lower levels than this point are discarded, thus achieving the objective of self-adaptive adjustment of the level of compression. Our case study results demonstrate that, compared with existing ship trajectory compression methods based on empirically set thresholds, the proposed method has the following advantages: (1) it does not require presetting a fixed threshold and adaptively determines the degree of compression by identifying the trajectory point that leads to the largest step change in similarity, and (2) under the condition of a similar data compression rate, the DTW calculation value is reduced by approximately 25%, significantly enhancing the similarity of the trajectory before and after compression.

Improved ACOM pattern matching in 4D-STEM through adaptive sub-pixel peak detection and image reconstruction

The technique known as 4D-STEM has recently emerged as a powerful tool for the local characterization of crystalline structures in materials, such as cathode materials for Li-ion batteries or perovskite materials for photovoltaics. However, the use of new detectors optimized for electron diffraction patterns and other advanced techniques requires constant adaptation of methodologies to address the challenges associated with crystalline materials. In this study, we present a novel image-processing method to improve pattern matching in the determination of crystalline orientations and phases. Our approach uses sub-pixel adaptive image processing to register and reconstruct electron diffraction signals in large 4D-STEM datasets. By using adaptive prominence and linear filters, we can improve the quality of the diffraction pattern registration. The resulting data compression rate of 103 is well-suited for the era of big data and provides a significant enhancement in the performance of the entire ACOM data processing method. Our approach is evaluated using dedicated metrics, which demonstrate a high improvement in phase recognition. Several features are extracted from the registered data to map properties such as the spot count, and various virtual dark fields, which are used to enhance the handling of the results maps. Our results demonstrate that this data preparation method not only enhances the quality of the resulting image but also boosts the confidence level in the analysis of the outcomes related to determining crystal orientation and phase. Additionally, it mitigates the impact of user bias that may occur during the application of the method through the manipulation of parameters.

Three-dimensional quantum imaging of dynamic targets using quantum compressed sensing

Quantum imaging based on entangled light sources exhibits enhanced background resistance compared to conventional imaging techniques in low-light conditions. However, direct imaging of dynamic targets remains challenging due to the limited count rate of entangled photons. In this paper, we propose a quantum imaging method based on quantum compressed sensing that leverages the strong correlation characteristics of entangled photons and the randomness inherent in photon pair generation and detection. This approach enables the construction of a compressed sensing system capable of directly imaging high-speed dynamic targets. The results demonstrate that our system successfully achieves imaging of a target rotating at a frequency of 10 kHz, while maintaining an impressive data compression rate of 10−6. This proposed method introduces a pioneering approach for the practical implementation of quantum imaging in real-world scenarios.

Prediction-based coding with rate control for lossless region of interest in pathology imaging

Online collaborative tools for medical diagnosis produced from digital pathology images have experimented an increase in demand in recent years. Due to the large sizes of pathology images, rate control (RC) techniques that allow an accurate control of compressed file sizes are critical to meet existing bandwidth restrictions while maximizing retrieved image quality. Recently, some RC contributions to Region of Interest (RoI) coding for pathology imaging have been presented. These encode the RoI without loss and the background with some loss, and focus on providing high RC accuracy for the background area. However, none of these RC contributions deal efficiently with arbitrary RoI shapes, which hinders the accuracy of background definition and rate control. This manuscript presents a novel coding system based on prediction with a novel RC algorithm for RoI coding that allows arbitrary RoIs shapes. Compared to other methods of the state of the art, our proposed algorithm significantly improves upon their RC accuracy, while reducing the compressed data rate for the RoI by 30%. Furthermore, it offers higher quality in the reconstructed background areas, which has been linked to better clinical performance by expert pathologists. Finally, the proposed method also allows lossless compression of both the RoI and the background, producing data volumes 14% lower than coding techniques included in DICOM, such as HEVC and JPEG-LS.

Research on the integration and development of public management strategies and rural e-commerce based on the integration model

Abstract This paper focuses on the symbiotic development of rural e-commerce and logistics by exploring the intersection of fusion modeling with public management. By leveraging data fusion and logistics optimization, we address the burgeoning needs of rural e-commerce, presenting a deep learning approach that incorporates Deep Belief Networks (DBNs) and weight-sharing to achieve a 40% data compression rate. This advancement significantly lowers storage and energy costs while enhancing computational efficiency. Furthermore, we devise a rural e-commerce logistics and distribution model optimized through an ant colony algorithm to maximize profits and demand coverage. The application of this Model has led to a 20% increase in profits and a surge in demand coverage to 90%, showcasing the efficacy of fusion models in streamlining rural e-commerce logistics. These insights contribute to the broader discourse on innovative public management strategies.

Research on Energy Harvesting Mechanism and Low Power Technology in Wireless Sensor Networks.

Wireless sensor networks (WSNs) are widely used in various fields such as military, industrial, and transportation for real-time monitoring, sensing, and data collection of different environments or objects. However, the development of WSNs is hindered by several limitations, including energy, storage space, computing power, and data transmission rate. Among these, the availability of power energy plays a crucial role as it directly determines the lifespan of WSN. To extend the life cycle of WSN, two key approaches are power supply improvement and energy conservation. Therefore, we propose an energy harvesting system and a low-energy-consumption mechanism for WSNs. Firstly, we delved into the energy harvesting technology of WSNs, explored the utilization of solar energy and mechanical vibration energy to ensure a continuous and dependable power supply to the sensor nodes, and analyzed the voltage output characteristics of bistable piezoelectric cantilever. Secondly, we proposed a neighbor discovery mechanism that utilizes a separation beacon, is based on reply to ACK, and can facilitate the identification of neighboring nodes. This mechanism operates at a certain duty cycle ratio, significantly reduces idle listening time and results in substantial energy savings. In comparison to the Disco and U-connect protocols, our proposed mechanism achieved a remarkable reduction of 66.67% and 75% in the worst discovery delay, respectively. Furthermore, we introduced a data fusion mechanism based on integer wavelet transform. This mechanism effectively eliminates data redundancy caused by spatiotemporal correlation, resulting in a data compression rate of 5.42. Additionally, it significantly reduces energy consumption associated with data transmission by the nodes.

Accurate ECG monitoring by Gaussian feature streaming

Wearable cardiac monitors can usefully contribute to early detection of potential cardiovascular pathologies. However ECG trace data streaming over wireless links creates some significant challenges, due to the amount of data to be transmitted. We employ a signal analysis approach based on a Gaussian dictionary to model ECG traces in a compressed way. The algorithm operates on fixed-length segments, and achieves effective compression for wireless data transmission, associating just 6 bytes to each Gaussian feature. At the same time it enables accurate reconstruction of ECG traces from the reduced data set. We tested our method on a set of 46 ECG recordings taken from the Physionet MIT-BIH Arrhythmia Database, obtaining 90% data compression rates, while percent relative deviation of reconstructed traces is always below 5%.

Compact agile Tchebycheff transform variant for temporal compression of neural signals on brain-implantable microsystems

This paper proposes a new approximate variant of the Tchebycheff transform for the compression of intra-cortically-recorded neural signals on brain-implantable microsystems. The proposed approximate transform is achieved using an innovative algorithm that guarantees ‘partial orthogonality’ of the basis functions corresponding to the coefficients conveying the majority of the signal energy. This significantly reduces inter-coefficient energy leakage in the proposed transform variant, which helps meaningfully enhance the data compression rate. While keeping the signal reconstruction accuracy within the acceptable range, the proposed Tchebycheff transform variant is maximally truncated in order to reduce the hardware cost of the associated implementation. Based on the resulting truncated approximate transform, a hardware-efficient 256-channel neural signal compressor is designed, prototyped, and tested using pre-recorded neural signals. Operated at a clock rate of 960 kHz, the prototyped neural signal compressor exhibits a true compression rate of 1880 with an RMS reconstruction error of 2.8%.

Unified Quality-Aware Compression and Pulse-Respiration Rates Estimation Framework for Reducing Energy Consumption and False Alarms of Wearable PPG Monitoring Devices

Objective: Due to the high demands of tiny, compact, lightweight and low-cost photoplethysmogram (PPG) monitoring devices, these devices are resource-constrained including limited battery power. Consequently, it highly demands frequent charge or battery replacement in the case of continuous PPG sensing and transmission. Further, PPG signals are often corrupted severely under ambulatory and exercise recording conditions that leads to frequent false alarms. Method: In this paper, we propose a unified quality-aware compression and pulse-respiration rates estimation framework for reducing energy consumption and false alarms of wearable and edge PPG monitoring devices by exploring predictive coding technique for jointly performing signal quality assessment (SQA), data compression and pulse rate (PR) and respiration rate (RR) estimation without use of different domains of signal processing techniques that can be achieved by using the features extracted from the smoothed prediction error signal. Results: By using the five standard PPG databases, the performance of the proposed unified framework is evaluated in terms of compression ratio (CR), mean absolute error (MAE), false alarm reduction rate (FARR), processing time (PT) and energy saving (ES). The compression, PR and RR estimation and SQA results are compared with that of the existing methods and also with results of uncompressed PPG signals with sampling rates of 125 Hz and 25 Hz. Conclusion: The proposed unified quality-aware framework achieves an average CR of 4%, SQA (Se of 92.00%, FARR of 84.87%), PR (MAE: 0.46 ±1.20) and RR (MAE: 1.75 (0.65-4.45), PT (sec) of 15.34 ±0.01) and ES of 70.28% which outperforms the results of uncompressed PPG signal with a sampling rate of 125 Hz. Significance: Arduino Due computing platform based implementation demonstrates the real-time feasibility of the proposed unified quality-aware PR-RR estimation and data compression and transmission framework on the limited computational resources. Thus, it has great potential in improving energy-efficiency and trustworthiness of wearable and edge PPG monitoring devices.

Research on Lightweighting and Rendering Optimization Techniques for Building Information Models

Traditional Building Information Modeling (BIM) services, using a client-server (C/S) architecture, struggle to adapt to the new demands and changes in the field of architectural informatization. The development of a comprehensive BIM visualization system, which is web-based, requires no installation or downloading, is cross-platform, and facilitates easy sharing, has become a new pathway for BIM advancement. However, when rendering the vast amount of data in BIM three-dimensional scenes on the web, several issues persist, including low transmission efficiency, slow loading speed, laggy screen display, and low rendering frame rates, particularly on mobile devices and terminals with limited hardware performance. Therefore, this paper proposes BIM lightweighting techniques based on geometric data and texture information, as well as a web-based rendering optimization method for three-dimensional models. In terms of BIM lightweighting, more efficient methods and algorithms are employed to simultaneously lighten the geometric data and texture information of the models. For rendering optimization, an efficient view frustum culling algorithm is introduced, along with a design for an adaptive rendering strategy to enhance rendering performance.Through testing the loading efficiency of different scale BIM models before and after optimization was tested on the web. Results show that, while maintaining display accuracy, the average loading time of the optimized models was reduced by 40% compared to the unoptimized models. The average data compression rate reached 45%, and the average memory usage decreased by 32.55%. After stable rendering, the frame rate was close to 60fps.

Broadband radio frequency signal measurement based on quantum compression sensing

With the rapid development of radio frequency technology such as radar, electronic warfare and 5G communication, the measurement and real-time spectrum characterization of broadband radio frequency signals become increasingly important. The traditional radio frequency signal real-time measurement technology is limited by the sampling rate of analog-to-digital converter and the ability to process digital signals, and encounters the problems of narrow measurement band, large data volume, and susceptibility to electromagnetic interference. This work is to study a radio frequency signal measurement technology based on quantum compression sensing, which uses integrated electro-optical crystal as radio frequency sensor, and constructs a compression sensing machine by modulating the photon wave function of the measured radio frequency signal to realize the compression measurement of broadband radio frequency signal, significantly improving the spectrum sensing bandwidth. The experiment demonstrates the long-term spectrum monitoring of power frequency and intermediate frequency high voltage signals, and the real-time spectrum measurement of high frequency radio frequency signals. Under the Fourier limit spectrum resolution, the real-time spectrum analysis bandwidth of GHz magnitude is realized, and the data compression rate reaches 1.7×10<sup>–5</sup>, which can meet the needs of 5G wireless communication, cognitive radio and other applications for broadband radio frequency signal spectrum measurement, and provide a new technical path for developing the next-generation broadband spectrum sensing technology.

A Data Aggregation Approach Exploiting Spatial and Temporal Correlation among Sensor Data in Wireless Sensor Networks

Wireless sensor networks (WSNs) have various applications which include zone surveillance, environmental monitoring, event tracking where the operation mode is long term. WSNs are characterized by low-powered and battery-operated sensor devices with a finite source of energy. Due to the dense deployment of these devices practically it is impossible to replace the batteries. The finite source of energy should be utilized in a meaningful way to maximize the overall network lifetime. In the space domain, there is a high correlation among sensor surveillance constituting the large volume of the sensor network topology. Each consecutive observation constitutes the temporal correlation depending on the physical phenomenon nature of the sensor nodes. These spatio-temporal correlations can be efficiently utilized in order to enhance the maximum savings in energy uses. In this paper, we have proposed a Spatial and Temporal Correlation-based Data Redundancy Reduction (STCDRR) protocol which eliminates redundancy at the source level and aggregator level. The estimated performance score of proposed algorithms is approximately 7.2 when the score of existing algorithms such as the KAB (K-means algorithm based on the ANOVA model and Bartlett test) and ED (Euclidian distance) are 5.2, 0.5, respectively. It reflects that the STCDRR protocol can achieve a higher data compression rate, lower false-negative rate, lower false-positive rate. These results are valid for numeric data collected from a real data set. This experiment does not consider non-numeric values.

Improvement of hierarchical matrices for 3D elastodynamic problems with a complex wavenumber

It is well known in the literature that standard hierarchical matrix (\({\mathscr{H}}\)-matrix)-based methods, although very efficient for asymptotically smooth kernels, are not optimal for oscillatory kernels. In a previous paper, we have shown that the method should nevertheless be used in the mechanical engineering community due to its still important data compression rate and its straightforward implementation compared to \({\mathscr{H}}^{2}\)-matrix, or directional, approaches. Since in practice, not all materials are purely elastic, it is important to be able to consider visco-elastic cases. In this context, we study the effect of the introduction of a complex wavenumber on the accuracy and efficiency of \({\mathscr{H}}\)-matrix-based fast methods for solving dense linear systems arising from the discretization of the elastodynamic (and Helmholtz) Green’s tensors. Interestingly, such configurations are also encountered in the context of the solution of transient purely elastic problems with the convolution quadrature method. Relying on the theory proposed in Börm et al. (IMA Journal of Numerical Analysis 12, 2020) for \({\mathscr{H}}^{2}\)-matrices for Helmholtz problems, we study the influence of the introduction of damping on the data compression rate of standard \({\mathscr{H}}\)-matrices. We propose an improvement of \({\mathscr{H}}\)-matrix-based fast methods for this kind of configuration and illustrate how the introduction of a complex wavenumber can, as expected, improve further the efficiency of such methods. This work is complementary to the recent report (Börm et al., IMA Journal of Numerical Analysis 12, 2020). Here, in addition to addressing another physical problem, we consider standard \({\mathscr{H}}\)-matrices, derive a simple criterion to introduce additional compression and we perform extensive numerical experiments.

Standardized database of 400 complex abstract fractals.

In experimental settings, characteristics of presented stimuli influence cognitive processes. Knowledge about stimulus features is important to manipulate or control the influence of stimuli. To date, there are a lack of standardized data incorporating such information for complex abstract stimuli. Thus, we provide norms for a database of 400 abstract and complex stimuli. Grey-scaled fractals were rated by 512 participants on the stimulus features of abstractness, animacy, verbalizability, complexity, familiarity, favorableness, and memorability. Moreover, 111 participants labeled the fractals, enabling us to calculate indices of naming agreement and modal names. Overall, the results confirmed high abstractness and low verbalizability of the provided stimuli. To establish external validation for selected stimulus features, we evaluated (a) classifier probability of a deep neural network labeling the fractals, negatively correlated with ratings of abstractness and positively with verbalizability and naming agreement; (b) data compression rate of fractal image files, positively correlated with the rating of complexity; and (c) performance of 212 participants in a recognition-memory task, positively correlated with the rating of memorability. The present work fills the gap of a standardized database for abstract stimuli and provides a database with valid norms for abstract and complex stimuli based on ratings and external validation measures. This database can be used to control and manipulate these stimulus features in experimental settings using abstract stimuli. Such a database is essential in experimental research using abstract stimuli for instance to control for verbal influence and strategy or to control for novelty and familiarity.

Implementation of Image Compression by Using High-Precision In-Memory Computing Scheme Based on NOR Flash Memory

A novel image processing scheme based on flash memory is proposed as a highly efficient pre-processing unit capable of Discrete Cosine Transform (DCT) operation for image compression and feature extractions. By using matrix splitting algorithms and optimizations of operation bias, the DCT operation is demonstrated successfully based on the 65 nm NOR flash memory. The processing accuracy, as well as its variations, can be well controlled by setting the working points at the saturation region with further circuit optimizations. The data compression rate is ~15.6%, showing that the proposed scheme can provide a fundamental solution to accelerate various neural network training, image recognitions, and retrievals.

ACI: a bar chart index for non-linear visualization of data embedding and aggregation capacity in IoMT multi-source compression

Visualization of numerical results in computer communications is very important such that some very small differences are sometimes crucial, distinguishable, and descriptive for comparison among some state-of-the-art techniques. For the issue of data quality evaluation and compression rates in internet of multimedia things, there are many metrics traditionally, for instance, peak signal-to-noise ratio (PSNR) is strongly able to describe non-sensitive (and relatively ambiguous) results of mean square error and since PSNR is normally between 10 and 100 for most of the lossy techniques, it can plotted with using any graphical/visualization tool. However, the results of compression rates for aggregation techniques may be a little complicated on which using a non-flexible mathematical operator like logarithm may have an unsuitable effect with ignoring the small differences while plotting the results. The aim behind this paper is to introduce a new metric entitled average capacity index (ACI), as a non-linear visualization approach/scaling mechanism, to be usable in evaluating capacity results of data hiding and aggregation algorithms based on bar charts. Some examples with synthetic and real data will show that the proposed metric outperforms the existing conventional tools in terms of statistical measures and visual presentation.

Perceptual Quality Assessment of Compressed Vibrotactile Signals Through Comparative Judgment.

In this article, we present a comprehensive scheme for the quality assessment of compressed vibrotactile signals with human assessors. Inspired by the multiple stimulus test with hidden reference and anchors (MUSHRA) from the audio domain, we designed a method in which each compressed signal is compared to its original signal and rated on a numerical scale. For each signal tested, the hidden reference and two anchor signals are used to validate the results and provide assessor screening criteria. Differing from previous approaches, our method is hierarchically structured and strictly timed in a sequential manner to avoid experimental confounds and provide precise psychophysical assessments. We validated our method in an experiment with 20 human participants in which we compared two state-of-the-art lossy codecs. The results show that, with our approach, the performance of different codecs can be compared effectively. Furthermore, the method also provides a measure of subjective quality at different data compression rates. The proposed procedure can be easily adapted to evaluate other vibrotactile codecs.

A Smart multi-view panoramic imaging integrating stitching with geometric matrix relations among surveillance cameras (SMPI)

Reducing data stored and transferred is a critical topic in the modern era, particularly after the evolution in multimedia applications and surveillance systems worldwide. Motivated by the massive amount of data generated by surveillance cameras and the enormous number of redundant pixels produced among them, this paper introduces a novel model entitled: “A Smart Multi-View Panoramic Imaging integrating stitching with geometric matrix relations among surveillance cameras (SMPI).” The introduced model aims to create a novel feedback real-time stitching system to reduce the storing and transferring of redundant data generated by neighboring surveillance cameras for an extra level of compression. Moreover, the panoramic view is mostly a better monitoring option rather than multiple monitors in complicated surveillance cameras’ control rooms. The proposed system, in this paper, merges feature extraction stitching techniques with geometric relational matrix calculations to reduce the time complexity limitations of traditional mosaicking. Additionally, the proposed work introduces a real-time algorithm to reconstruct images of each camera from the panoramic view, and a novel algorithm for ordering cameras’ frames before stitching is recommended for producing a panoramic view without any human interference. The experimental work tests numerous state of the art feature extraction algorithms for stitching, Scale Invariant Feature Transform (SIFT), Speed Up Robust Feature (SURF) and Oriented FAST and Rotated BRIEF (ORB) with different orders of stitching. The amount of compression per image after reconstruction is also analyzed. The suggested model was implemented and tested using a vast number of benchmark datasets. Evaluation measures have been used to indicate the efficiency of the recommended system. The proposed model’s algorithm has recorded a low time processing per frame while keeping high accurate results. It was found that the recommended Efficient Stitching Algorithm (ESA) produced an average of 46 panoramas per second, and the reconstruction phase could reach a rate of 90 frames per second, which is significantly higher than the 30 frames per second standard video format system. These results give our model an excellent advantage for the effective processing of more scalable systems with a higher number of frames per second. The proposed system created panoramas with an average of 99% similarities with the traditional mosaicking systems while being highly faster than these conventional methods. Compression ratios and data rate savings, reflecting the gain in data stored and transferred, were calculated, reporting an average of 2.66 and 0.62 per frame, respectively, when applied to standard datasets. The results illustrated that the proposed system gives a dramatic reduction in the volume of data stored/transferred and showed that the creating of mosaics and the reconstruction was made in proper processing time. Experimental outcomes also showed that, for the suggested methods, the produced frames after reconstruction have a high similarity percentage compared with original ones before stitching, which indicates that the proposed approach is efficient enough to preserve the essential features of cameras’ frames without significant information loss.

ZDC: A Zone Data Compression Method for Solid State Drive Based Flash Memory

Solid-state drive (SSD) with flash memory as the storage medium are being widely used in various data storage systems. SSD data compression means that data is compressed before it is written to Not-And (NAND) Flash. Data compression can reduce the amount of data written in NAND Flash and improve the performance and reliability of SSDs. At present, the main problem facing data compression of SSD is how to improve the efficiency of data compression and decompression. In order to improve the performance of data compression and decompression, this study proposes a method of SSD data deduplication based on zone division. First, this study divides the storage space of the SSD into zones and divides them into one hot zone and multiple cold zones according to the different erasing frequency. Second, the data in each zone is divided into hot data and cold data according to the number of erasures. At the same time, the address mapping table in the hot zone is loaded into the cache. Finally, when there is a write or read request, the SSD will selectively compress or decompress the data according to the type of different zones. Through simulation tests, the correctness and effectiveness of this study are verified. The research results show that the data compression rate of this research result can reach 70–95%. Compared with SSD without data compression, write amplification is reduced by 5 to 30%, and write latency is reduced by 5 to 25%. The research results have certain reference value for improving the performance and reliability of SSD.