Large Data Compression Research Articles

The image-based multiscale multigrid solver, preconditioner, and reduced order model

We present a novel image-based multiscale multigrid solver that can efficiently address the computational complexity associated with highly heterogeneous systems. This solver is developed based on an image-based, multiresolution model that enables reliable data flow between corresponding computational grids and provides large data compression. A set of inter-grid operators is constructed based on the microstructural data which remedies the issue of missing coarse grid information. Moreover, we develop an image-based multiscale preconditioner from the multiscale coarse images which does not traverse through any intermediate grid levels and thus leads to a faster solution process. Finally, an image-based reduced order model is designed by prolongating the coarse-scale solution to approximate the fine-scale one with improved accuracy. The numerical robustness and efficiency of this image-based computational framework is demonstrated on a two-dimensional example with high degrees of data heterogeneity and geometrical complexity.

High-Sensitivity High-Speed Compressive Spectrometer for Raman Imaging

Compressive Raman is a recent framework that allows for large data compression of microspectroscopy during its measurement. Because of its inherent multiplexing architecture, it has shown imaging speeds considerably higher than conventional Raman microspectroscopy. Nevertheless, the low signal-to-noise (SNR) of Raman scattering still poses challenges for high-sensitivity bio-imaging exploiting compressive Raman: (i) the idle solvent acts as a background noise upon imaging small biological organelles, (ii) current compressive spectrometers are lossy precluding high-sensitivity imaging. We present inexpensive high-throughput spectrometer layouts for high-sensitivity compressive hyperspectroscopy. We exploit various modalities of compressive Raman allowing for up to 80X reduction of data storage and 2X microspectroscopy speed up at a 230-nm spatial resolution. Such achievements allowed us to chemically image sub-diffraction-limited biological specimens (lipid bilayers) in few seconds.

Big Data Compression Technology Based on Internet of Vehicles

To ensure the high efficiency of the development of car networking technology, large data compression technology based on car networking was studied. First, RFID technology and vehicle networking, big data technology in vehicle networking, RFID path data compression technology in the Internet of vehicles were introduced. Then, RFID path data compression verification experiments were performed. The results showed that when the data volume was relatively small, there was no obvious change in the compression ratio under the fixed threshold and the threshold change. However, when the amount of data gradually increased, the compression ratio under the condition of changing the threshold was slightly higher than the fixed threshold. Therefore, RFID path big data processing is feasible, and compression technology is efficient.

A large ratio data compression method for pipeline inspection based on EMAT-generated guided wave

In order to meet the engineering requirement of gas pipeline inline inspection for cracking defects, a new ultrasonic guided wave (GW) method based on electromagnetic acoustic transducer (EMAT) has emerged. In consideration of the large amount of original inspection data and the limited capacity of the storing device, the research on rapid and large ratio data compression is needed. This paper proposes a new large ratio data compression method based on subsection adaptive method (SAM) and wavelet neural network (WNN) algorithm. The influences of parameter selection on the compression quality, speed and ratio and the method of enhancing the compression speed of this method is studied. The experiment results indicate that compared with the traditional WWN compression algorithm, the proposed method has higher compression quality and speed, and a large compression ratio of 110:1 is achieved by the proposed method. It is thus suitable for gas pipeline crack inline inspection based on EMAT-generated GW.

Daubechies 웨이블릿 필터를 사용한 볼륨 데이터 인코딩

데이터 압축 기술은 대용량의 데이터를 효율적으로 저장하고 전송할 수 있게 해주는 기술로, 요구되는 데이터의 용량이 커지고 네트워크의 트래픽이 증가함에 따라 그 중요도가 점점 더 높아지고 있다. 특히 다양한 응용과학과 공학 분야에서 산출되는 볼륨 데이터는 컴퓨팅 기술의 발전에 힘입어 그 용량이 점점 더 증가하는 추세에 있다. 본 논문에서는 Daubechies 웨이블릿 변환을 적용해서 볼륨 데이터를 압축하는 기법을 제안한다. 구현된 D4 웨이블릿 필터 기반 압축 기법은 3차원 볼륨 데이터에 대한 손실 압축과 블록 단위의 무작위 추출 복원을 지원한다. 본 기법은 기존의 Harr 필터를 이용한 압축 방식에 비해 복원 데이터의 손실율이 낮기 때문에, 정밀한 복원 영상이 중요시되는 대용량 데이터의 압축 및 인터렉티브 가시화 응용에 유용하게 사용될 수 있다. Data compression technologies enable us to store and transfer large amount of data efficiently, and become more and more important due to increasing data size and the network traffic. Moreover, as a result of the increase of computing power, volumetric data produced from various applied science and engineering fields has been getting much larger. In this Paper, we present a volume compression scheme which exploits Daubeches wavelet transform. The proposed scheme basically supports lossy compression for 3D volume data, and provides unit-wise random accessibility. Since our scheme shows far lower error rates than the previous compression methods based on Haar filter, it could be used well for interactive visualization applications as well as large volume data compression requiring image fidelity.

Hybrid Three-Dimensional Supercompact Multiwavelets for Computational Fluid Dynamics

ABSTRACT Supercompact wavelets are extended to hybrid-type applications for practical fluid dynamics data management. Beam and Warming's supercompact multiwavelet method is regarded as one proper tool for representing fluid simulation data because of its compact support. This wavelet compactness significantly reduces storage requirements as well as data processing time, and can prevent unnecessary interaction with closely located data across singularities such as shocks and vortices. For effective domain subdivision, a hybrid method that combines 3-D, 2-D, and 1-D versions of supercompact multiwavelets is developed. The hybrid 3-D type of supercompact multiwavelet shows high data compression ratio in practical fluid simulation data because the mother wavelets are highly effective and encompass subdivision methods. With the present method, efficient data management for datasets of huge size is enabled without losing important features. Several numerical tests substantiate large data compression for flow field simulation.

3차원 유동 시뮬레이션을 위한 Supercompact 다중 웨이블릿

본 논문에서는 supercompact 다중 웨이블릿 기법과 이 기법의 유동 시뮬레이션 데이터에의 적용을 발표한다. Supercompact 웨이블릿 방법은 간결한 지원(support)을 제공할 수 있고 또 속성이 다른 떨어져 있는 데이터와(예: 충격파의 불연속구간 또는 와동을 가로지르는 부분) 불필요한 상호작용을 피할 수 있는 점에서 유동 시뮬레이션 데이터를 위한 적합한 웨이블릿 방법이라 할 수 있다. 데이터 압축을 위한 임계처리법(thresholding)은 다중 웨이블릿의 공분산 벡터 구조 기반 하에 적용된다. 본 논문은 3차원으로의 기법 확장이 설명 분석되었다. 수치실험은 본 방법이 여러 이론적인 이점을 제공할 수 있고 실제 결과에 있어서 큰 데이터 압축 비율을 산출 할 수 있음을 보여준다. This paper presents a supercompact multi-wavelet scheme and its application to fluid simulation data. The supercompact wavelet method is an appropriate wavelet for fluid simulation data in the sense that it can provide compact support and avoid unnecessary interaction with remotely located data (e.g. across a shock discontinuity or vortices). thresholding for data compression is applied based on a covariance vector structure of multi-wavelets. The extension of this scheme to three dimensions is analyzed. The numerical tests demonstrate that it can allow various analytic advantages as well as large data compression ratios in actual practice.

A study on CFD data compression using hybrid supercompact wavelets

A hybrid method with supercompact multiwavelets is suggested as an efficient and practical method to compress CFD dataset. Supercompact multiwavelets provide various advantages such as compact support and orthogonality in CFD data compression. The compactness is a crucial condition for approximated representation of CFD data to avoid unnecessary interaction bet­ween remotely spaced data across various singularities such as shock and vortices. But the supercompact multi wavelet method has to fit the CFD grid size to a product of integer and power of two. m×2^n. To resolve this problem, the hybrid method with combination of 3.2 and 1 dimensional version of wavelets is studied. With the hybrid method, any arbitrary size can be handled without any shrinkage or expansion of the original problem. The presented method allows high data compression ratio for fluid simulation data. Several numerical tests substantiate large data compression ratios for now field simulation successfully.

MULTIDIMENSIONAL SUPERCOMPACT WAVELETS FOR FLUID DYNAMICS

Highly compact wavelets called supercompact multiwavelets are extended to higher dimensions for the purpose of numerical fluid solution data compression. Due to the compact nature of the wavelets, the proposed multiresolution approach with the supercompact wavelets offers numerous benefits such as high data compression. Supercompact wavelets are well suitable for gas flow in which some local complex features are embedded in an overall smooth solution. Supercompact wavelets provide advantages in that they allow higher-order accurate representation with compact support and therefore avoid unnecessary interaction with remotely located data across singularities such as shocks and vortices. The numerical implementation is relatively easy and less expensive than the comparable interpolator multiresolution scheme. A practical demonstration of multidimensional supercompact wavelets is presented. Several numerical tests demonstrate large data compression ratios for the outputs of flow field simulation.

Supercompact multiwavelets for flow field simulation

A supercompact multiwavelet scheme for computational fluid dynamics is presented. Beam and Warming's supercompact wavelet method is an appropriate wavelet for fluid simulation data in the sense that it can provide compact support. The compactness of the wavelets avoids unnecessary interaction with remotely located data (e.g. across a shock discontinuity or vortex) and significantly reduces computational data processing time. Thresholding for data compression with the supercompact wavelets is applied based on a covariance vector structure of multiwavelets. The extension of this scheme to three dimensions is analyzed. Numerical tests demonstrate that the analytic advantages actually result in large data compression ratios.