Data Compression Research Articles

Smart grid data compression and reconstruction by wavelet packet transform

A smart grid is a power network from generation to consumers and provides beneficial, steady, and safe electricity. It utilizes smart meters for billing, phasor measurement units to check the system's health., etc. As a result, it contains enormous volumes of real-time data that may be shared and stored by users, control centers, and services in a smart grid. It weakens the smart grid's communication networks. The size of the data in a smart grid will grow extremely in the future. As a result, it must reduce distortion in data compression and denoise while minimizing the demand on storage and communication networks. The goal of data compression and denoising should be to maximally conserve the useful data while accurately reflecting the state of the system and providing sufficient data regeneration at the receiving end. This paper has used lower-order different wavelets to represent a design to compress and reconstruct data at level three using wavelet Packet Transform. It works on the phasor measurement unit's current magnitude and voltage sag signals.•The proposed design has a better compression ratio.•Low reconstruction error.•This design is easy to access, systematic, profitable, and not time-consuming.

A signal-adaptive measurement matrix construction algorithm for compressed sensing of sEMG data

Abstract Surface electromyography (sEMG), serving as a pivotal wearable technology, is a promising tool to assess and monitor muscle function. Yet, the efficacy of a sEMG system faces inevitable constraints, primarily stemming from the challenges of transmission and energy consumption induced by big data. Compressed sensing (CS) is a promising data acquisition solution that takes advantage of the signal sparseness in a particular basis to significantly reduce the number of samples. Current CS methods usually employ random or deterministic measurement matrix to compress sEMG signal. However, these measurement matrices do not integrate the signal feature, which limits the performance of these CS methods. To address this problem, this paper proposes an improved CS method for sEMG data compression. This proposed method introduces a measurement matrix construction algorithm to produce a deterministic matrix tailored for processing sEMG signals. The deterministic measurement matrix integrates the characteristics of the magnitudes of sEMG signals. The sEMG signals acquired from the upper limb muscles of the stroke survivors were applied to evaluate the proposed CS method, with results showing that it achieves better reconstruction accuracy and robustness than the CS methods with other measurement matrices. The proposed method employing basis pursuit (BP) in the signal reconstruction presents better performance than that employing orthogonal matching pursuit (OMP). Hence, we can conclude that the proposed CS algorithm is of key importance for the popularization of sEMG in the wearable health monitoring devices.

Flexible grouping of linear segments for highly accurate lossy compression of time series data

Flexible grouping of linear segments for highly accurate lossy compression of time series data

Comparative analysis of optional hunting behavior in Cricetinae hamsters using the data compression approach

Research into the hunting behavior in members of the Cricetidae family offers an opportunity to reveal what changes in the predatory behavioral sequences occur when a rodent species shifts from an omnivorous to a predatory lifestyle. The study tests the following hypotheses: are there phylogenetic differences in the divergence of species’ predatory lifestyles in hamsters or do ecological factors lead to shaping their hunting behavior? We applied the data compression approach for performing comparative analysis of hunting patterns as biological “texts.” The study presents a comparative analysis of hunting behaviors in five Cricetinae species, focusing on the new data obtained for the desert hamster Phodopus roborovskii whose behavior has never been studied before. The hunting behavior of P. roborovskii appeared to be the most variable one. In contrast, behavioral sequences in P. campbelli and Allocricetulus curtatus display more significant order and predictability of behavior during hunting. Optional hunting behavior in the most ancient species P. roborovskii displayed similarities with obligate patterns in “young” Allocricetulus species. It thus turned out to be the most advanced hunter among members of the Phodopus genus. Differences in hunting sequences among Phodopus representatives suggest that the hunting behavior of these species, despite its optional mode, was subject to selection during species splitting within the genus. These results did not reveal the role played by phylogenetic differences in the divergence of species’ predatory lifestyles. They suggested that ecological conditions are the main factors in speciation of the hunting behavior in hamsters.

Computed tomography of chemiluminescence using a data-driven sparse sensing framework

The aim of this work is to demonstrate the use of data-driven sparse sensing for the reconstruction of the three-dimensional chemiluminescence field of a flame from the two-dimensional line-of-sight integrated chemiluminescence signal. By leveraging the intrinsic low-dimensionality of physical phenomena, sparse sensing can be employed to reconstruct a signal from few samples. This makes it a good candidate to be employed in computed tomography of chemiluminescence, an imaging technique used to reconstruct the chemiluminescence field from chemiluminescence images. In the proposed methodology, the transforming basis used to project the system’s state in the low-dimensional manifold is computed using the proper orthogonal decomposition, which is a popular machine-learning technique for data compression. The methodology is demonstrated on a virtual experiment based on the data coming from the large eddy simulation of a jet flame in a vitiated coflow, where OH is employed as a surrogate of OH∗. This methodology is then compared to conventional techniques that are generally employed to solve tomographic problems. The results show that the proposed technique is able to reconstruct the chemiluminescence field using only one view, compared to the multiple views required by conventional methods. Moreover, our techniques is also capable of predicting the distribution of other features such as temperature and species concentrations.

Data Compression Solution for ϕ -OTDR: Encoding the Extracted Spectrum

Data Compression Solution for ϕ -OTDR: Encoding the Extracted Spectrum

Optoelectronic Devices for In-Sensor Computing.

The demand for accurate perception of the physical world leads to a dramatic increase in sensory nodes. However, the transmission of massive and unstructured sensory data from sensors to computing units poses great challenges in terms of power-efficiency, transmission bandwidth, data storage, time latency, and security. To efficiently process massive sensory data, it is crucial to achieve data compression and structuring at the sensory terminals. In-sensor computing integrates perception, memory, and processing functions within sensors, enabling sensory terminals to perform data compression and data structuring. Here, vision sensors are adopted as an example and discuss the functions of electronic, optical, and optoelectronic hardware for visual processing. Particularly, hardware implementations of optoelectronic devices for in-sensor visual processing that can compress and structure multidimensional vision information are examined. The underlying resistive switching mechanisms of volatile/nonvolatile optoelectronic devices and their processing operations are explored. Finally, a perspective on the future development of optoelectronic devices for in-sensor computing is provided.

CFNet: An infrared and visible image compression fusion network

Image fusion aims to acquire a more complete image representation within a limited physical space to more effectively support practical vision applications. Although the currently popular infrared and visible image fusion algorithms take practical applications into consideration. However, they did not fully consider the redundancy and transmission efficiency of image data. To address this limitation, this paper proposes a compression fusion network for infrared and visible images based on joint CNN and Transformer, termed CFNet. First of all, the idea of variational autoencoder image compression is introduced into the image fusion framework, achieving data compression while maintaining image fusion quality and reducing redundancy. Moreover, a joint CNN and Transformer network structure is proposed, which comprehensively considers the local information extracted by CNN and the global long-distance dependencies emphasized by Transformer. Finally, multi-channel loss based on region of interest is used to guide network training. Not only can color visible and infrared images be fused directly but more bits can be allocated to the foreground region of interest, resulting in a superior compression ratio. Extensive qualitative and quantitative analyses affirm that the proposed compression fusion algorithm achieves state-of-the-art performance. In particular, rate–distortion performance experiments demonstrate the great advantages of the proposed algorithm for data storage and transmission. The source code is available at https://github.com/Xiaoxing0503/CFNet.

High-Performance Binocular Disparity Prediction Algorithm for Edge Computing.

End-to-end disparity estimation algorithms based on cost volume deployed in edge-end neural network accelerators have the problem of structural adaptation and need to ensure accuracy under the condition of adaptation operator. Therefore, this paper proposes a novel disparity calculation algorithm that uses low-rank approximation to approximately replace 3D convolution and transposed 3D convolution, WReLU to reduce data compression caused by the activation function, and unimodal cost volume filtering and a confidence estimation network to regularize cost volume. It alleviates the problem of disparity-matching cost distribution being far away from the true distribution and greatly reduces the computational complexity and number of parameters of the algorithm while improving accuracy. Experimental results show that compared with a typical disparity estimation network, the absolute error of the proposed algorithm is reduced by 38.3%, the three-pixel error is reduced to 1.41%, and the number of parameters is reduced by 67.3%. The calculation accuracy is better than that of other algorithms, it is easier to deploy, and it has strong structural adaptability and better practicability.

A data compression algorithm with the improved SRLE for high-throughput neural signal acquisition device.

Multi-channel acquisition systems of brain neural signals can provide a powerful tool with a wide range of information for the clinical application of brain computer interfaces. High-throughput implantable systems are limited by size and power consumption, posing challenges to system design. To acquire more comprehensive neural signals and wirelessly transmit high-throughput brain neural signals, a FPGA-based acquisition system for multi-channel brain nerve signals has been developed. And the Bluetooth transmission with low-power technology are utilized. To wirelessly transmit large amount of data with limited Bluetooth bandwidth and improve the accuracy of neural signal decoding, an improved sharing run length encoding (SRLE) is proposed to compress the spike data of brain neural signal to improve the transmission efficiency of the system. The functional prototype has been developed, which consists of multi-channel data acquisition chips, FPGA main control module with the improved SRLE, a wireless data transmitter, a wireless data receiver and an upper computer. And the developed functional prototype was tested for spike detection of brain neural signal by animal experiments. From the animal experiments, it shows that the system can successfully collect and transmit brain nerve signals. And the improved SRLE algorithm has an excellent compression effect with the average compression rate of 5.94%, compared to the double run-length encoding, the FDR encoding, and the traditional run-length encoding. The developed system, incorporating the improved SRLE algorithm, is capable of wirelessly capturing spike signals with 1024 channels, thereby realizing the implantable systems of High-throughput brain neural signals.

Compressed sensing: a discrete optimization approach

We study the Compressed Sensing (CS) problem, which is the problem of finding the most sparse vector that satisfies a set of linear measurements up to some numerical tolerance. CS is a central problem in Statistics, Operations Research and Machine Learning which arises in applications such as signal processing, data compression, image reconstruction, and multi-label learning. We introduce an ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _2$$\\end{document} regularized formulation of CS which we reformulate as a mixed integer second order cone program. We derive a second order cone relaxation of this problem and show that under mild conditions on the regularization parameter, the resulting relaxation is equivalent to the well studied basis pursuit denoising problem. We present a semidefinite relaxation that strengthens the second order cone relaxation and develop a custom branch-and-bound algorithm that leverages our second order cone relaxation to solve small-scale instances of CS to certifiable optimality. When compared against solutions produced by three state of the art benchmark methods on synthetic data, our numerical results show that our approach produces solutions that are on average 6.22%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$6.22\\%$$\\end{document} more sparse. When compared only against the experiment-wise best performing benchmark method on synthetic data, our approach produces solutions that are on average 3.10%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.10\\%$$\\end{document} more sparse. On real world ECG data, for a given ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _2$$\\end{document} reconstruction error our approach produces solutions that are on average 9.95%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$9.95\\%$$\\end{document} more sparse than benchmark methods (3.88%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.88\\%$$\\end{document} more sparse if only compared against the best performing benchmark), while for a given sparsity level our approach produces solutions that have on average 10.77%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$10.77\\%$$\\end{document} lower reconstruction error than benchmark methods (1.42%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1.42\\%$$\\end{document} lower error if only compared against the best performing benchmark). When used as a component of a multi-label classification algorithm, our approach achieves greater classification accuracy than benchmark compressed sensing methods. This improved accuracy comes at the cost of an increase in computation time by several orders of magnitude. Thus, for applications where runtime is not of critical importance, leveraging integer optimization can yield sparser and lower error solutions to CS than existing benchmarks.

PENGGUNAAN ALGORITMA GEOSHASHING DALAM PENCARIAN LETAK INFORMASI GEOGRAFIS PENGGANTI KOOORDINAT LATITUDE DAN LONGITUDE

In the digital age when the search for geographic data is becoming increasingly important in location applications, including mapping, navigation, and analysis of the environment of a place within a location. Geohashing algorithms offer an efficient approach to encoding and storing geographic data in string format, these strings use alphanumeric generated by geohashing of numbers and letters in the search for location points by coordinates, it can also offer an access layer of sensitive location data. This study aims to explore the application of geohashing algorithms in geographic data management. Geohashing algorithm in the search for the location of geographic information as an alternative to latitude and longitude coordinates. In particular, this study analyzes the efficiency of geohashing in data compression, the localization accuracy of geohashing use allows for better grouping of geographic data, and supports spatial data search such as location search and coordinate point search within a specific region. Geohashing algorithms are implemented in geographic data and tested for various search scenarios. From the results of the study showing that geohashing can significantly increase the speed of searching for geographic information, the geohashing algorithm proves its action as an effective tool in the management and search of geographic data, in the search for location coordinate data, it can also provide an alternative to get the latitude and longitude of a location. The results show that geohashing can significantly increase the speed of searching for geographic information, especially in reaching a location with a small and large data box grid level according to the entered string characters. The use of geohashing not only improves the efficiency and speed of searches but also provides additional benefits in terms of the management and protection of geographic data. Testing on this system is carried out by providing geographic access to the location application API that has been created on demand to obtain geographic data information and location points.

Macrophenological dynamics from citizen science plant occurrence data

Abstract Phenological shifts across plant species is a powerful indicator to quantify the effects of climate change. Today, mobile applications with automated species identification open new possibilities for phenological monitoring across space and time. Here, we introduce an innovative spatio‐temporal machine learning methodology that harnesses such crowd‐sourced data to quantify phenological dynamics across taxa, space and time. Our algorithm links individual phenological responses across thousands of species and geographical locations, using a similarity measure. The analysis draws on nearly ten million plant observations collected through the AI‐based plant identification app Flora Incognita in Germany from 2018 to 2021. Our method quantifies changes in synchronisation across the annual cycle. During the growing season, synchronised behaviour can be encoded by a few characteristic macrophenological patterns. Nonlinear spatio‐temporal changes of these patterns can be efficiently quantified using a data compressibility measure. Outside the growing season, the phenological synchronisation diminishes introducing noise into the patterns. Despite biases and uncertainties associated with crowd‐sourced data, for example due to human data collection behaviour, our study demonstrates the feasibility of deriving meaningful indicators for monitoring plant macrophenology from individual plant observations. As crowd‐sourced databases continue to expand, our approach holds promise to study climate‐induced phenological shifts and feedback loops.

ALGORITHME DE CLUSTERING AGGLOMÉRATIF BASÉ SUR MOJO AVEC STRATÉGIE EDMT POUR LES RÉSEAUX DE CAPTEURS SANS FIL À GRANDE ÉCHELLE

The wireless sensor network (WSN) is a distributed sensor network that monitors and stores environmental data wirelessly by connecting dispersed sensor nodes. Since wireless sensor nodes rely on batteries for energy, energy consumption, and limitations are considered fundamental problems. A novel multi-objective jellyfish optimization based on energy, degree, distance, mobility, and time parameters (MOJO-ED2MT) technique has been proposed to overcome these challenges. Three phases are involved in the proposed method: selection of cluster heads, compression of data, and routing of the data. In this first phase, a fuzzy agglomerative clustering algorithm is employed to choose an optimal dual cluster head from inter-cluster and intra-cluster. In the second phase, a neighborhood indexing sequence (NIS) algorithm can compress the number of bits in the data before it is transmitted. In the third phase, jellyfish optimization selects the shortest path based on multi-objective parameters. The simulation analysis and result statistics show that the suggested MOJO-E MT approach performs better than the state-of-the-art algorithms across various performance measures. The proposed MOJO-E MT framework achieves 11.5, 15.4 %, and 17.99 % more network lifetime than EOR-iABC, C3HA, and ML-AEFA algorithms.

A 62.2dB SNDR Event-Driven Level-Crossing ADC with SAR-Assisted Delay Compensation Loop for Time-Sparse Biomedical Signal Acquisition.

This paper proposed an event-driven clockless level-crossing ADC (LC-ADC) suitable for biomedical applications. Thanks to the LC loop, the sampling rate of the converter automatically adapts to the input activities. Activity-dependent power consumption and data compression can thus be realized, saving system power, especially during time-sparse signal acquisition. Meanwhile, a SAR-assisted loop is exploited to resolve the loop-delay-induced distortion in conventional LC-ADC. Therefore, the resolution and power efficiency of the LC-ADC are improved effectively while maintaining the event-driven feature. Implemented in a 55nm process, the proposed LC-ADC achieves a scalable power consumption and a peak SNDR of 62.2dB for a 20kHz input. It also achieves a Walden FoM of 29.7fJ/conv.-step and a Schreier FoM of 158.6dB, which is best in class, without using off-chip calibration. Sub μW power is realized when the input frequency is below 1.5kHz. The proposed LC-ADC is also verified by simulated electrocardiogram (ECG), neural spike, and electromyogram (EMG) signals. It provides a ~7X data compression for ECG input, providing an attractive solution for time-sparse signal acquisition in biomedical applications.

Sharing massive biomedical data at magnitudes lower bandwidth using implicit neural function

Efficient storage and sharing of massive biomedical data would open up their wide accessibility to different institutions and disciplines. However, compressors tailored for natural photos/videos are rapidly limited for biomedical data, while emerging deep learning-based methods demand huge training data and are difficult to generalize. Here, we propose to conduct Biomedical data compRession with Implicit nEural Function (BRIEF) by representing the target data with compact neural networks, which are data specific and thus have no generalization issues. Benefiting from the strong representation capability of implicit neural function, BRIEF achieves 2[Formula: see text]3 orders of magnitude compression on diverse biomedical data at significantly higher fidelity than existing techniques. Besides, BRIEF is of consistent performance across the whole data volume, and supports customized spatially varying fidelity. BRIEF's multifold advantageous features also serve reliable downstream tasks at low bandwidth. Our approach will facilitate low-bandwidth data sharing and promote collaboration and progress in the biomedical field.

Deep learning model for fiber optic distributed acoustic sensing seismic data compression based on autoencoder and recurrent neural networks

Because of the extremely high sampling rate of fiber optic distributed acoustic sensing (DAS) equipment, the amount of DAS seismic data collected is enormous, which poses great challenges to the transmission and storage of DAS seismic data. Therefore, it is essential to study the compression methods of DAS seismic data. Existing data compression methods such as wavelet transform, cosine transform, and convolutional autoencoder (CAE) still have room for improvement in the compression performance and compression ratio (CR). Thus, we have proposed what we believe to be is a novel deep learning model called the recurrent autoencoder (RAE) for high-performance compression of DAS seismic data. Under different CRs, we have designed performance evaluation experiments for RAE models based on different RNN modules with different loss functions. When the CR of the RAE model is 8, the signal-to-noise ratio (SNR) of the reconstructed DAS seismic data reaches 40.60 dB, which is better than that of the CAE’s 11.58 dB. The ultimate CR was increased to 512 without reducing the compression quality, which is 4.12 times higher than the CAE model. The LSTM with a weighted loss function improves the SNR to 43.66 dB at a CR of 8, which is 3.06 dB higher than the LSTM with additive loss function. The results show that the RAE model proposed with a weighted loss function in this paper has excellent DAS seismic data compression performance and provides a high CR, which can be widely applied in large-scale DAS seismic data compression.

A Comprehensive Survey for Hiding Medical Data Based DNA Lossless Technique

The health information data includes reports on the patient’s condition, including addresses, names, tests, treatments, diagnoses, and medical history. It is sensitive information for patients, and all means of protection must be provided to prevent third parties from manipulation or fraudulent use. The goal of this paper presents studies of safeguard the transfer of sensitive data over an unsecured network using DNA techniques to enhance security. A comparison analysis is used to show the security issues of the various DNA-based data (ciphering, concealing, and compression) techniques that are studied and investigated. Also, in our study is to provide future researchers with the necessary knowledge to conduct data security research to develop or improve DNA-based data- ciphering, compression-hiding strategies that are more reliable, secure, and efficient. Thus, future researchers need to propose a trustworthy DNA-based data (ciphering, hiding, and compression) techniques. The result of our study appears the perceived vulnerability of medical data and health insurance to concerns of security as well as common security issues, also the nature of these common security concerns, for reducing medical data security risks.

AstroDLLC: Efficiently Reducing Storage and Transmission Costs for Massive Solar Observation Data via Deep Learning-based Lossless Compression

Effective data compression technology is essential for addressing data storage and transmission needs, especially given the escalating volume and complexity of data generated by contemporary astronomy. In this study, we propose utilizing deep learning-based lossless compression techniques to improve compression efficiency. We begin with a qualitative and quantitative analysis of the temporal and spatial redundancy in solar observation data. Based on this analysis, we introduce a novel deep learning-based framework called AstroDLLC for the lossless compression of astronomical solar images. AstroDLLC first segments high-resolution images into blocks to ensure that deep learning model training does not rely on high-computation power devices. It then addresses the non-normality of the partitioned data through simple reversible computational methods. Finally, it utilizes Bit-swap to train deep learning models that capture redundant features across multiple image frames, thereby enhancing compression efficiency. Comprehensive evaluations using data from the New Vacuum Solar Telescope reveal that AstroDLLC achieves a maximum compression ratio of 3.00 per image, surpassing Gzip, RICE, and other lossless technologies. The performance of AstroDLLC underscores its potential to address data compression challenges in astronomy.

Biostatistical Aspects of Whole Genome Sequencing Studies: Preprocessing and Quality Control.

Rapid advances in high-throughput DNA sequencing technologies have enabled large-scale whole genome sequencing (WGS) studies. Before performing association analysis between phenotypes and genotypes, preprocessing and quality control (QC) of the raw sequence data need to be performed. Because many biostatisticians have not been working with WGS data so far, we first sketch Illumina's short-read sequencing technology. Second, we explain the general preprocessing pipeline for WGS studies. Third, we provide an overview of important QC metrics, which are applied to WGS data: on the raw data, after mapping and alignment, after variant calling, and after multisample variant calling. Fourth, we illustrate the QC with the data from the GENEtic SequencIng Study Hamburg-Davos (GENESIS-HD), a study involving more than 9000 human whole genomes. All samples were sequenced on an Illumina NovaSeq 6000 with an average coverage of 35× using a PCR-free protocol. For QC, one genome in a bottle (GIAB) trio was sequenced in four replicates, and one GIAB sample was successfully sequenced 70 times in different runs. Fifth, we provide empirical data on the compression of raw data using the DRAGEN original read archive (ORA). The most important quality metrics in the application were genetic similarity, sample cross-contamination, deviations from the expected Het/Hom ratio, relatedness, and coverage. The compression ratio of the raw files using DRAGEN ORA was 5.6:1, and compression time was linear by genome coverage. In summary, the preprocessing, joint calling, and QC of large WGS studies are feasible within a reasonable time, and efficient QC procedures are readily available.