Large-scale Genome Sequencing Research Articles

Localized Amino Acid Enrichment Analysis as a Tool for Understanding Protein Extremophilicity.

Sequence conservation analyses offer us a powerful glimpse of natural selection at work. Standard tools for measuring sequence conservation report conservation as a function of a specific location in a multiple sequence alignment and have proven indispensable in identifying highly constrained features such as active site residues. The advent of large-scale genomic sequencing efforts allows researchers to expand this paradigm and investigate more nuanced relationships between sequence and function. Here, we present a simple tool (SWiLoDD: Sliding Window Localized Differentiation Detection) that allows researchers to analyze local, rather than site-specific, conservation using a sliding window approach. Our tool accepts multiple sequence alignments partitioned based on a biological differentiator and returns alignment position-based, localized differential enrichment metrics for amino acids of choice. We present two case studies of this analysis in action: local-but-diffuse glycine enrichments in the ATPase subunits of thermophilic and psychrophilic bacterial gyrase homologs, and ligand- and interface-specific amino acid enrichments in halophilic bacterial crotonyl-CoA carboxylases/reductases. Though we have described examples of extremophilic bacterial proteins in this study, our tool may be used to investigate any set of homologous sequences from which sub-groups can be meaningfully partitioned. Our results suggest that investigating differential localized conservation in partitioned MSAs will expand our understanding of how sequence conservation and protein function are connected.

Leveraging large-scale Mycobacterium tuberculosis whole genome sequence data to characterise drug-resistant mutations using machine learning and statistical approaches

Tuberculosis disease (TB), caused by Mycobacterium tuberculosis (Mtb), is a major global public health problem, resulting in > 1 million deaths each year. Drug resistance (DR), including the multi-drug form (MDR-TB), is challenging control of the disease. Whilst many DR mutations in the Mtb genome are known, analysis of large datasets generated using whole genome sequencing (WGS) platforms can reveal new variants through the assessment of genotype-phenotype associations. Here, we apply tree-based ensemble methods to a dataset comprised of 35,777 Mtb WGS and phenotypic drug-susceptibility test data across first- and second-line drugs. We compare model performance across models trained using mutations in drug-specific regions and genome-wide variants, and find high predictive ability for both first-line (area under ROC curve (AUC); range 88.3–96.5) and second-line (AUC range 84.1–95.4) drugs. To aggregate information from low-frequency variants, we pool mutations by functional impact and observe large improvements in predictive accuracy (e.g., sensitivity: pyrazinamide + 25%; ethionamide + 10%). We further characterise loss-of-function mutations observed in resistant phenotypes, uncovering putative markers of resistance (e.g., ndh 293dupG, Rv3861 78delC). Finally, we profile the distribution of known DR-associated single nucleotide polymorphisms across discretised minimum inhibitory concentration (MIC) data generated from phenotypic testing (n = 12,066), and identify mutations associated with highly resistant phenotypes (e.g., inhA − 779G > T and 62T > C). Overall, our work demonstrates that applying machine learning to large-scale WGS data is useful for providing insights into predicting Mtb binary drug resistance and MIC phenotypes, thereby potentially assisting diagnosis and treatment decision-making for infection control.

Cardiovascular Disease Pathogenicity Predictor (CVD-PP): A Tissue-Specific In Silico Tool for Discriminating Pathogenicity of Variants of Unknown Significance in Cardiovascular Disease Genes.

Interpretation of variants of uncertain significance (VUSs) remains a challenge in the care of patients with inherited cardiovascular diseases (CVDs); 56% of variants within CVD risk genes are VUS, and machine learning algorithms trained upon large data resources can stratify VUS into higher versus lower probability of contributing to a CVD phenotype. We used ClinVar pathogenic/likely pathogenic and benign/likely benign variants from 47 CVD genes to build a predictive model of variant pathogenicity utilizing measures of evolutionary constraint, deleteriousness, splicogenicity, local pathogenicity, cardiac-specific expression, and population allele frequency. Performance was validated using variants for which the ClinVar pathogenicity assignment changed. Functional validation was assessed using prior studies in >900 identified VUS. The model utility was demonstrated using the Catheterization Genetics cohort. We identified a top-ranked model that accurately prioritized variants for which ClinVar clinical significance had changed (n=663; precision-recall area under the curve, 0.97) and performed well compared with conventional in silico methods. This model (CVD pathogenicity predictor) also had high accuracy in prioritizing VUS with functional effects in vivo (precision-recall area under the curve, 0.58). In Catheterization Genetics, there was a greater burden of higher CVD pathogenicity predictor scored VUS in individuals with dilated cardiomyopathy compared with controls (P=8.2×10-15). Of individuals in Catheterization Genetics who harbored highly ranked CVD pathogenicity predictor VUS meeting clinical pathogenicity criteria, 27.6% had clinical evidence of disease. Variant prioritization using this model increased genetic diagnosis in Catheterization Genetics participants with a known clinical diagnosis of hypertrophic cardiomyopathy (7.8%-27.2%). We present a cardiac-specific model for prioritizing variants underlying CVD syndromes with high performance in discriminating the pathogenicity of VUS in CVD genes. Variant review and phenotyping of individuals carrying VUS of pathogenic interest support the clinical utility of this model. This model could also have utility in filtering variants as part of large-scale genomic sequencing studies.

Chinese Populations of Magnaporthe oryzae Serving as a Source of Human-Mediated Gene Flow to Asian Countries: A Population Genomic Analysis.

Magnaporthe oryzae, a filamentous heterothallic ascomycete fungus that serves as the causative agent of rice blast disease, is globally distributed in rice-growing regions. Populations shaped by environmental factors and human intervention play important roles in the formation of genetic structure. In this study, population structures and spatiotemporal dynamics were investigated based on large-scale whole genomic sequences of rice-infecting M. oryzae around the world. By analyzing these genetic structures, we identified divergent clades that crossed geographic boundaries. While we observed associations between the isolates and their geographic origins, we also found that there were frequent migration events occurring across Asia in main rice cultivation regions. Within Asia, China was the migration origin, facilitating gene flows to Japan and South Korea. Since the 1970s, the genetic diversity of M. oryzae populations in China has also shown a steadily increasing trend, continuing through to the 2020s. Additionally, our analysis of the evolutionary history of Asian M. oryzae populations provided insights into the population expansion that has taken place in recent decades. Overall, our findings indicate that human-mediated gene flows played a pivotal role in shaping the genetic structure of M. oryzae.

Quest for Orthologs in the Era of Biodiversity Genomics.

The era of biodiversity genomics is characterized by large-scale genome sequencing efforts that aim to represent each living taxon with an assembled genome. Generating knowledge from this wealth of data has not kept up with this pace. We here discuss major challenges to integrating these novel genomes into a comprehensive functional and evolutionary network spanning the tree of life. In summary, the expanding datasets create a need for scalable gene annotation methods. To trace gene function across species, new methods must seek to increase the resolution of ortholog analyses, e.g. by extending analyses to the protein domain level and by accounting for alternative splicing. Additionally, the scope of orthology prediction should be pushed beyond well-investigated proteomes. This demands the development of specialized methods for the identification of orthologs to short proteins and noncoding RNAs and for the functional characterization of novel gene families. Furthermore, protein structures predicted by machine learning are now readily available, but this new information is yet to be integrated with orthology-based analyses. Finally, an increasing focus should be placed on making orthology assignments adhere to the findable, accessible, interoperable, and reusable (FAIR) principles. This fosters green bioinformatics by avoiding redundant computations and helps integrating diverse scientific communities sharing the need for comparative genetics and genomics information. It should also help with communicating orthology-related concepts in a format that is accessible to the public, to counteract existing misinformation about evolution.

Linkage Disequilibrium-Informed Deep Learning Framework to Identify Genetic Loci for Alzheimer's Disease Using Whole Genome Sequencing Data.

The exponential growth of genomic datasets necessitates advanced analytical tools to effectively identify genetic loci from large-scale high throughput sequencing data. This study presents Deep-Block, a multi-stage deep learning framework that incorporates biological knowledge into its AI architecture to identify genetic regions as significantly associated with Alzheimer's disease (AD). The framework employs a three-stage approach: (1) genome segmentation based on linkage disequilibrium (LD) patterns, (2) selection of relevant LD blocks using sparse attention mechanisms, and (3) application of TabNet and Random Forest algorithms to quantify single nucleotide polymorphism (SNP) feature importance, thereby identifying genetic factors contributing to AD risk. The Deep-Block was applied to a large-scale whole genome sequencing (WGS) dataset from the Alzheimer's Disease Sequencing Project (ADSP), comprising 7,416 non-Hispanic white participants (3,150 cognitively normal older adults (CN), 4,266 AD). First, 30,218 LD blocks were identified and then ranked based on their relevance with Alzheimer's disease. Subsequently, the Deep-Block identified novel SNPs within the top 1,500 LD blocks and confirmed previously known variants, including APOE rs429358 and rs769449. The results were cross-validated against established AD-associated loci from the European Alzheimer's and Dementia Biobank (EADB) and the GWAS catalog. The Deep-Block framework effectively processes large-scale high throughput sequencing data while preserving interactions between SNPs in performing the dimensionality reduction, which can potentially introduce bias or lead to information loss. The Deep-Block approach identified both known and novel genetic variation, enhancing our understanding of the genetic architecture of and demonstrating the framework's potential for application in large-scale sequencing studies.

Large-scale genome sequencing of giant pandas improves the understanding of population structure and future conservation initiatives

The extinction risk of the giant panda has been demoted from "endangered" to "vulnerable" on the International Union for Conservation of Nature Red List, but its habitat is more fragmented than ever before, resulting in 33 isolated giant panda populations according to the fourth national survey released by the Chinese government. Further comprehensive investigations of the genetic background and in-depth assessments of the conservation status of wild populations are still necessary and urgently needed. Here, we sequenced the genomes of 612 giant pandas with an average depth of ~26× and generated a high-resolution map of genomic variation with more than 20 million variants covering wild individuals from six mountain ranges and captive representatives in China. We identified distinct genetic clusters within the Minshan population by performing a fine-grained genetic structure. The estimation of inbreeding and genetic load associated with historical population dynamics suggested that future conservation efforts should pay special attention to the Qinling and Liangshan populations. Releasing captive individuals with a genetic background similar to the recipient population appears to be an advantageous genetic rescue strategy for recovering the wild giant panda populations, as this approach introduces fewer deleterious mutations into the wild population than mating with differentiated lineages. These findings emphasize the superiority of large-scale population genomics to provide precise guidelines for future conservation of the giant panda.

How exome sequencing improves the diagnostics and management of men with non-syndromic infertility.

Male infertility affects approximately 17% of all men and represents a complex disorder in which not only semen parameters such as sperm motility, morphology, and number of sperm are highly variable, but also testicular phenotypes range from normal spermatogenesis to complete absence of germ cells. Genetic factors significantly contribute to the disease but chromosomal aberrations, mostly Klinefelter syndrome, and microdeletions of the Y-chromosome have remained the only diagnostically and clinically considered genetic causes. Monogenic causes remain understudied and, thus, often unidentified, leaving the majority of the male factor couple infertility pathomechanistically unexplained. This has been changing mostly because of the introduction of exome sequencing that allows the analysis of multiple genes in large patient cohorts. As a result, pathogenic variants in single genes have been associated with non-syndromic forms of all aetiologic sub-categories in the last decade. This review highlights the contribution of exome sequencing to the identification of novel disease genes for isolated (non-syndromic) male infertility by presenting the results of a comprehensive literature search. Both, reduced sperm count in azoospermic and oligozoospermic patients, and impaired sperm motility and/or morphology, in asthenozoospermic and/or teratozoospermic patients are highly heterogeneous diseases with well over 100 different candidate genes described for each entity. Applying the standardized evaluation criteria of the ClinGen gene curation working group, 70 genes with at least moderate evidence to contribute to the disease are highlighted. The implementation of these valid disease genes in clinical exome sequencing is important to increase the diagnostic yield in male infertility and, thus, improve clinical decision-making and appropriate genetic counseling. Future advances in androgenetics will continue to depend on large-scale exome and genome sequencing studies of comprehensive international patient cohorts, which are the most promising approaches to identify additional disease genes and provide reliable data on the gene-disease relationship.

Chromatin remodellers as therapeutic targets.

Large-scale cancer genome sequencing studies have revealed that chromatin regulators are frequently mutated in cancer. In particular, more than 20% of cancers harbour mutations in genes that encode subunits of SWI/SNF (BAF) chromatin remodelling complexes. Additional links of SWI/SNF complexes to disease have emerged with the findings that some oncogenes drive transformation by co-opting SWI/SNF function and that germline mutations in select SWI/SNF subunits are the basis of several neurodevelopmental disorders. Other chromatin remodellers, including members of the ISWI, CHD and INO80/SWR complexes, have also been linked to cancer and developmental disorders. Consequently, therapeutic manipulation of SWI/SNF and other remodelling complexes has become of great interest, and drugs that target SWI/SNF subunits have entered clinical trials. Genome-wide perturbation screens in cancer cell lines with SWI/SNF mutations have identified additional synthetic lethal targets and led to further compounds in clinical trials, including one that has progressed to FDA approval. Here, we review the progress in understanding the structure and function of SWI/SNF and other chromatin remodelling complexes, mechanisms by which SWI/SNF mutations cause cancer and neurological diseases, vulnerabilities that arise because of these mutations and efforts to target SWI/SNF complexes and synthetic lethal targets for therapeutic benefit.

Population structure and antibiotic resistance of swine extraintestinal pathogenic Escherichia coli from China

Extraintestinal Pathogenic Escherichia coli (ExPEC) pose a significant threat to human and animal health. However, the diversity and antibiotic resistance of animal ExPEC, and their connection to human infections, remain largely unexplored. The study performs large-scale genome sequencing and antibiotic resistance testing of 499 swine-derived ExPEC isolates from China. Results show swine ExPEC are phylogenetically diverse, with over 80% belonging to phylogroups B1 and A. Importantly, 15 swine ExPEC isolates exhibit genetic relatedness to human-origin E. coli strains. Additionally, 49 strains harbor toxins typical of enteric E. coli pathotypes, implying hybrid pathotypes. Notably, 97% of the total strains are multidrug resistant, including resistance to critical human drugs like third- and fourth-generation cephalosporins. Correspondingly, genomic analysis unveils prevalent antibiotic resistance genes (ARGs), often associated with co-transfer mechanisms. Furthermore, analysis of 20 complete genomes illuminates the transmission pathways of ARGs within swine ExPEC and to human pathogens. For example, the transmission of plasmids co-harboring fosA3, blaCTX-M-14, and mcr-1 genes between swine ExPEC and human-origin Salmonella enterica is observed. These findings underscore the importance of monitoring and controlling ExPEC infections in animals, as they can serve as a reservoir of ARGs with the potential to affect human health or even be the origin of pathogens infecting humans.

Biostatistical Aspects of Whole Genome Sequencing Studies: Preprocessing and Quality Control.

Rapid advances in high-throughput DNA sequencing technologies have enabled large-scale whole genome sequencing (WGS) studies. Before performing association analysis between phenotypes and genotypes, preprocessing and quality control (QC) of the raw sequence data need to be performed. Because many biostatisticians have not been working with WGS data so far, we first sketch Illumina's short-read sequencing technology. Second, we explain the general preprocessing pipeline for WGS studies. Third, we provide an overview of important QC metrics, which are applied to WGS data: on the raw data, after mapping and alignment, after variant calling, and after multisample variant calling. Fourth, we illustrate the QC with the data from the GENEtic SequencIng Study Hamburg-Davos (GENESIS-HD), a study involving more than 9000 human whole genomes. All samples were sequenced on an Illumina NovaSeq 6000 with an average coverage of 35× using a PCR-free protocol. For QC, one genome in a bottle (GIAB) trio was sequenced in four replicates, and one GIAB sample was successfully sequenced 70 times in different runs. Fifth, we provide empirical data on the compression of raw data using the DRAGEN original read archive (ORA). The most important quality metrics in the application were genetic similarity, sample cross-contamination, deviations from the expected Het/Hom ratio, relatedness, and coverage. The compression ratio of the raw files using DRAGEN ORA was 5.6:1, and compression time was linear by genome coverage. In summary, the preprocessing, joint calling, and QC of large WGS studies are feasible within a reasonable time, and efficient QC procedures are readily available.

COPO - Managing sample metadata for biodiversity: considerations from the Darwin Tree of Life project

Large-scale reference genome sequencing projects for all of biodiversity are underway and common standards have been in place for some years to enable the understanding and sharing of sequence data. However, the metadata that describes the collection, processing and management of samples, and link to the associated sequencing and genome data, are not yet adequately developed and standardised for these projects. At the time of writing, the Darwin Tree of Life (DToL) Project is over two years into its ten-year ambition to sequence all described eukaryotic species in Britain and Ireland. We have sought consensus from a wide range of scientists across taxonomic domains to determine the minimal set of metadata that we collectively deem as critically important to accompany each sequenced specimen. These metadata are made available throughout the subsequent laboratory processes, and once collected, need to be adequately managed to fulfil the requirements of good data management practice. Due to the size and scale of management required, software tools are needed. These tools need to implement rigorous development pathways and change management procedures to ensure that effective research data management of key project and sample metadata is maintained. Tracking of sample properties through the sequencing process is handled by Lab Information Management Systems (LIMS), so publication of the sequenced data is achieved via technical integration of LIMS and data management tools. Discussions with community members on how metadata standards need to be managed within large-scale programmes is a priority in the planning process. Here we report on the standards we developed with respect to a robust and reusable mechanism of metadata collection, in the hopes that other projects forthcoming or underway will adopt these practices for metadata.

Real-time genomic surveillance for enhanced control of infectious diseases and antimicrobial resistance

This article advocates for mobilizing pathogen genomic surveillance to contain and mitigate health threats from infectious diseases and antimicrobial resistance (AMR), building upon successes achieved by large-scale genome sequencing analysis of SARS-CoV-2 variants in guiding COVID-19 monitoring and public health responses and adopting a One Health approach. Capabilities of laboratory-based surveillance and epidemic alert systems should be enhanced by fostering (i) universal access to real-time whole genome sequence (WGS) data of pathogens to inform clinical practice, infection control, public health policies, and vaccine and antimicrobial drug research and development; (ii) integration of diagnostic microbiology data, data from testing asymptomatic individuals, pathogen sequence data, clinical data, and epidemiological data into surveillance programs; (iii) stronger cross-sectorial collaborations between healthcare, public health, animal health, and environmental surveillance and research using One Health approaches, toward understanding the ecology and transmission pathways of pathogens and AMR across ecosystems; (iv) international collaboration and interconnection of surveillance networks, harmonization of laboratory methods, and standardization of surveillance methods for global reporting, including on pathogen genomic variant or strain nomenclature; (v) responsible data sharing between surveillance networks, databases, and platforms according to FAIR (findability, accessibility, interoperability, and reusability) principles; and (vi) research on genomic surveillance system implementation and its cost-effectiveness for different pathogens and AMR threats across different settings. Regional and global One Health policies and governance initiatives should foster the concerted development and efficient utilization of pathogen genomic surveillance to protect the health of humans, animals, and the environment.

Dynamic analysis of SARS-CoV-2 evolution based on different countries

Since late 2019, COVID-19 has significantly impacted the world. Understanding the evolution of SARS-CoV-2 is crucial for protecting against future infectious pathogens. In this study, we conducted a comprehensive chronological analysis of SARS-CoV-2 evolution by examining mutation prevalence from the source countries of VOCs: United Kingdom, India, Brazil, South Africa, plus two countries: United States, Russia, utilizing genomic sequences from GISAID. Our methodological approach involved large-scale genomic sequence alignment using MAFFT, Python-based data processing on a high-performance computing platform, and advanced statistical methods the Maximal Information Coefficient (MIC), and also Long Short-Term Memory (LSTM) models for correlation analysis.Our findings elucidate the dynamics of SARS-CoV-2 evolution, highlighting the virus's changing behaviour over various pandemic stages. Key results include the discovery of three temporal mutation patterns—lineage distinct, long-span, and competitive mutations—with varying levels of impact on the virus. Notably, we observed a convergence of advantageous mutations in the spike protein, especially in the later stages of the pandemic, indicating a substantial evolutionary pressure on the virus.One of the most significant revelations is the predominant role of natural immunity over vaccination-induced immunity in driving these evolutionary changes. This emphasizes the critical need for regular vaccine updates to maintain efficacy against evolving strains.In conclusion, our study not only sheds light on the evolutionary trajectory of SARS-CoV-2 but also underscores the urgency for robust, continuous global data collection and sharing. It highlights the necessity for rapid adaptations in medical countermeasures, including vaccine development, to stay ahead of pathogen evolution. This research provides valuable insights for future pandemic preparedness and response strategies.

Species-aware DNA language models capture regulatory elements and their evolution

BackgroundThe rise of large-scale multi-species genome sequencing projects promises to shed new light on how genomes encode gene regulatory instructions. To this end, new algorithms are needed that can leverage conservation to capture regulatory elements while accounting for their evolution.ResultsHere, we introduce species-aware DNA language models, which we trained on more than 800 species spanning over 500 million years of evolution. Investigating their ability to predict masked nucleotides from context, we show that DNA language models distinguish transcription factor and RNA-binding protein motifs from background non-coding sequence. Owing to their flexibility, DNA language models capture conserved regulatory elements over much further evolutionary distances than sequence alignment would allow. Remarkably, DNA language models reconstruct motif instances bound in vivo better than unbound ones and account for the evolution of motif sequences and their positional constraints, showing that these models capture functional high-order sequence and evolutionary context. We further show that species-aware training yields improved sequence representations for endogenous and MPRA-based gene expression prediction, as well as motif discovery.ConclusionsCollectively, these results demonstrate that species-aware DNA language models are a powerful, flexible, and scalable tool to integrate information from large compendia of highly diverged genomes.

Abstract 6937: High throughput single cell simultaneous DNA and RNA sequencing identified cell-of-origin of breast cancer

Abstract Understanding the relationship between genotype and phenotype in breast cancer cells has been challenging at single cell resolution, mainly because existing high-throughput methods are limited to measuring a single modality and data must be integrated indirectly via computational methods. To address this challenge, we developed wellDR-seq, a high-throughput single cell method that can simultaneously measure the single cell whole genome and transcriptome directly from thousands of single cells in parallel. Using this method, we profiled 17,427 single cells in 6 different ER+ breast cancer patients with either premalignant disease (Ductal-carcinoma-in-situ) or invasive ductal carcinoma (IDC). From these data we identified the epithelial cell-of-origin in 3 cases, showing that ER+ breast cancer cells originated from Luminal Hormone Responsive (LumHR) in the normal breast tissues. We also found that autosomal somatic copy number aberrations were exclusively present in LumSec (< 2%) and LumHR epithelial cells, while chrX gain or loss also occurred in stromal cells (eg fibroblast and endothelial) but a low frequency (<2%) in our datasets. Additionally, our data show the impact of subclonal copy number profiles on gene expression programs, which reflects both gene dosage effects in CNA regions and more complex deregulation in copy-neutral regions. wellDR-seq offers a powerful tool for large-scale single cell genome and transcriptome simultaneous sequencing across different clinical samples, opens new avenue to investigate genotype and phenotypes interactions, identify tumor cells origins, reveal somatic copy number and mutation events, quantify the gene dosage effect and discerning differential genes expression between tumor subclones. Citation Format: Kaile Wang, Rui Ye, Shanshan Bai, Zhenna Xiao, Lei Yano, Jianzhuo Li, Yiyun Lin, Emi Sei, Steven Lin, Alastair Thompson, Savitri Krishnamurthy, Nicholas Navin. High throughput single cell simultaneous DNA and RNA sequencing identified cell-of-origin of breast cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 6937.

A catalogue of chromosome counts for Phylum Nematoda.

Nematodes are important biological models in genetics and genomics, with research driven by basic biological as well as applied questions. The presence of holocentric chromosomes, clades with frequent polyploidy and the phenomenon of programmed DNA elimination make nematode karyotypic diversity of particular interest. Here we present a catalogue of published karyotypes of nematode species, rationalising and normalising descriptions from the previous 135 years. Karyotypes of 257 species are presented in taxonomic context. Nuclear chromosome counts range from 2 to 60. Tylenchina is identified as particularly diverse in karyotype. We highlight that Rhabditida and especially parasitic Rhabditina are well-represented, but there is a paucity of data from Enoplea, Dorylaimia, and from free-living marine groups in Chromadorea. The data have been uploaded to the Genomes on a Tree (GoaT) datasystem ( https://goat.genomehubs.org/) for integration with ongoing, large-scale genome sequencing efforts.

Mobilisation and analyses of publicly available SARS-CoV-2 data for pandemic responses.

The COVID-19 pandemic has seen large-scale pathogen genomic sequencing efforts, becoming part of the toolbox for surveillance and epidemic research. This resulted in an unprecedented level of data sharing to open repositories, which has actively supported the identification of SARS-CoV-2 structure, molecular interactions, mutations and variants, and facilitated vaccine development and drug reuse studies and design. The European COVID-19 Data Platform was launched to support this data sharing, and has resulted in the deposition of several million SARS-CoV-2 raw reads. In this paper we describe (1) open data sharing, (2) tools for submission, analysis, visualisation and data claiming (e.g. ORCiD), (3) the systematic analysis of these datasets, at scale via the SARS-CoV-2 Data Hubs as well as (4) lessons learnt. This paper describes a component of the Platform, the SARS-CoV-2 Data Hubs, which enable the extension and set up of infrastructure that we intend to use more widely in the future for pathogen surveillance and pandemic preparedness.

A randomized optimal k-mer indexing approach for efficient parallel genome sequence compression

Next Generation Sequencing (NGS) technology generates massive amounts of genome sequence that increases rapidly over time. As a result, there is a growing need for efficient compression algorithms to facilitate the processing, storage, transmission, and analysis of large-scale genome sequences. Over the past 31 years, numerous state-of-the-art compression algorithms have been developed. The performance of any compression algorithm is measured by three main compression metrics: compression ratio, time, and memory usage. Existing k-mer hash indexing systems take more time, due to the decision-making process based on compression results. In this paper, we propose a two-phase reference genome compression algorithm using optimal k-mer length (RGCOK). Reference-based compression takes advantage of the inter-similarity between chromosomes of the same species. RGCOK achieves this by finding the optimal k-mer length for matching, using a randomization method and hashing. The performance of RGCOK was evaluated on three different benchmark data sets: novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), Homo sapiens, and other species sequences using an Amazon AWS virtual cloud machine. Experiments showed that the optimal k-mer finding time by RGCOK is around 45.28 min, whereas the time for existing state-of-the-art algorithms HiRGC, SCCG, and HRCM ranges from 58 min to 8.97 h.

GSC: efficient lossless compression of VCF files with fast query.

With the rise of large-scale genome sequencing projects, genotyping of thousands of samples has produced immense variant call format (VCF) files. It is becoming increasingly challenging to store, transfer, and analyze these voluminous files. Compression methods have been used to tackle these issues, aiming for both high compression ratio and fast random access. However, existing methods have not yet achieved a satisfactory compromise between these 2 objectives. To address the aforementioned issue, we introduce GSC (Genotype Sparse Compression), a specialized and refined lossless compression tool for VCF files. In benchmark tests conducted across various open-source datasets, GSC showcased exceptional performance in genotype data compression. Compared with the industry's most advanced tools (namely, GBC and GTC), GSC achieved compression ratios that were higher by 26.9% to 82.4% over GBC and GTC on the datasets, respectively. In lossless compression scenarios, GSC also demonstrated robust performance, with compression ratios 1.5× to 6.5× greater than general-purpose tools like gzip, zstd, and BCFtools-a mode not supported by either GBC or GTC. Achieving such high compression ratios did require some reasonable trade-offs, including longer decompression times, with GSC being 1.2× to 2× slower than GBC, yet 1.1× to 1.4× faster than GTC. Moreover, GSC maintained decompression query speeds that were equivalent to its competitors. In terms of RAM usage, GSC outperformed both counterparts. Overall, GSC's comprehensive performance surpasses that of the most advanced technologies. GSC balances high compression ratios with rapid data access, enhancing genomic data management. It supports seamless PLINK binary format conversion, simplifying downstream analysis.