Mykrobe Predictor Research Articles

Accurate and rapid prediction of tuberculosis drug resistance from genome sequence data using traditional machine learning algorithms and CNN

Effective and timely antibiotic treatment depends on accurate and rapid in silico antimicrobial-resistant (AMR) predictions. Existing statistical rule-based Mycobacterium tuberculosis (MTB) drug resistance prediction methods using bacterial genomic sequencing data often achieve varying results: high accuracy on some antibiotics but relatively low accuracy on others. Traditional machine learning (ML) approaches have been applied to classify drug resistance for MTB and have shown more stable performance. However, there is no study that uses deep learning architecture like Convolutional Neural Network (CNN) on a large and diverse cohort of MTB samples for AMR prediction. We developed 24 binary classifiers of MTB drug resistance status across eight anti-MTB drugs and three different ML algorithms: logistic regression, random forest and 1D CNN using a training dataset of 10,575 MTB isolates collected from 16 countries across six continents, where an extended pan-genome reference was used for detecting genetic features. Our 1D CNN architecture was designed to integrate both sequential and non-sequential features. In terms of F1-scores, 1D CNN models are our best classifiers that are also more accurate and stable than the state-of-the-art rule-based tool Mykrobe predictor (81.1 to 93.8%, 93.7 to 96.2%, 93.1 to 94.8%, 95.9 to 97.2% and 97.1 to 98.2% for ethambutol, rifampicin, pyrazinamide, isoniazid and ofloxacin respectively). We applied filter-based feature selection to find AMR relevant features. All selected variant features are AMR-related ones in CARD database. 78.8% of them are also in the catalogue of MTB mutations that were recently identified as drug resistance-associated ones by WHO. To facilitate ML model development for AMR prediction, we packaged every step into an automated pipeline and shared the source code at https://github.com/KuangXY3/MTB-AMR-classification-CNN.

Antibiotic resistance prediction for Mycobacterium tuberculosis from genome sequence data with Mykrobe

Two billion people are infected with Mycobacterium tuberculosis, leading to 10 million new cases of active tuberculosis and 1.5 million deaths annually. Universal access to drug susceptibility testing (DST) has become a World Health Organization priority. We previously developed a software tool, Mykrobe predictor, which provided offline species identification and drug resistance predictions for M. tuberculosis from whole genome sequencing (WGS) data. Performance was insufficient to support the use of WGS as an alternative to conventional phenotype-based DST, due to mutation catalogue limitations. Here we present a new tool, Mykrobe, which provides the same functionality based on a new software implementation. Improvements include i) an updated mutation catalogue giving greater sensitivity to detect pyrazinamide resistance, ii) support for user-defined resistance catalogues, iii) improved identification of non-tuberculous mycobacterial species, and iv) an updated statistical model for Oxford Nanopore Technologies sequencing data. Mykrobe is released under MIT license at https://github.com/mykrobe-tools/mykrobe. We incorporate mutation catalogues from the CRyPTIC consortium et al. (2018) and from Walker et al. (2015), and make improvements based on performance on an initial set of 3206 and an independent set of 5845 M. tuberculosis Illumina sequences. To give estimates of error rates, we use a prospectively collected dataset of 4362 M. tuberculosis isolates. Using culture based DST as the reference, we estimate Mykrobe to be 100%, 95%, 82%, 99% sensitive and 99%, 100%, 99%, 99% specific for rifampicin, isoniazid, pyrazinamide and ethambutol resistance prediction respectively. We benchmark against four other tools on 10207 (=5845+4362) samples, and also show that Mykrobe gives concordant results with nanopore data. We measure the ability of Mykrobe-based DST to guide personalized therapeutic regimen design in the context of complex drug susceptibility profiles, showing 94% concordance of implied regimen with that driven by phenotypic DST, higher than all other benchmarked tools.

Antibiotic resistance prediction for Mycobacterium tuberculosis from genome sequence data with Mykrobe.

Two billion people are infected with Mycobacterium tuberculosis, leading to 10 million new cases of active tuberculosis and 1.5 million deaths annually. Universal access to drug susceptibility testing (DST) has become a World Health Organization priority. We previously developed a software tool, Mykrobe predictor, which provided offline species identification and drug resistance predictions for M. tuberculosis from whole genome sequencing (WGS) data. Performance was insufficient to support the use of WGS as an alternative to conventional phenotype-based DST, due to mutation catalogue limitations. Here we present a new tool, Mykrobe, which provides the same functionality based on a new software implementation. Improvements include i) an updated mutation catalogue giving greater sensitivity to detect pyrazinamide resistance, ii) support for user-defined resistance catalogues, iii) improved identification of non-tuberculous mycobacterial species, and iv) an updated statistical model for Oxford Nanopore Technologies sequencing data. Mykrobe is released under MIT license at https://github.com/mykrobe-tools/mykrobe. We incorporate mutation catalogues from the CRyPTIC consortium et al. (2018) and from Walker et al. (2015), and make improvements based on performance on an initial set of 3206 and an independent set of 5845 M. tuberculosis Illumina sequences. To give estimates of error rates, we use a prospectively collected dataset of 4362 M. tuberculosis isolates. Using culture based DST as the reference, we estimate Mykrobe to be 100%, 95%, 82%, 99% sensitive and 99%, 100%, 99%, 99% specific for rifampicin, isoniazid, pyrazinamide and ethambutol resistance prediction respectively. We benchmark against four other tools on 10207 (=5845+4362) samples, and also show that Mykrobe gives concordant results with nanopore data. We measure the ability of Mykrobe-based DST to guide personalized therapeutic regimen design in the context of complex drug susceptibility profiles, showing 94% concordance of implied regimen with that driven by phenotypic DST, higher than all other benchmarked tools.

Evaluation of whole genome sequencing and software tools for drug susceptibility testing of Mycobacterium tuberculosis

ObjectivesCulture-based assays are currently the reference standard for drug susceptibility testing for Mycobacterium tuberculosis. They provide good sensitivity and specificity but are time consuming. The objective of this study was to evaluate whether whole genome sequencing (WGS), combined with software tools for data analysis, can replace routine culture-based assays for drug susceptibility testing of M. tuberculosis. MethodsM. tuberculosis cultures sent to the Finnish mycobacterial reference laboratory in 2014 (n = 211) were phenotypically tested by Mycobacteria Growth Indicator Tube (MGIT) for first-line drug susceptibilities. WGS was performed for all isolates using the Illumina MiSeq system, and data were analysed using five software tools (PhyResSE, Mykrobe Predictor, TB Profiler, TGS-TB and KvarQ). Diagnostic time and reagent costs were estimated for both methods. ResultsThe sensitivity of the five software tools to predict any resistance among strains was almost identical, ranging from 74% to 80%, and specificity was more than 95% for all software tools except for TGS-TB. The sensitivity and specificity to predict resistance to individual drugs varied considerably among the software tools. Reagent costs for MGIT and WGS were €26 and €143 per isolate respectively. Turnaround time for MGIT was 19 days (range 10–50 days) for first-line drugs, and turnaround time for WGS was estimated to be 5 days (range 3–7 days). ConclusionsWGS could be used as a prescreening assay for drug susceptibility testing with confirmation of resistant strains by MGIT. The functionality and ease of use of the software tools need to be improved.

Dissecting whole-genome sequencing-based online tools for predicting resistance in Mycobacterium tuberculosis: can we use them for clinical decision guidance?

Whole-genome sequencing (WGS)-based bioinformatics platforms for the rapid prediction of resistance will soon be implemented in the Tuberculosis (TB) laboratory, but their accuracy assessment still needs to be strengthened. Here, we fully-sequenced a total of 54 multidrug-resistant (MDR) and five susceptible TB strains and performed, for the first time, a simultaneous evaluation of the major four free online platforms (TB Profiler, PhyResSE, Mykrobe Predictor and TGS-TB).Overall, the sensitivity of resistance prediction ranged from 84.3% using Mykrobe predictor to 95.2% using TB profiler, while specificity was higher and homogeneous among platforms. TB profiler revealed the best performance robustness (sensitivity, specificity, PPV and NPV above 95%), followed by TGS-TB (all parameters above 90%). We also observed a few discrepancies between phenotype and genotype, where, in some cases, it was possible to pin-point some “candidate” mutations (e.g., in the rpsL promoter region) highlighting the need for their confirmation through mutagenesis assays and potential review of the anti-TB genetic databases.The rampant development of the bioinformatics algorithms and the tremendously reduced time-frame until the clinician may decide for a definitive and most effective treatment will certainly trigger the technological transition where WGS-based bioinformatics platforms could replace phenotypic drug susceptibility testing for TB.

Whole genome sequencing of clinical strains of Mycobacterium tuberculosis from Mumbai, India: A potential tool for determining drug-resistance and strain lineage

Amplification of drug resistance in Mycobacterium tuberculosis (M.tb) and its transmission are significant barriers in controlling tuberculosis (TB) globally. Diagnostic inaccuracies and delays impede appropriate drug administration, which exacerbates primary and secondary drug resistance. Increasing affordability of whole genome sequencing (WGS) and exhaustive cataloguing of drug resistance mutations is poised to revolutionise TB diagnostics and facilitate personalized drug therapy. However, application of WGS for diagnostics in high endemic areas is yet to be demonstrated. We report WGS of 74 clinical TB isolates from Mumbai, India, characterising genotypic drug resistance to first- and second-line anti-TB drugs. A concordance analysis between phenotypic and genotypic drug susceptibility of a subset of 29 isolates and the sensitivity of resistance prediction to the 4 drugs was calculated, viz. isoniazid-100%, rifampicin-100%, ethambutol-100% and streptomycin-85%. The whole genome based phylogeny showed almost equal proportion of East Asian (27/74) and Central Asian (25/74) strains. Interestingly we also found a clonal group of 9 isolates, of which 7 patients were found to be from the same geographical location and accessed the same health post. This provides the first evidence of epidemiological linkage for tracking TB transmission in India, an approach which has the potential to significantly improve chances of End-TB goals. Finally, the use of Mykrobe Predictor, as a standalone drug resistance and strain typing tool, requiring just few minutes to analyse raw WGS data into tabulated results, implies the rapid clinical applicability of WGS based TB diagnosis.

Mycobacterium tuberculosis resistance prediction and lineage classification from genome sequencing: comparison of automated analysis tools

Whole-genome sequencing (WGS) has the potential to accelerate drug-susceptibility testing (DST) to design appropriate regimens for drug-resistant tuberculosis (TB). Several recently developed automated software tools promise to standardize the analysis and interpretation of WGS data. We assessed five tools (CASTB, KvarQ, Mykrobe Predictor TB, PhyResSE, and TBProfiler) with regards to DST and phylogenetic lineage classification, which we compared with phenotypic DST, Sanger sequencing, and traditional typing results for a collection of 91 strains. The lineage classifications by the tools generally only differed in the resolution of the results. However, some strains could not be classified at all and one strain was misclassified. The sensitivities and specificities for isoniazid and rifampicin resistance of the tools were high, whereas the results for ethambutol, pyrazinamide, and streptomycin resistance were more variable. False-susceptible DST results were mainly due to missing mutations in the resistance catalogues that the respective tools employed for data interpretation. Notably, we also found cases of false-resistance because of the misclassification of polymorphisms as resistance mutations. In conclusion, the performance of current WGS analysis tools for DST is highly variable. Sustainable business models and a shared, high-quality catalogue of resistance mutations are needed to ensure the clinical utility of these tools.

The variability and reproducibility of whole genome sequencing technology for detecting resistance to anti-tuberculous drugs.

BackgroundThe emergence of resistance to anti-tuberculosis drugs is a serious and growing threat to public health. Next-generation sequencing is rapidly gaining traction as a diagnostic tool for investigating drug resistance in Mycobacterium tuberculosis to aid treatment decisions. However, there are few little data regarding the precision of such sequencing for assigning resistance profiles.MethodsWe investigated two sequencing platforms (Illumina MiSeq, Ion Torrent PGM™) and two rapid analytic pipelines (TBProfiler, Mykrobe predictor) using a well characterised reference strain (H37Rv) and clinical isolates from patients with tuberculosis resistant to up to 13 drugs. Results were compared to phenotypic drug susceptibility testing. To assess analytical robustness individual DNA samples were subjected to repeated sequencing.ResultsThe MiSeq and Ion PGM systems accurately predicted drug-resistance profiles and there was high reproducibility between biological and technical sample replicates. Estimated variant error rates were low (MiSeq 1 per 77 kbp, Ion PGM 1 per 41 kbp) and genomic coverage high (MiSeq 51-fold, Ion PGM 53-fold). MiSeq provided superior coverage in GC-rich regions, which translated into incremental detection of putative genotypic drug-specific resistance, including for resistance to para-aminosalicylic acid and pyrazinamide. The TBProfiler bioinformatics pipeline was concordant with reported phenotypic susceptibility for all drugs tested except pyrazinamide and para-aminosalicylic acid, with an overall concordance of 95.3%. When using the Mykrobe predictor concordance with phenotypic testing was 73.6%.ConclusionsWe have demonstrated high comparative reproducibility of two sequencing platforms, and high predictive ability of the TBProfiler mutation library and analytical pipeline, when profiling resistance to first- and second-line anti-tuberculosis drugs. However, platform-specific variability in coverage of some genome regions may have implications for predicting resistance to specific drugs. These findings may have implications for future clinical practice and thus deserve further scrutiny, set within larger studies and using updated mutation libraries.Electronic supplementary materialThe online version of this article (doi:10.1186/s13073-016-0385-x) contains supplementary material, which is available to authorized users.

Unveiling the molecular basis of antimicrobial resistance in Staphylococcus aureus from the Democratic Republic of the Congo using whole genome sequencing

Staphylococcus aureus from sub-Saharan Africa is frequently resistant to antimicrobial agents that are commonly used to treat invasive infections in resource-limited settings. The underlying mechanisms of resistance are largely unknown. We therefore performed whole genome sequencing (WGS) on S. aureus from the Democratic Republic of the Congo (DRC) to analyse the genetic determinants of antimicrobial resistance. One hundred S. aureus samples were collected from community-associated asymptomatic nasal carriers in the metropolitan area of Kinshasa, DRC, between 2013 and 2014. Phenotypic resistance against 15 antimicrobial agents was compared to the genotypic results that were extracted from WGS data using Mykrobe predictor and the SeqSphere+ software that screened for 106 target genes associated with resistance. Isolates were phenotypically resistant against penicillin (97%, n=97), trimethoprim (72%, n=72) and tetracycline (54%, n=45). Thirty-three isolates (33%) were methicillin-resistant S. aureus (MRSA). Of these, nine isolates (27.3%) were oxacillin-susceptible MRSA (OS-MRSA) and belonged to ST8 (t1476). The Y195F mutation of FemA was associated with OS-MRSA (p 0.015). The majority of trimethoprim resistant isolates carried dfrG. Tetracycline resistance was associated with tet(K). The concordance between phenotypic susceptibility testing and both WGS analysis tools was similar and ranged between 96% and 100%. In conclusion, a high proportion of OS-MRSA in the DRC was linked to mutations of FemA. Genotypic and phenotypical antimicrobial susceptibility testing showed high concordance. This encourages the future use of WGS in routine antimicrobial susceptibility testing.

Rapid antibiotic-resistance predictions from genome sequence data for Staphylococcus aureus and Mycobacterium tuberculosis.

The rise of antibiotic-resistant bacteria has led to an urgent need for rapid detection of drug resistance in clinical samples, and improvements in global surveillance. Here we show how de Bruijn graph representation of bacterial diversity can be used to identify species and resistance profiles of clinical isolates. We implement this method for Staphylococcus aureus and Mycobacterium tuberculosis in a software package (‘Mykrobe predictor') that takes raw sequence data as input, and generates a clinician-friendly report within 3 minutes on a laptop. For S. aureus, the error rates of our method are comparable to gold-standard phenotypic methods, with sensitivity/specificity of 99.1%/99.6% across 12 antibiotics (using an independent validation set, n=470). For M. tuberculosis, our method predicts resistance with sensitivity/specificity of 82.6%/98.5% (independent validation set, n=1,609); sensitivity is lower here, probably because of limited understanding of the underlying genetic mechanisms. We give evidence that minor alleles improve detection of extremely drug-resistant strains, and demonstrate feasibility of the use of emerging single-molecule nanopore sequencing techniques for these purposes.