Performance assessment of genomic island prediction tools with an improved version of Design-Island

Abstract

Genomic Islands (GIs) play an important role in the evolution and adaptation of prokaryotes. The origin and extent of ecological diversity of prokaryotes can be analyzed by comparing GIs across closely or distantly related prokaryotes. Understanding the importance of GI and to study the bacterial evolution, several GI prediction tools have been generated. An unsupervised method, Design-Island, was developed to identify GIs using Monte-Carlo statistical test on randomly selected segments of a chromosome. Here, in the present study Design-Island was modified with the incorporation of majority voting, multiple hypothesis testing correction. The performance of the modified version, Design-Island-II was tested and compared with the existing GI prediction tools. The performance assessment and benchmarking of the GI prediction tools require experimentally validated dataset, which is lacking. So, different datasets, generated or taken from literature were utilized to compare the sensitivity (SN), specificity (SP), precision (PPV) and accuracy (AC) of Design-Island-II. It showed substantial enhancement in term of SN, SP, PPV and AC, and significantly reduced the computation time of the algorithm. The performance of Design-Island-II has also been compared with several GI prediction tools using curated dataset of putative horizontally transferred genes. Design-Island-II showed the highest sensitivity and F1 score, comparable specificity, precision and accuracy in comparison to the other available methods. IslandViewer4 and Islander outperformed all the available methods in terms of AC and PPV respectively. Our study suggested Design-Island-II, IslandViewer4 and GIHunter among the top performing GI prediction tools considering both sensitivity and specificity of the methods.