Benchmark of computational methods for predicting microRNA-disease associations

Zhou Huang,Jiangcheng Shi,Leibo Liu,Qinghua Cui,Yuan Zhou,Jianwei Li,Yuanxu Gao

doi:10.1186/s13059-019-1811-3

Abstract

BackgroundA series of miRNA-disease association prediction methods have been proposed to prioritize potential disease-associated miRNAs. Independent benchmarking of these methods is warranted to assess their effectiveness and robustness.ResultsBased on more than 8000 novel miRNA-disease associations from the latest HMDD v3.1 database, we perform systematic comparison among 36 readily available prediction methods. Their overall performances are evaluated with rigorous precision-recall curve analysis, where 13 methods show acceptable accuracy (AUPRC > 0.200) while the top two methods achieve a promising AUPRC over 0.300, and most of these methods are also highly ranked when considering only the causal miRNA-disease associations as the positive samples. The potential of performance improvement is demonstrated by combining different predictors or adopting a more updated miRNA similarity matrix, which would result in up to 16% and 46% of AUPRC augmentations compared to the best single predictor and the predictors using the previous similarity matrix, respectively. Our analysis suggests a common issue of the available methods, which is that the prediction results are severely biased toward well-annotated diseases with many associated miRNAs known and cannot further stratify the positive samples by discriminating the causal miRNA-disease associations from the general miRNA-disease associations.ConclusionOur benchmarking results not only provide a reference for biomedical researchers to choose appropriate miRNA-disease association predictors for their purpose, but also suggest the future directions for the development of more robust miRNA-disease association predictors.

Highlights

MicroRNAs are ~ 22 nt RNAs that regulate gene expression mainly by targeting the 3′UTR regions of mRNAs [1, 2]
Independent benchmarking of miRNA-disease association prediction methods on novel HMDD v3.1 data By manual investigation of the related literature from PubMed and Google Scholar, 90 published miRNA-disease association predictors were collected (Additional file 1: Table S1)
(2) All of the high-ranked predictors exhibited acceptable overall performance in the benchmarking test, with the top 13 predictors reaching areas under the precision-recall curve (AUPRC) > 0.2, and the MCLPMDA, LFEMDA, and LPLNS achieved the best overall performance (Fig. 1). (3) Users should be cautious of the potential bias toward the overrepresented diseases

Summary

Introduction

MicroRNAs (miRNAs) are ~ 22 nt RNAs that regulate gene expression mainly by targeting the 3′UTR regions of mRNAs [1, 2]. These small non-coding RNAs are widely involved in important biological processes such as cell division, differentiation, apoptosis, cell cycle regulation, inflammation, and stress response [3, 4]. HMDD v3.1, the most updated miRNA-disease association dataset for (released in January 2019), covers only 35,547 miRNA-disease associations between 893 diseases and 1206 miRNA genes [8] These statistics indicate that ~ 30% and ~ 80% of human miRNAs and diseases respectively have not been reported by experimental investigations. Independent benchmarking of these methods is warranted to assess their effectiveness and robustness

Methods

Results

Conclusion