Completeness and accuracy of WWW-based catalogues of medical E-learning modules

Abstract

Catalogues of medical E-learning modules offer easy access to learning material that is available on the World Wide Web (WWW) free of charge. Therefore, a study was conducted on the retrieval performance of four WWW-based catalogues: CAL reviews, KELDAmed, LMU, and LRSMed. LRSMed is run by the authors. Completeness was calculated pairwise. Two reviewers checked the availability of 80 modules independently. Five criteria were chosen to calculate accuracy: gynaecology and microbiology as medical fields; case study as type of learning resource; diabetes mellitus and AIDS as free-text diagnoses. Also, two reviewers evaluated independently whether the module is really an appropriate resource for that criterion or not. The analysis is based on a consensual decision about the correct votes. From the URLs, 93% were available at the evaluation of completeness, 92% at the evaluation of accuracy. The kappa values for inter-rater reliability were 0.83 and 0.36. The best service offers 60.8% of the pooled E-learning resources. The resources retrieved by the five criteria were rated as correct in 69.3% (LRSMed), 76.6% (KELDAmed), and 82.7% (CAL reviews). The overall accuracy was 76.7%. Medical students and other potential users of E-learning modules should be aware that the completeness of WWW-based catalogues in this area is not satisfying. The retrieval accuracy is better; four of five resources offered correspond with the search criterion. Up to now, most of the services miss an application-programming interface that could be used for a meta-search to improve completeness.