Measuring the Performance of Computer Vision Systems in Malaria Studies

Juan Valentín Lorenzo-Ginori,Niurka Mollineada-Diogo,Lyanett Chinea-Valdés,Alfredo Meneses-Marcel

doi:10.3991/ijoe.v16i11.16035

Abstract

Digital image processing-computer vision (DIP-CV) systems are used to automate malaria diagnosis through microscopy analysis of thin blood smears. Some variability is observed in the experimental design to evaluate the statistical measures of performance (SMP) of such systems. The objective of this work is assessing good practices when using SMP to evaluate DIP-CV systems for malaria diagnosis. A mathematical model was built to characterize diagnosis using DIP-CV systems and used to obtain curve families showing the relationships among various SMP of these systems, both using theoretical equations and computer simulation. Curve families showing (a) the relationships among the minimum number of positive erythrocytes (RBCs) to be observed, the per object (RBC) sensitivity and the probability to detect at least one positive, (b) per specimen sensitivity vs. total number of RBCs observed for a typical per object sensitivity and a range of parasite densities (c) per object positive predictive value vs. per object specificity for a typical per object sensitivity and various parasite densities. When determining the per specimen sensitivity, the parasite density p showed to have more influence on the number of RBCs that must be analyzed than the per object sensitivity. Measuring p accurately depends heavily upon the per object positive predictive value of the classifier. For low p values, this would require very high per object specificity and a high enough value of observed RBCs to measure this accurately.

Highlights

Malaria continues being one of the largest health problems faced by our planet, where in 2018 228 million cases of malaria occurred worldwide, with 405 000 deaths, mostly children under 5 years old and with a high prevalence in the Sub-Saharan Africa region [1]
An effective way to diagnose malaria and to determine the infection rate is the analysis of thin blood smears in a microscope, during which the infected erythrocytes are detected and can be counted to determine the parasite density p which will designate in this work the proportion of infected RBCs
Research on the application of digital image processing-computer vision (DIPCV) systems to analyze the thin blood smears during malaria studies, constitutes a current topic that has produced in the last few years numerous scientific publications

Summary

Introduction

Malaria continues being one of the largest health problems faced by our planet, where in 2018 228 million cases of malaria occurred worldwide, with 405 000 deaths, mostly children under 5 years old and with a high prevalence in the Sub-Saharan Africa region [1]. The issue of the practical meaning of the statistical measures of performance (SMP) of these systems and the influence on them of the errors due to various sources usually present, has not been considered with enough interest and deserves a more detailed analysis, which is the object of this work. Analyzing a representative set of published papers reveals the lack of uniformity with which the system’s effectiveness is evaluated This problem is increased due to the lack of available, public annotated databases that could be used for this purpose

Objectives

Methods

Results

Discussion

Conclusion