Algorithmic Enhancements to Big Data Computing Frameworks for Medical Image Processing

Abstract

Large-scale medical imaging studies to date have predominantly leveraged in-house, laboratory-based or traditional grid computing resources for their computing needs, where the applications often use hierarchical data structures (e.g., NFS file stores) or databases (e.g., COINS, XNAT) for storage and retrieval. The resulting performance for laboratory-base approaches reveal that performance is impeded by standard network switches since they can saturate network bandwidth during transfer from storage to processing nodes for even moderate-sized studies. On the other hand, the grid may be costly to use due to the dedicated resources used to execute the tasks and lack of elasticity. With increasing availability of cloud-based Big Data frameworks, such as Apache Hadoop, cloud-based services for executing medical imaging studies have shown promise. Despite this promise, our preliminary studies have revealed that existing Big Data frameworks illustrate different performance limitations for medical imaging applications, which calls for new algorithms that optimize their performance and suitability for medical imaging. For instance, Apache HBase's load distribution strategy of region split and merge is detrimental to the hierarchical organization of imaging data (e.g., project, subject, session, scan, slice). To address these challenges, this doctoral research is developing a range of performance optimization algorithms. This paper describes preliminary research we have conducted in this realm and presents a list of research tasks that will be undertaken as part of this doctoral research.