Performance optimization for data intensive grid applications

Abstract

The ability to effectively use computational grids for data intensive applications is becoming increasingly important. The distributed, heterogeneous, shared nature of the computing resources provides a significant challenge in developing support for computationally demanding applications. In this paper we describe several performance optimization techniques we have developed for the filter-stream programming framework that we have designed and implemented for programming data intensive applications on the Grid. We present performance results for multiple versions of a medical imaging application on various distributed machine configurations that show the benefits of the optimizations, and also provide evidence that filter-stream programming can be implemented to both efficiently utilize available Grid resources and to provide scalable application performance as additional resources are made available.