A dynamic programming approach to dependent task clustering

Abstract

In Complex Data Management Systems, the program generator must consider segmentation of the data processing program resulting from the generation. There are well-defined breaking points where the only dependency between program segments is the use of external data files. If program segments sharing data files were to be grouped together in the same processing unit, a record constructed for, or read from, a shared file could be routed to each segment using that file, eliminating redundant reading of the file. This paper considers the optimization of clustering program segments into processing units. It is a dynamic or selective trial and error approach which considers constraints of execution order, both implicit and explicit.