An enhanced parallel processing algorithm based on TOP-K decomposition of hypercube model

Abstract

Parallel processing technology has been widely used in many fields. We will discuss the technology of large-scale data parallel computing based on network. The parallel processing method based on hypercube model could divide large-scale data into a large number of sub-datasets, which will be distributed to each processing unit. But empty hypercube units existed because of uneven segmentation. To solve this question, an enhanced parallel processing algorithm based on TOP-K (it is equal to selecting the kth data from the ordered data) decomposition of hypercube model was proposed to evenly divide large-scale data in parallel processing. Experiment result shows that the proposed algorithm has some enhancement on time complexity, scalability and speedup in contrast with the parallel processing method based on hypercube model.