Parallel DEA-Dantzig-Wolfe algorithm for massive data applications

Abstract

The application of data envelopment analysis (DEA) in large-scale datasets raises computational concerns, and many novel algorithms have been proposed. However, limitations of the existing algorithms such as the computational difficulties due to data volume and privacy issues still exist when the datasets under evaluation are massive and possess a high-density feature. The existing algorithms have not mentioned the potential conflict between a requirement of full data for implementation in applications and the reality that data privacy could prevent a full data application. To address the above-mentioned issues, we integrate DEA and the Dantzig-Wolfe (DW) decomposition algorithm and propose a parallel DEA-DW algorithm to facilitate the computing of efficiency scores. Furthermore, the computing time of the algorithm is analyzed. Finally, we perform numerical experiments with different datasets to demonstrate the feasibility and effectiveness of the proposed algorithm, and analyze the interactions of the master problem (MP) and the sub-problems (SPs) of the algorithm.