Introduction to big data becomes very important with dealing in high-performance parallel distributed computing, especially in those systems where communication among a number of processors is required. The current paper takes into consideration different topologies for the Shuffle Exchange Network (SEN) in order to attain optimal data transfer among such scenarios. SEN topologies are a key feature in connecting several processors and implementing the data transfer among them where a single processor fails to handle the load. The study hereby reports in the performance, reliability, and cost analysis of these topologies. These topologies, some of which are advocated to have better performance in many studies, include replicated networks. Our research demystifies claims made in earlier papers that replicated networks have inflated reliability and higher costs due to their additional links. Researchers are therefore guided on accurate performance data that will lead them to make optimum choices of SEN topologies for targeted applications. These findings further highlight that the trade-offs between reliability and cost must be carefully considered during network design so as to arrive at better results for big data communication and computation in parallel and distributed systems. This work provides important insights into the correct evaluation of SEN topologies, helping to correct the misleading facts and to have better network selection for various real-time application realizations.
Read full abstract