Streaming Graph Theory

Abstract

Graph‐based deep‐learning accelerators have been developed. They function as the Multiple Instructions Multiple Data machines to perform massive parallel operations. Blaize Graph Streaming Processor (GSP) proposes the novel architecture based on Streaming Graph model for deep learning applications. The streaming graph is widely applied to software development ranging from network traffic to database applications. The research focuses on streaming graph algorithms and different partitions to enhance the overall efficiency. The GSP achieves four‐level parallelism in architectural design: task parallelism, thread parallelism, data parallelism, and instruction parallelism. Graphcore develops Intelligence Processing Unit which adopts a similar approach as Blaize GSP that applies the graph theory to perform the finegrained operation with a massive parallel thread for deep learning applications. The operation of IPU is based on Bulk Synchronous Parallel model. It divides the operation into the local computation phase, communication phase, and barrier synchronization.