Scale Up or Scale Out for Graph Processing?

Abstract

This column explores a simple question: scale up or scale out for graph processing? Should we simply throw beefier individual multi-core, large-memory machines at graph processing tasks and focus on developing more efficient multi-threaded algorithms, or are investments in distributed graph processing frameworks and accompanying algorithms worthwhile? For rhetorical convenience, I adopt customary definitions, referring to the former as scale up and the latter as scale out. Under what circumstances should we prefer one approach over the other?