The Art of Scaling: Distributed and Connected to Sustain the Golden Age of Computation

Abstract

The history of computers was driven by maximizing computing power. In just about thirty years, CPU performance has skyrocketed by 5, 000 times [1], taking computers from the level of an almost mechanical calculator to a gadget to run 3D games. This boost was mainly due to the exponential development of process technology under Moore's Law [2]. While this progress also applies to the mobile world, current CPUs in mobile phones are merely comparable to the nervous systems of a jellyfish, not even capable of replicating connectomes [3] of a honeybee. Besides, Moore's Law and Dennard Scaling are already facing their limitations, if not declared is over, widening the gap between endless human imagination and physical computing levels. Despite a number of technical suggestions to tackle this barrier, few papers have managed to capture the essence of its causes and consequences, or to understand that increase in computing power comes from the combined interaction of design and economic factors as well as process technology improvements. This paper takes a general approach to analyze how each area has come to its limitations and review possible suggestions for innovation, to find if there are certain common and general principles under these innovations. Focusing on the law of change hidden in general evolution, we demonstrate how these laws influence the process, design, and economic factors facing limitation, and try to derive the general principles of integration, distribution, and connection.