9 - Parallel Processors

Abstract

This chapter provides an overview of parallel processors. It discusses single instruction stream, multiple data streams (SIMD) computers, multiple instruction streams, multiple data streams (MIMD) computers, programming MIMDs, and MIMDs connected by a single bus and a network. The virtues of SIMD are that all the parallel execution units are synchronized, and they all respond to a single instruction that emanates from a single program counter (PC). SIMD works best when dealing with arrays in for loops. Hence, for massive parallelism to work in SIMD, there must be massive data or data parallelism. SIMD is at its weakest in case or switch statements, where each execution unit must perform a different operation on its data, depending on what data it has. The chapter highlights the characteristics of some well-known SIMD computers. Computer architects have always sought the El Dorado of computer design: to create powerful computers simply by connecting many existing smaller ones. This is the idea behind MIMDs. MIMDs define high performance as high throughput for independent tasks. The chapter also reviews future directions for parallel processors and highlights facts of life for large-scale parallel processors. A key characteristic of programs for parallel machines is frequency of synchronization and communication. Large-scale parallel machines have distributed physical memory; the higher bandwidth and lower overhead of local memory compared to nonlocal memory strongly rewards parallel processing programmers who utilize locality.