Face processing supports our ability to recognize friend from foe, form tribes, and understand the emotional implications of changes in facial musculature. This skill relies on a distributed network of brain regions but how these regions interact is poorly understood. Here, we integrate anatomical and functional connectivity measurements with behavioral assays to create a global model of the face connectome. We dissect key features such as the network topology and fiber composition. We propose a neurocognitive model with three core streams, and face processing along these streams occurs in a parallel and reciprocal fashion. While long-range fiber paths are important, face network is dominated by short-range fibers. Last, we provide some evidence that the well-known right lateralization of face processing arises from imbalanced intra/interhemispheric connections. In sum, the face network relies on dynamic communication across highly structured fiber tracts, which enables coherent face processing that underpins behavior and cognition.