The mammalian genome is intricately folded in a three-dimensional topology believed to be important for the orchestration of gene expression regulating development, differentiation and tissue homeostasis. Important features of spatial genome conformation in the nucleus are promoter-enhancer contacts regulating gene expression within topologically-associated domains (TADs), short- and long-range interactions between TADs and associations of chromatin with nucleoli and nuclear speckles. In addition, anchoring of chromosomes to the nuclear lamina via lamina-associated domains (LADs) at the nuclear periphery is a key regulator of the radial distribution of chromatin. To what extent TADs and LADs act in concert as genomic organizers to shape the three-dimensional topology of chromatin has long remained unknown. A new study addressing this key question provides evidence of (i) preferred long-range associations between TADs forming TAD “cliques” which organize large heterochromatin domains, and (ii) stabilization of TAD cliques by LADs at the nuclear periphery after induction of terminal differentiation. Here, we review these findings, address the issue of whether TAD cliques exist in single cells and discuss the extent of cell-to-cell heterogeneity in higher-order chromatin conformation. The recent observations provide a first appreciation of changes in 4-dimensional higher-order genome topologies during differentiation.