The unique properties of graphene have made it a promising material for integration in future electronic applications. The idealized surface of graphene, atomically-flat and without dangling bonds, offers the opportunity to understand the assembly of organic and inorganic molecules to form a wide range of ordered architectures and functional graphene-based heterostructures. In this review, we summarize recent progress in the growth of hierarchical nanostructures on graphene. The self-assembly of organic molecules and inorganic two-dimensional (2D) layers on graphene for the construction of various types of heterostructures are highlighted. Van der Waals interactions between the assembled molecules and graphene are shown to allow the formation of highly-ordered structures with preferred molecular orientations and stacking configurations that circumvent the strict lattice-matching requirements in traditional epitaxial growth. Finally, we briefly discuss representative applications of graphene-based heterostructures in electronic and optoelectronics.