Self-assembly is a critical process that can greatly expand the existing structures and lead to new functionality of nanoparticle systems. Multicomponent superstructures self-assembled from nanocrystals have shown promise as multifunctional materials for various applications. Despite recent progress in assembly of homogeneous nanocrystals, synthesis and self-assembly of Janus nanocrystals with contrasting surface chemistry remains a significant challenge. Herein, we designed a novel Janus nanocrystal platform to control the self-assembly of nanoparticles in aqueous solutions by balancing the hydrophobic and hydrophilic moieties. A series of superstructures have been assembled by systematically varying the Janus balance and assembly conditions. Janus Au-Fe3O4 dumbbell nanocrystals (<20 nm) were synthesized with the hydrophobic ligands coated on the Au lobe and negatively charged hydrophilic ligands coated on the Fe3O4 lobe. We systematically fine-tune the lobe size ratio, surface coating, external conditions, and even additional growth of Au nanocrystal domains on the Au lobe of dumbbell nanoparticles (Au-Au-Fe3O4) to harvest self-assembly structures including clusters, chains, vesicles, and capsules. It was discovered that in all these assemblies the hydrophobic Au lobes preferred to stay together. In addition, these superstructures clearly demonstrated different levels of enhanced surface plasmon resonance that is directly correlated with the Au coupling in the assembly structure. The strong interparticle plasmonic coupling displayed a red-shift in surface plasmon resonance, with larger structures formed by Au-Au-Fe3O4 assembly extending into the near-infrared region. Self-assembly of Janus dumbbell nanocrystals can also be reversible under different pH values. The biphasic Janus dumbbell nanocrystals offer a platform for studying the novel interparticle coupling and open up opportunities in applications including sensing, disease diagnoses, and therapy.