AbstractControlled assembly of plasmonic particles is vital for catalyst and sensor applications. Because light is considered a promising external stimulus, various methods for assembling plasmonic particles using photoswitching molecules are investigated. However, the light used to photoisomerize such molecules is often limited to the UV region. In this study, anionic ortho‐fluoroazobenzene is synthesized to reversibly control the self‐assembly of plasmonic nanoparticles under only green and blue visible light. This azobenzene alters the dispersibility of cubic gold nanorattles protected by a cationic surfactant via visible‐light irradiation, causing a visible change in the solution color. It is possible that the cationic surfactant and anionic azobenzene interact to neutralize the surface charge of the gold nanorattles. Because the interaction is changed by the photoisomerization of azobenzene, the aggregation and redispersion of the gold nanorattles are repeatable. Through this method, the dispersibility of nonspherical plasmonic particles can be controlled using only visible‐light irradiation.