Controllable manipulation of propagating graphene plasmons (GPs) has important significance in the application of nanophotonic and optoelectronic technologies. In this paper, a simple graphene-semiconductor hybrid structure is proposed, in which unidirectional emission of propagating GPs can be achieved and controlled under circularly polarized light illumination. The impact of the distance between the nanorod and graphene and the radius of the nanorod on the excitation efficiency of GPs, as well as the impact on the unidirectional ratio are theoretically calculated in detail. The results show that considering both the unidirectional ratio and the excitation efficiency of GPs, the radius of the nanorod can be selected between 15 and 50 nm. It is also found that the maximum unidirectional ratio η of GPs does not depend on the different distances between the nanorod and graphene. This study not only provides optimized parameters for further experimental implementation, but also offers theoretical support for nanoscale information transmission and processing.