A novel three-dimensional graphene aerogel composite Nb2O5-g-C3N4/rGA was prepared, with a composition of nanorods-nanosheets Nb2O5-g-C3N4 (NbNR-CN) heterojunction and graphene aerogel (rGA). The 1D-2D NbNR-CN heterojunction was first prepared by a simple grinding and calcination method. Afterwards, different amounts of NbNR-CN (30%, 60%, 90%) were loaded onto graphene nanosheets by chemical reduction self-assembly method to obtain the 3D composites Nb2O5-g-C3N4/rGA (NbNR-CN/rGA). The compositions, morphologies, and optical properties of all materials were characterized. The results showed that the obtained 1D-2D heterojunctions were uniformly loaded onto rGA and the NbNR-CN/rGA composites had good visible light response. Compared with g-C3N4, NbNR and NbNR-CN, NbNR-CN/rGA showed the lowest charge transfer resistance and the highest photocurrent density. Photodegradation experiments indicated that the NbNR-CN/rGA composites exhibited remarkably enhanced visible-light photocatalytic activity for degradation of Rhodamine B (RhB). The removal rate for the 60%NbNR-CN/rGA composite was 94.8% within 100 min. The enhanced photoactivity could be explained by prolonged lifetime of photogenerated carriers due to the formation of heterojunction as well as the good electron transfer ability of rGA. Our research demonstrated that the NbNR-CN/rGA composite is a promising candidate for visible-light photocatalyst.