Hybrid plasmonic waveguide provided high confinement of the propagating light and it is compatible with silicon on insulator technology. Graphene with unique optical properties has been used in optical modulator to give a high extinction ratio, broadband operation, and small footprint. In this paper, we used graphene as an absorber layer in a hybrid plasmonic waveguide structure to get sub-wavelength field confinement and better interaction between the graphene and the propagating light to reach maximum absorption of the graphene. We studied the effects of a different number (2–6) of the graphene layers on the extinction ratio, bandwidth, and energy consumption. The modulation extinction ratio is as high as 36 dB/μm and it has a flat response in a wide range of wavelength. The power consumption is about 14 fJ/bit. This modulator has a good response in all communication bands.