This study investigates the feasibility of using combination of Cement Kiln Dust (CKD) which is classified as a hazardous solid waste material and Silica Fume (SF), as a sustainable precursor to make binary binder based geopolymer paste. Furthermore, Rice Husk Ash (RHA) an agricultural waste is used as the main source for preparing amorphous silica in order to produce sustainable sodium silicate solution and also to replace it with the standard commercial sodium silicate. In this study alkaline activator solution was prepared through the combination of RHA powder with 10-M sodium hydroxide solution (NaOH) to adjust the Na2O/SiO2 and H2O/Na2O molar ratio. Geopolymer paste was produced by mixing different SF/CKD proportion (15/85%, 30/70%, 45/55%, 50/50%, 55/45%, 70/30% and 85/15% by weight) with the RHA-NaOH alkaline activator solution. The compressive strength test, Fourier transform infrared spectroscopy (FTIR), mini-slump test, setting time and X-ray diffraction (XRD) were carried out to characterize the geopolymer. The results show that it was feasible to make sustainable geopolymer with combination of SF/CKD binder and sodium silicate solution from RHA. By increasing CKD up to the optimum amount (45%) the compressive strength increased linearly and then the compressive strength decreased when it exceeds the optimum amount. The results also showed that the setting time of the geopolymer can be modified by changing the SF/CKD proportion. Moreover, the XRD presented that the geopolymer paste has an amorphous to semi-crystalline phase. Also, FTIR analysis illustrated that by increasing silica content, Si-O-Si (Al) bonds which are considered as the geopolymeric fingerprint area, will shift to the higher wavenumber.