Fly ash has the potential to be used as the precursor in the acid-activated geopolymer preparation, fly ash can react with phosphorus-containing activators to form geopolymers, but different types of activators present different activation effects on fly ash. In this study, common liquid phosphoric acid (PA), liquid aluminum dihydrogen phosphate (ADP), solid sodium dihydrogen phosphate (SDP) and solid potassium dihydrogen phosphate (PDP) are used as a single-component or composite activators and fly ash as raw material to prepare phosphate-based geopolymers (PGEOs). The effect of different activators on setting time, pH, electrical conductivity (EC) and compressive strength of PGEOs were assessed. Moreover, the activation mechanism of different activators on fly ash was revealed by using X-ray diffraction, Fourier transform infrared spectroscopy, Mercury Intrusion Porosimetry, X-ray Photoelectron Spectroscopy and Scanning Electron Microscopy-Energy Dispersive Spectroscopy. The results showed that geopolymer prepared using ADP exhibited the highest compressive strength among the different activators, but combining ADP and PA would prolong setting time. However, the unreacted SDP and PDP particles were detected in geopolymers activated by PA+SDP and PA+SDP composite activator due to insufficient geopolymerization. Nonetheless, combining PDP or SDP with PA as composite activator is more effective than the single-component PDP or SDP activator in improving compressive strength. The pH values of the geopolymers increased with the curing time while EC showed a decreasing trend which was independent of the activator type. AlPO4 phase was found in all samples, and Al-containing phases are the main strength-given component of PGEOs.