Achieving proton-conducting solids with high proton conductivity at subzero temperatures remains a serious challenge. Thus, it is indispensable to develop the composites with high conductivity ranging from subzero to medium temperatures (243–343 K). Graphene oxide (GO) as an alternative matrix can provide the rich oxygen-containing groups. Thus, we prepared a series of proton conductors by covalently linking phosphate units onto the amino-functionalized GO monoliths (denoted NGOM-P), the structures of which are determined by FT-IR, XPS, XRD and Raman characterization techniques. Further, the fast proton transfer in effective acid-base networks in NGOM-P make the monolith achieve high proton conductivity from 2.61 × 10−2 S cm−1 (243 K) to 0.68 S cm−1 (343 K) under 98% relative humidity (RH).