Carbon nanotubes (CNTs) are of particular interest because of their ability to enhance the mechanical strength in the material; however, processing difficulties of CNTs often restrict utilization up to its full potential. To resolve this, in the current study, we have developed a simple and efficient method to functionalize the surface of a multiwalled carbon nanotube (MWCNT) with precise functional polymer chains, which were covalently grafted on the surface of the MWCNT, and delved into an application to demonstrate this material as an efficient nanofiller in developing a proton conducting membrane (PEM) from polybenzimidazole (PBI). At first, the MWCNT surface was converted to a polymerizable surface by attaching a trithiocarbonate based chain transfer agent (CTA). Then, a N-heterocyclic block copolymer, namely poly-N-vinyl-1,2,4-triazole-b-poly-N-vinyl imidazole (pNVT-b-pNVI), was grown from the CTA anchored surface with a one-pot surface initiated reversible addition–fragmentation chain transfer (SI-RAFT) technique. Grafting of block copolymer chain was confirmed by GPC, NMR, TGA, TEM, FESEM, and EDX studies. To the best of our knowledge, this will be the first report of a growing block copolymer structure grafted covalently on the surface of the MWCNT. The novelty of the work was further enhanced by incorporating pNVT-b-pNVI-g-MWCNT as a nanofiller into the oxypolybenzimidazole (OPBI) membrane to obtain homogeneous nanocomposite membranes with excellent thermomechanical and tensile properties, thermal stability, superior proton conduction when doped with phosphoric acid (PA), and PA holding capacity. The nanocomposite membrane with 2.5 wt % nanofiller loading displayed a tensile stress of 1.8 MPa and a strain of 176% at break. The basic N-heterocyclic rings dangling from the block copolymer chains grafted on the MWCNT surface allowed formation of strong H-bonding, acid–base interaction with PA, which is responsible for high acid uptake and superior PA retention, and also exhibited proton conductivity as high as 0.164 S cm–1 at 180 °C, which is a 2.6-fold increment when compared with a pristine OPBI membrane. This significant increase in conductivity is attributed to the proton conducting nanochannel pathway generated along the polymer-g-MWCNT surface.