Developing an environmentally benign, low-cost, and upgraded adsorbent possesses a great demand for arsenic removal from natural water systems. In this study, cellulose nanofibers (CNF) modified with iron and zirconium oxide nanoneedles (Fe-Zr-NN-CNF) were successfully synthesized for the rapid removal of As(V). Fe-Zr-NNs were homogeneously distributed on the surface of CNF using a simple hydrothermal synthesis route, which helped to overcome the agglomeration and loss of active sites of as-synthesized adsorbents. The vertical arrangement of the nanoneedles makes the active sites easily accessible to As(V), enabling its rapid and efficient removal. The surface morphology of the adsorbent and the successful chemical modification of CNF by impregnating iron and zirconium oxide nanoneedles were confirmed by field-emission scanning electron microscopy, Fourier-transform infrared spectroscopy, X-ray absorption near-edge structure, and X-ray powder diffraction. In addition, the metal concentration on the CNF surface was optimized by spectroscopic analysis to attain the superior efficiency of the adsorbent. The experimental kinetic data were fitted to a pseudo-second-order kinetic model, which suggested that the As(V) adsorption was chemisorption. The adsorption isotherm data were more closely fitted by the Langmuir model than by the Freundlich model. The maximum As(V) adsorption capacity (213 µmol g−1) was observed at pH 7.0 and at 25 °C. The kinetic study revealed that the As(V) adsorption rate was rapid in the first minute, and reached to equilibrium within 20 min. These results indicate that the adsorbent can efficiently and rapidly remove As(V) from natural water.