In this paper, carbon nanofibers (CNFs) as a novel solid phase extraction sorbent were developed for speciation preconcentration and separation of inorganic arsenic species As(III) and As(V) prior to determination by inductively coupled plasma mass spectrometry (ICP-MS). It was found that during all the steps of the separation, As(III) was selectively sorbed on the microcolumn packed with CNFs within a pH range of 1.0–3.0 in the presence of ammonium pyrroinedithiocarbamate (APDC), while As(V) was passed through the microcolumn without the retention. Various experimental parameters affecting the separation and determination of As(III) and As(V) have been investigated in detail. Under the optimized conditions, the detection limits of this method for As(III) were 0.0045 ng mL −1 with an enrichment factor of 33 and 0.24 ng mL −1 for As(V), and the relative standard deviations for As(III) and As(V) were 2.6% and 1.9% ( n = 9, c = 1.0 ng mL −1), respectively. In order to verify the accuracy of the method, a certified reference of water sample was analyzed, and the results obtained were in good agreement with the certified values. The proposed method was applied for the analysis of inorganic arsenic species in groundwater and lake water with the recovery of 92–106%.