A simple rubbing treatment was used to mechanochemically modify the surface of polyacrylonitrile-based carbon fibers and its effect on their surface structure and functional groups was studied using several surface characterization techniques. To control the mechanochemical effect, the shear forces accompanying rubbing were kept constant. Scanning electron microscopy tests and the peak positions and widths of the main Raman spectroscopy bands indicated that there were no morphological changes to the carbon fibers following rubbing. In contrast, X-ray photoelectron spectroscopy showed an increase in oxygen-containing functional groups; in addition to hydroxyl species, the main groups introduced were alkoxide, carbonyl, and carboxyl groups. The ratio of carboxyl groups on the carbon fiber surface increased with the shear force magnitude, indicating carbon surface oxidation. The difference between the Raman and X-ray photoelectron spectroscopy results indicates that the modification was confined to the first few atomic layers; therefore, this rubbing method is capable of producing efficient mechanochemical surface modification of carbon fibers. This technique is simple, is relatively inexpensive, and is applicable to carbon fiber-reinforced plastic processing techniques.