This study develops a new methodology based on computational metrology techniques to measure rail surface roughness from vertical axle-box accelerations. A VIA rail passenger rail car, operating in eastern Ontario, Canada, was instrumented with accelerometers on the car body, trucks, and axle-boxes. The rail surface was measured by a heavy track geometry inspection car. A Gaussian filter was applied to the measured rail surface data and the rail surface roughness was quantified by the root mean square (RMS), as recommended by ISO 11562 for evaluating engineering surfaces. Values of rail surface RMS roughness calculated from axle-box accelerometer data are verified with the measured surface RMS roughness. The overlap ratio and length of the moving window over which the rail surface RMS roughness is calculated are studied with respect to roughness wavelengths, statistical considerations, and maintenance planning purposes. The effect of rail car operating speed as well as the difference between axle-box accelerations measured at two axles of the instrumented rail car on the accuracy of estimating the rail surface RMS roughness are assessed. Filtering techniques and application limitations for calculating roughness are also discussed. The results of this study suggest axle-box acceleration data provide a useful assessment of rail surface roughness for the typical wavelengths between 3 and 25 m and a complementary technique to light and heavy track geometry inspections.