Abstract Pavement smoothness was recently identified as the most significant factor the motoring public uses to judge the quality of U.S. roadways. A new generation of lightweight profilers has been developed that have the potential of providing nearly instantaneous smoothness measurements, giving contractors a tool to identify and address process control issues promptly and cost effectively. This new technology was cooperatively evaluated by a team of Purdue University and Indiana Department of Transportation (INDOT) researchers to assess the performance of the devices, as well as the potential for their use in Indiana. A successful field test was conducted using four lightweight profilers that performed five replicate measurements at each of six sites: three, hot mix asphalt (HMA) and three, portland cement concrete (PCC). Smoothness data were reported in terms of International Roughness Index (IRI) and Profile Index with 0.0 and 0.2 in. (PI-0.0 and PI-0.2) blanking bands. The field test data showed good repeatability and poor reproducibility in terms of the smoothness indices considered, the profiles generated, and the identification of must-grind locations. Estimates of precision for the devices were determined in accordance with ASTM standards. The observed precision statements as well as statistical tests of between-vendor performance suggest that the performance of the devices, especially the reproducibility, should be improved prior to implementation of their use in construction specifications.