To understand the relationship between the surface color and ageing of asphalt, the evaluation method for asphalt surface color was established with the color parameter of Munsell Value, then two asphalts were adopted to study the influence of asphalt ageing on its surface color, and the mechanism of whitening was investigated by the four generic fractions experiment, color spectrum tests and scanning electron microscopy (SEM) test. The results show that the surface color of asphalt will be changed by some factors, while ageing is an important factor. The thermal-oxidative ageing (TOA) of asphalt would lead to the asphalt to be darker in color due to the component migration caused by ageing, while the ultraviolet ageing (UVA) of asphalt will result in a grey or white asphalt surface because of the fact that UVA makes the surface wrinkled and rough which would cause the asphalt surface to change from specular reflection to diffuse reflection. Since the whitening of asphalt surface caused by UVA is much more serious than the blacking caused by TOA, the asphalt pavement always tends to appear grey or white during its service life. To verify the close relationship between the surface color and UVA degree of asphalt, two modified asphalts with anti-ultraviolet ageing agents, carbon black and layered double hydroxides, were suffered from UVA, and their ageing degrees were assessed with the parameters of Munsell Value and conventional physical properties, results indicate the two evaluations on UVA degree have much consistency, which means it is feasible to adopt Munsell Value, a parameter of asphalt surface color, to evaluate the UV ageing degree of asphalt.