Ultra-smooth surfaces with low contamination and little damage are a great challenge for aluminum optical fabrication. Ion beam sputtering (IBS) has obvious advantages of low contamination and non-contact that make it a perfect method for processing aluminum optics. However, the evolution laws of aluminum surface morphology are quite different from conventional amorphous materials, which affects the roughness change and needs systematic research. Thus, in this paper, the roughness evolution of an aluminum optical surface (i.e., aluminum mirror) subjected to IBS has been studied with experimental and theoretical methods. The surface morphology evolution mechanisms of turning marks and second phase during IBS are revealed. The newly emerging relief morphology and its evolution mechanism are studied in depth. The experimental results find that IBS causes the coarsening of optical surfaces and the appearance of microstructures, leading to the surface quality deterioration. Turning marks have been through the process of deepening and vanish, while second phase generates microstructures on the original surface. The corresponding mechanism is discussed exhaustively. Preferential sputtering, curvature-dependent sputtering and material properties play important roles on surface quality deterioration. A modified roughness evolution mechanism and an improved binary sputtering theory are proposed to describe the polycrystalline sputtering phenomena. The current research can provide a guidance for the application of IBS in aluminum optics manufacture fields.