The aims of this study were: (i) the mapping of asbestos cement roofs in an urban area; and (ii) the development of a spectral index related to the roof weathering status. Aerial images were collected through the Multispectral Infrared and Visible Imaging Spectrometer (MIVIS) sensor, which acquires data in 102 channels from the visible to the thermal infrared spectral range. An image based supervised classification was performed using the Spectral Angle Mapper (SAM) algorithm. The SAM was trained through a set of pixels selected on roofs of different materials. The map showed an average producer’s accuracy (PA) of 86% and a user’s accuracy (UA) of 89% for the asbestos cement class. A novel spectral index, the “Index of Surface Deterioration” (ISD), was defined based on measurements collected with a portable spectroradiometer on asbestos cement roofs that were characterized by different weathering statuses. The ISD was then calculated on the MIVIS images, allowing the distinction of two weathering classes (i.e., high and low). The asbestos cement map was handled in a Geographic Information System (GIS) in order to supply the municipalities with the cadastral references of each property having an asbestos cement roof. This tool can be purposed for municipalities as an aid to prioritize asbestos removal, based on roof weathering status.