Abstract: Road signs are infrastructure assets crucial for transportation safety. Transportation agencies need to inventory signs regularly, e.g., annually or every 2 years, for asset management. This article proposes a framework of intelligent road sign inventory (IRSI) to enhance sign data collection, compared with existing manual or semiautomatic methods. The two key technical problems of sign recognition and attribute computation are addressed for IRSI. Because sign colors are important for both image segmentation and color features analysis, a novel kernel‐density‐estimation‐based statistical color model (KDE‐SCM) is proposed to describe the widespread and multimodel sign color distributions. The KDE‐SCM is embedded into a sign recognition algorithm by integrating sign shapes, textures, and other features. Furthermore, sign attributes, including the distance, local position, global positioning system (GPS) position, height, tilt angle, etc., are computed from image and GPS data. The proposed algorithms have been tested with the actual video log images collected on 35 km road segments in Kyoto City, Japan. For the KDE‐SCM color models, the false positive rates for the red and blue colors are 0.9% and 0.7%, while the false negative rates are 0.8% and 0.5%, respectively. The sign recognition algorithm was configured and applied to recognize three regulatory sign types of stop, no‐crossing, and no‐parking signs with 92.6% precision and 91.9% recall on average. The sign attribute computation model was also tested with the actual video log. The computation errors for distance, position, tilt angle, and height, etc., increase with the measurement distances. The sign‐to‐camera distances have the computation errors of less than 22 cm if the measurement distances are less than 15 m. The proposed algorithms demonstrate good potential for developing IRSI to enhance road asset data collection and processing.