Road Sign Detection and Recognition (RSDR) is aimed to enable drivers maintain basic functionality with the aim of identifying and notifying driver through the existing restrictions so that the process is a success on the present widened road. Examples for RSDR include ‘traffic light ahead’ or ‘pedestrian crossing’ signs.An innovative RSDR system has been introduced which comprises of pre-processing, edge detection, feature extraction, features selection and Ensemble Fuzzy Support Vector Machine (EFSVM) classifier. Feature selection is carried out successfully by deployment of Ant Colony Optimization (ACO) algorithm to determine most prominent and definitive features. These features are then fed into the ensemble SVM to enable both road side traffic detection as well as recognition. Suggested system’s performance is analyzed and evaluated with respect to road signs having a capable recognition rate.