This paper describes an automatic road sign recognition system by using matching pursuit (MP) filters. The system consists of two phases. In the detection phase, it finds the relative position of road sign in the original distant image by using a priori knowledge, shape and color information and captures a closer view image. Then it extracts the road sign image from the closer view image by using conventional template-matching. The recognition phase consists of two processes: training and testing. The training process finds a set of best MP filter bases for each road sign. The testing process projects the input unknown road sign to different set of MP filter bases (corresponding to different road signs) to find the best match.