The traffic symbol system has been studies for many years. The system mainly has two phases, 1) Sign detection, 2) Recognition. It is one of the most important research area for enabling vehicle driving assistances. The automatic driving system should be simple in order to detect symbol with high responses. The challenge gets more difficult in order to make system simple while avoiding complex image processing techniques to detect symbol. The propose system consist of three main phases, 1) Frame selection, 2) Symbol detection, 3) Symbol Recognition. In frame selection phase the best frame possibly consisting of traffic symbol is selected from number of frames. In detection phase color image segmentation is perform on given frame in order detect the symbol. In recognition phase the detected traffic symbol is recognized using joint tranceform correlation (JTC) technique which is generally use for pattern matching.