A hierarchical system of classifiers for recognizing traffic signs from their images is proposed in the article. Only signs that fit into a square are considered. The traffic signs set is analyzed by their shape, color and basic features. A hierarchy of classes for traffic signs on the details of their images is proposed. The traffic sign image recognition algorithm uses this hierarchy. The algorithm only works with localized signs. The localization algorithm is not considered. Image preprocessing is performed at each level of the hierarchy for traffic sign features. Different classifiers are used at different levels of the hierarchy. Preprocessing at the first level uses the segmentation method. Signs are classified by their form in the first level. The edge segment is highlighted in the image of the traffic sign. The Euclidean distance in color space is used to calculate the edge segment. The edge segment outline defines the shape of the sign. Classification of the sign by its shape is carried out based on comparison with the standards. The classifier defines the pattern closest to the sign contour. The efficiency of the classification in the first level is 99.8 %. The second level classifies signs by the color of the edge segment. The effectiveness of this classification is close to 100 %. Internal sign images are obtained after the edge segment is removed from the image. The following levels classify by internal images. These classifiers use artificial neural networks. The efficiency of the system on the GTSRB collection is 98 %.