This paper deals with a similarity decision method between the shape of hand-postures and their structures to improve performance of the vision-based hand-posture recognition system. Hand-posture recognition by vision sensors has difficulties since the human hand is an object with high degrees of freedom, and hence grabbed images present complex self-occlusion effects and, even for one hand-posture, various appearances according to viewing directions. Therefore many approaches limit the relative angle between cameras and hands or use multiple cameras. The former approach, however, restricts user`s operation area. The latter requires additional considerations on the way of merging the results from each camera image to get the final recognition result. To recognize hand-postures, we use both of appearance and structural features and decide the similarity between the two types of features by learning.