With the development of computer technology, the means of human‐computer interaction are constantly updated. The development trend is from human to computer to constantly adapt human habits. In the human‐computer interaction, the somatosensory sensor technology is used to track and analyze human actions through computer vision technology. This paper adopts somatosensory sensor to implement gesture recognition, which is an important way in human‐computer interaction. The somatosensory sensor can obtain the depth information and bone information in the human‐computer interaction. First, the gesture is decomposed into a gesture sequence consisting of micro gestures to describe different directions. Then, the gesture sequence is compared with the gesture templates. The gesture is determined by the pattern matching result. The experimental results show that more than 92% gestures can be identified by proposed recognition method.